Scope 3 carbon emissions – indirect emissions from across a firm’s value chains – are the toughest to evaluate and predict. A recent study found a clear divergence across Scope 3 data from the three biggest providers, making it hard for investors to know their ‘real’ exposure to climate-related risk. However, algorithms can improve the prediction accuracy of these emissions.
This article is part of our series on current academic research into a range of sustainability-related investment topics. The papers discussed were presented at the latest annual Global Research Alliance for Sustainable Investment and Finance conference. We believe in science-led sustainable investment. Partnering with academic researchers can add value since thorough research helps us to grasp the scope of climate change and biodiversity loss, to quantify risk, and to develop fit-for-purpose solutions. This is why we sponsor GRASFI’s annual conference and share relevant scientific findings with investors, clients and the wider asset management industry on our websites.
The study described in the paper “Scope 3 emissions: Data quality and machine learning prediction accuracy”explores the quality of Scope 3 emissions data and how machine-learning models perform in predicting these emissions. Using datasets of three major providers, Refinitiv Eikon, Bloomberg and ISS, it finds the datasets diverge for emissions values reported by firms.
The authors conclude that users of Scope 3 emissions datasets should consider data quality and prediction errors when using input from third-party providers in their risk analyses.
The authors also find that the application of machine-learning algorithms can improve the prediction accuracy of aggregated Scope 3 emissions, but the improvement is limited.
Carbon footprints useful in assessing climate risk
Corporate carbon footprints indicate how much a company contributes to atmospheric greenhouse gas emissions and to global warming.
They are generally preferred over other climate transition risk metrics because they can be easily converted to dollar losses (using the effective carbon price) or hidden costs (using the future costs of carbon).
As such, carbon footprints are viewed as one of the most plausible metrics in assessing climate transition risks, even in the face of limited, inconsistent and inaccurate reporting.
The GHG Protocol put together by the World Resources Institute and the World Business Council for Sustainable Development in 2020 divides carbon emissions into three categories:
- Scope 1 – direct emissions from sources and assets controlled by a firm
- Scope 2 – indirect emissions from purchased electricity
- Scope 3 – indirect emissions from across a firm’s value chains.
The protocol requires all firms to report Scope 1 and 2 emissions; reporting Scope 3 emissions is left to their discretion.
While there is little dispute as to the importance of quantifying Scope 3 emissions, doing so accurately along the entire value chain (sometimes stretching to thousands of units and subsidiaries) can be challenging. Scope 3 covers areas ranging from acquiring and pre-processing raw materials (upstream) to distributing, storing, using and disposing of the end-products sold to customers (downstream).
Its breadth means Scope 3 represents the most significant emission reduction opportunities. A full assessment of these emissions is critical to understanding the end-to-end impacts of carbon tax and climate policies on individual firms.
Scope 3: problems
However, analyses of firm-level emissions by external stakeholders are usually limited to Scope 1 and Scope 2 emissions, as Scope 3 carries several problems:
1. No regulation and lack of clear guidance. Internationally, there are no binding rules on Scope 3 emissions disclosures. As measurement and disclosure of Scope 3 are inconsistent and unsystematic, the quality and accuracy of firms’ voluntary disclosures remain unclear.
2. Incomplete composition/activity exclusion. Firms are not required to disclose the full composition of their Scope 3 emissions and may choose to report only on areas where they are performing well or that are easier to measure, so using aggregated Scope 3 emissions data can be misleading.
3. Measurement divergence. Scope 3 emissions diverge because firms may set different operational boundaries on the same emissions category or report different values across different communication channels such as sustainability reports or annual fillings or third-party initiatives (such as the Carbon Disclosure Project).
These difficulties make it hard for third-party data providers to provide consistent measures. Asset managers and institutional investors should be aware of measurement divergences when performing analysis and constructing investment portfolios using Scope 3 data.
Data divergence
The proportion of identicaldata points (within 1% error) between Bloomberg and Refinitiv-Eikon was unexpectedly low (68%), given that reported emissions are supposed to be similar. The divergences persist over time.
Data inconsistency was also more prominent among sectors that are relatively green (e.g., real estate, financials, information technology, and communication services) or those that are notably brown (e.g., utilities). Nevertheless, this divergence is not expected to have a major impact on the formation of low-carbon portfolios.
However, the authors found considerable divergences when comparing Bloomberg or Refinitiv-Eikon to ISS. The latter adjusts firm-reported values to address inconsistencies in reporting and differentiate between upstream and downstream emissions. As a result, only 5% of ISS’s Scope 3 emissions is similar to the other two datasets.
If adjusted emissions from ISS are used, only 22% of ISS’s emissions data falls into the same ranking decile as the other two datasets, meaning that the constituents in a carbon-sensitive portfolio could be quite different.
Machine learning has limited prediction benefits
The study found that the use of machine learning, even the most sophisticated machine-learning algorithms, does not significantly improve prediction accuracy when compared to simpler methods such as industry fill and naïve regression models.
While the industry fill and naïve regressions models are not perfectly accurate, they are still useful for predicting Scope 3 emissions, especially when emissions estimates from all individual categories are obtained and aggregated.
However, the results are better for upstream emissions than for downstream emissions which remain difficult to predict.
Room for improvement
Given that firms tend to only disclose those Scope 3 emission categories that are easier to calculate, the authors emphasise the need to improve Scope 3 emissions disclosure. In particular, they recommend that binding mandates be established, and that more guidance is needed to derive accurate calculations. Firms should expand their operational reporting boundaries to include the categories most relevant to their businesses and source primary data from their value chain partners.
Moreover, the authors’ findings suggest that researchers and industry practitioners should be wary of the potential errors when using third-party data, whether publicly disclosed emissions data or estimated data based on prediction models. The authors see a need for data providers to be more transparent on their estimation methodologies and prediction performance.
“Given the challenges of using Scope 3 data , investors and stakeholders should be careful with such data and be aware of the potential biases and limitations of the sources they use. Researchers, regulators and industry practitioners should collaborate more closely to improve the accuracy and transparency of the data and develop more effective methods for estimating and predicting these emissions, especially for downstream categories where data quality now is often poor.”
Raul Leote de Carvalho, Deputy Head Quant Research Group, BNPP AM
Disclaimer
