Benchmarking · Glossary

Benchmarking refers to the methodically regulated comparison of costs, performance, emissions, technical parameters or processes against a reference value. This reference value may derive from the average of comparable actors, from the best observed values, from technical target figures or from politically set thresholds. In the electricity system, benchmarking makes efficiency, quality and target achievement measurable. This applies especially where normal competitive signals are absent or where political goals have to be translated into concrete requirements.

A benchmark is not just any comparison value. It serves as a yardstick against which a plant, a network operator, a business process or a regulatory target path is measured. Typical metrics are costs per kilometre of line, costs per connected customer, outage minutes per final consumer, CO₂ emissions per kilowatt hour, investment costs per kilowatt of installed capacity, or operating costs per quantity of electricity transported. How meaningful a benchmark is depends on whether the units compared are truly comparable and whether the reference variable reflects the relevant performance.

Comparison is not yet benchmarking

Benchmarking is often equated with ranking, monitoring or key-figure analysis. The terms are close to one another but describe different functions. A ranking orders actors by a value. Monitoring observes developments over time. A key figure condenses data into a measurable quantity. Benchmarking combines these elements with an evaluation rule: a value is not merely collected but measured against a yardstick, and a consequence follows from it.

This consequence can be regulatory, economic or strategic. A network operator can check whether its maintenance costs are conspicuously high compared with similar network areas. A regulatory authority can derive efficiency requirements for revenue caps from this. An industrial company can compare its electricity consumption per production unit with other sites. In the European emissions trading system, product benchmarks determine how many free allowances certain industrial installations receive.

A benchmark is therefore never entirely neutral. It contains assumptions about which performance counts, which costs are recognised, which differences are adjusted out and which deviations are regarded as explainable. Anyone who wants to understand the effect of a benchmark must look at the rule by which it is formed.

Role in the regulation of electricity networks

Benchmarking is particularly important for electricity networks because transmission and distribution networks are natural monopolies. Households, businesses and generators cannot freely switch their network operator the way they can switch an electricity supplier. Without regulation there would be a risk that inefficient costs are passed on to connected users via network charges. At the same time, regulation must allow enough revenue for networks to be maintained, expanded and modernised.

In German incentive regulation, benchmarking is called the "Effizienzvergleich" (efficiency comparison). Under Section 12 and Annex 3 of the Incentive Regulation Ordinance (ARegV), two methods are applied: non-parametric data envelopment analysis (DEA) and parametric stochastic frontier analysis (SFA). The efficiency frontier is formed by the network operators with the best ratio between network performance delivered and expenditure. They receive an efficiency score of 100 percent, all others a correspondingly lower score. If the results of the two methods differ, the higher value counts for the network operator (best-of-four... best-of-two settlement, "Bestabrechnung"). If the result is below 60 percent, a minimum value of 60 percent is applied under Section 12(4) ARegV. The identified inefficiency must be reduced arithmetically via a reduction path. From this follows the revenue cap, i.e. the amount the network operator may refinance through network charges.

The legal framework is currently changing. The Bundesnetzagentur (Federal Network Agency) is gradually replacing the ARegV with its own determinations: the framework determination "RAMEN Strom" and the methodology determination "Effizienzvergleich Strom" (GBK-25-02-1#2). The methodology for the fifth regulatory period for electricity is based on these determinations. Here too, two methods are applied. The expenditure parameter is total expenditure (TOTEX), less the cost components that are not subject to the efficiency comparison. Small network operators can take part in the simplified procedure instead of the efficiency comparison. Details such as the minimum efficiency score should therefore be checked for the respective regulatory period in the current text of the determination.

The real difficulty lies in comparability. An urban distribution network with many connections in a small area has different cost structures from a rural network with long lines and few connections. A network area with many decentralised generation plants, heat pumps or charging points places different demands on planning, control and operation than an area with little electrification. For this reason the efficiency comparison works with structural parameters, such as line lengths, connection points, area, loads or installed decentralised generation capacity. If such characteristics are not captured, or only inadequately, differences arising from geography, settlement structure or connection tasks wrongly appear as inefficiency.

Benchmarking is thus closely linked to network charges, security of supply and investment incentives. A yardstick that is too soft protects inefficient costs. A yardstick that is too hard or poorly adjusted can make investment difficult, crowd out maintenance or shift costs into areas that are treated more favourably by regulation. For this reason incentive regulation supplements the pure cost comparison with a quality element that takes into account, for example, the reliability of supply.

Technical and economic reference variables

In the electricity system, benchmarks apply at very different levels. For power plants and generation technologies, comparisons often involve costs per megawatt hour, CO₂ emissions per kilowatt hour, availability, efficiency or investment costs per kilowatt of installed capacity. For storage, the focus is on charging and discharging power, storage capacity, efficiency, cycle stability and costs per usable kilowatt hour. For networks, line lengths, number of connections, load density, supply quality and operating costs are the most important comparison dimensions.

A common oversimplification arises when a single key figure is read as a complete efficiency verdict. Low costs per megawatt hour may result from high utilisation, good technology, favourable sites, low capital costs or incompletely captured follow-on costs. A low emissions value per kilowatt hour says little about whether a plant is available at times of high residual load. And inexpensive network expansion per kilometre does not reveal whether the network was reinforced in the right place.

The choice of system boundary also changes the result. A benchmark for generation costs can exclude grid connection costs. A comparison of heat pumps can look only at electricity consumption or additionally include the fuel saved. An industrial benchmark can take the electricity price at the site into account or be limited to technical consumption figures per product unit. For the same plants, different system boundaries yield different conclusions.

Benchmarking in climate policy and emissions assessment

In climate policy, benchmarking serves to compare emission intensities and to translate reduction targets into concrete requirements. The best-known example is the EU Emissions Trading System. Since 2013, free allowances have gone mainly to manufacturing industry. Electricity generation must in principle buy its allowances at auction. For industrial products there are performance benchmarks that reflect the average emission intensity per product unit of the 10 percent most efficient installations of a sector in the EU. Free allocation results, put simply, from production volume times the benchmark value. Corrections are added, for example through the cross-sectoral correction factor or the carbon leakage classification.

It is therefore misleading to say that installations "close to the benchmark" are more efficient. The benchmark is itself a demanding best-performance value. Whoever meets or falls below it in principle covers their needs with free allocation. All others must buy additional allowances and thereby have an incentive to reduce emissions. The benchmark values are regularly adjusted to technical progress. For sectors less exposed to carbon leakage, free allocation phases out by 2030.

This logic is relevant for the electricity system because electricity increasingly replaces other energy sources. When assessing electromobility, heat pumps or electrolysis, an isolated look at the additional electricity consumption is not sufficient. What is decisive for the emissions effect is which electricity generation is additionally needed, which load profiles arise and which fossil energy sources are replaced. An average emission factor per kilowatt hour helps when comparing quantities of electricity. But it does not automatically explain how additional demand at a particular point in time affects the system.

Particularly in electrification, misunderstandings arise when benchmarks are used statically. Today's emission factor of the electricity mix is an average value. For investment decisions, the expected future value is often more important. For network operation and flexibility, the timing of demand can matter more than the annual average. A device with a good annual balance can be unfavourable for the network if it places its load in hours that are already critical.

Typical misinterpretations

Benchmarking appears precise because it produces numbers. The methodological decisions behind these numbers easily remain invisible. Which costs are included? Are capital and operating costs considered separately or together? Are regional differences in wages, land or permits taken into account? How are extraordinary events and statistical outliers treated? How good is the data? Whether a benchmark provides reliable guidance or merely creates spurious precision depends on these questions.

Another misunderstanding concerns best values. The best observed value is not automatically a sensible target value for everyone. It may be based on special site conditions, one-off effects or shifted risks. Very low maintenance costs at a network operator can indicate an efficient organisation. But they can also mean that renewals are being postponed. Without quality indicators such as the frequency and duration of supply interruptions or the condition of the equipment, a cost comparison remains incomplete.

Politically, too, benchmarks are sometimes used as simple proof of the superiority of a technology. A cost benchmark for photovoltaics, wind power, gas-fired power plants or storage is only meaningful, however, if it compares the same system task. Firm capacity, short-term controllability, energy over many hours, seasonal availability and grid-serving behaviour are different functions. A kilowatt hour from two plants can have the same energy value on the market and yet contribute differently to network operation and security of supply.

Incentives and side effects

Benchmarking changes behaviour. When revenues, support levels, allowance allocations or management targets are tied to comparison values, organisations align their decisions accordingly. This can trigger the desired efficiency gains. But it can also lead actors to rebook costs, postpone investments or design processes so that they look better in the benchmark without actually becoming more productive.

In regulated electricity networks, this particularly affects the distinction between operating and investment costs, the recognition of innovation measures and the handling of new tasks arising from decentralised generation, storage, controllable consumers and electromobility. If a benchmark mainly reflects past network structures, it captures new requirements only inadequately. Regulation must then clarify which additional costs are inefficient and which result from changed connection and control tasks. This is also why it separately specifies which costs are regarded as non-controllable and are not included in the efficiency comparison.

Benchmarking is thus an instrument for disciplining costs and for translating political or regulatory goals into measurable comparison variables. It replaces neither technical planning nor market design nor decisions about responsibilities. Its strength lies in making deviations visible. Its limit lies where different tasks, risks or system boundaries are squeezed into a single key figure. A robust benchmark makes comparability transparent, discloses adjustments and links cost comparisons to the performance that is actually to be delivered in the electricity system.

Sources

Anlage 3 (zu § 12) Anreizregulierungsverordnung (ARegV) — Bundesministerium der Justiz / gesetze-im-internet.de; unbekannt. Anlage 3 Nr. 1–3
Effizienzvergleich – Methodik für die 5. Regulierungsperiode Strom — Bundesnetzagentur, Beschlusskammer 8; unbekannt. Abschnitte Methodik, Vereinfachtes Verfahren, Effizienzvergleichsverfahren
About free allocation — Europäische Kommission, GD Klimapolitik; unbekannt. Abschnitte Free allocation to installations; How is free allocation calculated
Carbon leakage — Europäische Kommission, GD Klimapolitik; unbekannt. Abschnitt Free allocation
Anreizregulierungsverordnung (ARegV), Gesamtausgabe als PDF — Bundesministerium der Justiz / gesetze-im-internet.de; unbekannt. § 12 Abs. 3 und 4 ARegV

Related glossary entries

Edited by eCMS on 11.11.2026 15:28