Energy-Aware Cloud Architecture in the AI Era

Energy-aware architecture is not a separate ESG workstream and it is not achieved by selecting a region with a green label. It is an engineering discipline: measure the energy and emissions associated with a defined unit of useful work, then change software, models, schedules and infrastructure without breaching reliability, security or data-location requirements.

The decision must begin with a functional unit such as an order completed, a document processed, a successful inference or an artefact built. Without that denominator, lower consumption may simply reflect lower traffic, while a better data-centre PUE can coexist with an inefficient application.

Treat power as a capacity constraint

The IEA Energy and AI report describes the growing dependency between digital infrastructure, electricity generation and grid capacity. For an architect, the practical implication is that compute plans cannot assume power is available on demand. Location, delivery lead time, cooling and accelerator utilisation affect both cost and the ability to launch a service.

Product, platform, FinOps and energy owners need one demand model. It should connect expected requests or jobs, the hardware profile, running time, required headroom, energy characteristics and regional constraints. Reserving more accelerators before understanding the workload can lock low utilisation into a multi-year procurement decision.

Separate online services with hard latency objectives from batch and experimental work. Online capacity needs predictable response and operational headroom. Batch training, analytics and evaluation can often be queued, combined, paused or moved to another execution window. That flexibility is an architectural property and should be defined before a scheduler tries to exploit it.

Define the measurement boundary

ISO/IEC 21031:2024 specifies Software Carbon Intensity as a method for calculating emissions per functional unit of a software system. The method considers energy consumed, the carbon intensity of that energy and an allocated share of embodied hardware emissions. A cloud invoice or provider sustainability statement is therefore not a sufficient application measure.

The boundary should match the decision. For an inference service it may include the gateway, preprocessing, model serving, cache, storage and data transfer. For an analytics pipeline it may include orchestration, execution, result retention and retries. Record exclusions explicitly and preserve the model, hardware and configuration version behind each observation.

Normalise by useful work. Cluster energy per hour is operationally useful, but it cannot compare systems producing different volumes or quality of results. For AI, energy needs to be read alongside response quality, throughput and latency. An optimisation that causes more retries or incorrect outcomes can increase total consumption despite making one execution cheaper.

Remove unnecessary work before tuning hardware

The lowest-energy operation is the one that does not run. Review data retention, recomputation frequency, duplicate pipelines, idle development environments and experiments left online. Event-driven updates may replace periodic full scans. A cache may prevent repeated inference. A smaller input or shorter retention policy may eliminate storage and downstream processing together.

For AI services, order the decisions deliberately. Select the smallest model that meets the quality requirement, then examine quantisation, batching, context length, caching and routing between model classes. Only after those choices are understood can the team select an accelerator and autoscaling strategy with credible demand assumptions.

Measure actual traffic distributions rather than daily averages. A GPU may appear allocated while spending substantial time waiting for data or communication. Useful signals include utilisation distributions, queue depth, initialisation time, idle intervals and energy per correctly completed unit. Combine them with the causes of failed or repeated work.

Schedule flexibly within the service contract

Carbon-aware scheduling is appropriate when a workload can tolerate delay or a change of location. Give each workload an explicit flexibility class: allowed execution window, eligible regions, minimum capacity, data constraints and maximum waiting time. A scheduler can then choose based on agreed boundaries rather than making an opaque trade-off on behalf of the product.

Do not sacrifice reliability or data residency for a better carbon estimate. Grid-intensity forecasts are estimates, and moving a large dataset consumes resources. Define a fallback rule: when a deadline approaches or no compliant region is available, execute in the primary location. Keep the decision and the forecast used so its effect can be reviewed.

Account for rebound effects. Cheaper, more efficient execution can increase usage enough to raise total consumption. Measure the efficiency of a functional unit and the total workload volume over the same period. If demand expands, distinguish product growth from unnecessary repeated work.

Keep facility and software measures distinct

The EU Delegated Regulation 2024/1364 defines indicators for reporting data centres including Power Usage Effectiveness, Water Usage Effectiveness, Energy Reuse Factor and Renewable Energy Factor. They describe important facility characteristics, but they answer different questions from workload efficiency.

PUE compares total data-centre energy with energy used by IT equipment. It does not establish whether that equipment performs useful work. WUE describes water use, ERF energy reuse and REF renewable energy. Improving one measure can create a trade-off elsewhere, so an architectural decision should not collapse them into an unsupported single score.

An application dashboard can combine the functional unit, cost, energy, estimated emissions, quality and service objectives. Facility or regional indicators remain context. When a provider does not expose sufficiently granular data, label the estimate, source and uncertainty instead of presenting it as direct measurement.

Turn experiments into operating defaults

Start with one material workload that has a clear owner and a reproducible test. Freeze the dataset or traffic profile, record a baseline for quality, latency, cost and energy, then change one factor at a time. If a result improves without violating the service contract, capture it in a supported platform default rather than leaving it in an optimisation document.

Ongoing measures can include energy and cost per functional unit, accelerator utilisation, the share of flexible work completed in optimised windows, delayed jobs missing their deadline, total period consumption and measurement-data quality. Each measure needs an owner and a defined response to an adverse change.

Energy-aware architecture is credible when it produces an auditable choice: a different model, schedule, code path, retention rule or hardware class. A broad claim about a sustainable cloud does not tell engineers what to change or decision-makers what outcome they purchased.

Sources

• Energy and AI — International Energy Agency, 10 April 2025.
• ISO/IEC 21031:2024 — Information technology — Software Carbon Intensity specification — ISO/IEC, 22 March 2024.
• Commission Delegated Regulation (EU) 2024/1364 establishing a data-centre rating scheme — European Commission, adopted 14 March 2024 and published 17 May 2024.