On 4 July 2024 I went to a sustainability meetup in Amsterdam, held in a venue with an âA LABâ sign on the stage. Two talks are in my photos: an open-source high performance computing (HPC) framework that schedules work by carbon intensity, and a practical session on reducing cloud emissions in Azure. I donât name the organiser, since it isnât on the slides I photographed. This is a throwback built from those slides.
The stage at the start of the HPC talk.
Sailfish-HPC: HPC on Kubernetes with KEDA
An âAbout usâ slide introduced the first speakers as Lisette van Leeuwen and Zeid Adabel, both software engineers. Their talk was about Sailfish, an HPC framework built from Kubernetes components such as KEDA and a Knative gateway, as their design slide showed. The demo slide said âItâs open-source!â.
The âFramework Designâ slide listed the principles: open-source based, bring your own container, managed operators for components, Kubernetes for orchestration, and an infrastructure layer that âdoes not matterâ. Another slide showed the flow: a Knative gateway API takes a job from the user, a job queue holds tasks, managers split them and workers pick them up.
Framework Design: open source, bring your own container, operators on Kubernetes.
KEDA is the Kubernetes event-driven autoscaler, a CNCF graduated project. Its site highlights scale-to-zero and sustainability. That fits an HPC queue: workers scale with the number of waiting tasks instead of idling.
Carbon intensity and space shifting
The talk used Green Software Foundation material. The Green Software Foundation is a Linux Foundation nonprofit that publishes standards such as the Software Carbon Intensity metric. The slides built the argument in steps:
- A slide on global data-centre electricity consumption showed a rise from 460 TWh in 2022 to 1000 TWh in 2026.
- âQuantifying carbon emissions in cloudâ combined energy consumption with carbon intensity to give emissions.
- A world map showed carbon intensity of electricity grids.
Global electricity consumption of data centres: 460 TWh in 2022, 1000 TWh in 2026.
Then came the Sailfish Operator and space shifting. The slide showed the rule MIN(grid_intensity[EU_West], grid_intensity[EU_North]). In other words, run the work in the region whose grid is cleaner right now. A âSustainable Developmentâ slide said the platform âdeploys across time and spaceâ, meaning the same operator can also shift when jobs run. A later screenshot showed queries mixing grid carbon intensity with Azure region availability.
Space shifting: pick the region with the lower grid intensity.
The demo slide for sailfish-hpc.
My take: location shifting only works for jobs that tolerate moving, such as batch and HPC runs with no data-residency limits. Latency-bound services canât take part. I wrote about the scheduler side in Carbon-aware Kubernetes scheduling.
Reducing carbon emissions in Azure
The second talk came from the founder of CloudXcellence, per the bio slide. It started with the simplest lever: âPick your region wiselyâ, with a slide comparing the CO2 of different Azure regions.
âWhat is the simplest way to reduce any cloud emission? Pick your region wisely.â
Other slides covered:
- Azureâs carbon optimisation view keeps only a 12-month sliding window of scope 1, 2 and 3 emissions, while preview APIs give up to five years.
- Top Azure emissions on one slide: storage, virtual machines, Databricks/SQL, logging and Azure Data Factory.
- Storage tips: move non-production from GRS to LRS and archive after 180+ days.
- Gartnerâs prediction, quoted on a slide, that 75% of organisations will have a data-centre infrastructure sustainability programme by 2027.
A FinOps versus GreenOps slide quoted that FinOps gets better deals on the number of machines but canât say whether those machines are needed. The overlap was right-sizing and clean-up.
FinOps vs GreenOps: savings plans and reserved instances on one side, low-carbon regions on the other.
The take-aways slide was short: check your own footprint, start sustainability groups within your company, follow meetups, and learn to write green software, for example in hackathons.
