Climate models are vital for understanding how Earth's climate is changing and how it may evolve in the future. But, checking how well these simulations are able to reflect the real-world – a key factor in their success – has been slow, often taking months to complete, despite the growing need for timely climate information to support decision-making and action.
Now, a newly available and free-to-use tool referred to as the Rapid Evaluation Framework (REF), makes the previously cumbersome comparison of model outputs against real-world observations much faster and easier by automating a series of standardised diagnostic checks.
By accelerating evaluation of climate model performance, the REF helps to ensure the latest advances in climate science inform key assessments such as those produced by the Intergovernmental Panel on Climate Change (IPCC) and decision-making processes.
A community-built tool for climate-model evaluation
The REF has been developed by a team of scientists working on the Coupled Model Intercomparison Project (CMIP), a global collaboration that develops, compares, and improves the latest climate models, and draws on feedback and requirements from the scientific community, and is detailed in a new research article.
The platform has been released in time to support assessment of the latest generation of Earth system models currently being finalised as part of the CMIP7 Assessment Fast Track (AFT) - a set of priority experiments designed to deliver evidence for the IPCC's Seventh Assessment Report on the state of the climate.
By shortening the time needed for systematic model evaluation, the REF helps ensure the latest climate science can be assessed and incorporated more quickly.
“The REF brings together climate scientists and Earth observation researchers to quickly check how accurately climate simulations reflect reality. The better we can do that, the more reliable our picture of future climate change becomes, and the better equipped policymakers and communities are to respond to it,” says Dr Ranjini Swaminathan, co-lead of the Model Benchmarking Task Team and a scientist at the UK's National Centre for Earth Observation, University of Reading.
Records from Space
Earth observation data records represent a key reference for model comparison. Recognising their importance, the European Space Agency through its Climate Change Initiative has developed a suite of satellite climate records tailored to cater for the modelling community’s requirements. These high-quality records provide the observational evidence at the appropriate spatio-temporal resolution and are both fully validated and uncertainty characterised – all features required by modellers.
Four ESA-supported datasets are already in use in the CMIP7 AFT version of the REF, providing information on Arctic and Antarctic sea ice, snow cover and atmospheric ozone, with further satellite records for sea surface temperature, snow water equivalent and cloud properties being prepared for future inclusion.
As part of its on-going commitment to strengthening links between Earth observation and modelling communities, to ensure the utility and uptake of satellite records for model assessment, the European Space Agency has both co-funded the development of the REF and hosts the team responsible for coordinating CMIP at its UK European Centre for Space Applications and Telecommunications facility.
“Satellites provide an independent, global view of Earth's changing climate. By bringing ESA's Climate Change Initiative records into the framework, we can assess models being prepared for the next IPCC assessment against consistent, quality-assured observations. This makes model evaluation more timely and comparable,” says Eleanor O'Rourke, Director of the CMIP International Project Office.
Beyond providing observational datasets for model evaluation, ESA is also helping improve the CMIP7 models through the development of forcing datasets for greenhouse gases, ozone and volcanic aerosols. Based on the latest Earth observation, and ground-based records, these datasets provide a more accurate representation of the factors driving climate change – compared to model assumptions – and so will help improve the consistency and reliability of climate projections.
The REF is freely available online and can also be deployed locally at modelling centres. It provides transparent performance information to assess model performance including the identification of biases and can be used by modelling groups to compare model iterations and improvements. Results are delivered through an interactive dashboard, alongside downloadable outputs in formats such as NetCDF, CSV and PNG, enabling more bespoke and in-depth analysis. The REF also includes an API, allowing users to run evaluation metrics from established community tools.
Full details and access to the Rapid Evaluation Framework are available from the World Climate Research Project CMIP website.