Accurate weather forecasting can save lives. Know where flash flooding is about to occur or where a hurricane is likely to tear through a town and you can warn people to flee the area and save them from harm. It’s the case with Hurricane Dorian, which has wreaked havoc across the Bahamas, killing at least 30, with many more deaths expected.
But what do you do when models differ in their predictions about how the weather will unfold? Two competing models – one from Europe and the other from the US – are battling to provide the most accurate weather reports possible.
The European Centre for Medium-Range Weather Forecasts (ECMWF), currently based in Reading, divides the globe into nine kilometre boxes, with 150 points vertically measuring up to eight kilometres in altitude. The National Weather Service at the US’s National Oceanic and Atmospheric Administration (NWS) has a similar system that it recently introduced, called the FV-3 system – Finite Volume on a Cubed-Sphere.
“On the organisational level, the ECMWF and NWS are designed to do two very different things,” says Jack Sillin, a forecaster at weather.us. “The ECMWF is focused purely around producing the very best medium-range (three to ten days) weather model that they can, while the NWS has a mission to protect life and property with watches, warnings, and other forecast products.” Both systems are used by forecasters to try and look ahead into the future. The US National Hurricane Centre utilises both.
The battle is a high-stakes one, involving not just lives but plenty of money: the United States faced 14 weather or climate incidents in 2018 that caused more than $1 billion (£811 million) worth of damage each. The last three years have seen the number of big money incidents double the long-run average. Even the most minor forecasting mistakes can have a huge impact, and critics of the NWS have long said that the American system is simply not good enough in comparison to the European one.
“If you want to know what the weather will be like in five or six days time, you need to know what the atmosphere is like now,” says Tony McNally, principal scientist at the ECMWF, which uses a constellation of between 30 and 40 satellites orbiting Earth making hundreds of millions of atmospheric and oceanic measurements, such as air pressure, temperature and the level of moisture, specific to weather forecasting every hour.
The US system at the NWS uses that same source data from the same satellites – but what happens with it after that is where the two models diverge. While the fundamental laws of physics are the same, what each model focuses on is different. They’re powered by calculations that simulate atmospheric physics. “Many of the equations can’t be solved absolutely,” says McNally. “You have to make approximations. You can do it in different ways, and they’ve adopted a different way to solving those equations.”
That’s what causes the vast differences in forecasts produced using each model. The forecasts for Hurricane Dorian have been relatively similar, with the ECMWF thinking the storm would track further southwest than it did in comparison to the NWS model. However previous examples, including 2015’s Hurricane Sandy, have shown massive differences. US modelling saw Sandy staying offshore, while the European model accurately predicted landfall in New Jersey.
“Very small changes in how you start to do things can rapidly amplify,” McNally says. “These details are critical – particularly when looking at long range forecasts.” And while we know a lot about how weather works, some things we don’t. “We still depend on some data that is not available all the time and every time,” says Vijay Tallapragada, the NWS’s chief of modelling and data assimilation. “We’re trying to construct the real atmosphere using available data and that introduces a source of uncertainty.”
For years, the ECMWF has proven more accurate. A 2016 analysis of forecast accuracy showed that the US model scores 0.857 on a scale from zero to one (where one is perfect forecasting), while the European model scores 0.905. “That doesn't mean it gets every storm's forecast ‘correct’, nor is it even the most correct model for any given storm,” says Sillin. “However, over the course of many years, it generally does a little bit better than the other global models, especially the one run by the NWS.”
Both the ECMWF and NWS acknowledge the strengths (and differences of each other system). “Each model has its own strengths and weaknesses,” says Tallapragada. “In certain aspects NWS is better, in certain aspects the European one is better. We don’t see it as one is better than the other.” McNally adds: “There is healthy competition.”
The NWS has recently overhauled the way it models the atmosphere, which feeds into forecasts made using its data. “It took three years to build it, test it, and show it’s performing better than the previous version,” says Tallapragada. The change in the model mimics the European model in dividing up the globe into cubes and modelling them individually - a shift away from the old US method of portraying the atmosphere using mathematical waves.
The upgrade is the first step in a series of constant iterations of the FV-3 model, the NWS explains, with the next upgrade, to bring its resolution closer in line with the European one, coming in 2021. But while those behind the rework say it’ll be a major shift, others are more muted about its potential to bridge the gap.
“The new FV-3 model has made little difference,” says Clifford Mass of the University of Washington, a longstanding critic of the NWS model. “The National Weather Service still does data assimilation the same way, and poor data assimilation is the crucial reason why ECMWF has better forecasts.”
That could be unfair criticism, argues Sillin. People expected the new FV-3 model to provide massive advances in forecast accuracy. “But that immediate boost in skill hasn't been as dramatic as some might have expected because the purpose of the update was primarily to set the stage for more improvements to come, rather than to deliver a major short term gain,” he says.
Criticism has been laid at the accuracy of both models in forecasting where Dorian would go and land – but is unfair, says Mass. “Dorian was a particularly difficult storm. It was very small and entered a region of very weak steering currents, which made it difficult to forecast for several days.”
In addition, tiny perturbations in the atmosphere can have outsized impact – akin to the butterfly effect. Forecasts published in the middle of last week by the National Hurricane Centre expected Dorian to make landfall in Florida. However, intense thunderstorms over Kansas and Nebraska nudged the storm north. “After day one you won’t notice very small changes to that initial state, but later in the forecast range, those small things can suddenly explode into large differences,” explains McNally.
We’re also increasing our knowledge of how weather systems work every year. “It’s a complex process that we try to make incremental advancements each year as we gain more understanding,” says Tallapragada.
Besides, what’s important is less the fine detail, but the general information provided. “People take for granted we are forecasting these things and pick up on small differences like 50 miles north or south,” explains McNally. “You have to realise we are consistently giving six, seven or eight days advanced notice of these things happening. We’re helping save tens of thousands of lives per year.”