Note 05 · Model comparison
Which forecast model is closest?
Over one week at 185 airports, the AIFS as shown in Marble had the smallest temperature error of the models we tested, the IFS came second and NOAA GFS last; for wind and rain, no model was clearly best.
A summary of internal testing. Methods are described only in outline. Not an official forecast or warning.
- 1.21 °Caverage temperature error one day ahead, AIFS as Marble shows itIFS 1.37 °C, GFS 1.93 °C
- 116 / 185airports where the AIFS was closestIFS 52, GFS 17
- 2,245comparisons per model and lead time13 times, 15–21 Nov 2026
- 3.1 → 1.7 °CIFS error at the 18 airports above 1,500 m, raw versus as shownone day ahead
What we compared, and what we did not
Most weather maps show the same few global models, run by national and international centres. A fair comparison is therefore a comparison of those models, using only data their producers publish openly, against the same independent observations. This is not a test of any other app: other products may process the same models differently, and could be more accurate than the raw model data tested here.
We scored the ECMWF IFS and AIFS (raw and as Marble shows them, with its terrain adjustment), NOAA GFS at the same 0.25° resolution (raw), and, at 27 stations, NOAA’s 3 km HRRR model over the United States. The observations are reports from 185 airports chosen to spread across six regions, a third of them above 1,000 m where available, at 13 times between 15 and 21 November 2026, for forecasts one, three and five days ahead. The score is the average error (mean absolute error).
Average temperature error one day ahead
Chart in words
Mean absolute error of 2 m temperature at +24 h, 185 airports, 13 times on 15–21 November 2026, 2,245 comparisons per model: AIFS as Marble shows it 1.21 °C, IFS as shown 1.37 °C, IFS raw 1.59 °C, GFS raw 1.93 °C.
Temperature
The AIFS as Marble shows it had the lowest temperature error at every lead: 1.21 °C one day ahead, 1.58 °C at five days. The IFS followed (1.37 and 1.83 °C), and GFS had the largest error (1.93 and 2.32 °C). The terrain adjustment mattered most in the mountains: at the 18 airports above 1,500 m it cut the IFS error from 3.1 to 1.7 °C.
Average temperature error at 1, 3 and 5 days
Chart in words
Mean absolute error of 2 m temperature, 185 airports. AIFS as Marble shows it: 1.21, 1.32, 1.58 °C. IFS as shown: 1.37, 1.54, 1.83 °C. GFS raw: 1.93, 2.07, 2.32 °C.
Closest model at each of 185 airports
Chart in words
One dot per airport, by the model with the lowest average temperature error over all three lead times: AIFS at 116 airports, IFS at 52, GFS at 17.
Humidity, wind and rain
Dew point followed the temperature ranking: AIFS 1.43 °C, IFS 1.58 °C, GFS 2.09 °C one day ahead, with GFS too dry by about 1.2 °C. For wind speed the three models were statistically indistinguishable (1.45 to 1.50 m/s). For whether it rains in a six-hour window, no model was clearly best: one day ahead, all three scored between 0.32 and 0.33 on the standard threat score. The AIFS forecasts light rain too often, so its rain layer looks wetter than the IFS layer in areas of light rain.
The 3 km HRRR, tested at 27 US airports one day ahead, was not more accurate on this small sample: temperature error 1.60 °C, against 1.56 °C for the IFS and 1.42 °C for the AIFS as Marble shows them. High-resolution models are built for the first hours and for thunderstorms, which a week of mostly calm November weather does not test.
Data and attribution
Forecast data: ECMWF open data (IFS, AIFS), CC BY 4.0; NOAA GFS and HRRR via the NOAA Open Data Dissemination program. Observations: airport reports via the NOAA Aviation Weather Center.
A summary of internal testing. Methods are described only in outline. Not an official forecast or warning.
