ENPT
Open Marble

Note 05 · Model comparison

Which forecast model is closest?

Over one week at 185 airports, the AIFS as shown in Marble had the smallest temperature error of the models we tested, the IFS came second and NOAA GFS last; for wind and rain, no model was clearly best.

November 20264 min readSuperlumine

A summary of internal testing. Methods are described only in outline. Not an official forecast or warning.

  • 1.21 °Caverage temperature error one day ahead, AIFS as Marble shows itIFS 1.37 °C, GFS 1.93 °C
  • 116 / 185airports where the AIFS was closestIFS 52, GFS 17
  • 2,245comparisons per model and lead time13 times, 15–21 Nov 2026
  • 3.1 → 1.7 °CIFS error at the 18 airports above 1,500 m, raw versus as shownone day ahead

What we compared, and what we did not

Most weather maps show the same few global models, run by national and international centres. A fair comparison is therefore a comparison of those models, using only data their producers publish openly, against the same independent observations. This is not a test of any other app: other products may process the same models differently, and could be more accurate than the raw model data tested here.

We scored the ECMWF IFS and AIFS (raw and as Marble shows them, with its terrain adjustment), NOAA GFS at the same 0.25° resolution (raw), and, at 27 stations, NOAA’s 3 km HRRR model over the United States. The observations are reports from 185 airports chosen to spread across six regions, a third of them above 1,000 m where available, at 13 times between 15 and 21 November 2026, for forecasts one, three and five days ahead. The score is the average error (mean absolute error).

Average temperature error one day ahead

Average temperature error one day aheadMean absolute error of 2 m temperature at +24 h, 185 airports, 13 times on 15–21 November 2026, 2,245 comparisons per model: AIFS as Marble shows it 1.21 °C, IFS as shown 1.37 °C, IFS raw 1.59 °C, GFS raw 1.93 °C.0.0°0.5°1.0°1.5°2.0°AIFS as Marble shows it1.21 °CIFS as Marble shows it1.37 °CIFS, no height adjustment1.59 °CGFS, no height adjustment1.93 °C
Average temperature error one day ahead. Marble’s terrain adjustment lowers the IFS error by 0.22 °C; the AIFS as shown is lowest.
Chart in words

Mean absolute error of 2 m temperature at +24 h, 185 airports, 13 times on 15–21 November 2026, 2,245 comparisons per model: AIFS as Marble shows it 1.21 °C, IFS as shown 1.37 °C, IFS raw 1.59 °C, GFS raw 1.93 °C.

Temperature

The AIFS as Marble shows it had the lowest temperature error at every lead: 1.21 °C one day ahead, 1.58 °C at five days. The IFS followed (1.37 and 1.83 °C), and GFS had the largest error (1.93 and 2.32 °C). The terrain adjustment mattered most in the mountains: at the 18 airports above 1,500 m it cut the IFS error from 3.1 to 1.7 °C.

Average temperature error at 1, 3 and 5 days

Average temperature error at 1, 3 and 5 daysMean absolute error of 2 m temperature, 185 airports. AIFS as Marble shows it: 1.21, 1.32, 1.58 °C. IFS as shown: 1.37, 1.54, 1.83 °C. GFS raw: 1.93, 2.07, 2.32 °C.1.01.52.02.51 day3 days5 daysAverage error, °CLead timeGFS raw 2.32IFS 1.83AIFS 1.58
Average temperature error by lead time. The ranking holds at one, three and five days.
Chart in words

Mean absolute error of 2 m temperature, 185 airports. AIFS as Marble shows it: 1.21, 1.32, 1.58 °C. IFS as shown: 1.37, 1.54, 1.83 °C. GFS raw: 1.93, 2.07, 2.32 °C.

Closest model at each of 185 airports

Closest model at each of 185 airportsOne dot per airport, by the model with the lowest average temperature error over all three lead times: AIFS at 116 airports, IFS at 52, GFS at 17.AIFS 116IFS 52GFS 17
Which model was closest at each airport, over all three lead times.
Chart in words

One dot per airport, by the model with the lowest average temperature error over all three lead times: AIFS at 116 airports, IFS at 52, GFS at 17.

Humidity, wind and rain

Dew point followed the temperature ranking: AIFS 1.43 °C, IFS 1.58 °C, GFS 2.09 °C one day ahead, with GFS too dry by about 1.2 °C. For wind speed the three models were statistically indistinguishable (1.45 to 1.50 m/s). For whether it rains in a six-hour window, no model was clearly best: one day ahead, all three scored between 0.32 and 0.33 on the standard threat score. The AIFS forecasts light rain too often, so its rain layer looks wetter than the IFS layer in areas of light rain.

The 3 km HRRR, tested at 27 US airports one day ahead, was not more accurate on this small sample: temperature error 1.60 °C, against 1.56 °C for the IFS and 1.42 °C for the AIFS as Marble shows them. High-resolution models are built for the first hours and for thunderstorms, which a week of mostly calm November weather does not test.

Data and attribution

Forecast data: ECMWF open data (IFS, AIFS), CC BY 4.0; NOAA GFS and HRRR via the NOAA Open Data Dissemination program. Observations: airport reports via the NOAA Aviation Weather Center.

A summary of internal testing. Methods are described only in outline. Not an official forecast or warning.