The navigation map on the bar at the head of the site may be used to access various components of the official Bureau of Meteorology site.

Cognitive decision making processes in forecasting

Stern H (2005) Defining cognitive decision making processes in forecasting: a knowledge based system to generate weather graphics. Accepted for presentation to 21st Conference on Weather Analysis and Forecasting/17th Conference on Numerical Weather Prediction. American Meteorological Society, Washington, 1-5 August, 2005.

INTRODUCTION

Decision making processes leading to a weather forecast are multi-faceted. Some of the issues considered by forecasters in arriving at their decision include:

1. Climatology;
2. The relevance of recent weather patterns to the evolution of the current weather pattern;
3. The weather suggested by:
   • Guidance from the direct numerical prediction model (NWP) output;
   • Guidance that is based upon a statistical interpretation of the NWP model output; and,
   • Subjective interpretation of the NWP model output, based upon forecaster experience;
4. Confidence in the synoptic evolution suggested by the NWP output;
5. Strategies for addressing issues such as conflicting guidance; and,
6. Criteria for amending the current forecast previously issued by their colleagues.

PURPOSE

It is the purpose of this work to attempt to objectively describe these decision making processes via a computerised weather forecasting system that operates in a manner to replicate the aforementioned processes. In this context, a test of how successfully the said processes are being replicated by the computerised weather forecasting system lies in the extent to which the output (and overall performance) of the computerised weather forecasting system mimics the output (and overall performance) of the officially issued product.

A KNOWLEDGE BASED WEATHER FORECASTING SYSTEM

Over recent years, the present author has been involved in the development of a knowledge based weather forecasting system. Various components of the system may be used to automatically generate:

• Worded weather forecasts for the general public;
• Terminal aerodrome forecasts (TAFs) for aviation interests; and,
• Marine forecasts for the boating fraternity.

The knowledge based system generates these products by using a range of forecasting aids to interpret NWP model output in terms of such weather parameters as:

• Precipitation amount and probability;
• Maximum and minimum temperature;
• Fog, low cloud, and thunderstorm probability;
• Wind direction and speed;
• Swell.

EXAMPLE OF A TAF GENERATED BY THE SYSTEM

LIMITS OF PREDICTABILITY

Recently, the knowledge based system was used to explore the limits of predictability. The outcome of this investigation was to reveal a modest level of skill at predicting the temperature up to 10 days in advance, but skill at predicting precipitation was limited to 7 days in advance:

THE CURRENT APPLICATION

The system is utilised to define and evaluate the cognitive decision making processes involved in the generation of extended-range day-to-day weather forecasts. Specifically, the weather forecasting product examined in the present work is that of weather graphics. Fourteen different weather graphics (selected from Australian Bureau of Meteorology and World Meteorology Organisation graphics) are utilised. These are:

Sunny

Dry

Haze

Fog

Partly Cloudy

Cloudy

Windy

Dust

Drizzle

Shower

Snow

Rain

Thunder

Cyclone

SCOPE OF APPLICATION

These forecasts are generated for Melbourne, and also for 18 nearby locations:

Forecasts may be generated for all locations shown on the map
except for Fawkner Beacon and South Channel Island.

THE ALGORITHM

The weather graphics are generated from an algorithm that has a logical process to combine features of the temperature, precipitation, and phenomena (for example, fog, thunder, and dust) components of the forecasts with features of the forecast synoptic type (strength, direction, and cyclonicity of the surface flow).

EXTRACT OF CODE TO GENERATE PRECIPITATION PROBABILITY

EXTRACT OF CODE TO GENERATE FOG PROBABILITY

EXTRACT OF CODE TO GENERATE MINIMUM TEMPERATURE

EXTRACT OF CODE TO GENERATE WEATHER GRAPHIC ICONS

NWP UNCERTAINTY

STRATEGY

The National Oceanic and Atmospheric Administration's (NOAA) Global Forecasting System (GFSx) NWP model (http://www.arl.noaa.gov/ready/metdata.html) provides output that includes forecast data forecast data every 6 hours from forecast hours 0 to 180 on a 1 degree latitude/longitude grid covering the globe. In addition, this data is mapped to northern and southern hemispheric 257 x 257 grids. The data is updated 4 times per day. The knowledge based forecasting system was utilised to objectively interpret the output of the GFSx model statistically in terms of local weather at Melbourne (maximum temperature, minimum temperature, probability of precipitation, amount of precipitation, a weather graphic depicting the expected weather during the morning, and a weather graphic depicting the expected weather during the afternoon) in order to rigorously establish how successfully the system incorporates NWP uncertainty into forecasts.

The criteria that forecasters utilise in amending the current forecast previously issued by their colleagues, is replicated in the knowledge based system by preventing the system from changing temperature forecasts, from one preparation to the next, by less than 2 deg C.

EXAMPLE OF GENERATED HTML CODE

EXAMPLE OF GENERATED WEATHER GRAPHIC

Day & Date	Morning	Afternoon	Min Temp (deg C)	Max Temp (deg C)	Precip Amount (mm)	Precip Prob (%)
Mon-16-5-2005	Fog.	Partly Cloudy.	8	18	0	44
Tue-17-5-2005	Fog.	Haze.	8	19	0	11
Wed-18-5-2005	Partly Cloudy.	Partly Cloudy.	8	18	0	31
Thu-19-5-2005	Sunny.	Partly Cloudy.	6	17	0	6
Fri-20-5-2005	Partly Cloudy.	Cloudy.	6	18	0	24
Sat-21-5-2005	Cloudy.	Shower.	9	18	0.4	58
Sun-22-5-2005	Shower.	Rain.	10	20	3.2	73

A 100-DAY TRIAL

A 100-day trial of its performance was conducted, with the knowledge based system generating twice-daily forecasts out to seven days in advance. To illustrate, the forecasts leading up to Saturday April 2 are now given (the observed weather was fine, mainly sunny and windy with a minimum temperature of 20.2 deg C, and a maximum temperature of 31.6 deg C):

Forecasts leading up to Saturday April 2
Time Forecast Produced	Date Forecast Produced	Morning Graphic	Afternoon Graphic	Minimum (deg C)	Maximum (deg C)	Precip Amount (mm)	Precip Prob (%)
PM	Mar 26	Cloudy.	Cloudy.	16	26	0	40
AM	Mar 27	Cloudy.	Cloudy.	18	30	0	46
Mar 27	Shower.	Shower.	20	30	1.3	60
AM	Mar 28	Shower.	Rain.	20	30	3.4	71
PM	Mar 28	Cloudy.	Cloudy.	18	32	0	39
AM	Mar 29	Windy.	Windy.	20	30	0	24
PM	Mar 29	Sunny.	Partly Cloudy.	20	30	0	14
AM	Mar 30	Sunny.	Partly Cloudy.	20	32	0	9
PM	Mar 30	Sunny.	Partly Cloudy.	18	32	0	7
AM	Mar 31	Sunny.	Partly Cloudy.	18	30	0	9
PM	Mar 31	Sunny.	Partly Cloudy.	18	32	0	6
AM	Apr 1	Sunny.	Partly Cloudy.	18	32	0	4
PM	Apr 1	Sunny.	Sunny.	16	32	0	2
AM	Apr 2	Windy.	Windy.	18	32	0	4

By contrast, the forecasts leading up to Thursday April 14 (an overcast and wet day, with 22.0 mm of rain, a minimum temperature of 15.2 deg C, and a maximum temperature of 16.0 deg C) are now given:

Forecasts leading up to Thursday April 14
Time Forecast Produced	Date Forecast Produced	Morning Graphic	Afternoon Graphic	Minimum (deg C)	Maximum (deg C)	Precip Amount (mm)	Precip Prob (%)
PM	Apr 7	Shower.	Shower.	15	23	7.7	68
AM	Apr 8	Rain.	Rain.	15	21	6.5	78
PM	Apr 8	Rain.	Rain.	17	23	8.3	85
AM	Apr 9	Rain.	Rain.	17	23	8.5	87
PM	Apr 9	Rain.	Rain.	15	25	5.7	85
AM	Apr 10	Rain.	Rain.	15	23	7.2	84
PM	Apr 10	Rain.	Rain.	17	21	6.6	81
AM	Apr 11	Rain.	Rain.	15	23	9.3	87
PM	Apr 11	Rain.	Rain.	17	19	11.7	85
AM	Apr 12	Rain.	Rain.	15	21	13.2	83
PM	Apr 12	Rain.	Rain.	13	19	10.9	86
AM	Apr 13	Rain.	Rain.	13	19	8.3	88
PM	Apr 13	Rain.	Rain.	16	19	7.1	85
AM	Apr 14	Rain.	Rain.	18	19	7.3	83

RESULTS

After the 100 days of the trial (Feb 14, 2005 to May 24, 2005), the overall percentage variance of observed weather explained by the forecasts so generated (the system's forecasts) is 43.24% (compared with 42.31% for the official forecasts). That the knowledge based system has achieved some success in its attempt to replicate the cognitive decision making processes in forecasting is confirmed by the closeness of the overall percentage variances explained by the two sets of forecasts.

Specifically for temperature, the percentage variance explained by the 1000 minimum temperature (Day 1 to Day 7 00 UTC forecasts, and Day 2 to Day 4 12 UTC forecasts) and 1100 maximum temperature forecasts (Day 1 to Day 7 00 UTC forecasts, and Day 1 to Day 4 12 UTC forecasts) so generated is 59.71% (compared with 59.55% explained by the official forecasts).

The Root Mean Square (RMS) Error of the forecasts so generated is 2.55 deg C (compared with 2.45 deg C for the official forecasts).

Specifically for precipitation, the percentage variance explained by the 1100 quantitative precipitation (Day 1 to Day 7 00 UTC forecasts, and Day 1 to Day 4 12 UTC forecasts) and 1100 probability of precipitation forecasts (Day 1 to Day 7 00 UTC forecasts, and Day 1 to Day 4 12 UTC forecasts) so generated is 26.78% (compared with 25.07% explained by the official forecasts).

On a rain/no rain basis, the percentage correct forecasts so generated is 78.82% (compared with 77.64% of the official forecasts).

However, the overall percentage variance of official forecasts explained by the system's forecasts is only 45.91% (this being made up of 63.59% of the variance of officially forecast temperature, and 28.23% of the variance of officially forecast precipitation), indicating that, on a day-to-day basis, there are significant aspects of the processes employed in deriving the official forecasts that are not taken into account by the system's forecasts, and vice versa.

CONSENUS FORECASTS

Regarding the two sets of forecasts as partially independent and utilising linear regression to optimally combine the estimates of minimum temperature, maximum temperature, precipitation amount, and precipitation probability lifts the overall percentage variance explained and suggests that adopting such a strategy of optimally combining the official and system predictions has the potential to deliver a set of forecasts that are substantially more accurate than those currently issued officially:

• The overall percentage variance of observed weather explained by the consensus forecasts is lifted to 50.21% from 43.24% (system) and 42.31% (official);
• Specifically for temperature, the percentage variance explained by the consensus forecasts is lifted to 66.33% from 59.71% (system) and 59.55% (official), whilst the RMS Error is reduced to 2.25 deg C from 2.55 deg C (system) and 2.45 deg C (official);
• Specifically for precipitation, the percentage variance explained by the consensus forecasts is lifted to 34.09% from 26.78% (system) and 25.07% (official), whilst on a rain/no rain basis, the percentage correct forecasts is lifted to 83.55% from 78.82% (system) and 77.64% (official).

FUTURE WORK

It is planned to incorporate the aforementioned optimal consensus process into the system.

REFERENCES

Stern H (2005) Using a knowledge based forecasting system to establish the limits of predictability. 21st Conference on Interactive Information and Processing Systems, San Diego, California, USA 9-13 Jan., 2005.

Stern H (2004a) Using a knowledge based system to predict thunderstorms. International Conference on Storms, Storms Science to Disaster Mitigation, Brisbane, Queensland, Australia 5-9 Jul., 2004.

Stern H (2004b) Incorporating an ensemble forecasting proxy into a knowledge based system. 20th Conference on Interactive Information and Processing Systems, Seattle, Washington, USA 11-15 Jan., 2004.

Stern H (2003) Progress on a knowledge-based internet forecasting system. 19th Conference on Interactive Information and Processing Systems, Long Beach, California, USA 9-13 Feb., 2003.

Dawkins S S (2002) A web-based swell and wind forecasting tool. 9th Conference of the Australian Meteorological and Oceanographic Society, Melbourne, Australia, 18-20 Feb., 2002.

Stern H and Parkyn K (2001) A web-based Melbourne Airport fog and low cloud forecasting technique. 2nd Conference on Fog and Fog Collection, St John's, New Foundland, Canada 15-20 Jul.,2001.

WEB SITES

1. The graphic generator:
http://www.weather-climate.com/graphicgenerator_sevendays_version5.html

2. Sample forecast from the graphic generator:
http://www.weather-climate.com/sampleforecast.html

3. Future work - the new graphic generator, incorporating the aforementioned consensus process:
http://www.weather-climate.com/graphicgenerator_sevendays_version3jul2005.html