NEWxSFC - FINAL Summary...01-APR-11

AVG SUMSQ

AVG STP

AVG Total Absolute

AVG Absolute

Mean RSQ

 

Previous Ranks

Rank

Forecaster

Class

Total STN 4casts

Error ( " )

Error Z

% MPRV over AVG

Rank

4cast ( " )

Error

Error Z

% MPRV over AVG

Rank

Error (")

Error Z

% MPRV over AVG

Rank

Error (")

Error Z

%MPRV over AVG

Rank

RSQ

RSQ Z

% MPRV over AVG

Rank

Forecaster

1,1,1,1,1

1

donsutherland1

Senior

154

255

-0.910

47%

2

120.1

20.2

-0.451

36%

5

52.1

-1.142

32%

1

2.0

-1.134

31%

1

63.8%

1.092

45%

3

donsutherland1

2,2,2,2,2

2

Donald Rosenfeld

Senior

148

217

-0.834

56%

3

111.1

12.9

-0.574

49%

6

49.8

-0.835

33%

3

1.9

-0.868

32%

3

64.1%

1.009

37%

3

Donald Rosenfeld

6,3,3,3,3

3

Brad Yehl

Rookie

146

234

-0.702

45%

4

101.5

23.9

-0.381

36%

5

48.6

-0.697

25%

5

2.0

-0.697

24%

5

64.6%

0.812

25%

4

Brad Yehl

5,4,4,4,4

4

ejbauers

Intern

147

313

-0.641

43%

5

134.3

33.5

-0.089

12%

7

59.4

-0.620

24%

5

2.3

-0.631

24%

5

66.8%

0.735

31%

3

ejbauers

4,8,5,5,5

5

Mitchel Volk

Senior

148

380

-0.548

26%

5

113.8

34.0

-0.297

20%

6

63.0

-0.520

9%

5

2.5

-0.565

11%

5

50.2%

0.208

15%

6

Mitchel Volk

10,9,8,7,7

6

TQ

Senior

148

290

-0.495

38%

5

85.5

28.7

-0.099

13%

7

51.2

-0.654

26%

5

2.0

-0.673

24%

5

57.3%

0.422

16%

6

TQ

8,7,7,7,6

7

herb@maws

Senior

145

315

-0.417

32%

6

99.5

33.5

-0.061

8%

7

56.3

-0.340

15%

6

2.4

-0.304

12%

7

58.5%

0.348

14%

6

herb@maws

9,10,9,9,9

8

MarkHofmann

Journeyman

157

728

0.794

-34%

10

149.0

22.0

-0.042

-14%

7

96.3

1.018

-27%

10

3.6

1.024

-26%

11

32.1%

-0.743

-31%

10

MarkHofmann

12,12,11,12,10

9

Roger Smith

Intern

159

786

0.833

-32%

11

151.9

54.2

0.484

-41%

10

100.3

0.941

-21%

11

3.8

0.834

-16%

11

27.6%

-1.357

-47%

12

Roger Smith

 

There were eight (8) snowstorm-forecasting contests during the’10 / ’11 season …as of 01-APR-11.  Under the ‘two-thirds’ rule…forecasters who entered at least six (6) forecasts are included in the final standings.

 

A forecaster must enter at least two-thirds of all Contests to qualify for ranking in the Interim and final ‘End-of-Season’ standings.  If a forecaster has made more than enough forecasts to qualify for ranking…only the lowest SUMSQ Z-scores necessary to qualify are used in the computing the average.  IOW…if you made nine forecasts…only your six best SUMSQ Z-scores are used to evaluate your season-to-date performance.  You can think of it as dropping the worst quiz score before your final grade is determined.  The reason we have this rule is to 1) make it possible to miss entering a forecast or two throughout the season and still be eligible for Interim and ‘End-of Season’ ranking and 2) encourage forecasters to take on difficult and/or late-season storms without fear about how a bad forecast might degrade their overall 'season-to-date' performance score(s).

 

---

The mean normalized ‘SUMSQ error’ is the Contest/s primary measure of forecaster performance.  This metric measures how well the forecaster/s expected snowfall 'distribution and magnitude' for _all_ forecast stations captured the 'distribution and magnitude' of _all_ observed snowfall amounts.  A forecaster with a lower average SUMSQ Z Score has made more skillful forecasts than a forecaster with higher average SUMSQ Z Score.

 

The 'Storm Total Precipitation error’ statistic is the absolute arithmetic difference between a forecaster/s sum-total snowfall for all stations and the observed sum-total snowfall.  This metric…by itself…is not a meaningful measure of skill…but can provide additional insight of forecaster bias.

 

The 'Total Absolute error' statistic is the average of your forecast errors regardless of whether you over-forecast or under-forecast.  This metric measures the magnitude of your errors.

 

The 'Average Absolute Error' is the forecaster/s ‘Total Absolute Error’ divided by the number of stations where snow was forecast or observed.

 

The ‘RSQ error’ statistic is a measure of the how well the forecast captured the variability of the observed snowfall.  Combined with the SUMSQ error statistic…RSQ provides added information about how strong the forecaster/s ‘model’ performed.