Posted by: @cathoderay

↑

Posted by: @majordennisbloodnok

↑

Indeed, I'd be worried if the Met Office didn't update their forecast based on new data coming in.

Of course, but that is not the point I am getting at here, which is the volatility over short time frames, in this case 24 hours or less. The bottom line is that forecasts that change cannot all be correct, in reality only one (if that) gets it right, which means all the others were wrong. You might imagine with all the hype that modern forecasting can do rather better than this. There's also the collapse of the probability function thing I have mentioned before (in effect, applying group probabilities to an individual becomes meaningless in the event): a 60% probability of rain does suggest rain is more likely that a 30% probability, but in the event, it either rains or it doesn't, and you have no way of knowing in advance, given say a 50% probability, whether you will be in the 50% that gets rain or the 50% that doesn't. You might as well toss a coin...
It's also not just about probabilities, but also the type of precipitation forecast, which in this example ranges from none through light showers and light rain to heavy rain. No rain is good, light showers can sometimes be worked around, but heavy rain is a wash out. 
Another way of looking at such forecasts is that they are a Schroedinger's cat problem: you don't actually know whether it is going to rain or not until you observe it, by which time you already know whether it is raining or not!     

...except that's not how probabilities work.
The chances of me reaching my 100th birthday are roughly 9.2%. If I actually do get a telegram from the King, that doesn't mean the forecast was wrong; it was still correct that the odds were against me even if I beat those odds. Similarly, if a forecast predicts an 80% chance of rain and it turns out not to rain that doesn't make the estimate of chances wrong.
Let's not forget, though, why we are using forecasts. If we want to plan for a future event, we are effectively gambling. The forecast doesn't stop it being a gamble but gives us an idea of which gambles are worth taking and which are not. The forecast isn't telling us what will happen; it's saying how likely something is to happen. Whether those odds make it a good choice to make are dependent on our viewpoint and on context. If, for example, I'd like to invite everyone round for a barbecue, I'd obviously be likely to pick a day with a good chance of no rain and if the odds aren't good then I'd be well advised to make contingency plans. If I've a terminal condition and the only operation which could save me has a 10% chance of success, that's still better than certain death.
What you've posted is suggesting a forecast is either right or wrong and that's a misinterpretation. Forecasts attempt to provide an improvement on random chance and so can be better or worse but that's all they're there for. If you don't find that useful, it's still your choice not to use them. In the case of weather forecasts, as you mentioned earlier, it is quite difficult to discern how much of an improvement if at all weather forecasts provide over random guessing so it's up to each of us to subjectively decide whether the forecasts are adding value for us. 
 

Posted by: @majordennisbloodnok

↑

What you've posted is suggesting a forecast is either right or wrong and that's a misinterpretation.

I can't agree with this. A conventional forecast is a prediction, and we can compare the prediction to what actually happens, and if the forecast and actual weather are the same, then the forecast was right, if they differ, the forecast was wrong.

The relatively recent introduction of probabilities to some forecasts - not ubiquitous, eg neither the Shipping Forecast not the Inshore Waters Forecast use them - does introduce more complexity. If the 0600 Shipping Forecast this morning says 'rain later' that means it predicts it will rain tonight (and can be judged right or wrong on whether it does or doesn't rain tonight). Were the Shipping Forecast to say 'rain (40%) later' then the proper interpretation of that is probably that on balance, it is less likely to rain than rain. I suppose the same information could also be expressed by saying 'fair (60%) later'. Using probabilities does give the Met Office some wiggle room: they can always say we only said it might rain/not rain, not that it will/will not rain. But presenting what will in the event be a binary outcome (rain/no rain) as a prediction with an attached probability (40%) may appear to give more information, but I am not sure in practice it does, because back in the real world I still have to treat it as a binary prediction - I have to work on the basis that it will or won't rain tonight (the collapse of the probability function, there is no such thing as 40% rain in the real world).    

I have used the rain forecast in this example. What if instead it was something that goes beyond inconvenience (ordinary as opposed to torrential rain)  to potential danger, eg the wind forecast? How do I use a forecast that says 'SW 6 to Gale 8 (40%)'? It is I suggest more useful in the real world (as opposed to the imagined world of probabilities) for the Met Office to make a judgement call, and declare what in its opinion it thinks will happen, which I can then use in my planning. The point I am getting at here is pragmatism, making real decisions in the real world.  

I agree that if the probability of rain this weekend is 10% on Saturday and 90% on Sunday then it would be a better bet to have my barbecue on Saturday, ie I can use the probabilities to make comparisons between two times (or places for that matter). But the problem with this, as I tried to point out, is the volatility of the forecast. What if, as often happens (and happened in the examples I used), the forecast constantly changes? On Monday the forecast for the weekend might be fair throughout (90%), on Tuesday the forecast is light showers on Sunday evening (30%), on Wednesday is is fair throughout again (60%), on Thursday it is heavy rain all day Saturday (80%), and so on, right up until the weekend itself? Which day do I hold my barbecue, and when do I make that decision?

The underlying problem is the current forecast models, especially those with probabilities, are simply not very good, hence the volatility. If a timely observation of bee's fart in Brazil can change the forecast for the whole of the UK, is it any surprise that the forecast is so volatile? As I have said before, I am inclined to think weather forecasting will only improve significantly with a major paradigm shift, as opposed to making do with ever more complex models run on ever bigger computers.  

Posted by: @majordennisbloodnok

↑

If I've a terminal condition and the only operation which could save me has a 10% chance of success, that's still better than certain death.

Alas, medical decision making is not that simple. All the operation will do is (possibly) delay death (there are only two certainties in life...), and what if the operation is itself a horrendoplasty, with a 'successful' outcome leaving the patient in a state worse than death itself? There is a saying in medicine, thou shall not kill, nor strive officiously to keep alive, to guide the doctor against pointless heroics among other things, though I do of course accept that at the end of the day it is the patient's decision on whether to go ahead or not with the operation, provided always that that is an informed decision.    

 

Posted by: @cathoderay

↑

I can't agree with this. A conventional forecast is a prediction, and we can compare the prediction to what actually happens, and if the forecast and actual weather are the same, then the forecast was right, if they differ, the forecast was wrong.

But if you compare what was predicted with what happened, how tight or fuzzy do you make your judgement criteria? If a forecast predicts rain between 9 and 11 but the rain occurs between 12 and 2 your analysis would show that as a complete failure whilst I would happily defend it as an accurate prediction offset by three hours.
I believe that most people now use a weather forecasting service via an app or website and most of those give the forecasts as probabilities. As such that is now a de facto “conventional” forecast and so I see no issue with determining accuracy as a spectrum rather than your binary right or wrong. Even when probabilities weren’t routinely shown, people used their subjective “on the money”, “fairly close”, “not great” and “completely off” evaluations based on much more than whether the forecast was right or wrong.

Posted by: @cathoderay

↑

Alas, medical decision making is not that simple. All the operation will do is (possibly) delay death (there are only two certainties in life...), and what if the operation is itself a horrendoplasty, with a 'successful' outcome leaving the patient in a state worse than death itself?

Whilst I agree with the nuances you raise, all I was trying to illustrate with the medical scenario was that probabilities have to be taken in context, and that context can vary wildly. In terms of weather, that context has to include time offsets as well as what someone views a reasonable variances of temperature or precipitation, and even then viewed within the context of how much those variances affect whatever you’re trying to plan for. None of that lends itself towards a binary right/wrong evaluation.
 

This is all a bit too frequentist for my liking. The probability of rain is, imo, better understood as an expression of the degree of belief in it raining.
Also I must say I have found the Met Office forecasts to be very accurate and dependable.