No-code, plug-and-play monitoring for your heat pump

Planet Data

The integration of smart technology in UK home energy management, especially for the control and oversight of heat pump systems, is an evolving field. However, this progress is shadowed by increasing frustration among homeowners due to the inadequacy of current heat pump control and oversight capabilities.

As highlighted in multiple discussions on the Renewable Heating Hub forums, some homeowners have ventured into DIY solutions, such as modbus/RS-485 monitoring.

However, these endeavours often prove to be complex and intimidating. For example, cathodeRay describes his experience setting up a modbus monitoring system. This process involves intricate steps like hard wiring a RS-485 to USB converter into the heat pump controller and utilising open-source software for data reading. Although this approach could be effective, it’s not a feasible or realistic solution for the average homeowner.

Amidst these challenges, there is a clear demand for monitoring systems that are both straightforward and affordable. Homeowners are looking for solutions that require minimal hardware and software knowledge. The emergence of smart technologies, like PlanetData by Planet Devices, aims to address these challenges in home energy management.

Planet Devices

PlanetData functions as a WiFi-enabled Modbus communication gateway, enabling remote access to RTU Modbus enabled heat pumps. It presently supports over 150 heat pump models, with the list continuously expanding, showing its adaptability to various systems.

One of the key features of PlanetData is its ease of installation and use. It’s a no-code, plug-and-play device that connects directly to the heat pump hardware and only requires power. This simplicity contrasts sharply with the complex DIY solutions some homeowners have attempted, involving intricate wiring and programming.

PlanetData not only sends and receives data but also allows users to visualise the performance of their heat pumps via a user-friendly dashboard. This dashboard notifies users of any hardware errors, enabling timely interventions. The device can also handle peripherals like CT clamps and indoor temperature probes, providing additional data points crucial for calculating accurate CoP and other efficiency values.

The device also features an optional mains voltage meter to provide a more accurate coefficient of performance calculation. These features empower homeowners with detailed insights into their system’s performance, far surpassing the basic metrics provided by standard heat pump installations.

The ability to store historic data – hourly, daily, weekly, monthly and yearly averages – allows homeowners to engage in predictive maintenance. Applications can optimise energy use and predict maintenance needs, enhancing control over energy consumption.

For homeowners, PlanetData offers a solution that balances sophistication with user-friendliness. It addresses the gap in the market for an effective, accessible heat pump monitoring system. For installers, it provides a comprehensive tool for tracking, maintaining and controlling installations.

PlanetData signifies a significant step in home energy management technology. While innovative, it also focuses on addressing the practical needs of homeowners and installers. As the UK moves towards more sustainable energy solutions, technologies like PlanetData could play a crucial role in making home energy systems more efficient, manageable and user-friendly.

Related posts

Made in the UK: challenges facing heat pump installers

Alec Morrow

Jürgen Klopp: Similarities Between Heat Pumps & Football

Mars

Unleashing Carbon Savings: The Untapped Potential of Renewable Heating in Rural Homes

Sune
Subscribe
Notify of
guest

49 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
noburn
465 kWhs
5 months ago

Very interesting. The Planet Devices site indicates it is fully compatible with my Ecodan 8.5kw system but is very short on information regarding actual interfacing to the system. Maybe it is all very obvious to people with more ASHP experience. Is it possible to provide some basic details describing how this integrates into an existing system? Thanks
 

bontwoody
3126 kWhs
Reply to  noburn
5 months ago

@editor Mars, is this one of those advertising features you mentioned?  It reads like one. What would be more helpful is an unbiased review perhaps comparing pros and cons to a well tested system like openenergymonitor.

bontwoody
3126 kWhs
Reply to  Mars
5 months ago

@Mars that would be brilliant Mars! An objective comparison of how accurate the modbus readings are compared to a heat meter would be very informative

cathodeRay
Editor
6919 kWhs
5 months ago

Very interesting indeed. I think at a basic level it is identical to my system: modbus connection to the heat pump, then read and write data over that connection, possibly even using the same coding language (python). They even have the same tricky to collect data, energy in and indoor air temperature, with similar fixes (use an additional sensor). Interestingly, they don’t list Midea as a compatible system, though I am sure it must be.

There are at least three significant differences however:

(1) it is proprietary, ie a black box system (you have no idea what goes on in the black box)

(2) it uses wifi rather than a wire to transmit the data

(3) your data, reading between the lines ends up on their servers

Given (3), it appears that your installer, and very likely Planet Devices, will have access over the system to all your data. This obviously raises questions about privacy (will they sell your data to third parties or even the government?), and who’s data is it anyway. Since you can write as well as read over modbus, it also means your installer, and very possibly Planet Devices, can control your heat pump remotely. Potentially useful if these things don’t worry you, but in the wrong hands? Pesky customer? Turn up their heating to max at 2am and let them sweat a bit. Of course, no one would do that in practice, would they? But they might do something else. The problem is not where do these things start, but where do they end.

The lack of detail on their website is a bit worrying, but I suppose it is early days. There is also no information that I can see on prices. The business model appears to be Planet Devices sell to installers, and installers sell it to end users. 

 

scrchngwsl
1505 kWhs
Reply to  cathodeRay
5 months ago


Very interesting indeed. I think at a basic level it is identical to my system: modbus connection to the heat pump, then read and write data over that connection, possibly even using the same coding language (python). They even have the same tricky to collect data, energy in and indoor air temperature, with similar fixes (use an additional sensor). Interestingly, they don’t list Midea as a compatible system, though I am sure it must be.
There are at least three significant differences however:
(1) it is proprietary, ie a black box system (you have no idea what goes on in the black box)
(2) it uses wifi rather than a wire to transmit the data
(3) your data, reading between the lines ends up on their servers
Given (3), it appears that your installer, and very likely Planet Devices, will have access over the system to all your data. This obviously raises questions about privacy (will they sell your data to third parties or even the government?), and who’s data is it anyway. Since you can write as well as read over modbus, it also means your installer, and very possibly Planet Devices, can control your heat pump remotely. Potentially useful if these things don’t worry you, but in the wrong hands? Pesky customer? Turn up their heating to max at 2am and let them sweat a bit. Of course, no one would do that in practice, would they? But they might do something else. The problem is not where do these things start, but where do they end.
The lack of detail on their website is a bit worrying, but I suppose it is early days. There is also no information that I can see on prices. The business model appears to be Planet Devices sell to installers, and installers sell it to end users. 

I agree with these points – if they exposed the resulting data over MQTT or something (the way that Hildebrand do with their Glow aftermarket smart meters) then this would mitigate point 3, as then the data never needs to leave your home network. It would also make me trust (1) a bit more.
The first thing I would do is deny internet access to this device on my router. There is no need for the data to end up on their servers at all.
Still though this is an order of magnitude better than the current situation, where there is almost no data available to you, and any data you have access to is, well, via your manufacturer’s cloud server… I hope they do well, but I also hope that they realise (1) that the sort of weird nerds who are interested in this would also like the ability to keep it all “in house" and tinker with the outputs locally, and (2) that if anyone is thinking of buying a heat pump (or already has one and wants to know if it’s working well), they will talk to their weird nerd friend, who may well have bought this device and may well recommend it, should it pass muster. Appealing to the weird nerd crowd at the start then pivoting to a more mainstream offering is a really effective strategy that a lot of tech startups have followed successfully.
 

cathodeRay
Editor
6919 kWhs
5 months ago

I’ve asked Planet Devices to visit the site and answer questions.

Good thinking. I’ve also had a look at the connections in the photo, and it looks like a standard two wire modbus/RS485 hookup plus DC power supply:

image
cathodeRay
Editor
6919 kWhs
5 months ago

Is it compatible with Samsung?

Apparently so, according to their website. Samsung units have a modbus connection via an add-on board.

bontwoody
3126 kWhs
Reply to  cathodeRay
5 months ago

@cathodeRay I ran a wire just in case when I installed it 😁

Lenny
485 kWhs
Reply to  cathodeRay
2 months ago



Is it compatible with Samsung?

Apparently so, according to their website. Samsung units have a modbus connection via an add-on board.

 
No mention of Samsung on the compatibility page ?
 

iancalderbank
3640 kWhs
5 months ago

@mars interesting you picked up on this too. I read up on it a few weeks back. so far as I can tell, its method of connecting to the HP is fundamentally the same as homely so I didn’t see it as widely different, just a competitor.
It needs the exact same type modbus connection to the heat pump’s internal data bus as homely. Some HP’s have this natively, some need an extra module (samsung).
what’ll be interesting is if it gives more of a pared-back “monitor" approach rather than the whole super-efficiency-cost-optimised-control of homely.
even better if it exposes it nicely to the home owner. but from what I read on linkedin, they are focusing on the installer market though :-(. be nice if they come on here and discuss.
agree with @scrchngwsl a web front end or api for local data visibility would suit many of us.
there will always be the argument made by some that a Class1 heat meter approach is more accurate than relying on the numbers containing within the heat pump (which is all this will ever get you), but if at least you can reliably and methodically get to those heat pump’s numbers, and use them intelligently, then that is a bloomin good start.
 

Toodles
6274 kWhs
5 months ago

The Homely controller seems to be settling in nicely here; for the ‘householder’ the Homely takes all the work out of controlling the heat pump day to day of course. I wanted the additional information that is there for the installer and EvergreenHomely to optimise via the network connection. This access was given to me and I have found it quite fascinating to see how all the factors involved integrate!
At this stage, I have no particular concerns about my user data being available to my installer and EvergreenHomely – err…SHOULD I?! For use with the Daikin range, there is a ModBus unit with its’ own PSU and this all sits next to the Daikin MMI controller. Of course, the Homely is a smart controller as well as a monitor and at present, EvergreenHomely are monitoring mine to find a little pesky creature that causes my temperature node to ‘nod off’ for a minute or two occasionally. I note that the COP between 24th. and 26th. Jan. is 4.5 which seems reasonable to me. All else being equal, if I can rely on the date being measured accurately and interpreted well, I’m happy with that. Regards, Toodles.

James Davidson
61 kWhs
Reply to  Mars
2 months ago

@Mars We’re still interested in getting a device out to an RHH member. Please do let Mars and I know if you’re game.

James Davidson
61 kWhs
Reply to  Mars
2 months ago

@Mars Thanks Mars for highlighting this. We’ll be updating the compatibility table in-line with our testing pipeline soon.

James Davidson
61 kWhs
2 months ago

Samsung and others are coming soon. We only list heat pumps which are 100% ready for general use on the compatibility table (for now). 😎 

MPHB
360 kWhs
2 months ago

@James Davidson  isn’t the Clivet machine that is listed as compatible a Midea clone? I am operating a 10kw Midea clone which is modbus-enabled and would be happy to be part of the test. I already have a Easton modbus-enabled power meter for dedicated to my HP. I am in Germany though. 

James Davidson
61 kWhs
Reply to  MPHB
2 months ago

@MPHB The Clivet WSAN-YMi is indeed a Midea M-Thermal R32 under the hood, like so many other whitelabels. We’re currently serving the UK market only, though we are already ramping up to sell in the wider European and North American markets. Thanks for your interest 🙂

cathodeRay
Editor
6919 kWhs
2 months ago

it may well be straightforward, @cathodeRay, but it can be intimidating to anyone who hasn’t already taken that first step

I agree, new stuff can easily be intimidating, though for me the most intimidating part was getting my head round the modbus/hardware side of things – it took ages to work out what I needed, and how to connect things up. In the event, it turned out to be surprisingly straightforward – anyone familiar with the technology could do it in their sleep but the thing is I wasn’t familiar with the technology. Thankfully human beings don’t yet have a modbus connection that medics can connect to, though I believe Hatt Mancock wants to change all that. Good luck to him.   

The thing about python is it isn’t that complicated – honestly! – at the level I am using it, and the programming is relatively high level, ie almost human readable. I have been lamentably slow in completing the ‘How to’ thread, initially because I had some final tests to do, now because I didn’t really take enough notes at each stage, meaning I will have to remember some of the steps. I might do an abbreviated version, and let people ask questions if they are interested.    

That said, I do now have a stable monitoring and basic control system (the auto-adaption script) that I am comfortable with, and has now run for a complete heating season. The only missing bit is an independent outside air temperature sensor. I have the hardware, but have so far lacked the inspiration on where to put it. It needs to be somewhere that can have a wire (the modbus cable) run to it.

I have also found that the mini PC that manages the system can also do other things. For example, I use it to take a daily snapshot of the Shipping Forecast, part of an attempt to see just how (in-)accurate modern weather forecasting is. You will know where I am coming from on this, given that weather forecasting is whatiffery on an industrial scale… 

But really, neither PlanetData, my solution, your solution, any of the many rich and varied solutions out there, should not really be necessary. In a sane world, heat pumps would reliably collect this data, and keep historical records, as part of their core functions. But as we all know, we don’t live in a sane world.

The trouble is that the company has not provided any access on its web site to any product documentation (unless it’s hidden behind the login) – no user manual, no installation guide, no tech specs, nothing.

I am naturally wary of handing over my data to anyone. Basically I operate a white list system: data only goes out when I have white listed the receiver of the data. Who knows what some lunatic AI zombie will one day try to do with your data? One advantage of using my system, and others that are home based not cloud based, is that your data stays at home with you. Horses for courses of course, but another factor folks may want to consider. 

Jancold
724 kWhs
Reply to  cathodeRay
2 months ago

@cathodeRay Are you dissing the Shipping Forecast ? It is only for sailors, surely nobody else understands it anyway. 🤣 .

scrchngwsl
1505 kWhs
2 months ago

For what it’s worth, after my Mitsubishi Ecodan heat pump’s controls being inaccessible via the MelCloud app for nearly a week, I will not be installing anything that depends on the cloud to control or monitor something as fundamental to my life as heating and hot water. It is unfortunate that this device simply swaps one cloud with another, rather than empowering users with local-first access, and then bolting on a cloud offering for those who want a simple life. I’m going to be looking at @cathodeRay ‘s work very closely and try to do my own DIY modbus thing on my Mitsubishi Ecodan.

cathodeRay
Editor
6919 kWhs
2 months ago

It is only for sailors, surely nobody else understands it anyway.

Unless I happen to be a sailor!

The other reason for using the shipping forecast (actually the Inshore Waters Forecast) is that, compared to the normal land forecasts, which are full of waffle, including coverage of weather that has already happened, the shipping forecasts are in a standard format, and relatively concise, and they use key words eg if today’s 1200 forecast says SW 6 veering NW 4 later, the key word later means the NW 6 if forecast to happen 12 hours or more after the forecast was issued ie from midnight tonight until 1200 tomorrow (the numbers are the wind speed on the Beaufort Scale, veering means clockwise around the compass). If I want to head SW, I may do better to wait until tomorrow morning, rather than starting out this afternoon.

Even though it is relatively concise, translating the text into something a computer can use is still a challenge. Take the above forecast, that can be coded as two 6 hour blocks of SW 6, then two 6 hour blocks of NW 4, but how about ‘S 3 becoming cyclonic 4 for a time’? Cyclonic is shorthand for the typical wind pattern observed while a low passes over, but how do you code ‘for a time’? 

I’ve only found one paper in the literature that tries to compare forecast weather with actual weather. It was done by the Met Office in 2013, and as it happens, it was done using the Shipping Forecasts, for much the same reasons. They stumbled over the actual weather though, the problem being relatively few consistent weather observation posts, and they gave up, and used instead a ‘nowcast model’, and wrote a classic example of ‘no need to bother with real data round here, let’s use a model instead’ waffle (even if it does include the rather sinister concept of ‘truth data’):

“Ideally, real observations should be used to verify forecasts. However, real observations can only be used if they are relatively equally spaced and their coverage is sufficient to represent all the wind speeds within the geographical domain of the forecast. Unfortunately, this is rarely the case so the AFVS [Automatic Forecast Verification System] uses nowcast model analyses as the truth data because they contain the latest observations and the most recent model data." [emphasis added]

Using their version of the ‘truth data’, they managed to claim the main shipping forecast was accurate within plus or minus one Beaufort scale 88% of the time, while the inshore waters forecasts were accurate within plus or minus one Beaufort scale 94% of the time. Based on casual observations of the real ‘truth data’, these estimates seem on the high side, perhaps an example of an organisation marking its own homework. If you use a criteria of correct Beaufort scale prediction for the inshore waters forecast, rather than with plus or minus one Beaufort scale, the accuracy drops to around 60-70%, ie the forecast is only accurate two thirds of the time.

Another possible angle on the accuracy is to compare yesterday’s inshore waters forecast 24 hour outlook (covers 24-48 hours) with today’s actual forecast (covers 0-24 hours). By and large, they should be the same, but if they differ, then at least one of them is wrong. I have yet to try this particular analysis.

All of this does have some relevance to home heating control systems that make use of forecast weather rather than actual weather data to control the heating system. If the actual weather varies from the forecast weather, then the control system will be out of step with the actual conditions, to the extent the actual weather varies from the forecast weather.

Edit: link to the Met Office paper (pdf): Verification of marine forecasts using an objective area forecast verification system.

Toodles
6274 kWhs
Reply to  cathodeRay
2 months ago

@cathodeRay Don’t know about these days but, Southern Tv had an office in their Southampton premises for the weather forecaster; on the door was a hook, over which hung a piece of seaweed… Regards, Toodles.

Jancold
724 kWhs
Reply to  cathodeRay
2 months ago

@cathodeRay Thanks for the informative reply to my flippant remark. I have been following the forum for a while now and although most goes over my head I hope gradually, bit by bit, to reach an understanding. In the near future I will be going to discuss a design for an ASHP for my house so the more I know the better. The forecast today for Sidmouth said rain stopping by 10:00 , it has not and the “prediction" changed radically just now. There are so many apps that forecast marine weather but they rarely agree with each other. Anyway I am going way off topic.

cathodeRay
Editor
6919 kWhs
2 months ago

on the door was a hook, over which hung a piece of seaweed

and the curtains were drawn, to prevent the forecaster being distracted by the actual weather… 

Toodles
6274 kWhs
Reply to  cathodeRay
2 months ago

@cathodeRay I find it very irritating that when they waste some of the brief amount of allotted time on telling us what weather we have been having, they then have to rush through the forecast.😒 I don’t think I need them to tell me what weather I have been experiencing really! Harrumph Toodles.

cathodeRay
Editor
6919 kWhs
2 months ago

I find it very irritating that when they waste some of the brief amount of allotted time on telling us what weather we have been having, they then have to rush through the forecast

I agree completely. Typical of the BBC. The supposed forecast also goes by different names these days, eg the weather situation, or sometimes the ongoing weather situation. Since when was the weather not an ongoing situation? The announcer also needs to read Clive James’s Mr Fish’s ongoing freezing fog situation report, or situation. Soon the Today pongos will be asking the forecasters for their sense of the ongoing weather situation where they are. God help us all!  

Majordennisbloodnok
4527 kWhs
2 months ago


“Ideally, real observations should be used to verify forecasts. However, real observations can only be used if they are relatively equally spaced and their coverage is sufficient to represent all the wind speeds within the geographical domain of the forecast. Unfortunately, this is rarely the case so the AFVS [Automatic Forecast Verification System] uses nowcast model analyses as the truth data because they contain the latest observations and the most recent model data." [emphasis added]

I know this is probably a silly question but isn’t a forecast based on observed data to start with? i.e. data from the various weather monitoring stations? If so, why can they not use those same observations to compare against the forecast from x number of days ago?
That said, I seem to remember reading that percieved accuracy is based on the predicted weather for now vs the actual weather for now whilst in fact the prediction could be remarkably astute but an hour or two out (and therefore still an accurate forecast within reasonable time variations) and therefore appearing bad. I hasten to add that I am in no way an expert so am happy to be disabused of this viewpoint by anyone prepared to debunk it convincingly.

Toodles
6274 kWhs
Reply to  Mars
2 months ago

@Mars Some of the most powerful supercomputers have been built and installed for meteorological prediction systems. ECMWF have their centre just a mile or so from my heat pump and a French family who are friends of ours have recently retired from using and maintaining one of these beasts, Didier in particular has been the butt of many jokes about the megabucks these things cost and how, as soon as a new model has been installed and commissioned, they are looking at a newer, faster, flashier replacement for it! 😄 Regards, Toodles.

Jancold
724 kWhs
Reply to  Mars
2 months ago

@Mars

Weather models Predict wind

These are the computer weather models I can choose from in Predict Wind, my sailing weather app. They can be startlingly different so I look to see what suits and then a look out of the porthole and make my best guess. 😎

Majordennisbloodnok
4527 kWhs
2 months ago


Didier in particular has been the butt of many jokes about the megabucks these things cost and how, as soon as a new model has been installed and commissioned, they are looking at a newer, faster, flashier replacement for it

In fairness, your average gaming teenager will be functioning in much the same way….

bontwoody
3126 kWhs
2 months ago
Majordennisbloodnok
4527 kWhs
2 months ago


I think the next generation of forecasts will probably be AI driven

…so it can predict bad weather, criticise the Met Office and swear at you all in the same sentence. 😉 

Derek M
Editor
14285 kWhs
Reply to  Majordennisbloodnok
2 months ago

I think the next generation of forecasts will probably be AI driven

…so it can predict bad weather, criticise the Met Office and swear at you all in the same sentence. 😉 

No. Surely you just select the type of weather that you desire!!! 😋 

 

Toodles
6274 kWhs
Reply to  Majordennisbloodnok
2 months ago

@Majordennisbloodnok …and in an American accent. 😉 Toodles

cathodeRay
Editor
6919 kWhs
2 months ago

There are so many apps that forecast marine weather but they rarely agree with each other. Anyway I am going way off topic.

I think it is on topic enough, in that weather is the main determinant of our heating demand, and an ability to predict future weather could, if it can be done, be very useful in managing a heating system.

The fact that many apps produce different forecasts despite starting with the same data (past observations) is telling. It tells us that when it comes to weather forecasting, the science of modelling is far from perfect. Meteorologists will probably say ‘ah, that’s what happens with complex systems, you know, the butterfly’s wing in Brazil and all that’ but really that is just an excuse to explain why their models fail. Weather is a very complicated thing, and its prediction is uncertain. The modern trend of forecasting probabilities (10% chance of precipitation at 1200 today in this location) adds a spurious air of pseudo-confidence, but in the event, there is no such thing as 10% precipitation, just as there is no such thing as 2.2 children. At any one time in any one place there either is or isn’t precipitation, just as parents have say two or three children, never 2.2 children. This failure of the aggregate to predict the individual outcome is inherent in modelling. It predicts (but in no way assures) the average outcome, not the individual outcome, and unfortunately, it doesn’t even get the average right very often.

In passing, predicting tidal streams in a complex area like the Solent is a similar challenge. I have no less than four (maybe more!) tidal stream atlases for the Solent, and they all differ to a greater or lesser degree. The more traditional ones based on just observed data are generally better, especially when ‘local knowledge’ is added, the worst by far is an atlas that uses a computed modelled gridded system (‘data scientists’ love grids, it makes them feel in control, like a drill sergeant gridding his raw recruits; the AFVS paper I mentioned also uses a gridded modelling system to make up the ‘truth data’ for missing cells). 

The general point is beware modellers bearing gifts of rich data, because they are often wrong.

I know this is probably a silly question but isn’t a forecast based on observed data to start with? i.e. data from the various weather monitoring stations? If so, why can they not use those same observations to compare against the forecast from x number of days ago?

That said, I seem to remember reading that percieved accuracy is based on the predicted weather for now vs the actual weather for now whilst in fact the prediction could be remarkably astute but an hour or two out (and therefore still an accurate forecast within reasonable time variations) and therefore appearing bad. I hasten to add that I am in no way an expert so am happy to be disabused of this viewpoint by anyone prepared to debunk it convincingly.

I rather suspect they don’t want to look too closely, at least in public, at the observed weather vs the expected weather (yes, that is observed vs expected again…) because they know it is a recipe for red faces in high places. But in principle it is very doable, and is what I am trying to do. The problem is how to automate the process. I can already manually compare yesterday’s forecast weather with yesterday’s actual weather for a given time and location, but to build up a more astute (sic) picture, I need to do that for a lot of time/place combinations, and that is why I need to work out how to automate the process, and the problem there is converting what is in effect free text (the forecast) into something a computer can store and analyse. It’s a coding problem, how to code the free text into something systematic that a computer can use.

Timing is indeed a major headache for weather forecasters. They may have a good idea of the expected weather pattern, eg a low moving across the UK will tend to cause SE veering SW veering NW winds in the Channel, but what they are far less sure of is when that pattern will happen, because it depends very much on the track and speed of the low as it crosses the UK, and that is much harder to predict. They end up playing a variation of Eric Morecombe’s tune, all the right notes, but not necessarily in the right order. 

I looked into weather models last year because I found it annoying how, for example, the BBC weather website would have one forecast, and the BBC weather report on TV would have another and it turns they all use different modelling data (not sure why they do that).

There is a rumour the BBC stopped using the Met Office to provide its forecasts, and swapped to some budget option a few years ago, and as with budget airlines, you all too often end up in the wrong place at the wrong time. Also worth noting the variations arise not so much because they are using different modelling data (obs are obs), as different modelling models. The choice of model, and what the modeller chose to include and exclude, lies behind the variations. Websites like windy provide an easy way to switch models by clicking on the list of models at the bottom of the page.

Some of the most powerful supercomputers have been built and installed for meteorological prediction systems.

I think I am right in saying the Met Office’s super computer is the mother of all super computers. The problem is complex systems, too many variables, and errors that compound over time. Beyond a very short time frame, perhaps 12 to 24 hours, the whole thing is probably an absurd hubristic endeavour. The modelling paradigm – use past data to build a model, add some gridding to fill in the gaps, and then do some whatiffery to get a forecast – is simply not up to the job, and I doubt it ever will be. It’s a bit like using a steam engine to get to the moon, a technology doomed to fail in that endeavour, and no amount of better faster more powerful steam engines will ever succeed. We humans have yet to invent a new and revolutionary way of making better weather (and many other) predictions. There will certainly be a Nobel prize for anyone who comes up with something that works.

I think the next generation of forecasts will probably be AI driven

God help us! Surely I don’t need to point out that AI is just modelling by another name, and the whole point is the models don’t work much of the time. The powers that be are just doing a ‘the paradigm is dead, long live the paradigm’ dance. It is also if you look carefully a bit of an own goal, if ‘Artificial intelligence (AI) has the potential to revolutionise weather forecasting’, then the implication is surely that current forecasts must be pretty dire.

Endnote: reading and replying to the thread post by post, I see that I have covered points already made by others – sorry!        

Toodles
6274 kWhs
Reply to  cathodeRay
2 months ago

@cathodeRay The Nobel Prize will never happen as just as the final stage is being tested …. Funding will be Cut!
As to what and how Tesla are using for their weather predictions to decide on how much charge should be put in my battery – they have it all wrong! Charged overnight at cheapest times (five hours available), I wake up to find just 68% charge and it is grey now, wet and predicted to remain so all day! Harrumph! Toodles.

Toodles
6274 kWhs
Reply to  Toodles
2 months ago

Anyway, so far we have all missed the main protagonist … Bank Holidays! 😉 Toodles.

cathodeRay
Editor
6919 kWhs
1 month ago

Back to forecasts: I have a particular interest at the moment in whether it is going to rain or not (location is central south coast). Yesterday morning I am pretty sure the forecast (Met Office) was dry weather for the next few days, and I made my (foolish) plans accordingly. This morning we have rain forecast for all morning, some of it heavy:

image

 

For interest’s sake, I had a look at google’s cached version, which is timed at 22:01:06 GMT yesterday evening, ie less than 12 hours ago. The forecast then was rather different, only light showers between 1000 and 1300:

image

 

Who knows what will actually happen! No rain, light rain, or heavy rain? Also note how the ‘probabilities’ have changed. This volatility in the forecast is the product of modelling by one of the world’s biggest supercomputers. Another way of describing it may be to say it is the world’s most expensive random number generator.  

Majordennisbloodnok
Reply to  cathodeRay
1 month ago


Back to forecasts: I have a particular interest at the moment in whether it is going to rain or not (location is central south coast). Yesterday morning I am pretty sure the forecast (Met Office) was dry weather for the next few days, and I made my (foolish) plans accordingly. This morning we have rain forecast for all morning, some of it heavy:
— Attachment is not available —
 
For interest’s sake, I had a look at google’s cached version, which is timed at 22:01:06 GMT yesterday evening, ie less than 12 hours ago. The forecast then was rather different, only light showers between 1000 and 1300:
— Attachment is not available —
 
Who knows what will actually happen! No rain, light rain, or heavy rain? Also note how the ‘probabilities’ have changed. This volatility in the forecast is the product of modelling by one of the world’s biggest supercomputers. Another way of describing it may be to say it is the world’s most expensive random number generator.  

Hmm.
Looking at the images you posted, I’m interpreting it differently. Your first one (closer to the event) forecasts a 50% probability of rain between 8am and 2pm (jumping up to 80% chance during the middle). The second one (the earlier forecast from twelve hours earlier) forecasts a 40% probability between 10am and 1pm. So they’re both forecasting a reasonable chance of rain and the closer to the event it gets the clearer it becomes what’s likely to happen. I don’t see any inconsistency in that. Indeed, I’d be worried if the Met Office didn’t update their forecast based on new data coming in.
 

cathodeRay
Editor
6919 kWhs
1 month ago

Indeed, I’d be worried if the Met Office didn’t update their forecast based on new data coming in.

Of course, but that is not the point I am getting at here, which is the volatility over short time frames, in this case 24 hours or less. The bottom line is that forecasts that change cannot all be correct, in reality only one (if that) gets it right, which means all the others were wrong. You might imagine with all the hype that modern forecasting can do rather better than this. There’s also the collapse of the probability function thing I have mentioned before (in effect, applying group probabilities to an individual becomes meaningless in the event): a 60% probability of rain does suggest rain is more likely that a 30% probability, but in the event, it either rains or it doesn’t, and you have no way of knowing in advance, given say a 50% probability, whether you will be in the 50% that gets rain or the 50% that doesn’t. You might as well toss a coin…

It’s also not just about probabilities, but also the type of precipitation forecast, which in this example ranges from none through light showers and light rain to heavy rain. No rain is good, light showers can sometimes be worked around, but heavy rain is a wash out. 

Another way of looking at such forecasts is that they are a Schroedinger’s cat problem: you don’t actually know whether it is going to rain or not until you observe it, by which time you already know whether it is raining or not!     

Toodles
6274 kWhs
Reply to  cathodeRay
1 month ago

@cathodeRay If you can see them hills, it is going to rain – if you can’t see them hills, it is raining. 😉

Majordennisbloodnok
Reply to  cathodeRay
1 month ago



Indeed, I’d be worried if the Met Office didn’t update their forecast based on new data coming in.

Of course, but that is not the point I am getting at here, which is the volatility over short time frames, in this case 24 hours or less. The bottom line is that forecasts that change cannot all be correct, in reality only one (if that) gets it right, which means all the others were wrong. You might imagine with all the hype that modern forecasting can do rather better than this. There’s also the collapse of the probability function thing I have mentioned before (in effect, applying group probabilities to an individual becomes meaningless in the event): a 60% probability of rain does suggest rain is more likely that a 30% probability, but in the event, it either rains or it doesn’t, and you have no way of knowing in advance, given say a 50% probability, whether you will be in the 50% that gets rain or the 50% that doesn’t. You might as well toss a coin…
It’s also not just about probabilities, but also the type of precipitation forecast, which in this example ranges from none through light showers and light rain to heavy rain. No rain is good, light showers can sometimes be worked around, but heavy rain is a wash out. 
Another way of looking at such forecasts is that they are a Schroedinger’s cat problem: you don’t actually know whether it is going to rain or not until you observe it, by which time you already know whether it is raining or not!     

…except that’s not how probabilities work.
The chances of me reaching my 100th birthday are roughly 9.2%. If I actually do get a telegram from the King, that doesn’t mean the forecast was wrong; it was still correct that the odds were against me even if I beat those odds. Similarly, if a forecast predicts an 80% chance of rain and it turns out not to rain that doesn’t make the estimate of chances wrong.
Let’s not forget, though, why we are using forecasts. If we want to plan for a future event, we are effectively gambling. The forecast doesn’t stop it being a gamble but gives us an idea of which gambles are worth taking and which are not. The forecast isn’t telling us what will happen; it’s saying how likely something is to happen. Whether those odds make it a good choice to make are dependent on our viewpoint and on context. If, for example, I’d like to invite everyone round for a barbecue, I’d obviously be likely to pick a day with a good chance of no rain and if the odds aren’t good then I’d be well advised to make contingency plans. If I’ve a terminal condition and the only operation which could save me has a 10% chance of success, that’s still better than certain death.
What you’ve posted is suggesting a forecast is either right or wrong and that’s a misinterpretation. Forecasts attempt to provide an improvement on random chance and so can be better or worse but that’s all they’re there for. If you don’t find that useful, it’s still your choice not to use them. In the case of weather forecasts, as you mentioned earlier, it is quite difficult to discern how much of an improvement if at all weather forecasts provide over random guessing so it’s up to each of us to subjectively decide whether the forecasts are adding value for us. 
 

cathodeRay
Editor
6919 kWhs
1 month ago

What you’ve posted is suggesting a forecast is either right or wrong and that’s a misinterpretation.

I can’t agree with this. A conventional forecast is a prediction, and we can compare the prediction to what actually happens, and if the forecast and actual weather are the same, then the forecast was right, if they differ, the forecast was wrong.

The relatively recent introduction of probabilities to some forecasts – not ubiquitous, eg neither the Shipping Forecast not the Inshore Waters Forecast use them – does introduce more complexity. If the 0600 Shipping Forecast this morning says ‘rain later’ that means it predicts it will rain tonight (and can be judged right or wrong on whether it does or doesn’t rain tonight). Were the Shipping Forecast to say ‘rain (40%) later’ then the proper interpretation of that is probably that on balance, it is less likely to rain than rain. I suppose the same information could also be expressed by saying ‘fair (60%) later’. Using probabilities does give the Met Office some wiggle room: they can always say we only said it might rain/not rain, not that it will/will not rain. But presenting what will in the event be a binary outcome (rain/no rain) as a prediction with an attached probability (40%) may appear to give more information, but I am not sure in practice it does, because back in the real world I still have to treat it as a binary prediction – I have to work on the basis that it will or won’t rain tonight (the collapse of the probability function, there is no such thing as 40% rain in the real world).    

I have used the rain forecast in this example. What if instead it was something that goes beyond inconvenience (ordinary as opposed to torrential rain)  to potential danger, eg the wind forecast? How do I use a forecast that says ‘SW 6 to Gale 8 (40%)’? It is I suggest more useful in the real world (as opposed to the imagined world of probabilities) for the Met Office to make a judgement call, and declare what in its opinion it thinks will happen, which I can then use in my planning. The point I am getting at here is pragmatism, making real decisions in the real world.  

I agree that if the probability of rain this weekend is 10% on Saturday and 90% on Sunday then it would be a better bet to have my barbecue on Saturday, ie I can use the probabilities to make comparisons between two times (or places for that matter). But the problem with this, as I tried to point out, is the volatility of the forecast. What if, as often happens (and happened in the examples I used), the forecast constantly changes? On Monday the forecast for the weekend might be fair throughout (90%), on Tuesday the forecast is light showers on Sunday evening (30%), on Wednesday is is fair throughout again (60%), on Thursday it is heavy rain all day Saturday (80%), and so on, right up until the weekend itself? Which day do I hold my barbecue, and when do I make that decision?

The underlying problem is the current forecast models, especially those with probabilities, are simply not very good, hence the volatility. If a timely observation of bee’s fart in Brazil can change the forecast for the whole of the UK, is it any surprise that the forecast is so volatile? As I have said before, I am inclined to think weather forecasting will only improve significantly with a major paradigm shift, as opposed to making do with ever more complex models run on ever bigger computers.  

If I’ve a terminal condition and the only operation which could save me has a 10% chance of success, that’s still better than certain death.

Alas, medical decision making is not that simple. All the operation will do is (possibly) delay death (there are only two certainties in life…), and what if the operation is itself a horrendoplasty, with a ‘successful’ outcome leaving the patient in a state worse than death itself? There is a saying in medicine, thou shall not kill, nor strive officiously to keep alive, to guide the doctor against pointless heroics among other things, though I do of course accept that at the end of the day it is the patient’s decision on whether to go ahead or not with the operation, provided always that that is an informed decision.    

Majordennisbloodnok
4527 kWhs
1 month ago

 


I can’t agree with this. A conventional forecast is a prediction, and we can compare the prediction to what actually happens, and if the forecast and actual weather are the same, then the forecast was right, if they differ, the forecast was wrong.

But if you compare what was predicted with what happened, how tight or fuzzy do you make your judgement criteria? If a forecast predicts rain between 9 and 11 but the rain occurs between 12 and 2 your analysis would show that as a complete failure whilst I would happily defend it as an accurate prediction offset by three hours.
I believe that most people now use a weather forecasting service via an app or website and most of those give the forecasts as probabilities. As such that is now a de facto “conventional” forecast and so I see no issue with determining accuracy as a spectrum rather than your binary right or wrong. Even when probabilities weren’t routinely shown, people used their subjective “on the money”, “fairly close”, “not great” and “completely off” evaluations based on much more than whether the forecast was right or wrong.

Alas, medical decision making is not that simple. All the operation will do is (possibly) delay death (there are only two certainties in life…), and what if the operation is itself a horrendoplasty, with a ‘successful’ outcome leaving the patient in a state worse than death itself?

Whilst I agree with the nuances you raise, all I was trying to illustrate with the medical scenario was that probabilities have to be taken in context, and that context can vary wildly. In terms of weather, that context has to include time offsets as well as what someone views a reasonable variances of temperature or precipitation, and even then viewed within the context of how much those variances affect whatever you’re trying to plan for. None of that lends itself towards a binary right/wrong evaluation.
 

scrchngwsl
1505 kWhs
1 month ago

This is all a bit too frequentist for my liking. The probability of rain is, imo, better understood as an expression of the degree of belief in it raining.
Also I must say I have found the Met Office forecasts to be very accurate and dependable.

49
0
Please leave a comment.x
()
x
x  Powerful Protection for WordPress, from Shield Security
This Site Is Protected By
Shield Security