Posted by: @editor

↑

There have been several discussions about accessing and analysing data from your heat pump interspersed in other forums discussions and I thought it would be a good idea to have a dedicated topic.

Indeed, I have contributed to these other discussions.

There is no doubt the data is there, and it also seems equally beyond doubt that the manufacturers and and heat pump sellers want to block us from getting the data. For example, on my Midea controller, just like you, I can see many measurements being updated in real time - LWT, RWT, flow rate, capacity (which is output), power consumption and supply voltage (so can calculate watts) and many more. But only a fraction (or maybe more gets uploaded, but we aren't allowed to see it) of these get uploaded by the Midea app to its server, and only can only access a pitifully small set of aggregated data which is mostly ephemeral, the next day's hourly data overwrites the previous days hourly data etc. The only way to capture this data, unbelievably in the day and age, is by using pencil and paper, and even then it is severely limited by practical considerations. The fact the minimum inits of time on the app is one hour means it is useless for looking at things like defrost cycles.

And then there is the Freedom Midea app, which uses an 'incredibly simple' piece of hardware to upload your heat pump's date to their (or Midea's, it's not entirely clear) which they can play with - but their seems to be no way for us to access it. We get a dummed down 'consumer' dashboard - but it is not at all clear how we can even get to that, or what limited data it has, or what the cost is (it should be free, it's our data, after all). But the important bit of information here is that an 'incredibly simple' bit of hardware can do the magic of accessing and logging the data.

This set me thinking. The data is there, there is a 'incredibly simple' way of getting to it - why can't we do it ourselves? The Midea controller Operation Manual has a schematic showing wiring connections of a sort, and gives the communications ports and protocols (RS-485, Modbus RTU). Home automation software like domticz can apparently connect to and read these ports and protocols. Surely it shouldn't be that difficult to join it all up??? Independent (of anything Midea) energy consumption can be monitored it real time if you have a meter with a pulsing LED (pulses every so many watts of energy) by using an optical sensor.

I happen to have a Midea unit, but I wouldn't mind betting most other heat pumps brands have similar characteristics, and could be accessed in similar ways. The bottom line is this is our data, we should have free and easy access to it, with an option to sell it to the heat pump manufacturers and sellers, if we so wish, not the other way round.

I have a latest generation Daikin LT Altherma unit. I have got a basic setup going by using a custom integration on Home Assistant to get some basic information over the cloud from Daikin's systems. It's no more than you get with their app really and whilst it includes energy consumption it doesn't include heat delivery or COP, and there's limited control. Link to the integration for anyone interested: https://github.com/speleolontra/daikin_residential_altherma there's a whole family of similar integrations for different setups so definitely worth having a poke around even if that one isn't compatible.

In the future I plan on adding hardware to read directly from the ports on the controller to get all the kinds of information @editor mensioned using the method and code described here: https://github.com/Arnold-n/P1P2Serial the person who made that makes a nifty little box that you can email him to buy. It is ment

There's also the option of plugging directly into the heat pump itself which I'm a little bit more wary of, but might end up doing because it would give me cheap access to "smart grid" features eg boosting hot water when my solar panels have a high output. As with the above option, it also gives live information that I can see and analyse remotely. Link for that option:  https://github.com/raomin/ESPAltherma

Obviously this is all a bit more involved than just using Daikin's app, but their app is a bit rubbish and for stuff like smart grid features you have to pay hundreds of pounds for additional hardware from Daikin themselves. No thanks!

Of course the extreme way of getting all the data is by purchasing an external monitor for all of the info you want. These guys sell a kit to do just that: https://shop.openenergymonitor.com/level-3-heat-pump-monitoring-bundle-emonhp/ I'm not going with this option personally because as listed above I can do it cheaper using Daikin's own measurements, and I saw a few comments from people online saying their external monitors matched quite closely with Daikin's own figures.

OK, this was part of the reason for going for Mitsubishi Ecodan. I currently have access by using "pymelcloud" to connect to the same json-over-https services that the "melcloud" app uses, but those figures are delayed a few minutes and cached, which is annoying but does not matter a great deal in practice because heat pump heating is not exactly fast-response anyway. I've a logging program that takes those numbers and feeds them to MQTT (for home automation) and emoncms which is published at https://github.com/mjray/melcloud-mqtt.py as well as taking commands from MQTT and sending them to melcloud (which reminds me that I've a new version which can control flow temperatures that I should get on and publish).

It's reportedly also possible to install a device to offer a direct connection similar to what's described in https://chrdavis.github.io/hacking-a-mitsubishi-heat-pump-Part-1/ and https://github.com/SwiCago/HeatPump but I didn't realise at first that it uses the same connector as the melcloud adapter, so that's something I may try later, if melcloud goes away or charges or whatever.

It's disappointing that the melcloud adapter (MAC-567IF-E) seems to have a http interface on the LAN but I've never found any documentation of what it does or what the username/password is. The equivalent device for Aus/NZ (MAC-568IF) has an option to enable an "EchonetLite" protocol which seems to work on the LAN, without requiring working internet, but the European one doesn't have it.

emoncms is the best tool I've seen for detailed analysis. It comes with a "MyHeatPump" app and there's another one called "MMSPHeatPump" which is fiddly to install from details on the forums but offers a different view and real-time (well, as near as the melcloud delay permits) monitoring of several measurements.

@oswiu - interesting, all gist to the mill.

Here are some screen grabs of some things I have come across. From a Midea M-Thermal 'consumer' brochure:

What the wired controller can do:

Evidence of existence of a USB connection somewhere (I think this may be in the outdoors unit):

Split over two posts to get round image limit per post...

@cathoderay I think one day you're just going to have to dive right into doing it from modbus yourself if there's not a starting point out there to help. Maybe Midea would even help you with the documentation if you sweet talked them

@oswiu - I've got a lot to learn. I come from the Morris Minor era of electrical engineering, where the most complex thing I had to do was spray WD40 on the HT leads in damp weather to try to get a decent spark...

I'm afraid I'm with Mars on the manufacturers being helpful, somehow I get the impression they don't want us to access this data. If they did, it would be freely available.

@cathoderay have you considered getting a Homely?

And obviously I agree that the manufacturers should do better. It winds me up that they all have the data and they all have apps to show you the data in, but they just choose not to. 

All this talk of obfuscation..... I just sent an email to the MD of my ASHP supplier and said 'can I have the modbus register map please' - got a PDF back within 5 mins.

Not that I've done anything with that yet... but it's encouraging

Posted by: @mjr

↑

OK, this was part of the reason for going for Mitsubishi Ecodan. I currently have access by using "pymelcloud" to connect to the same json-over-https services that the "melcloud" app uses, but those figures are delayed a few minutes and cached, which is annoying but does not matter a great deal in practice because heat pump heating is not exactly fast-response anyway. I've a logging program that takes those numbers and feeds them to MQTT (for home automation) and emoncms which is published at https://github.com/mjray/melcloud-mqtt.py as well as taking commands from MQTT and sending them to melcloud (which reminds me that I've a new version which can control flow temperatures that I should get on and publish).

It's reportedly also possible to install a device to offer a direct connection similar to what's described in https://chrdavis.github.io/hacking-a-mitsubishi-heat-pump-Part-1/ and https://github.com/SwiCago/HeatPump but I didn't realise at first that it uses the same connector as the melcloud adapter, so that's something I may try later, if melcloud goes away or charges or whatever.

It's disappointing that the melcloud adapter (MAC-567IF-E) seems to have a http interface on the LAN but I've never found any documentation of what it does or what the username/password is. The equivalent device for Aus/NZ (MAC-568IF) has an option to enable an "EchonetLite" protocol which seems to work on the LAN, without requiring working internet, but the European one doesn't have it.

emoncms is the best tool I've seen for detailed analysis. It comes with a "MyHeatPump" app and there's another one called "MMSPHeatPump" which is fiddly to install from details on the forums but offers a different view and real-time (well, as near as the melcloud delay permits) monitoring of several measurements.

Are you using pymelcloud to set the flow temperatures? I didn't realise it could do that! Or did you make your own code for that?

I, too, have tried accessing the melcloud adaptor over LAN, but to no avail. One of these days I'll have a go with Wireshark and see what I can find.

EDIT: Oh and this too, I want to play with this at some point: https://github.com/ncaunt/meldec

Posted by: @hughf

↑

All this talk of obfuscation..... I just sent an email to the MD of my ASHP supplier and said 'can I have the modbus register map please' - got a PDF back within 5 mins.

Not that I've done anything with that yet... but it's encouraging

Hugh, Midea go even further, they publish the modbus register in the Operation Manual that comes with the wired controller. But what they don't do is make the data easy to access unless you have a lot of know how, and that is going to be beyond the ken of probably 99%, maybe 99.9%, of home owners. Even when you do have the registers and know how, it is far from straightforward, as countless dead end trails on the internet show. There simply isn't a straightforward recipe to make things work, or is there is, I have yet to find it. Even those who do have know how struggle, as can be seen for example in the Midea Air-Con python libraries. No one even appears to have tried with Midea heat pumps.

The Midea modbus registers from the Operational manual were in the images I couldn't post earlier. Will try again shortly. 

It shouldn't be this hard. That's where the obfuscation is happening. The data is ours, we generated it, and it has the potential to be very useful for anyone who wants to fine tune their system, or even simply know more about it.  

Posted by: @oswiu

↑

@cathoderay have you considered getting a Homely?

This is one of the many hurdles, too much choice, making it very difficult to decide which one to go with. Homely, Home Assistant, Domoticz, emonCMS, dozens of python libaries all trying to solve the same problem, none of then quite doing it. I think I might start a thread on a beginner's account to trying to set up heat pump monitoring...