@batalto - Purely by chance, I spotted this yesterday evening on the Midea app and it is rather curious. It shows hourly usage earlier in the day, around the middle of the day. At about 1130 I noticed on my desktop my internet connection has dropped out, but as I was about to turn the desktop off and head out for lunch I didn't think anything of it, expecting it would be back by the time I got back around 1530. But it wasn't, I had to reboot a couple of times. What we have on the app screen is the period when the internet was down:
Unsurprisingly, there is no data for the period when there was no internet, the wired controller couldn't send anything to the Midea cloud so no data during that period. But what is curious is how it deals with the data before/after the dropout. Yesterday's usage was normally around 1-2 kWh per hour. What the app appears to have done is retrospectively added the internet outage period use to the hour before the outage. In fact, it is even more curious: it has been added to the 1200 bar, despite the fact that by noon there was definitely no internet, and the trace stays flat until 1700, despite the fact I definitely had internet back by 1600.
Apart from the obvious conclusion that the Chinese continue to want to ensure we live in interesting times, I am not sure what to make of this, except that perhaps in a roundabout way it does confirm that the Midea cloud/app determine hourly (and so by implication other interval) use indirectly, by subtracting one lifetime usage figure from another lifetime usage figure, and because the lifetime figures are always integers, the result of the subtraction is always a an intege. But to know, at 1200, what the usage would be over the next few hours, now truly that can only be the work of a Chinese genius!
Midea 14kW (for now...) ASHP heating both building and DHW
@batalto - Purely by chance, I spotted this yesterday evening on the Midea app and it is rather curious. It shows hourly usage earlier in the day, around the middle of the day. At about 1130 I noticed on my desktop my internet connection has dropped out, but as I was about to turn the desktop off and head out for lunch I didn't think anything of it, expecting it would be back by the time I got back around 1530. But it wasn't, I had to reboot a couple of times. What we have on the app screen is the period when the internet was down:
Unsurprisingly, there is no data for the period when there was no internet, the wired controller couldn't send anything to the Midea cloud so no data during that period. But what is curious is how it deals with the data before/after the dropout. Yesterday's usage was normally around 1-2 kWh per hour. What the app appears to have done is retrospectively added the internet outage period use to the hour before the outage. In fact, it is even more curious: it has been added to the 1200 bar, despite the fact that by noon there was definitely no internet, and the trace stays flat until 1700, despite the fact I definitely had internet back by 1600.
Apart from the obvious conclusion that the Chinese continue to want to ensure we live in interesting times, I am not sure what to make of this, except that perhaps in a roundabout way it does confirm that the Midea cloud/app determine hourly (and so by implication other interval) use indirectly, by subtracting one lifetime usage figure from another lifetime usage figure, and because the lifetime figures are always integers, the result of the subtraction is always a an intege. But to know, at 1200, what the usage would be over the next few hours, now truly that can only be the work of a Chinese genius!
The answer is quite simple. Internet was lost between 11:00 and 11:30, and restored by 16:00.
So the last data that the midea cloud would have received, prior to losing the internet connection, would have been the 10:00 to 11:00 period. It therefore stored the total energy consumption as measured at 11:00. It would appear that data is only sent, or stored, once per hour at around the hour mark.
When the internet connection was restored, which would appear to have been at a time just after the time when it would normally have transmitted the data, so the next data transmission occurred around 17:00, which is when the temperature traces reappear.
Upon receiving an updated Energy Consumed figure, the Midea Cloud performs the normal calculation of subtracting the previous total from the new total, to obtain the amount of energy used during the measurement period. In this case the measurement period is 6 hours rather than the normal 1 hour, so instead of obtaining a reading of 1-2kWh it calculates a value of 7kWh, which is placed in the next time slot in the Midea Cloud record, which is the 11:00 to 12:00 period.
When you retrieve the data from the Midea Cloud, that is what is displayed.
@derek-m - possibly, but if it puts it the 'next time slot in the Cloud record' then why doesn't the bar displayed in the 1800 slot end up in the 1300 slot?
I don't think we need to get too distracted with this, it was really just to show some quirky app behaviour, and that it appears to calculate interval usage by subtraction, hence the always integer values (two actual hours of 0.5kWh use will appear as 1kWh in the second hour etc) . More to the purpose of trying to get more data which we know is in there somewhere, but can't yet access, eg ambient temp and set LWT (and hopefully actual LWT), I have been looking again at the message/body type 4 messages that have the lifetime energy produced/consumed numbers. Looking the logs, these are indeed usually at hourly(-ish) intervals, but then there might be a few a lot closer together from time to time. Typical log entries look like this:
The first log entry is the message as received (and partly decoded, at least from Midea's proprietary encryption?), the second is what midea_ac_lan pulls out of the message and adds to the database, apart from the testvar entries which I added to the code (testvar1 = body[data_offset + a number] etc) to see what values emerged from various offsets. So far none have looked promising, and most are static over time.
The message '0401000023d700005f1809281e32302bff01' looks as though it has more than four boolean and four integer values in it, but maybe it hasn't, the string is 36 chars long, and if we divide that into four char chunks, there are only nine chunks... but they are interesting. With each set of four on one line we have, and lines three and five are plain hex representations of the energy consumed/produced (see above; last time I did those we didn't know the energy produced decimal value):
meaning it looks as though I can get Midea reported energy in/out (plus timestamps, and so COP at whatever interval I fancy). But the rest are less obviously interesting (hex on left, decimal on right):
0400 => 1024 (down from 1025, something going from on to off? ) 0000 => 0 23d9 => 9177 (energy consumed, up by 2, credible) 0000 => 0 5f1f => 24351 (energy produced, up by 7, credible, gives a current COP of 3.5) 0828 => 2088 (down from 2344, quite a drop - unless this is a number to two decimal places, 20.88 down from 23.44?) 1e32 => 7730 (unchanged) 302c => 12332 (up from 12331, something going from off to on?) ff01 => 65281 (unchanged)
All very baffling. And then there is the question of how to get the other message/body types, at the moment I only get types 01 and 04, suggesting there may be types 02 and 03, possibly even 05 and up. Cripes.
Midea 14kW (for now...) ASHP heating both building and DHW
but if it puts it the 'next time slot in the Cloud record' then why doesn't the bar displayed in the 1800 slot end up in the 1300 slot?
Although you state that your internet returned to life by 16:00, I suspect that it was after the 16:00 data should have been transmitted, so that transmission was missed, along with the ones for 12:00, 13:00, 14:00 and 15:00.
The data sent to the Midea Cloud will be 'time stamped', such that when the data arrives at 17:00, the calculation is performed subtracting the previous 11:00 data from the data that has just arrived at 17:00, which produces the consumed amount of 7kWh.
The calculated value would normally be put in the 16:00 record, but the 16:00 record does not exist, as neither does the 15:00, 14:00 or 13:00 records, so it goes in the 12:00 record.
From the energy consumed perspective, the next full hour of data was for the 17:00 to 18:00 period, which was therefore added to the 18:00 record, as shown on the bargraph.
The calculated value would normally be put in the 16:00 record, but the 16:00 record does not exist, as neither does the 15:00, 14:00 or 13:00 records, so it goes in the 12:00 record.
Thanks, that does makes sense. It also explains how the app appears to see into the future.
Midea 14kW (for now...) ASHP heating both building and DHW
@derek-m - possibly, but if it puts it the 'next time slot in the Cloud record' then why doesn't the bar displayed in the 1800 slot end up in the 1300 slot?
I don't think we need to get too distracted with this, it was really just to show some quirky app behaviour, and that it appears to calculate interval usage by subtraction, hence the always integer values (two actual hours of 0.5kWh use will appear as 1kWh in the second hour etc) . More to the purpose of trying to get more data which we know is in there somewhere, but can't yet access, eg ambient temp and set LWT (and hopefully actual LWT), I have been looking again at the message/body type 4 messages that have the lifetime energy produced/consumed numbers. Looking the logs, these are indeed usually at hourly(-ish) intervals, but then there might be a few a lot closer together from time to time. Typical log entries look like this:
The first log entry is the message as received (and partly decoded, at least from Midea's proprietary encryption?), the second is what midea_ac_lan pulls out of the message and adds to the database, apart from the testvar entries which I added to the code (testvar1 = body[data_offset + a number] etc) to see what values emerged from various offsets. So far none have looked promising, and most are static over time.
The message '0401000023d700005f1809281e32302bff01' looks as though it has more than four boolean and four integer values in it, but maybe it hasn't, the string is 36 chars long, and if we divide that into four char chunks, there are only nine chunks... but they are interesting. With each set of four on one line we have, and lines three and five are plain hex representations of the energy consumed/produced (see above; last time I did those we didn't know the energy produced decimal value):
meaning it looks as though I can get Midea reported energy in/out (plus timestamps, and so COP at whatever interval I fancy). But the rest are less obviously interesting (hex on left, decimal on right):
0400 => 1024 (down from 1025, something going from on to off? ) 0000 => 0 23d9 => 9177 (energy consumed, up by 2, credible) 0000 => 0 5f1f => 24351 (energy produced, up by 7, credible, gives a current COP of 3.5) 0828 => 2088 (down from 2344, quite a drop - unless this is a number to two decimal places, 20.88 down from 23.44?) 1e32 => 7730 (unchanged) 302c => 12332 (up from 12331, something going from off to on?) ff01 => 65281 (unchanged)
All very baffling. And then there is the question of how to get the other message/body types, at the moment I only get types 01 and 04, suggesting there may be types 02 and 03, possibly even 05 and up. Cripes.
To be able to decipher the body I will need the relevant data values from your controller for comparison.
The problem is that the message body does not just consist of integers of fixed length, but has bolean, integers, and fixed real numbers, which would not be a problem if we knew the structure of the message.
To be able to decipher the body I will need the relevant data values from your controller for comparison.
I can certainly manually record data values from the wired controller, but some of them are very volatile (actual LWT/RWT, current draw, capacity (outout)) and some are curiously static (flow rate). It doesn't report the set LWT, but that must be recorded, because it appears in the Midea app, but it does report the ambient temp, which is credible (given where it is, it may under-read a bit) and not too volatile, maybe a good target to try and find.
It is obtained by the code in midea_ac_lan, the custom python component I've added to HA. The inner workings of the code are not that clear, at least to my eye. There are some requests in the code (done over LAN to the wired controller IP address), and the logs (see below) report a messages as having been 'Received'. To me, this appears to suggests it makes explicit requests, rather than passively sniffing out stuff the wired controller is already sending out.
So far, I know there are two message types, (a) 'message type': '03', 'body type': '01' and (b) 'message type': '04', 'body type': '04'. We know most if not all of the content detail for the 03/01 messages, because the log shows both the message received string and the decoded data that updates the HA database. There are regular sending/received/status update triplets in the log like this (the [custom_components...] string identifies the file that logged the entry ie the Sending entry comes from midea_ac_lan\midea\core\device.py):
So far, all the code comes from the original author of the module. another author then forked the code, and added code that gets a second type on message, the 04/04 type:
This lacks a "Sending" log entry, but that of source doesn't mean a message wasn't sent, one may have been, it just wasn't logged. The 'testvar' variables, as I mentioned before are my additions, to pull out whatever is at various data offsets. The have not been illuminating. The code also appears to process these variables in a similar but no identical way to the way the 03/01 messages are processed.
The other thing I have tried to work out is what sets the timing of the messages (as a clue to what is actually doing the sending). The 03/01 messages to all intents and purposes happen every minute. The 04/04 messages are mostly every hour but then apparently random timed messages also get received during the hour, but presumably something triggers the message.
Here is the code from \midea\core\device.py that culminates in the Sending message in the log triplet above. The earlier functions in this code extract are I think making the initial connection, the Authentication etc log entries only appear very infrequently. You can also see references to encode_8370 - this is Midea's so called version 3 encryption, but I think mercifully by the time we get to see the message bodies in the log they have been decrypted:
def fetch_v2_message(msg):
result = []
while len(msg) > 0:
factual_msg_len = len(msg)
if factual_msg_len < 6:
break
alleged_msg_len = msg[4] + (msg[5] << 8)
if factual_msg_len >= alleged_msg_len:
result.append(msg[:alleged_msg_len])
msg = msg[alleged_msg_len:]
return result, msg
def connect(self, refresh_status=True):
try:
self._socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self._socket.settimeout(10)
self._socket.connect((self._ip_address, self._port))
_LOGGER.debug(f"[{self._device_id}] Connected")
if self._protocol == 3:
self.authenticate()
_LOGGER.debug(f"[{self._device_id}] Authentication success")
if refresh_status:
if self._sub_type is None:
self.get_sub_type()
self.refresh_status(wait_response=True)
self.enable_device(True)
return True
except socket.timeout:
_LOGGER.debug(f"[{self._device_id}] Connection timed out")
except socket.error:
_LOGGER.debug(f"[{self._device_id}] Connection error")
except AuthException:
_LOGGER.debug(f"[{self._device_id}] Authentication failed")
except ResponseException:
_LOGGER.debug(f"[{self._device_id}] Unexpected response received")
except RefreshFailed:
_LOGGER.debug(f"[{self._device_id}] Refresh status is timed out")
except Exception as e:
_LOGGER.error(f"[{self._device_id}] Unknown error {repr(e)}")
self.enable_device(False)
return False
def authenticate(self):
request = self._security.encode_8370(
self._token, MSGTYPE_HANDSHAKE_REQUEST)
_LOGGER.debug(f"[{self._device_id}] Handshaking")
self._socket.send(request)
response = self._socket.recv(512)
if len(response) < 20:
raise AuthException()
response = response[8: 72]
self._security.tcp_key(response, self._key)
def send_message(self, data):
if self._protocol == 3:
self.send_message_V3(data, msg_type=MSGTYPE_ENCRYPTED_REQUEST)
else:
self.send_message_V2(data)
def send_message_V2(self, data):
if self._socket is not None:
self._socket.send(data)
def send_message_V3(self, data, msg_type=MSGTYPE_ENCRYPTED_REQUEST):
data = self._security.encode_8370(data, msg_type)
self.send_message_V2(data)
def build_send(self, cmd):
data = cmd.serialize()
_LOGGER.debug(f"[{self._device_id}] Sending: {cmd}")
msg = PacketBuilder(self._device_id, data).finalize()
self.send_message(msg)
This is what psychiatrists might call a word salad, but we know it is code, and like all codes, it can be cracked. All we need to do is crack it!
The two github urls to find all the code are here (original) and here (fork).
Midea 14kW (for now...) ASHP heating both building and DHW
In SCADA systems there are normally two ways that data is passed from computer to computer, one is by timed polling of the various control computers by the display computer. So every second, or so, the display computer may ask each control computer, in turn, to send a defined list of data, which the display computer then records and uses to update the displays. Of course this is not that efficient, since much of the data may not have changed since the previous transmission. An alternative method is for the control computers to only send data that has changed by a predefined amount since the last transmission.
Unless I am mistaken, I suspect with that your HA system, data is only sent from the Midea when requested by HA.
I think that I understand quite a bit of the 'word salad', although I am not familiar with Python and its syntax, it is quite similar to other programming languages I have used.
def fetch_v2_message(msg):
result = []
while len(msg) > 0:
factual_msg_len = len(msg)
if factual_msg_len < 6:
break
alleged_msg_len = msg[4] + (msg[5] << 8)
if factual_msg_len >= alleged_msg_len:
result.append(msg[:alleged_msg_len])
msg = msg[alleged_msg_len:]
return result, msg
The above subroutine would appear to be checking the length of the received message, and that it meets certain criteria.
I will have a look through the rest of the code to see what I can identify. Can you recommend a good source of information on Python?
unit_of_measurement so I'm wondering should I just omit these lines?
Also, I was wondering did you get unknown data for both Total Energy Consumption and Production? Not sure why these don't have readings but the others do
I was wondering did you get this when you added the sensors to configuration.yaml (after running a validation in home assistant UI)
Yes I did, thanks for reminding me, but I don't recall it being a critical error, or maybe I didn't need to restart HA, and so I didn't trigger the configuration check. You are correct, HA Statistics don't appear to have a unit_of_measurement themselves, so just delete them. Presumably it just picks up the inits from whatever it is doing the stats on. I will change the previous post to reflect this.
Total Energy Consumption and Production - these take a shortish period, up to about an hour, to appear, as HA needs two readings (to subtract one from the other) to get a value, and HA only gets the data every hour or so. Sit tight, they should appear soon.
Midea 14kW (for now...) ASHP heating both building and DHW
Thinking about installing a heat pump but unsure where to start? Already have one but it’s not performing as expected? Or are you locked in a frustrating dispute with an installer or manufacturer? We’re here to help.
