@cathoderay

I suspect, though you will need to be the one to prove it, that each hour your Midea controller sends a stream of data to the Midea's server. The Python software monitors this data stream, and takes a copy of any of the parameters specified within the Python code, which it then places in the message body at the designated locations.

The first thing to be aware of, is that the message does not contain the full data, also the position of a variable within the message is set by the Python code, not by the data stream. You can test this theory by obtaining a copy of the message, then changing outdoor temperature from position '9' to position '10', within the Python code.

I believe the HA then logs the data obtained from the message.

It is possible that the Python code identifies the different parameters within the data stream by their name, such as 'indoor temperature'. This could be tested by changing the relevant entries in the Python code, from 'outdoor temperature' to 'indoor temperature', and note if there are any changes that occur to the data within the message at the next update.

Looking at the Modbus register information, I suspect that in the heat pump (C3), the outdoor temperature variable is stored as an integer occupying just 1 byte, not as shown for the AC unit. This may become obvious in the hexadecimal value obtained from the message body.

Posted by: @derek-m

↑

I suspect, though you will need to be the one to prove it, that each hour your Midea controller sends a stream of data to the Midea's server.

I think this has to be the case, given neither of us has found anything in the code that requests the 04/04 message data. The odd thing is the random during the hour messages - why to they get sent?

Here's the routine on the hour 1200 message. For some of reason it got (sent)/received twice, a few seconds apart:

2023-02-20 12:00:33.750 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.devices.c3.device] [deviceID] Received: {'header': 'aab9c300000000000004', 'body': '0401000026fd0000686d0b281e323028ff01000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f0000000000000000000000000000000000014', 'message type': '04', 'body type': '04'}
2023-02-20 12:00:33.751 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.core.device] [deviceID] Status update: {'status_heating': True, 'status_dhw': False, 'status_tbh': False, 'status_ibh': False, 'total_energy_consumption': 9981, 'total_produced_energy': 26733, 'outdoor_temperature': -1.0}
2023-02-20 12:00:39.943 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.devices.c3.device] [deviceID] Received: {'header': 'aab9c300000000000004', 'body': '0401000026fd0000686d0b281e323028ff01000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f0000000000000000000000000000000000014', 'message type': '04', 'body type': '04'}
2023-02-20 12:00:39.943 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.core.device] [deviceID] Status update: {'status_heating': True, 'status_dhw': False, 'status_tbh': False, 'status_ibh': False, 'total_energy_consumption': 9981, 'total_produced_energy': 26733, 'outdoor_temperature': -1.0}

As you can see, mystery vars removed (all relevant code has been commented out), and 'outdoor_temperature' remains at -1.0.

Posted by: @derek-m

↑

The first thing to be aware of, is that the message does not contain the full data, also the position of a variable within the message is set by the Python code, not by the data stream.

Sorry to be dim, but can you please elaborate on both points. I understand each message/body type has only certain elements, ie not the full set, but shouldn't each element in a message represent full data for that element? I'm also confused as how the python code sets a variable's position - surely it leaves the wired controller as a string, with the order, and so position, of the variables already set? 

@cathoderay

I cannot explain why the data transfer does not occur just once per hour, since I don't know if it is being instigated by the Midea server or your Midea controller.

Part of the message has changed at position '9', which yesterday was 08 = 8C possibly, it is now indicating 0b = 11C? What is the outdoor temperature showing on your controller?

I will modify the software text files that I produced, so that we can be certain that all the code has been modified, since the message still appears to  contain data from the total mystery tests. I will also explain my thoughts about the positioning of the variables within the message.

I have to go out later this afternoon, so I may not be able to do this until later today.

@derek-m - at the moment (13:24) the wired controller outdoor temp shows 12 degrees, maybe it was 11 degrees.

At 1308 (one of the random not on the hour messages) it has changed to 0c, decimal 12, I think you may have spotted the outdoor temp integer! And maybe that is all it is, just a plain integer, no manipulation or decimal needed. On the wired controller display and in the app, it is always just an integer.   

2023-02-20 13:08:18.111 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.devices.c3.device] [device_id] Received: {'header': 'aab9c300000000000004', 'body': '0404000026ff000068720c281e323027ff01000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f3000000000000000000000000000000000014', 'message type': '04', 'body type': '04'}
2023-02-20 13:08:18.112 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.core.device] [device_id] Status update: {'status_heating': False, 'status_dhw': True, 'status_tbh': False, 'status_ibh': False, 'total_energy_consumption': 9983, 'total_produced_energy': 26738, 'outdoor_temperature': -1.0}

Note also the change at the beginning of the message, 0401 to 0404, and the changes in status_heating and status_dhw, were true, false, now false, true. This is correct: I have the DHW timed to be on between 1300 and 1400.

I'm also busy this afternoon, pick this up again later.   

@cathoderay

It is not so much that I spotted the outdoor temperature, it was requested by adding it to the list. I will now be interested to see how the system deals with temperatures below zero, the '0c' should probably change to '8c', which I think would indicate -12C.

The next task will be to change the 'outdoor' to 'indoor', which will hopefully provide a different value. I suspect that it will be the actual indoor temperature * 2, since I do believe it has a resolution of 0.5C. Don't worry about the Python code maths, that can be sorted later.

If the 'indoor' test proves successful, then draw up a list of the other variables that you would like to capture on an hourly basis. The Modbus register has a comprehensive list, though I don't know if they are all in the data stream. Once your list has been compiled think above the probably memory space required, 1 byte for small integers up to 127, 2 bytes for larger integers and 4 bytes for very large. The Modbus register data also provides details of the likely maths that will need to be performed.

For variables that you may require more frequently than once per hour, we can look at modifying the type 01 message.

Posted by: @derek-m

↑

It is not so much that I spotted the outdoor temperature, it was requested by adding it to the list. I will now be interested to see how the system deals with temperatures below zero, the '0c' should probably change to '8c', which I think would indicate -12C.

I doubt we will get -12C in the near future! In fact, I think we are unlikely to get much below zero during this week, if that. I will watch the local Met Office weather reports and if/when a below zero reading happens, compare with the likely outdoor temp value from the message. 

I think you did spot the outdoor temperature (if it is, we have yet to confirm it, I will add some code to pull that byte). Maybe this is a bit semantic, but the request doesn't create it, it is already there, but it needs to be revealed by asking for the right data from the message, so there is an element of spotting it, in the sense that you need to spot the right byte(s) to request. Or perhaps I have completely misunderstood!

The Midea setup, at least mine, doesn't use indoor temperature, because it uses weather compensation. Neither the wired controller nor the app report indoor temp. The wired controller does have a line for Ta ROOM TEMP on page 3 of the operational parameter pages but the value shown is '--'. At the moment, hourly room temp is recorded on a totally separate standalone Elitech RC-4 monitor, which isn't integrated into HA. In due course I plan to add a separate HA room temp sensor, but I need to get the hardware first.

The remaining variables of interest are: set and actual LWT, actual RWT and flow rate. All are visible in the wired controller and/or app, and so should be in the data somewhere. The reason for getting the set LWT is that further down the road I want to try to add load compensation to the weather comp curve (eg if desired room temp - actual room temp > 5 then new set LWT = current set LWT + 3, or if the set LWT is read only, then tweak the weather curve itself). To be useful for control purposes, these probably need to be got fairly frequently, almost certainly more than once an hour.          

Posted by: @cathoderay

↑

Posted by: @derek-m

↑

It is not so much that I spotted the outdoor temperature, it was requested by adding it to the list. I will now be interested to see how the system deals with temperatures below zero, the '0c' should probably change to '8c', which I think would indicate -12C.

I doubt we will get -12C in the near future! In fact, I think we are unlikely to get much below zero during this week, if that. I will watch the local Met Office weather reports and if/when a below zero reading happens, compare with the likely outdoor temp value from the message. 

I think you did spot the outdoor temperature (if it is, we have yet to confirm it, I will add some code to pull that byte). Maybe this is a bit semantic, but the request doesn't create it, it is already there, but it needs to be revealed by asking for the right data from the message, so there is an element of spotting it, in the sense that you need to spot the right byte(s) to request. Or perhaps I have completely misunderstood!

The Midea setup, at least mine, doesn't use indoor temperature, because it uses weather compensation. Neither the wired controller nor the app report indoor temp. The wired controller does have a line for Ta ROOM TEMP on page 3 of the operational parameter pages but the value shown is '--'. At the moment, hourly room temp is recorded on a totally separate standalone Elitech RC-4 monitor, which isn't integrated into HA. In due course I plan to add a separate HA room temp sensor, but I need to get the hardware first.

The remaining variables of interest are: set and actual LWT, actual RWT and flow rate. All are visible in the wired controller and/or app, and so should be in the data somewhere. The reason for getting the set LWT is that further down the road I want to try to add load compensation to the weather comp curve (eg if desired room temp - actual room temp > 5 then new set LWT = current set LWT + 3, or if the set LWT is read only, then tweak the weather curve itself). To be useful for control purposes, these probably need to be got fairly frequently, almost certainly more than once an hour.          

 

I think that you did misunderstand. The python code 'builds' the message, by copying data from the data stream as specified by the programmed list within the Python code. If the required data is not in the list, then it does not go into the message. The Python code also specifies the location within the message where the individual data items will be placed.

I am assuming that the selection process is by the variable name, for some of which, I am having to make an educated guess.

I am assuming that you did not try changing the 'outdoor temperature' to 'indoor temperature?

My reference to a reading of -12C was as an example, since I am not certain how the Midea controller will store negative values. I seem to remember from the distant past the use of 2's Compliment, which involves changing all the 0's to 1's and vice versa, then subtracting 255 (ff) from the new value.

So hex 0c = 00001100 = +12 which in 2's compliment becomes 11110011 = f3 = +243. Now subtracting 255 from 243 produces -12.

I have included both LWT and RWT in the modified code in the files below. I had already added indoor temperature so please give it a try, if it does not provide data then I will see about changing it to flow rate.

Please post the results once you get some.

Posted by: @derek-m

↑

I think that you did misunderstand. The python code 'builds' the message, by copying data from the data stream as specified by the programmed list within the Python code. If the required data is not in the list, then it does not go into the message. The Python code also specifies the location within the message where the individual data items will be placed.

Aha, I think I've got it. Let me put it like this. I thought the message was the data stream, but in fact the message is built by plucking elements from the data stream, meaning the message is a separate entity. The contents of the message are determined by what gets plucked from the data stream, and because the code builds the message, it also determines where in the message each bit of data goes.

I haven't yet changed outdoor to indoor because I didn't think it was available, if it is, it will the the airing cupboard temp, which is where the wired controller is, and so won't be room temp, but it could be useful as a testing variable for developing the code, so will add it (see below).

What I have done is extracted the outdoor temp candidate (at position 9, but it is actually in position 11 counting from the left - there is +2 offset, but I can't see where that comes from?) on the hour values from last night, done a hex2dec conversion, and plotted them against a local Met Office record of hourly temps:

Unfortunately, the outdoor temp didn't vary by very much last night, but with that limitation, the Midea temp does seem generally to mirror the Met Office temp. I will redo the plot when there is more data with more variability.

For some unknown reason, HA isn't logging the position 9 data in the database, despite having code that should do that, it's only available in the message strings in the logs, making extracting the data is a painfully slow manual process. I will work on trying to get it written into the database so it is easier to work with.

Thanks for the new version of the code, will add it (including indoor temp) and post results when I have some, probably this evening as we want a reasonable number of values to look at. 

@derek-m - last night's hourly 'byte' values after hex2dec conversion and my assessment (colour bars) of what is what:

The current C3\message.py code:

        self.byte_09 = (body[data_offset + 9])

        if body[9] == 0xFF:
            self.indoor_temperature = None
        else:
            self.indoor_temperature = body[data_offset + 9] / 2

        if body[10] == 0xFF:
            self.outdoor_temperature = None
        else:
            self.outdoor_temperature = body[data_offset + 10]

        self.entering_water_temperature  = body[data_offset + 11]
        self.leaving_water_temperature  = body[data_offset + 12]

At the top, I am just pulling byte_09 so I can log it in the database, so I don't have to manually transcribe it from the logs. So far, it continues to show a credible outdoor temp (as in the overnight series).

Here are a couple of early message responses:

2023-02-21 11:28:20.502 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.devices.c3.device] [device_id] Received: {'header': 'aab9c300000000000004', 'body': '04010000271f000068e70a281e32302cff01000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f6000000000000000000000000000000000014', 'message type': '04', 'body type': '04'}
2023-02-21 11:28:20.503 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.core.device] [device_id] Status update: {'status_heating': True, 'status_dhw': False, 'status_tbh': False, 'status_ibh': False, 'total_energy_consumption': 10015, 'total_produced_energy': 26855, 'byte_09': 10, 'indoor_temperature': 5.0, 'outdoor_temperature': 40, 'entering_water_temperature': 30, 'leaving_water_temperature': 50}

2023-02-21 12:00:35.031 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.devices.c3.device] [device_id] Received: {'header': 'aab9c300000000000004', 'body': '040100002720000068ea08281e32302cff01000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f4000000000000000000000000000000000014', 'message type': '04', 'body type': '04'}
2023-02-21 12:00:35.032 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.core.device] [device_id] Status update: {'status_heating': True, 'status_dhw': False, 'status_tbh': False, 'status_ibh': False, 'total_energy_consumption': 10016, 'total_produced_energy': 26858, 'byte_09': 8, 'indoor_temperature': 4.0, 'outdoor_temperature': 40, 'entering_water_temperature': 30, 'leaving_water_temperature': 50}

There does not appear to be any cross interference between my code and yours: my byte_09 gets the outdoor temp, your indoor_temperature gets the same data and divides it by 2, so we get 10 and 5 then 8 and 4. The last three variables +10, +11 and +12 are static, at 40 30 and 50, and have been since last evening, see the spreadsheet snapshot. In that snapshot, I think we can see the set LWT, at position 16 (location 14) - compare that column's values with the set LWT showing in the app. Note the app only has hourly data for the current day, once the day is over, the app loses the data for ever, so I can't get yesterday evenings hourly data.

PS: the app ambient temp (which may or may not be an average for the hour) shows up with two decimal places, perhaps because it is an average (perhaps 60 minute reading over an hour divided by 60) - will need to see more data to confirm or refute this, the message data is presumably a spot reading at the time of the timestamp. 

@cathoderay

The task now is to try adding code to retrieve data, and then ascertain to what variable the data relates, and how it should be decoded.

Collecting data in the form of messages, and then decoding each message, is I feel, the most productive way forward. To speed up the process I would suggest that code is added to gether data using the type 01 message, which I believe you said is being retrieved at a frequency of 1 minute intervals. This message gathering may be being initiated by the HA logging routine.

I am not 100% certain, that my assumption that data is being collected by variable name, is in fact correct, which is another point that requires clarification.

The addition of the

self.byte_09 = (body[data_offset + 9])

code appears to have offset the data decoding by 1 byte, and also the indoor and outdoor readings are in reversed order. There may also be some doubt about whether the 'entering' and 'leaving' values are actually the correct ones. At least it is not quite as bad as deciphering the Enigma code. As a test, try moving the self.byte_09 code to the end of that section of code after

 self.leaving_water_temperature  = body[data_offset + 12]

I will update the code for the type 1 message generation and send a text copy, so it may pay to wait for that, and then make all the changes at the same time.

I believe the first byte in the message indicates the message type, and then the data position selection starts at the following byte, which is designated byte (0), which is why there appears to be an offset of 2.

@cathoderay

Try modifying the attached file to see if outdoor temperature can be added to the type 01 message.

Posted by: @derek-m

↑

At least it is not quite as bad as deciphering the Enigma code.

Only yesterday, I was thinking that anyone who manages to decipher the Midea code deserves a Turing award.

Posted by: @derek-m

↑

also the indoor and outdoor readings are in reversed order

They are in the same order as in the code you provided (indoor then outdoor), I did a simple copy and paste to transfer the code. There were a few minor typos, eg wrong indents, :: instead of : but soon corrected.

I'm not sure my byte_09 addition does offset the data decoding. In the code we have:

        self.byte_09 = (body[data_offset + 9])
         
        if body[9] == 0xFF:
            self.indoor_temperature = None
        else:
            self.indoor_temperature = body[data_offset + 9] / 2

and the message status update comes back as

 'byte_09': 10, 'indoor_temperature': 5.0, 

 'byte_09': 8, 'indoor_temperature': 4.0, 

etc - my byte_09 gets the value, your indoor_temperature gets the same value, and divides it by 2.

Posted by: @derek-m

↑

There may also be some doubt about whether the 'entering' and 'leaving' values are actually the correct ones.

I am sure they are not, they have been fixed at 30 and 50 throughout the period the code has been running. It is also a rather high delta t. However, I think the position 16/location 14 byte does have the Set LWT - see the spreadsheet snapshot in my earlier post.

I am also sure the set (not actual) LWT is in position 16 (location 14) - the decimal values there mirror the hourly Set LWT shown in the app exactly (with a one hour offset).

I will wait for your 01 message code and then make the changes to the code.