@derek-m and @marekk - the dreaded blasted code police are out in force this morning, can't post a post I've just written. Here it is in text form. Sorry for the nuisance.

We've got something!

Message and status update at 0900:

2023-02-19 09:00:33.243 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.devices.c3.device] [device_id] Received: {'header': 'aab9c300000000000004', 'body': '0400000026d2000067d808281e32302cff01000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000fa000000000000000000000000000000000014', 'message type': '04', 'body type': '04'}
2023-02-19 09:00:33.244 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.core.device] [device_id] Status update: {'status_heating': False, 'status_dhw': False, 'status_tbh': False, 'status_ibh': False, 'total_energy_consumption': 9938, 'total_produced_energy': 26584, 'total_mystery1': 34362367538, 'total_mystery2': 206161379073, 'outdoor_temperature': -1.0}

Current outdoor temp on the wired controller is 8 degrees, and there is no parameter in the operational parameter pages at or near -1.0, but at least it is a number that isn't absurd.

I'm trying to think of a way of running a loop in the python code to loop through the various bytes in the message and apply various calculations and log the results to a text file, partly because making one off changes one at a time is tedious and boring, and partly because the 04/04 messages only happen once an hour, making things very slow if only one thing is changed at a time.  

Posted by: @cathoderay

↑

We've got something!

Message and status update at 0900:

2023-02-19 09:00:33.243 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.devices.c3.device] [device_id] Received: {'header': 'aab9c300000000000004', 'body': '0400000026d2000067d808281e32302cff01000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000fa000000000000000000000000000000000014', 'message type': '04', 'body type': '04'}
2023-02-19 09:00:33.244 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.core.device] [device_id] Status update: {'status_heating': False, 'status_dhw': False, 'status_tbh': False, 'status_ibh': False, 'total_energy_consumption': 9938, 'total_produced_energy': 26584, 'total_mystery1': 34362367538, 'total_mystery2': 206161379073, 'outdoor_temperature': -1.0}

Current outdoor temp on the wired controller is 8 degrees, and there is no parameter in the operational parameter pages at or near -1.0, but at least it is a number that isn't absurd.

I'm trying to think of a way of running a loop in the python code to loop through the various bytes in the message and apply various calculations and log the results to a text file, partly because making one off changes one at a time is tedious and boring, and partly because the 04/04 messages only happen once an hour, making things very slow if only one thing is changed at a time.  

 

From what I can see in the midea_ac_lan code, the order of the message body is set by the request routine, so the decipher routine needs to use the same structure. i.e. Is this particular byte containing boolean information, a one byte integer, part of a two byte integer, part of a four byte integer, or a floating point number?

When I modified the 'midea_ac_lan' code I forgot that you had added the test variables and was also unaware of what code had been added and where, so I did not allow for this in the modified version. The thing that I was not fully certain about in the modified code was the structure of the 'outdoor temperature' reading, in the number of bytes being used, and their arrangement. I therefore used the same format as used by the AC code, which places the integer part of the outdoor temperature reading  at message body position [14], and the decimal part at position [18]. In the modified version this would equate to positions [9] and [13] in the code that I supplied, but of course the modified code does not contain your test variables, which may also be occupying the same positions within the message body.

I would therefore suggest that you remove your test variable code and see what difference that may make in the data received, and how it is interpreted.

Posted by: @cathoderay

↑

We've got something!

Message and status update at 0900:

2023-02-19 09:00:33.243 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.devices.c3.device] [device_id] Received: {'header': 'aab9c300000000000004', 'body': '0400000026d2000067d808281e32302cff01000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000fa000000000000000000000000000000000014', 'message type': '04', 'body type': '04'}
2023-02-19 09:00:33.244 DEBUG (Heat Pump Wi-Fi Controller) [custom_components.midea_ac_lan.midea.core.device] [device_id] Status update: {'status_heating': False, 'status_dhw': False, 'status_tbh': False, 'status_ibh': False, 'total_energy_consumption': 9938, 'total_produced_energy': 26584, 'total_mystery1': 34362367538, 'total_mystery2': 206161379073, 'outdoor_temperature': -1.0}

Current outdoor temp on the wired controller is 8 degrees, and there is no parameter in the operational parameter pages at or near -1.0, but at least it is a number that isn't absurd.

I'm trying to think of a way of running a loop in the python code to loop through the various bytes in the message and apply various calculations and log the results to a text file, partly because making one off changes one at a time is tedious and boring, and partly because the 04/04 messages only happen once an hour, making things very slow if only one thing is changed at a time.  

 

I have now had a chance to look at the message body in a little more detail.

So '04' is the message type

'00' is binary data in position '0' zero

'000026d2' is total energy consumption in positions 1 to 4.

'000067d8' is total produced energy in positions 5 to 8.

'08' could be the outdoor temperature integer portion in position 9, which would give a value of 8.

'30' could be the outdoor temperature decimal portion in position 13, which would give a value of 0.48.

These are assumptions that may or may not be correct.

I would suggest that you remove your test code and obtain a sequence of values as the outdoor temperature varies, to see if any of my assumptions are correct or not. It may also be necessary to look closer at the maths being performed, since the ' -50 / 2' doesn't appear to make a great deal of sense to me.

@derek-m - been doing things outside today as the weather is better, now it's dark it's time to look at the heat pump code again.

I still think we can't see a request in the code for the 04/04 message (the one we're currently working on), or have you spotted one, and it just doesn't get logged? There is a request routine for the other 03/01 message, and it shows up in the log, but the lack of a request for the 04/04 message means we can't use the request routine to decipher the response.

I can certainly comment out my test vars but I don't think they should make any difference, unless data can only be pulled from the message string once, which seems unlikely. As far as I can see, there is nothing, for example, to stop the code doing x on byte 8, and then the next bit of code doing y on byte 8, because the message string remains unchanged, it is just read.

I've got part of the way towards adding code that will do multiple manipulations on various bytes in one pass and log the results to a text file. Making changes on only a small number of items at hourly intervals is a very slow way of getting things done, so I see getting this code working as a worthwhile exercise. I can then run multiple variations of your code in one go, and have all the results easily available in a dedicated text file.

The current HA log shows 'outdoor_temperature' has stayed at '-1.0' all day. Whatever it is, is not very volatile.        

Posted by: @cathoderay

↑

@derek-m - been doing things outside today as the weather is better, now it's dark it's time to look at the heat pump code again.

I still think we can't see a request in the code for the 04/04 message (the one we're currently working on), or have you spotted one, and it just doesn't get logged? There is a request routine for the other 03/01 message, and it shows up in the log, but the lack of a request for the 04/04 message means we can't use the request routine to decipher the response.

I can certainly comment out my test vars but I don't think they should make any difference, unless data can only be pulled from the message string once, which seems unlikely. As far as I can see, there is nothing, for example, to stop the code doing x on byte 8, and then the next bit of code doing y on byte 8, because the message string remains unchanged, it is just read.

I've got part of the way towards adding code that will do multiple manipulations on various bytes in one pass and log the results to a text file. Making changes on only a small number of items at hourly intervals is a very slow way of getting things done, so I see getting this code working as a worthwhile exercise. I can then run multiple variations of your code in one go, and have all the results easily available in a dedicated text file.

The current HA log shows 'outdoor_temperature' has stayed at '-1.0' all day. Whatever it is, is not very volatile.        

I'm afraid I follow the doctrine 'more haste, less speed'. The only way to decipher how the data is stored and transmitted is by adopting a logical and systematic approach, which in the early period may require making one change and analysing the result.

Please let me know if you wish me to continue?

Posted by: @derek-m

↑

I'm afraid I follow the doctrine 'more haste, less speed'. The only way to decipher how the data is stored and transmitted is by adopting a logical and systematic approach, which in the early period may require making one change and analysing the result.

Please let me know if you wish me to continue?

The approach I am suggesting is 100% logical and systematic, just more efficient. Instead of doing x, waiting an hour, then doing y, waiting an hour, then doing z, waiting an hour etc, I am hoping to do x y and z in the same hour, all fully documented. We are not trying to alter anything in the heating system, we are just trying to collect data, so we don't need to wait to see results in the system, just what data we get back. Or think of it as being like taking three photos of a mystery object all at once, and then analysing them, rather than taking one photo and analysing that one, then returning to take a second photo and analysing that one, and then returning to take a third, and analysing that one. The photos and analysis are identical we just got them in less time (ie more hastily...) and you never know, having three photos of the same object at the same time might itself reveal something.

I see another commenter has appeared on github asking the original developers of the code to add other variables.   

Posted by: @derek-m

↑

since the ' -50 / 2' doesn't appear to make a great deal of sense to me.

The full line is 

temp_integer = int((body[14] - 50) / 2)

and because of the bracketing I read that as body[14] - 50 then divide result by 2, but what is odd is the placement of 'int'. Body[14] is almost certainly a string (even if by chance it has only digits), and normally python would choke on doing maths on a string - ie this would make more sense

temp_integer = (int(body[14]) - 50) / 2

but it doesn't choke. Maybe the int wrapper makes everything inside an integer, or maybe it is there to chop off any decimals, if body[14] - 50 evaluates to an odd number.

Posted by: @derek-m

↑

I therefore used the same format as used by the AC code

But I think yours is not quite the same: top half (commented out at the moment) is your code, bottom half is the AC code copied and pasted as is:

        # if body[9] != 0xFF:
            # self.outdoor_temperature = None
        # else:
            # temp_integer = int((body[9] - 50) / 2)
            # temp_decimal = ((body[13] & 0xF) * 0.1) if len(body) > 10 else 0
            # if body[9] > 49:
                # self.outdoor_temperature = temp_integer + temp_decimal
            # else:
                # self.outdoor_temperature = temp_integer - temp_decimal

        if body[14] == 0xFF:
            self.outdoor_temperature = None
        else:
            temp_integer = int((body[14] - 50) / 2)
            temp_decimal = (((body[18] & 0xF0) >> 4) * 0.1) if len(body) > 20 else 0
            if body[14] > 49:
                self.outdoor_temperature = temp_integer + temp_decimal
            else:
                self.outdoor_temperature = temp_integer - temp_decimal

The opening line changes != to ==, and the temp_decimal line is different in a number of ways. I know why you think it may be body[9 and 13] rather than body[14 and 18] but the maths/numbers are also different.  

Your code returned 'None' (the default, from earlier in the code, or alternatively body[9] is indeed not 0xFF (not that unlikely), in which case this code will return 'None'), the literal copy of the AC code returned '-1.0'  

Posted by: @cathoderay

↑

Posted by: @derek-m

↑

I'm afraid I follow the doctrine 'more haste, less speed'. The only way to decipher how the data is stored and transmitted is by adopting a logical and systematic approach, which in the early period may require making one change and analysing the result.

Please let me know if you wish me to continue?

The approach I am suggesting is 100% logical and systematic, just more efficient. Instead of doing x, waiting an hour, then doing y, waiting an hour, then doing z, waiting an hour etc, I am hoping to do x y and z in the same hour, all fully documented. We are not trying to alter anything in the heating system, we are just trying to collect data, so we don't need to wait to see results in the system, just what data we get back. Or think of it as being like taking three photos of a mystery object all at once, and then analysing them, rather than taking one photo and analysing that one, then returning to take a second photo and analysing that one, and then returning to take a third, and analysing that one. The photos and analysis are identical we just got them in less time (ie more hastily...) and you never know, having three photos of the same object at the same time might itself reveal something.

I see another commenter has appeared on github asking the original developers of the code to add other variables.   

I'm afraid that it does not matter how many photo's you take, they are of little use if it is the wrong object.

@derek-m - I don't know why you are so against a more efficient but still rigorous approach to data collection. There's no way we are photographing the wrong object - the object we are interested in is the 04/04 message body and that is what we are taking snapshots of (but that doesn't mean there may not be other object we have yet to find, but for now, we are only looking at that 04/04 message). There's also the blind men and the elephant dimension, there may be patterns to be seen in wider data that simply are not visible in single chunks of data. 

Have you found the 04/04 message request? I still can't see it anywhere in the code. The other odd thing is the pattern over time of received 04/04 messages. There is invariably one just after the hour, but then there are also occasional apparently random ones, even groups sometimes, during the hour. Are there any clues as to what triggers those apparently random ones? I ask, because like you I think if we can find what initiates the request, we may be a lot closer to knowing what is in those requests.  

Posted by: @cathoderay

↑

@derek-m - I don't know why you are so against a more efficient but still rigorous approach to data collection. There's no way we are photographing the wrong object - the object we are interested in is the 04/04 message body and that is what we are taking snapshots of (but that doesn't mean there may not be other object we have yet to find, but for now, we are only looking at that 04/04 message). There's also the blind men and the elephant dimension, there may be patterns to be seen in wider data that simply are not visible in single chunks of data. 

Have you found the 04/04 message request? I still can't see it anywhere in the code. The other odd thing is the pattern over time of received 04/04 messages. There is invariably one just after the hour, but then there are also occasional apparently random ones, even groups sometimes, during the hour. Are there any clues as to what triggers those apparently random ones? I ask, because like you I think if we can find what initiates the request, we may be a lot closer to knowing what is in those requests.  

I am all in favour of using an efficient and rigorous approach to problem solving, in fact that is what I was paid to do for over 50 years. Your total mystery 1 & 2, as far as I am aware, are failing to provide any useful data, and may in fact be muddying the waters, since total mystery 1 occupies the same message location as my proposed outdoor temperature measurement. If you are adding code you should at least keep me informed as to where and why, otherwise it is quite pointless for me to spend time and effort looking through the code and suggesting modifications, which may then conflict with modifications that you have made.

I would suggest that you remove the total mystery code and then post the next couple of messages.

Please let me know if you still require my assistance?

Posted by: @derek-m

↑

If you are adding code you should at least keep me informed as to where and why

I have, it is in yesterday's message1.txt file which I had to use as the WordPress code police wouldn't let me post the code verbatim in a post. The link is at the top of this page.

I have commented out the total mystery parameters (you are right, no useful info in current form) but I still can't see how they muddy the waters vis a vis your code because all the code we have both produced does is read from the message, and then optionally do some processing. In effect, and very simplified, all we are doing is this (dm vars are your code, cr vars are my code):

dm_var9_hex = body[10]

cr_var9_dec = int(body[10], 16)

which will give us (assuming body is an array because of the [] notation, and the values in the array are hex)

the 9th item (assuming zero based counting) in the body array still in hex in dm_var9_hex

and 

the 9th item (assuming zero based counting) in the body array in decimal in cr_var9_dec

We can then inspect the results. They remain independent, neither knows the other exists, all we have done is automated the hex to dec conversion, or whatever other test we want to try. Or we could shift the bytes being examined along by one each time (9 and 13, 10 and 14, 11 and 15 etc) in each iteration of the code. Or whatever. It is just a more efficient way of trying different things.

The other reason I want this to be as efficient as possible is that modifying the code isn't a trivial undertaking. It takes ages because it only takes effect once an hour, so ten changes one at a time takes ten hours, and it also means stopping and restarting HA (to load the changes). If I can do do ten new and independent reads of the message in one go, it only takes one hour, and one restart.