@derek-m - Home Assistant is a nightmare, from start to finish. It is only the fact that after bludgeoning it into shape, it just might do something useful, that keeps me hammering away at it. For example, here is recent history, since I got it up and running again after the database self destructed: 

We can all probably agree this sort of data is useful. The kWh chart dips on the left hand side are an artefact of how the values are calculated, current total - 24 hours ago total, there wasn't any 24 hours old data at the start of the period, so it used the oldest data available. This being HA, I have no idea how it made up the extreme left hand higher values.   

Posted by: @derek-m

↑

A further problem is that we are not looking at exactly the same software, since your version has the added code from wasilukm. How can I obtain a copy of your version?

Since I don't have any of the software loaded on any of my PC's, I can only pass information in Word files. Is that okay.

I'm afraid that the original code is not written in a particularly structured manner, in that making additions involves updating quite a number of routines, rather than just one or two obvious ones. It is also not particularly well documented.

There are several ways we can move forward.

You can carryout modifications and send a copy for me to check.

I can add highlighted code to my present copy for you to update your version and test.

I think that it is pointless for us both to be identifying the software changes at the same time, since we may be duplicating effort. So how would you like to proceed?

Looks like you have solved your HA database problem, which again should have been better documented.

To answer your other questions:

The wasilukum version of the code is here, in the custom_components/midea_ac_lan folder. There is also that useful page that compares his code with the original version here.

Yes, word files are fine.

I absolutely agree the code is all over the place, and poorly documented, this is standard HA practice. The 'official documentation' is even worse, and the forum is stuffed to the brim with ordinary people expressing extreme frustration at HA's inability to make even the simplest thing simple. I've just spent far too long discovering how HA keeps its stats. There's a crap bloatware InfluxDB addon (half a GB, FFS) that is supposed to make long term stats easy, but it is impossible to configure it, the documentation is either wrong or non-existent. I think the problem is there are a lot of HA 'enthusiasts' who put these hopeless things together and then promote them, when in fact they are redundant, because HA does what you want, but you don't know that, because it doesn't tell you that. I'll go into the details of all this in due course in the Beginner's Guide to Heat Pump Monitoring thread. 

I suggest the best next step is for you to see what you can come up with, a fresh pair of eyes looking at the problem, if that is OK with you. I have managed to add the code to get the total_mystery values (see in the docx file), but beyond that I am rather stumped as to what else to try.   

@cathoderay

No problem, I will go through the code as time allows, and send proposed changes that you can try out.

What additional parameters are you trying to capture? I suspect indoor and outside temperature, but is there anything else?

Posted by: @derek-m

↑

No problem, I will go through the code as time allows, and send proposed changes that you can try out.

What additional parameters are you trying to capture? I suspect indoor and outside temperature, but is there anything else?

Great, thanks. As time allows is of course fine, it is your time after all, and I very much appreciate your effort in trying to get this working.

Additional parameters: outdoor (ambient in Midea-speak) temp (but not indoor, it is not available in the wired controller), set and actual LWT, RWT and flow rate, compressor current and supply voltage. These are all (apart from indoor temp) visible in the wired controller, and some appear in the Midea app, so I'm guessing they are indeed in the data somewhere, especially those that appear in the app, otherwise how did they get there? The idea is to be able to do my own calcs, eg allow for antifreeze in the primary circuit for energy out, rather than rely on Midea's black box calculations (which I can compare with directly and transparently calculated values).

@cathoderay

I forgot to ask. Which of the data that you are obtaining at the moment is on an hourly bases, so possibly monitoring the hourly transmission to the Midea Cloud, and which can you request as required, so probably involves a request to, and a reply from, the Midea controller?

@derek-m - based on the entries in the HA logs, the total energy consumed and produced data appears every hour around about on the hour (the 04/04 messages), but there are also unexplained occasional during the hour entries, the rest of the data appears every minute (the 03/01 messages).

I don't personally request anything, any requests are somewhere in the code, if there are requests. The 03/01 received messages are always immediately preceded by a 'Sending' log entry, but the 04/04 received messages are not. I am sure this observation is significant, but can't see how just yet! At the same time, I suppose it is possible there might be a 04/04 request sent, but it doesn't get logged. The log only records what it is set to record.  

Posted by: @cathoderay

↑

@derek-m - based on the entries in the HA logs, the total energy consumed and produced data appears every hour around about on the hour (the 04/04 messages), but there are also unexplained occasional during the hour entries, the rest of the data appears every minute (the 03/01 messages).

I don't personally request anything, any requests are somewhere in the code, if there are requests. The 03/01 received messages are always immediately preceded by a 'Sending' log entry, but the 04/04 received messages are not. I am sure this observation is significant, but can't see how just yet! At the same time, I suppose it is possible there might be a 04/04 request sent, but it doesn't get logged. The log only records what it is set to record.  

Hi CathodeRay,

I suspect that the hourly transmission will be initiated by the Midea equipment, while the 1 minute request is probably initiated by HA, since I have not found any form of timing being carried out by the Midea_ac_lan software.

I have had a good look through the software routines and the additional code from wasilukm, and decided that rather than try to re-invent the wheel, I would initially follow the example provided by wasilukm.

I have made a number of assumptions, which you can confirm or correct.

I assume that you have added the additional code from wasilukm, since you are now obtaining power input and power output measurements.

The only routines that require modification are the ones within the wasilukm code.

Midea employs a standard set of names throughout its different devices to identify measurement variables. i.e. The name 'outdoor temperature' is used both in AC units and heat pumps.

You have stopped HA from wiping the database periodically.

I therefore added the code produced by wasilukm, to the relevant routines in the Midea_ac_lan software. I then added additional code to obtain the outdoor temperature measurement. This additional code has been added to the files below, with the text being light blue in colour. Check all the way through each text file since there may be more than one code addition.

I would suggest that you take a backup of the present software, then modify the operational copy to include the suggested additions. Allow the software to run and see if it provides the desired result. If it is successful it should then be possible to add the code to obtain further data. If it fails then please send any data for analysis.

@derek-m - thanks. Interestingly, I came up with a very similar set of modifications for my mystery/testvars. The main difference is the data handling in ..\C3\Messages.py, I just got the value eg self.mysteryvar1 = body[data_offset + 9] (and got the absurdly large integers) whereas you have decimal place processing.

Posted by: @derek-m

↑

I have made a number of assumptions, which you can confirm or correct.

See below:

I assume that you have added the additional code from wasilukm, since you are now obtaining power input and power output measurements.

Yes, it's been in there for a while.

The only routines that require modification are the ones within the wasilukm code.

I agree: we just extend waslikum's modifications. 

Midea employs a standard set of names throughout its different devices to identify measurement variables. i.e. The name 'outdoor temperature' is used both in AC units and heat pumps.

I had also noticed this, it gives us likely names for common variables.

You have stopped HA from wiping the database periodically.

I have read it the Riot Act but am not sure whether it was listening. The other interesting thing about the timing of the self destruction is that probably happened around 10 days after startup - the default auto-purge interval. Maybe it overdid the emetic. Meanwhile I have extended the auto purge interval to 45 days...   

I will add your code and report back!

Edit: the italics got extended to all the quote, have re-jigged it to hopefully make my replies clearer.

Hi, I can share some observations from the Modbus area, which I personally think have impact on Midea calculations.

First: found hidden value in registry 144-145, which seems to me a 'heat production' over time in kWh. It is barerly described in manuals, one says 'Heating capacity of the system', other just 'power output', no unit provided but what I noticed it is cummulative growing and corrensponding to power consumption vaule:

heating tank, consumption: 3kWh, production (??): 9kWh

interesting why is not shown on display... 

Second: current (Ampere) values are... integers(!!!)

while for voltage it's more or less ok (error is like 0.5%) but for current error may be even 50% (is it really 1 or 2A?).

If these values are used for power consumption/production calculation in the Midea software then you can forget about precission.

btw, I have rebuild the circuit with smaller ESP8266 which fit nicely in the display unit, using CN6 as 3.3V power:

@marekk - all very interesting. I wonder, are the hidden 144-145 addresses are a logical extension of the 8.1.2 modbus registers table in the 'Operational Manual', which ends at 140? If so, there may be other hidden above 140 eg 141 etc and maybe even above 145.

If you look at the Operational Parameter pages in the wired controller, the terminology is very similar to but not identical to what you have found, eg page 3 has HEAT PUMP CAPACITY which is (alleged) present output and POWER CONSUMPTION which is alleged lifetime energy use. It seems there may be more in the modbus data than appears in the wired controller, which makes sense, as the app for example has lifetime output (which isn't visible in the wired controller).

The amps and volts are also intriguing. They do appear in the wired controller (OP pages 6 and 7) but not in the app. I agree the integer values for amps are rather implausible, but it is the same in the wired controller, mine is currently showing 5A. I've noticed in the past it can be quite volatile, and maybe, just maybe, if consumption is taken over time, some of the errors just might cancel out. It depends how the rounding is done, eg normal (nearest integer) or ceiling (next highest integer) etc. I suppose it may also be the case the internal calcs use decimal places, which are dropped from the display.

You have done a great job of fitting the ESP8266 in the wired controller, and have been very brave soldering in the connections. Every time I look at my soldering iron and ask myself 'what could possibly go wrong' the answer isn't encouraging!    

@derek-m - I added your code shortly before 1900 hours and I now see an 04/04 message has been received, as usual almost exactly at the top of the hour (19:00:33.665). The good news is it does contain the outdoor_temperature label, but the value is given as 'None' ie the default. Here's the complete Status update message, it also shows my total_mystery vars which I have left in for now:

Status update: {'status_heating': False, 'status_dhw': False, 'status_tbh': False, 'status_ibh': False, 'total_energy_consumption': 9917, 'total_produced_energy': 26507, 'total_mystery1': 55837204018, 'total_mystery2': 206160985857, 'outdoor_temperature': None}

The current total_mystery vars use boolean operators (like the total energy figures) but I also tried just retrieving the value without doing any maths, still no obvious label for the value returned.

@cathoderay

Can you post the 'message body' so that I can try to analyse the data. It may be that HA does not know how to interpret the data.

@cathoderay

I have just realised that I did not add your testvar's to the code since I don't know what modifications you have carried out on your system, so there will be differences between your code and the copy that I have. Please provide a copy so that we are both singing from the same song sheet.

One thing that you could try is to use the # sign to make your testvar code into a comment, so that it does not run the code. Place the # at the start of each line of code that is not to be run.

Posted by: @cathoderay

↑

@marekk - all very interesting. I wonder, are the hidden 144-145 addresses are a logical extension of the 8.1.2 modbus registers table in the 'Operational Manual', which ends at 140? If so, there may be other hidden above 140 eg 141 etc and maybe even above 145.

well, the registry table ends at number 147, not much left there:

• there are few readings/settings more: 
  states of units in cascade, range 1000 -> 1042
• Password-protected unit parameters (these available in service menu), range 200 -> 272
   

all taken from the Clivet manual, start at page 143

Seems the values from registries 143-144 and 145-146 are coresponding to your status message values: "total_energy_consumption" and "total_produced_energy"

I suspect that "total_mistery" may represent combined bit values like in modbus registries 0,5,128 or 129:

but seems not easy to decode