@cathoderay

In the early days of computers they had limited memory, which itself could be quite expensive. Not only did the actual programmed code have to be written efficiently, but memory storage had to be used in the most efficient manner.

Large industrial control systems can have thousands of inputs and outputs, both analogue and digital, so to use 1 byte to store one digital input did not make sense, when 1 byte could be used to store eight digital inputs or outputs.

I think that I have figured out why the transmitted values for roomTempSet: parseInt(r[8]) is hex 30, which gives dec 48, rather than the actual value of 24C. The same applies to roomTempSetMax: parseInt(r[17]) roomTempSetMin: parseInt(r[18]).

The Python code shows it is necessary to perform a divide by 2 to obtain the true value, this, I believe, is because the value is not being stored as an integer, but in fixed point form, so hence has a resolution of 0.5C.

Consider the following:-

Binary

 128     64     32     16     8     4     2     1

   0        0       0       1     1     0     0     0

The above is the binary presentation of 24, which is of course an integer, but how to quantify and store 24.5 in binary format? The answer is in fixed point form as detailed below.

Binary

  64     32     16     8     4     2     1     0.5

   0       0       1      1     0     0     0      1

If the value 24 is stored in memory in the same format, that would give 00110000 in binary, 30 in hexadecimal but 24 in decimal. Because it is being assumed that the information in the data stream is integers, when converted from hexadecimal to decimal the displayed value becomes 48 rather than 24. Dividing the displayed value by 2 produces the true value of 24.

I would suggest that you try changing the room temperature setpoint to 24.5C and see if the displayed value in the data stream changes from hex 30 to hex 31.

@derek-m - intriguing. Unfortunately I can't change the room set temp without coming off weather comp and I am not keen to do that after this morning's fun and games, at least not while it is so cold, and the system is struggling. I had a very anxious hour or so this morning before I got things apparently working again. On weather comp, the wired controller claims to show the set LWT, but it doesn't, it just shows a fixed random two digit number (currently it 40, but the current set LWT will be significantly above that, as it is zero degrees outside at the moment, and the cold outdoors end of my weather curve is 55@-4. According to the app, the set LWT is currently 47 degrees).  

I'm currently experimenting with the other 'message' (the body-type 0x04 one which has the energy consumed/produced numbers) to see what else might be in it. I have found a way of getting 'testvar1' etc to appear in the Status update in the debug log, but so far, none of it makes sense, and the message only gets sent once an hour (the other one gets sent every minute) so it is very tedious and boring... I've tried to find out where the send interval is set, but so far without success. To make it even more tedious and boring, I have to reload the whole freaking Home Assistant thing every time I make a change in the code to enable the change. Perhaps I need to get out more often...but then again, to have live real time data for the heat pumps vital signs would be extremely useful. Here's the DHW tank temp for this morning showing where it went haywire, turning on the DHW heating at ~0900, when it is supposed to be off on the timer. I think the ~15m dip in the middle of the rise is a defrost cycle, crazy really, sucking all that heat out of the DHW only to have to put it back in again. The big dip late morning is me having a bath deliberately just before 1300, when the timer should, and did, heat the hot water. The curve, by the way, isn't perfect, as it smooths the trace, which can be misleading, in much the same way a GPS trace on a longish interval can show a boat sailing across dry land.     

At last, I may have made some progress! Yikes! Here is a Home Assistant dashboard (for those not familiar with HA, think dashboard on your car) showing a card (the HA name for these display boxes is cards...) with some key entries:

Crucially, this isn't being done by connecting to Midea MSmartHome, or the Midea cloud, instead it is being done by connecting to the Midea wired controller (in my airing cupboard, in case you hadn't guessed) over wifi using the midea_ac_lan HA custom component (works, despite Midea complexity, but very limited output at the moment) and then doing some maths on the available numbers, on a mini PC (details on this will shortly appear in my Beginners Guide to Heat Pump Monitoring thread). What the card shows:

Heat Pump Wi-Fi Controller Tank Actual Temperature: this is one of the default retrieved values, shows current DHW tank temp, shown as a reality check

Heat Pump Wi-Fi Controller Total energy consumption: also a default retrieved value, shows lifetime energy used

Heat Pump Wi-Fi Controller Total produced energy: also a default retrieved value, shows lifetime energy produced

Now, the energy used and produced are logged to HA's database at hourly intervals plus or minus a few minutes, That means we should be able to calculate the use over the past hour, current reading minus reading an hour ago, but this can produce silly results, because the exact timing of the readings varies, meaning the reading an hour ago might be the same as the current one. We can sort of fix this by using a longer interval, in the above example 3 hours. An error can still occur, but it will be a smaller percentage of the total. I have also set the interval used to 190 minutes, ie three and a bit hours, so I collect anything that is just a bit out. This gives us:

kWh used in last 3 hours

kWh produced in last 3 hours

and as we know those numbers, we also know the average COP for the last hours:

3 Hourly COP

HA calls things that producing readings sensors. The custom ones (the bottom three) are set up in HA's configuration.yaml file (a editable text file that can be used to, err, configure HA), and the code looks like this (my actual device ID redacted):  

sensor:
  - platform: statistics
    name: "kWh used in last 3 hours"
    entity_id: sensor.[myDeviceID]_total_energy_consumption
    state_characteristic: change
    max_age:
      minutes: 190

  - platform: statistics
    name: "kWh produced in last 3 hours"
    entity_id: sensor.[myDeviceID]_total_produced_energy
    state_characteristic: change
    max_age:
      minutes: 190

template:
  - sensor:
      - name: "3 Hourly COP"
        unit_of_measurement: "COP"
        state: >
          {%set used = states('sensor.kwh_used_in_last_3_hours') %}
          {%set produced = states('sensor.kwh_produced_in_last_3_hours') %}
          {{ ((int(produced)/int(used)) | string )[0:4] }}

The first two (sensor - platform statistics x 2) measure the difference (change) between the current value and the one 190 minutes ago to give the kWh  use/production over the last three hours. The third (template - sensor) is the one that calculates the COP from the used/produced values. This particular bit of code was a right royal PITA to get right, I almost gave up. For those who know a bit about coding, the states values are returned as strings, and so have to be converted to integers before doing the division. The result is then rounded to two decimal places. Result: a three hour trailing moving average for COP. Because of the way it is coded, it is flexible as to the time over which COP is calculated, by changes the max-age setting, which sets the interval (it is actually setting the max-age of values in the database that will be used in the calculation). Daily COP? Use max-age: hours: 24 etc.   

The HA Dashboard only shows current values. However, there is a module called History Explorer that can do some useful charts, which I will return to in another post. 

Again, the key thing is all this is being done locally. No need for that pesky Midea MSmart Home app, no need to peer at minuscule numbers on tiny screen, as this is all available in real time and as historical data on my desktop, with all my data stored, and accessible and exportable (HA uses a sqlite database), locally.           

Note: 6/2/23: edited (a) to remove 'unit_of_measurement: "kWh"' lines from platform: statistics entries as they cause configuration errors and (b) last line of template sensor to a more reliable way of getting a COP rounded to two decimal places

@cathoderay I cannot add a clapping gif. So bravo

@batalto - thanks! Later today, I hope to post the charts produced by the History Explorer add in. On the plus side, these are pretty much automated, and are dynamic (variable time frames, scrolling backwards and forwards in time) but on the minus side the Midea data as processed by History Explorer does not exactly match the Midea data as processed (and plotter) by the Midea app. We do however have the answer to why power use is always an integer, at least in History Explorer (and my calcs), the use is calculated as lifetime use current reading minus relevant lifetime use past reading, and as lifetime use is always an integer, the difference is always an integer. This may however only matter over small time frames (hours), because kWhs aren't actually lost, they just end up in the wrong time period, over longer time frames (days) most kWhs will end up in the right time slot.

Here's an image of the History Explorer tab/page, now that the COP monitoring has been running for just over 24 hours:

Most of it sort of makes sense, but there are some oddities:

(a) why did the DHW heating come on at around 2am yesterday? It's on a timer, only one between 1300 and 1400. RHS of chart shows a rise happening when it should, but none yesterday as tank temp wasn't low enough (40 degrees) to trigger a reheat

(b) some of the green and red bars, energy in and out, are odd, either none when there should be some, or unexpectedly high. I think these may be either Midea artefacts and/or History Explorer artefacts, because of the way they do the sums

(c) my calculated 3 hourly COP is definitely rather spiky, but a line smoothed in the mind's eye doesn't look too bad, you can even just about see COP varying with outdoor ambient temp.

What I want to do next is get LWT, RWT, flow rate on the output side and current and volts on the input side (all visible in the wired controller) and do some DIM (do it myself, not dim, hopefully) energy in / energy out calcs. I'll still be using Midea generated data, but will be getting from an earlier stage in the data chain, and so avoiding any tweaking/rounding/massaging they do during their later calcs. 

A bit further down the line I want to add a pulse counter to my dedicated heat pump electricity meter. It should be more accurate, and interestingly, it actually consistently shows lower energy use than Midea's own calcs suggest. Perhaps I am being unfair about Midea after all.   

@cathoderay what does the comparison of cop show?

@batalto - between the 'Midea' COP and my 3 hourly COP? Too early to say yet, but my 3 hourly one might be just a little bit higher, but not a lot. Also, don't forget the 'Midea' one isn't calculated by Midea, it is calculated by me from Midea's energy in/energy out data on the app.

@cathoderay yes that was my question. Just trying to understand the difference between them. As the cop moves up and down in the day I'd be interested in 

1. What's the difference between yours and the Midea one in general
2. How do the powers differ - do they round up?
3. How accurate is the Midea number over a day? E.g. is it rounded at the end? Or every hour and then those hours added up (compounding the error)

@batalto - see also my comments on the Keytone COPs thread, but to answer your very pertinent questions here:

1. For lifetime use, the only thing I can directly compare at the moment, something over 10% for energy used, at least comparing Midea's reported lifetime energy use vs the dedicated heat pump external electricity meter, current readings are Midea 9128kWh, external meter 7994kWh. This difference is not trivial, and it is probably not unreasonable to assume that, for energy used, all time frames, hourly, daily, weekly, have the same percentage error. What we have no way of knowing is whether the energy produced values also have the same degree of error. If they do, then, at least for COP calculations, they probably cancel out. The only way to get an independent measure of energy produced is to set up independent third party monitoring ie temp sensors and flow meters, a silver bullet I have yet to bite because (a) being silver, it is an expensive bullet, and (b) I am still not entirely sure how to feed the data into HA. I'm sure it is possible, others have done it, it is just that at this stage, I don't know the 'how to' details.

2 and 3. Rounding (almost certainly up) happens, that much we know, because the use numbers are always integers, something that can't be happening in real life, but where and when it happens is less clear. I think it may depend in the interval being displayed, ie it rounds to the interval, meaning there may not be compounding: an hourly view is rounded within the hour, but the daily view is rounded withing the day, if that makes sense, is the daily use is not the sum of the hourly use. What I do know is that the HA maths for energy in and out are all done by subtracting one value from another value (usually current lifetime use value minus the lifetime use value at interval x (one hour one day whatever) ago), and as lifetime use is always an integer, the interval use is also always an integer. Two things follow: (a) use never gets lost, but it may end up in the wrong time frame, because of when the readings are taken vis-a-vis the specific time and interval chosen, and (b) longer time frames will give more reliable results, because, although (a) type errors will still occur, they will account for a smaller amount of energy over a longer time frame.    

@cathoderay You are a hero! I also use home assistant and have this same heat pump (I saw you are wondering about HA hardware - I can recommend Home Assistant Yellow - I backed the project a while ago and how it's available). 

Now, how can we create a nice guide for beginners to replicate what you have done? I have gotten as far as getting the OOTB experience with the hot water temperature but got a bit lost. I'm mainly interesting in the power consumption values. If you do plan on writing a guide I'll happily beta test it, help you edit it

@tried2fail - thanks. I take it you are using georgezhao2010/midea_ac_lan which is brilliant in what it achieves, but is sadly lacking in useful heat pump sensor readouts. I then switched to using wasilukm/midea_ac_lan which is a fork, and adds the heat pump energy in/out sensors, again very useful, but, also again, that's as far as it goes. Nonetheless, these are the sensors behind the green and red bar charts in the History Explorer image I posted above. They also mean I can calculate a 3 hourly (or any other interval) COP, the brown/beige line chart at the bottom of the image. The code (goes in configuration.yaml) for that is in a post a few posts above the History Explorer image post. HA templating is neither easy nor intuitive, but it can be done.

I have already started a Beginner's Guide to Heat Pump Monitoring thread, see here, with step by step details and discussion along the way. Your offer to help is welcome and appreciated! I have written but not yet posted Part 3 which describes setting up the core hardware (I went for a mini PC with HA OS on it in the end), not yet posted because it is too long, so I am going to split it in two to make it more digestible.

The roadblock at the moment is how to read other data from the 'message' that contains the energy in/out data. Ideally I want to collect current in and voltage on the input side, so I can independently calculate energy in, and LWT/RWT and flow on the output side, so I can independently calculate energy out. All these variable are visible in the wired controller Operational Parameter pages, it is just a question of finding then in hopefully 'message/body' type 04 or if need be in yet another 'message type', but needless to say how to get other message types is also far from clear...

You can see the message body type 04 details in the HA main log file if you add

logger:
  default: warn
  logs:
    custom_components.midea_ac_lan: debug      

to your HA configuration.yaml file. Apparently the indents have to be correct, always two spaces.

The other big question, discussed above and elsewhere, is how much tinkering Midea does with the data. At the moment (Stage 1), I am doing everything between HS and the wired controller over wifi. Stage 2 if Stage 1 fails (which I suspect it will, at least in a sane time frame) will be doing it over modbus/RS-485 over a wired connection. It should be easier addressing modbus registers than decoding, and it should provide a more reliable way of setting as well as getting values, but I need to get my head fully round how all the hardware for this needs to be setup before I get my screwdrivers out. Stage 3, if I ever get there, will be independent third party hardware to take the independent measurements (pulse counter on the heat pump dedicated electricity meter = energy in, LWT/RWT and flow rates = energy out) that will confirm or deny the Midea readings.