Posted by: @cathoderay

↑

Next step is to get hourly stats and this seems to be working.

There's another similar concatenation of data which you might like to ponder at the same time.

When 'things go wrong', it's useful to have data for the preceding seconds in case it reveals the origin of the fault.

So you could do two rounds of data extraction:

• Read data every second
• Every other minute, filter that to obtain the average for the preceding minute
• If all still functional, discard the set of 60-readings from which you've just obtained the minute-average
• Every hour, take the set of 60 minute-averages, and calculate the hour-average

So it's basically the same routine,
but with the additional step to discard the seconds data.

Posted by: @transparent

↑

When 'things go wrong'

Yes, sooner or later that will happen, the question is when not whether, but so far my heat pump has been remarkably well behaved, and so far has run mechanically/hydraulically for over a year without a hitch, meaning that for now fault tracing on it is not a priority. The problems have all been in software, both the Midea app (crippleware) and HA, upon which I shall not here and now have another rant.

I think the simplest way of doing every second monitoring would be to have a separate script running that did just that, and only kept the last say five or ten minutes data. One thing I am still unsure about is what happens when polling requests collide, ie two read register events happen at the same instant, or a second one happens while a first one is still running. Python also has a plethora of timing modules, and it is not clear what the pros and cons of the various modules are. There is some stuff about memory leaks and thread collisions that I haven't got my head round. What I can say for now is that RAM usage (which I monitor) seems to be under control, around 20% most of the time.   

Yesterday the electrician completed the distribution board (consumer unit) upgrade started by having solar+batteries and Zappi EV charger installed at the end of January. I asked him install an Emporia Vue Energy monitor on all the circuits as he migrated them. I got the App up and running so for the first time I can see the ASHP in action. An obvious thing is that the circuit only draws about 8W when idle and looking for work. The pump starts immediately the heating demands (24/7 is not warranted now spring has sprung at last) - the pump draws another 150W. The pump takes a while to ramp up from cold. Here are a few photos from the Vue App.

During the startup I estimated the COP using the output energy reported by the wired controller. It was remarkably good at the specified figures -around 5. It seems to get running at a 50% before ramping up. I guess it wants the primary circulation loop running before being sure that higher output is justified.

There is a HA interface to the Emporia Cloud API. I am quite happy with the accuracy of the Emporia kit. It matches the Zappi figures. I doubt Midea include the secondary pump power figure in their figures. Compressor current is just that and seems to be integer, so I’d rather use measured power. There is a case for including all energy except the secondary pump when calculating the COP.

I’m very irritated with my installer for driving the secondary pump independently of the heat pump. It continues running even if the pump switches to DWP. 

Phil

@derek-m or indeed anyone - I have almost got the python code to do everything I want, including allocating energy in and out to space heating or DHW heating based on the three way valve position (@batalto that part of the code might interest you), apart from one rather tricky problem. In the immediate period after a DHW heating run, the LWT is still very high relative to what is now the space heating RWT, making energy out high because of the high delta t, but although the heat energy generated came from a DHW heating session, it gets allocated to space heating (because of the three way valve position), resulting in an absurdly high COP for space heating eg 11.6. in that hour. The highlighted section from the minute data file shows what happens: at 13:53 the DHW temp reaches the set temp (50 degrees), the dhw_on_off goes to 0 (off) and the htg_on_off to 1 (these two determined by valve position), but for the next few minutes the LWT and so delta t are still high until the cooler heating RWT brings the LWT back down again: 

The allocation to space heating or DHW heating is done by multiplying the calculated kWh in and out by the respective valve position column. When this is 0 (ie the valve is not supplying heat to that side), the result of the calculation is zero, and when this is 1, the calculation result remains as it is. The actual code looks like this (spht = specific heat, I'm using 4.05 to allow for a little glycol in the system):

    ## allocate kWh in/out to htg and dhw cols 
    df['htg_kWh_in'] = (df['amps_in'] * df['volts_in'] * (1/60) * df['htg_on_off'])/1000
    df['dhw_kWh_in'] = (df['amps_in'] * df['volts_in'] * (1/60) * df['dhw_on_off'])/1000
    df['htg_kWh_out'] = (df['flowrate']/3.6) * spht * (df['LWT'] - df['RWT']) * (1/60) * df['htg_on_off']
    df['dhw_kWh_out'] = (df['flowrate']/3.6) * spht * (df['LWT'] - df['RWT']) * (1/60) * df['dhw_on_off']

Brief explanation: df is a dataframe, in effect a spreadsheet table inside the code, df['...'] are the columns. Each of the four lines adds a calculated kWh in/out value to the kWh in/out columns. The "* (1/60)" is because each row is a minute, ie 1/60th of an hour, the minute values are summed later in the code to get hour values which are then saved to the hour data file.

What I can't figure out is a way to deal with those rogue high kWh_out numbers. The best I came up with is an "if" condition that compares the current valve position values (1,0) with those five minutes ago (0,1) and if that was true, then deep six the the kWh_out for that minute. But that is very kludgy, and doesn't totally fix the error, although it does get fairly close.

Such a nuisance, all the more so as apart from this, the code is rather neat in how it does the allocation. I wonder if you have any ideas?  

@ cathodeRay I could see how the instantaneous COP value could be in the order of 11.6, but I can't see how that could be the average over an hour due to the fact that your heat pump is running at less than 100% duty cycle; from time stamp 13.56.10 onwards the compressor appears to me to have stopped running (system not calling for heat). Do you need a flag to indicate when the heat pump is running or when heating is called for as part of the COP calculation?

@iaack - thanks. The 11.6, which is definitely for the hour not minute, is definitely an artefact, caused by the erroneously high energy out figures from 13:53 to 13:55 inclusive when the compressor was still running, so I can't use if(amps_in = zero) ie compressor isn't running then kWh_out = 0 to remove those high readings (the amps_in zeros that do then appear are because it's cycling at the moment because of the mild weather).

Unless the maestro (@derek-m) can think of a solution, I think I am going to have to identify those minutes by the valve position pattern, eg something like:

if htg_on_off = 1 and dhw_on_off 5mins ago = 1 then kWh_out = 0

but as I say that's rather kludgy and not easy to implement in data frame calculations which tend to be on single rows or columns. Maybe I can add another column called keep_kWh_out, set to 1 to keep, 0 to discard and use the same multiply by trick described above to keep or discard the calculated value.      

@cathodeRay,  Would I be right in thinking that your hourly COP value calculation includes periods (minutes) when the compressor is not running? Or more statistically correct derived from the summation and average of each of  60 individual minute intervals when the compressor is running?

@iaack - as this is a discussion forum, then please pitch in with any ideas you have for producing a 'fair result' from the available data.

Personally, I'm always dubious of statistics based on any data which includes a zero !
I need to think carefully whether this is

• a genuine reading from a sensor
• a number indicating that a sensor wasn't operational
• an anomaly caused by the questionable approach I'm taking
• a reading which should've been negative, but I haven't allowed for that possibility!

etc

So the prospect of @cathoderay writing
if htg_on_off = 1 and dhw_on_off 5mins ago = 1 then kWh_out = 0
sets my alarm bells ringing 🤨 

Posted by: @transparent

↑

I'm always dubious of statistics based on any data which includes a zero !

In the on_off columns, it is being used as a binary on/off value, but being a number it has the useful side effect it can be used to zero a calculation of leave it unchanged (x 0 = 0, x 1 = no change to value).

The other truly numerical values can also have zero values, eg if compressor is off, compressor amps_in will be zero (and calculated kWh in ie amps x volts x time will also be zero).

Posted by: @iaack

↑

Would I be right in thinking that your hourly COP value calculation includes periods (minutes) when the compressor is not running? Or more statistically correct derived from the summation and average of each of  60 individual minute intervals when the compressor is running?

The basic idea is readings every minute, and then do sums on the last 60 mins worth of minute readings every hour to both iron out ups and capture more accurate data. A lot can happen in an hour, and if I only took hourly reading, I would miss a lot, but minute data is too much for charts etc, hence the need for both.

Zeros do happen and are OK, as in the example above, when the compressor isn't running.

@cathoderay

I have been giving your problem some thought whilst mowing the lawns.

Correct me if I am wrong, but 'energy in' that has been allocated to producing DHW, actually was used for CH, which brings us to the problem of time delays.

It would be interesting to see what happens at the other end, when the system switches from CH to DHW heating.

A possible solution could be to offset the 'energy in' and 'energy out' records by 2 or 3 minutes, if the offset turns out to be fairly consistent. The only way to check that would be by comparing the above operating conditions on a number of occasions.

I put your data into a spreadsheet with the following results:-

I am uncertain how you calculated a COP of 11.3, unless you are seeing an 'Energy Out' value when the compressor is actually stopped.

The first table shows the data as supplied, the second one shows the same data with a 3 minute offset between Energy In and Energy Out. I know from my own heating system, that from the gas boiler and water pump switching on, it takes approximately 4 minutes before the radiator in my office reaches the calculated temperature, so time delays can become important for reasonably accurate measurement and control.

I doubt that your results will be that accurate, particularly over 1 minute time periods, since there are too many integers involved. Both LWT and RWT are integers, which if any form of rounding is involved, could produce a DT calculation with an accuracy of +/- 1. The electrical current measurement is also an integer, so could be accurate to +/- 0.5, and that is assuming that all the readings themselves are actually accurate, before being rounded.

I would suggest that you obtain several sets of data under similar and varying conditions and see if the results provide any degree of repeatability.

Posted by: @derek-m

↑

Correct me if I am wrong, but 'energy in' that has been allocated to producing DHW, actually was used for CH, which brings us to the problem of time delays.

In the yellow highlighted section, sort of. What happens is this. The space heating is on 24 hours a day, and uses weather comp to regulate heat output. At the moment, being mild, it also uses cycling (on/off regulation). As a result, there are times when the LWT/RWT delta t can be quite low, even zero. The DHW is on a timer, set to be on between 1300 and 1400 daily, and if, it isn't always the case, the DHW tank temp is below 40 degrees, then the DHW heating takes precedence over space heating, and the system switches to DHW heating. For some reason yesterday this happened in fits and starts, probably because at around 1230 I realised the Midea controller clock had not moved to BST, and so did the adjustment manually, so as far as the time was concerned, 1300 'never happened' on the Midea controller. Oh the wonders of BST, and making time disappear.

At the start of the DHW heating period (about 1311 BST), the transition is fairly smooth. At that time, the space heating was off (because of cycling), and the LWT/RWT were both around 34, zero delta t. The LWT then steadily rises, with the DHW tank following it, until at 1353 the LWT got to 56 degrees, and the DHW reached 50 degrees, the target temp, and so the system switched back to space heating.

Now this is the critical bit, visible in the yellow highlighted section, because the system had been heating the DHW, the LWT is necessarily high (well over 50 degrees), but the RWT is now coming from the much cooler rad circuit, nearer 40 degrees, creating a large delta t, and so a large energy out value (kWh out = flow rate x specific heat x delta t) and because the system is now in space heating mode, that high energy out figure becomes the numerator in the COP equation in which the denominator is the much lower heating energy in figure. Result: for a few mintutes the system appears to have a ludicrously high COP. Normally such spike occurring over a few minutes, if one happened, would get lost in an hours data, but because the space heating was only on for a short while during the 1300 to 1400 hour (only on and heating for around 7 minutes of that hour), the energy out spike features large and distorts the hourly COP.

The methodology I am using is based on two scripts. The first polls the Midea controller once a minute, and logs the data to a CSV file, an extract of which you see above. Included in that data is the binary 0 or 1 value for the position of the three way valve, with 0 being space heating, 1 being DHW heating, which the code then uses to set the htg_on_off and dhw_on_off values to 0 or 1, with 0 meaning off and 1 meaning on. The reason for collecting this data is that it offers a way to allocate the energy in and out figures (which don't of themselves know where they are headed) to either space or DHW heating. If the valve is in the space heating position, then necessarily both the energy in and out are being used for space heating, ditto for DHW. The idea behind collecting minute data is to capture the minute by minute variations, because a number of the variables are quite volatile, and a single reading may not represent that actual average value over a longer period.

The second script then aggregates the data over the last hour, to get a meaningful value for that hour. It takes the last 60 rows in the minute data csv file, and most of the time uses an average of the minute values to calculate an hourly value. For example, the energy in is based on average amps in x average volts in x time, and because of the interval is an hour the kW (amps x volts) over that hour become kWh used over that hour. It is this second script that uses the 0 or 1 value for the position of the three way valve to allocate the energy in and out to space of DHW heating, making use of the fact that as numbers multiplying by 0 means nothing gets allocated if the on_off value is 0 (off) and all gets allocated if the on_off value is 1 (on). You could also see the 0 and 1 as percentages, 0 means 0% gets allocated, 1 means 100% gets allocated.   

At least that was the theory. What it can't do is allow for spill over, as happens after a period of DHW heating, when some left over DHW energy in ends up as space heating energy out, thereby distorting the COP. If, as is likely to be the case in a DHW hour, the space heating is only on for a short period during that hour, then that spill over has a disproportionately large effect on the total space heating energy out for that hour, leading to a gross over-estimate of the space heating COP for that hour.

I think the only way to deal with this is to detect when the system is the few minutes immediately after a DHW heating period, which is when this spill over occurs, and either delete the offending energy out values altogether, of possibly re-allocate them to DHW heating, even though as far as the script logic is concerned, the system is in space heating mode. I think I can detect the transitional period - lets say it is five minutes long - by using the space and DHW on_off values: in the transitional period, both the space and the dhw five minutes previously on_off values will both be 1 (on). I can then set up a binary 0/1 column to mark whether the system is in this transitional period and deal with the energy out value accordingly. Provisionally I think transition = (htg_on_off + dhw_on_off five minutes earlier) - 1 should do it: in the transitional period both will be on (1) , and transition will be 1 + 1 - 1 = 1, at all other times transition will be (1 + 0 or 0 + 1) - 1 and so evaluate to 0.   

This will take a few days to verify, as the DHW heating doesn't come on every day, large tank, modest hot water use. Despite these complications, the code remains short and sweet. It looks as though I may yet be able to meet my 1st April deadline to have a viable heat pump monitoring system up and running.

Another thought, in reality a restatement of @derek-m's thinking, perhaps my implied assumption that one minute's energy in is directly responsible for that minute's energy out is wrong, because the system has an inevitable lag built into it, meaning, as @derek-m suggests, there should always be an offset of a few minutes, with that offset determined empirically, by observing the visible spill over in the minute data table. It might mean doing the hour calculation at say five minutes past the hour on the data from 65 to 5 minutes ago, to avoid #DIV/0! errors. It'll try various adjustments and see what happens.