Posted by: @cashback

↑

my professional opinion (20+ yrs in software engineering)

That's the clue, I think most if not all HA aficionados have a background in software/computing, and so already think like HA does, and so take naturally to it. My focus on the other hand has always been on beginners, intelligent beginners, but still beginners who do not think in computing terms. Based on my experience, HA is extremely hard work, and it is certainly not helped by its vast but mostly useless (as in doesn't provide and answer) forum. Then, looking at your first post, I see things like "many problems with the [HA/modbus] approach", "The fault tolerance in the HA modbus integration isn't the best, and the integration would quickly stop working after a few minutes". The beginner isn't going to have the wherewithal to deal with this, which is why I have to say I cannot hand on heart recommend HA for beginners. It is as I have said many times a pity because HA does have so much potential, but it lets itself down in its implementation.

That said, and rather against my own advice, I am currently still using parts of HA, chiefly because I had already installed them during my time trying to get HA to work. The minimalist Linux based OS runs my python code fine, the Samba add-on , the HACS not Official Terminal and SSH add-on, the google drive backup add-on and the History Explorer card are all useful, though the latter is rapidly being replaced by Highcharts JS. I am also currently using in effect a HA script to check my python scripts are running, and restart them if they are not. My current modbus setup, which in normal use runs totally outside HA, is ultra reliable and ultra minimalist. Compare your diagram with mine:

 That's it. Type in the mini PC IP address with the path to a local Highcharts webpage and you get something like this:

This is this morning's minute by minute data, bang up to date, not a drop out in sight. Behind the scenes, Highcharts reads the csv file that contains the logged modbus data, and charts it. Again, ultra simple, ultra reliable. 

Another in my view fundamental problem with HA is that it is too complex. It is like trying to use a Saturn V rocket to move your groceries from your local supermarket to your back door. What could possibly go wrong? As it turns out, quite a lot, which is why I recommend using the family car instead.

I too have an aversion to 'the cloud' (bad choice of word in my opinion, cloudy = murky, things get hidden in clouds etc) which is why I have gone for an all local setup.

Posted by: @cashback

↑

(for example, as you have no doubt found, heat output is in kW but needs multiplying by 10 to get the value in W, or dividing by 100 to get a float in kW).

@derek-m put me onto this, it is an ancient but efficient way of sending numbers with decimal places as integers which use less memory eg 1.35m^3/h gets sent as 135 and then divide by 100 on arrival to get the original value.

Posted by: @cashback

↑

I have a few gremlins I need to get to the bottom of (e.g. my hot water tank was reading 3333°C for a time today while it was 105°C outside)

Are you sure you haven't got a nuclear reactor tucked away in the airing cupboard? Or perhaps that Saturn V rocket?

Posted by: @cashback

↑

I haven't yet done any writing with modbus (my foray in to modbus began when my USB to RS485 stick arrived yesterday), but I have it working just fine after a bit of reverse engineering of the HTTP traffic from the FHP Cloud box - I'm able to toggle heating/hot water (which is how I have tado° act as the call for heat, bypassing tado°'s built-in modulation, and schedule hot water, obeying home/away detection and so on)

It can be done far more simply in python with the minimalmodbus module. After importing the module and a few lines of connection code, all it needs is read and write commands, again simple, and even a beginner like me can understand it (T1SetH1 is the LWT at the left hand cooler ambient end of the weather comp curve, 265 is the modbus register address for T1SetH1):

T1SetH1 = instrument.read_register(265, 0)     #gets the current value

T1SetH1 = instrument.write_register(265, 60, 0)       #set the temp to 60 degrees

Posted by: @cashback

↑

Posted by: @cashback

↑

I have a few gremlins I need to get to the bottom of (e.g. my hot water tank was reading 3333°C for a time today while it was 105°C outside)

Are you sure you haven't got a nuclear reactor tucked away in the airing cupboard? Or perhaps that Saturn V rocket?

That's another problem with HA.

It won't go above 1024 million°C  but I only discovered that after I'd installed the Tokamak in the orchard. 😥 

Posted by: @transparent

↑

Posted by: @cashback

↑

Posted by: @cashback

↑

I have a few gremlins I need to get to the bottom of (e.g. my hot water tank was reading 3333°C for a time today while it was 105°C outside)

Are you sure you haven't got a nuclear reactor tucked away in the airing cupboard? Or perhaps that Saturn V rocket?

That's another problem with HA.

It won't go above 1024 million°C  but I only discovered that after I'd installed the Tokamak in the orchard. 😥 

Some people are never satisfied. 🙄 

Posted by: @cathoderay

↑

Posted by: @cashback

↑

my professional opinion (20+ yrs in software engineering)

That's the clue, I think most if not all HA aficionados have a background in software/computing, and so already think like HA does, and so take naturally to it. My focus on the other hand has always been on beginners, intelligent beginners, but still beginners who do not think in computing terms. Based on my experience, HA is extremely hard work, and it is certainly not helped by its vast but mostly useless (as in doesn't provide and answer) forum.

I think a lot of it boils down to what you want it to do. For every-day "smart" home stuff (Hue bulbs, zigbee buttons, smart power switches & heating controls and the like), Home Assistant does a great job of gluing everything together in a nice neat UI. For more exotic things (like esoteric Chinese heat pumps for which no ready-to-roll integration exists), I can see it's an up-hill battle for sure. I gave up writing a proper Python modbus integration very early on (that's not to say I won't revisit it).

My use case is that I want to be able to measure how my heating system is performing AND tie it in to other things like more complex schedules, presence detection, solar & weather forecast, and so on.

Posted by: @cathoderay

↑

Then, looking at your first post, I see things like "many problems with the [HA/modbus] approach", "The fault tolerance in the HA modbus integration isn't the best, and the integration would quickly stop working after a few minutes". The beginner isn't going to have the wherewithal to deal with this, which is why I have to say I cannot hand on heart recommend HA for beginners. It is as I have said many times a pity because HA does have so much potential, but it lets itself down in its implementation.

With this particular problem, I am 99% convinced that this is down to my hardware implementation. I have two Modbus slaves that were never intended to operate in parallel with another (each has a 120 ohm resistor across A+ and B-, only the last one should), which I suspect is what's inducing the errors that whatever HA is using under the hood for Modbus is not dealing with with very well. Once I have reached feature parity with the FHP Cloud box, I'll be able to disconnect it and then we'll see if the same problems exist.

On that note, I received a DM from @Mookyfoo which I am unable to respond to having not yet hit the requisite number of posts, give me a day or two and I'll respond!

Posted by: @cashback

↑

Compare your diagram with mine:

-- Attachment is not available --

 

The difference is that the diagram I shared is the implementation; Literally drag and drop boxes, punch in some values (i.e. the address) and draw some lines of where you want the data to go, no code at all. I'd really recommend you taking a look as you can also build dashboards with it using the data you gather from arbitrary data sources, plus the implementation is then shareable/repeatable for beginners to import.

That's not to say for a second that I dislike your approach, nor think mine is better. Code is my day job, so doing it outside of work is just not fun (which is why I like drag & drop boxes and drawing lines), in the same way HA is not fun for you; whichever approach is more fun is the one that's likely to yield the best results. More power to you!

I'm mostly here to learn from everyone here and share what I've learned along a similar/parallel journey.

@cashback

There doesn't appear to be a great deal of data shown in your example, is your heat pump switched off, or the data collection not working?

I have been trying different ways of calculating energy in to compare the results. 

I have taken an hour when the heat pump was running in a fairly steady state ie no defrosts, no cycling, the 9th April 2023 0400 to 0500 hour from the chart I posted above (posted at 09/04/2023 8:47 am).

The standard Midea method (total energy consumed at 0500 minus total energy consumed at 0400) gives a value of 3 kWh, as ever rounded because the source numbers are integers. Actual value might be 2.5 to 3.49? 2.01 to 3.0? Who knows?

Now the calculated method ie amps times volts collected over modbus. The data (in csv format) is amps in and volts in at minute intervals ie

0401,10,240
0402,9,242
...
0500,11,239

I started with very basic calcs and then tried various area under the curve methods ("integrating" the values over time): 

Method 1: mean amps x mean volts

2.484 kWh

Method 2: amps x volts for every minute then sum

2.484 kWh

Method 3: standard spreadsheet trapezoidal area under curve (sum of (min1 kW + min2 kw)/2 plus (min2 kW + min3 kW)/2 etc for whole hour)

2.490 kWh

Method 4: trapezoidal rule in python (code is very simple, can be written as one line):

2.441 kWh

Method 5: Simpson's rule in python (ditto, very simple):

2.441 kWh

Method 6: I tried a Riemann Sum module for python but it was written by yet another python sadist (demo code falls over at first hurdle)

The methods that worked are all very close. No surprises method 1 and 2 yield the same result. Method 3 is the outlier, and methods 4 and 5 are probably the most accurate, as they are closer to a true area under the curve calculation. I plan to use one of them, probably the trapezoidal rule as it is more accessible as a concept.

Any thoughts/comments very welcome as ever. 

Jumping in rather over my depth, I did find two ways* of calculating the integral, one via a spreadsheet formula and the other by using an online calculator, and they both give the same result: 2.488 kWh. Again, very similar to the previous results. However you slice and dice it 2.4 or maybe 2.5 kWh got used in that hour...

* the sums will make your eyes pop: ((0.005/3)*(60^3)-(0.5624/2)*(60^2)+52.345*60) - (0.005/3)*(0^3)-(0.5624/2)*(0^2)+52.345*0) where all the decimal numbers are from the polynomial trend line equation, the online calculator is here (link should have query embedded so you should see the result).

https://www.symbolab.com/solver/definite-integral-calculator/%5Cint_%7B0%7D%5E%7B60%7D%200.005x%5E%7B2%7D-0.5624x%2B52.345?or=input

@cathoderay

I suppose one way to check the overall accuracy of the various calculation methods, would be to use specified data over different time periods.

Say 240v and 10A for the full 60 minutes, then on for 30 minutes and off for 30 minutes during the one hour period. Then try different combinations of on and off timing.

I assume you're not expecting to calculate the COP at an accuracy greater than 0.1 ?

Posted by: @transparent

↑

I assume you're not expecting to calculate the COP at an accuracy greater than 0.1 ?

The problem facing CR and probably other heat pump owners, is how to you know when the heat pump is operating at optimal performance.

The problem is exacerbated when the same parameters are measured using different equipment and calculated using different methods, since I am fairly certain that they will obtain different results. So if you obtain two values that are different, how do you know which, if any, is correct.

If a system displays one reasonable reading, then most will not question its accuracy, but will take it at face value. Provide two differing readings, and doubt will soon set in.

Posted by: @derek-m

↑

@cashback

There doesn't appear to be a great deal of data shown in your example, is your heat pump switched off, or the data collection not working?

The screenshot was taken in between cycles on a warm(ish) day, those are instantaneous values, each of which is logged over time.

Thought this graph might be of interest.

The purple line shows the power as reported by the heatpump (via modbus) polled every 10s (with Volts * Amps). The blue line shows the power as reported by the Shelly Pro3 EM at the consumer unit on the heat pump circuit every couple of seconds.

The Shelly is supposedly accurate to +/-1% on average (the accuracy increases with the power.

As you can see, the heatpump lags behind the Shelly (not surprising given the imprecision of the current) and consistently under-reports. I am yet to see it over report for more than a few minutes. This can be accounted for in part by the imprecision of the current as an integer (and who knows which direction it's rounding, my hunch is it's simply dropping the decimal), the rest can be accounted for in the ancillaries. The external pump on the other side of close-coupled tee uses between 40-100W depending on the state of the TRVs upstairs.

As such, I'll stick to reading the power in from the Shelly for now!

One of the things on my list is to report the power in/out to different sensors depending on whether the ASHP is in DHW or space heating mode. I'm less interested in the CoP of the heat pump for water, and more interested in the space heating performance, obviously DHW skews the results given the considerably higher flow temperatures.