@cathoderay

I haven't yet looked at HA in detail, being much more familiar with industrial systems that have Terabytes of storage, but such systems would be storing thousands of points, probably being measured and recorded at 1 second intervals.

The frequency of storage is therefore important along with the number of points being recorded. It also depends upon the efficiency with which the data is stored.

Much of the data within a heat pump system appears to use 16 bits or two bytes, so it should be possible to store a great number of values for one point in a Gigabyte of storage. Obviously the storage software may include a time stamp and other information which will increase the storage requirement, so that the two bytes of data could require ten bytes per record.

I suspect that HA may reserve storage capacity based upon the number of specified points and the frequency of recording, and also the period over which records are to be taken. Are there any settings within HA where these values can be specified?

@derek-m - HA uses a sqlite database by default, but for those who find that doesn't cause them enough pain, they can upgrade the pain by going for a more complicated (and in my opinion totally pointless) more convoluted database backend. My current HA sqlite db is 305MB, for about 6 months of data (by comparison, my python fed csv data file covering a slightly shorter period is 24MB). Both get backed up, meaning you can double or treble (or more) the amount of storage needed, depending on how you do your backups.

I think I'm right in saying HA uses a two layer system, a sort of day to day file (or rather table in the database) that records the current and recent data, which is then added to a separate long term history table that theoretically persists for ever, and yes, I think you can set an interval for how long the day to day data is kept for, using a typically cryptic entry in the configuration.yaml file:

recorder:
  purge_keep_days: 45

(ie about six weeks, can't remember why I set it to this, but there was a reason...) which is what I currently have. Even so, the states table, which I believe stores this data, still accounts for over 60% of the total size of the database, according to the command line sqlite3_analyzer tool.

The actual values in the tables are mostly human readable values ie 10 degrees C will appear as 10 apart from the timestamps which are in epoch time, just to keep you guessing/on your toes/scratching you head eg 1694751134.183572, Go figure...         

Oh, for heavens sake, this is absolutely why I hate Home Assistant. Running that sqlite3_analyzer tool on the HA database corrupted my HA database, and the 'live' database is now 4KB. The sqlite3_analyzer tool comes from the sqlite developers on the official sqlite web pages, and I think it most unlikely they would release a tool that fries databases, so yet again the finger of blame must, in my view, and until proved otherwise, point at HA deficiencies. A marble balanced on a pin is infinitely more stable than anything to do with Home Assistant...

I did once upon a time find a way of recovering a corrupted HA database file, but it wasn't easy.   

Regarding hardware requirements, I'm using a Raspberry Pi with 2GB or RAM and a 64GB SD card (Pi's are becoming available again after recent supply issues); the advantage they have is very low power usage (a Pi 4 uses 3-5W depending on what it's doing) and small footprint with no moving parts. I think all varieties have Bluetooth, LAN ports, USB, and WiFi.

There's an HA install available, although I've not tried this (see here).

My plan is to eventually use this as a simple HP controller, which is why I've added a screen with push buttons for input.

I'm not using HA, but my (sqlite) monitoring database is currently 100MB (5 months of data), and I'm using 210MB of the 2GB of RAM available (running the monitoring code and a website to view the data). Of the 64GB available on the SD card, I've used 3GB.

@mikefl - I'm sure that's right, and useful to know, thanks for the contribution, but I think a lot of people (including me) are just not ready to take on the raspberry pie thing. One of the hardest things I had to do at the start of all this was somehow distil down the overwhelming, contradictory, often badly written, sometimes just plain wrong 'information' out there. It is a miracle I didn't give up. In the end, I decided to use hardware that I was, and I suspect most people who are going to try to do this are, familiar with, ie a PC, using a thin client to make it affordable and low power use. Maybe not as low power use as a raspberry Pi, but still pretty low, and probably the same sort of price, maybe cheaper, and you get to use familiar tools eg I can use Windows Explorer to view/manipulate/edit the files on my headless PC using samba.   

Likewise, I am not sure we need to use a relational database, even a 'simple' one like sqlite (which I have used for other things, so I am not a sqlite novice). Heat pump data is about as basic as it gets, in essence a rows times columns table, with the rows being the timestamped entries, and the columns being the data readings. It is very well suited to being stored in a 'flat' csv file (which is actually a text file that uses a particular format to store the data, no more or less than that), that can be easily edited if need be, and the data plotted easily, either in a spreadsheet, which again most people attempting this sort of thing will be familiar with, or a python plotting library (my current preference). My csv data file holding about 5 months of data is paltry 24MB.

The other problem with HA (for those tempted) is its instability (breaking changes) and proneness to severe corruption, as I have just demonstrated. I have in fact just managed to recover the data, but I shouldn't have to, the system that looks after my precious home energy data should be before anything else robust, and HA is far from that.

If you are feeling a bit high, and think you could do with lowering your spirits a bit, then you could do worse than taking a look at this far from atypical HA 'comminity' thread. It has all the regulars, abusive attacks, contradictory advice, and, as far as I can, see no solution. It is on just the problem I had while doing the corrupt database recovery: you have to stop HA to do the final step (add the repaired db back to where it should be), but when you stop HA, you also stop the SSH/terminal connection that you used to stop HA ('ha core stop') and so can't do 'ha core start' to get it to start again. The only way so far as I see is you have to power cycle the headless PC to do the restart (after you have put the repaired db back in place).             

PS not for a moment suggesting you don't know what a csv file is, I just included the explanation for other readers who are less familiar with these things

PPS forgot add the link to the HA thread, now added

Hi All,

We moved into a new-build house with a Midea M-thermal heat pump last year and I’m now finally getting round to trying to work out exactly how efficient it is (or isn’t). I’ve spent a good day reading through this thread and all its predecessors (thanks very much for everyone that’s contributed and spent the time figuring this out) so think I now have a reasonable handle on what needs doing. I do have a couple of questions I was hoping someone could provide some answer to though (and apologies in advance if the answer to these are somewhere in this thread or another and I've missed them).

1. Is there any guidance out there on how to actually wire up the Midea controller? I did find this YouTube video which appears to show how to do this - is someone able to confirm this is correct, and not somethign specific to Homely?
2. I've done some searching through all the documentation I have to hand, but don't appear to have anything that contains the modbus register, which I’ve seen referenced. Does anyone know where there is a copy of this online.

Any help would be much appreciated.

Thanks

Posted by: @ctw

↑

is someone able to confirm this is correct

Basically, yes. It is a two wire connection to those terminals in the wired controller. I also added a simple 120 ohm resistor across the terminals.

Posted by: @ctw

↑

I've done some searching through all the documentation I have to hand, but don't appear to have anything that contains the modbus register

Your wired controller should have come with a small booklet which has the modbus register addresses in tables at the back of the booklet. If it didn't, perhaps ask you supplier for the booklet, or you can try searching online for something like 'midea wired controller manual' and you will find some pdf versions but be aware they are not identical, some do and some don't have the modbus tables, and some even document addresses that are available but are not mentioned in the model specific manual!

I do know I have been very lax in completing this thread, I wanted to wait until I had the room temperature sensor set up and working, which it now is. I'm also a bit wary of the next post which will be the how to do it (the wiring) because I'll be moving from the background and what you need to this is what you actually do, and I am wary of telling people how to blow up their heat pump by mistake. The post will have to be very much a 'this is what I did, if you follow in my footsteps you are doing it 100% on your responsibility and at your risk'. I'm sure you understand!  

@cathoderay My installation is a Daikin and Homely controller setup but the two connections that cathodray mentions are named P1 and P2 and I believe this is standard nemenclature; the terminating resistor is there to prevent ‘noise’ on the circuit - just saying… Regards, Toodles.

@toodles - thanks, in a Midea controller they are labelled H1 and H2. You can just make out H2 (bottom right) in this image of my wired controller with the wires and resistor connected (I also connected an earth wire, I am not sure that is needed):

Here's a better close up photo:

Perfect, thanks @cathoderay ,

I've been trying to get hold of the installer - unfortunately all this was done before we moved in and he's proving quite hard to get hold of; however, I think it's just FHP setup so I might try just approaching them directly.

Re: 'this is what I did, if you follow in my footsteps you are doing it 100% on your responsibility and at your risk' - Completely understand.

@ctw - Freedom can be very awkward, they have a B2B business model that means that as a rule (but not always) they ignore us mere end users. If you can find a model number somewhere on the wired controller, then you may have better luck finding an online pdf version of the manual that matches that model.