Part 2: Thinking about setup and hardware

Time for an update.

I've been working on the other thread, Connecting to Midea MSmartHome using a PC (<=link), as this is another way of getting the monitoring data. This approach uses wifi to connect with a heat pump's wifi which is there to send data to the manufacturer's cloud, but can also be interrogated over wifi to retrieve data. In theory, I guess any heat pump that sends data over wifi can probably be interrogated this way, but it is not easy. It needs dedicated python (a programming language) code to (a) make the connection (b) do the queries and (c) process the results into something we can read and understand. Luckily, others have already done some of the heavy lifting on Midea units to make this happen, and I now have the ability to see limited real time and historical data. This is logged in Home Assistant, and viewable in various ways. So far, I have only got DHW tank temperature (limited interest) plus total (lifetime) energy consumed and produced (of great interest, and also allow COP to be calculated). The details of how to do this are in the other thread, but it's basically install Home Assistant (HA, I'm currently using a limited Windows version, but its good enough to allow me to see what is possible), add the custom component (midea_ac_lan) to enable HA to connect to and query the Midea unit via wifi, and set up the displays you want in HA. Here's a so-called 'card' showing current real time data and then a 'History Explorer' page showing historical data (only a little, as this is all very new; these images are just to indicate what is possible). The lifetime COP is also a bit suspect, at 2.62, but that's another story (about the reliability of Midea data): 

There are a number of problems with what I am currently doing. Firstly, it happens on my PC, which isn't on all the time, see gaps in the energy consumed/produced charts, the use when it is off get aggregated into the first bar after it comes on, each bar is current reading minus immediate past reading, basically what has been added to the total during the interval, so the data, although 'complete', is far from perfect eg I can't see midnight to midnight ie daily use, because my PC is off at midnight. Secondly, I am using a 'cracked' version of HA that can run 'as is' on Windows, and it is cracked in more ways than one, not only has it been hacked to make this possible, but it is also hobbled, there are various HA things it can never do. (For those who want to know, it is basically a huge python program/sever set up that can run natively on Windows, but without the HA OS or supervisor being present.)

Because of these limitations, I want a dedicated box to run HA 24/7, and I want a full HA installation on this box, so I can do all the things I want to be able to do. I'm going to stick with HA as the least bad of the various home assistant type 'solutions' (headaches might be a better term) because it is the most mainstream, and seems to have the most options. But that also means it has the most rabbit holes, and the most threads on HA forums that start with my bread maker widget won't connect to my HA gizmo, what can I do to make it work, and end up unsolved.

I said dedicated box because there is a bewildering array of possibilities for this box. It needs to be a dedicated (only runs HA) box because in its purist form HA is an operating system (OS, the 'system' that runs on your computer, enabling everything to happen, or all too often not happen, eg Windows or Mac OS. When we talk about a Windows PC, we call it that because the OS on that machine is Windows. There are many other OSes all over the place eg Linux and Android (very common on smart phones) as well as a set of programs, and any one box can only run one OS at any one time. There is an exception, you can run a 'box within a box', a so called virtual machine as in you might run Windows in a box on a mac, or vice versa, but these 'solutions', or rather headaches, are complex, flakey, and generally they cripple the computer's performance, because in effect there is now a parasite sitting on your desktop. Or to put it another way, your hardware is now trying to serve two masters. I've tried using them a few times and always deep sixed them as being useless. Your mileage may very (that's for VM geeks) but that's my mileage. I'm going for a dedicated standalone box.

As it happens, HA is surprisingly undemanding when it comes to minimum hardware requirements. It absolutely does not need the latest turbo charged PC/Apple Mac to run, it can do so on much more modest hardware. This is both a blessing and a curse, a blessing because you can use just about anything, and a curse because you now have this bewildering choice of what to run it on. Early adopters of HA tended to be geeks, so they did geeky things, like use raspberry pies (ultra mini sort of computer) and strawberry flies (a figment of someone's imagination, he knows who he is), but these fruity options really are for geeks and the like, the rest of us will struggle, strawberry fields forever, until we fall down the infinite rabbit hole. Don't go there, I said to myself.

The other thing is that at the moment strawberry fields forever is actually a rather expensive option. Raspberry pies, like strawberry flies, are in short supply, and prices have rocketed. 

The answer, for me, is to use a low spec PC. A cheap laptop bought on ebay could do it, but as it is going to be on 24/7 I want something that uses as little power as possible. Rather surprisingly, or perhaps not surprisingly at all, there is no clear statement of what HA's minimum hardware/system requirements are. Instead, the HA developers follow standard operating procedure (SOP), why make things simple when you can make them complicated. How low end can I go, and still be able to run HA and its OS? I haven't been able to find the definitive answer, but there is a proxy answer, the spec's for HA's own offering of a dedicated hardware box on which to run HA. Again, SOP, why make this spec accessible when you can hide it? The web page plugging HA's own 'yellow box' hardware is very much in plain site (<=link), but nowhere does it say what the hardware spec is. A long way down the page it has 'Features and Specifications' (<=link) but this is a list of possibilities, not specific hardware details. 'Up to 8GB of RAM' does not tell you what the minimum RAM needed is (RAM is Random Access Memory, the systems working memory, where it does the sums, as opposed to storage memory, hard disk and solid state drives and microSD cards etc, where it stores things. In human brain terms, RAM is where you calculate how many bags of flour you need, storage memory is where you remember how many bags you need).

The actual yellow box spec (which by the way is way over my budget at $100-200 dollars, and furthermore it is only available to pre-order) is here, buried in a pdf (<=link). From this, we can see that HA and its OS will run on a system that has (this is technical, just note the names and numbers if you are not PC literate): a 64bit CPU running at around 1.5GHz, 2GB of RAM, and 16GB of storage (flash/SSD) memory. Iit might be able to run on less, but these are already pretty low end specs, and we know HA will run on them, because these are the specs used by the HA developers for their yellow box, so I am going to stick with them as a good enough minimum spec. Note the 64bit requirement. Most hardware is 64bit these days, but not all, and slightly older stuff may well be only 32bit, on which HA OS will not run.

The yellow box has various other things that mostly are requirements, including a wired ethernet network connection and wifi (wireless network connection), three USB ports, and high quality stereo (??? Do you listen to Mozart while setting it up?). It can connect to things, and play tunes, like Strawberry Fields Forever, though in practice this is not a minimum requirement, only the ports/connections are, and in fact even they are not all absolutely necessary, for example, only one form of internet connection is absolutely required.

Notice it does not have a screen or a keyboard or a mouse. This is indirectly why it needs at least one form of internet connection. In practice, what happens is your HA OS box runs as 'master' or 'server', and you connect to it over the internet using a browser (Internet explorer, Safari, Firefox, Edge etc) running on a 'slave' or 'client' device (PC, or possibly even a smart phone, but I don't 'do' smart phones, hate the frigging things). Don't worry too much about the terminology, prefer the 'architecture' of the setup. Think back to the days of huge IBM mainframe computers, and the operative sitting at a terminal connecting to and accessing the mainframe from the terminal. It is the same basic setup, all the gubbins is on the mainframe, or in our case, on the HA OS box, and we access the gubbins from our slave/client/terminal. The HA OS box has it's own IP address, one of those groups of numbers separated by three dots (eg 192.168.0.100), and you connect to that IP address by putting that IP address in your browser's address bar, in much the same way you connect to a phone by entering its phone number into a phone. There are of course some minor complications, but that is the basic idea, or architecture, behind the setup.

I now have a spec (see above). A lot of hardware will meet and exceed this spec. There are lots of small form factor and mini computers that will all do, but they are all either over-priced (because they can do more than I need) or use more power than I want to use. In the end, I decided to get something called a thin client PC. Thin means they are lean and do not consume huge amount of power, because they are intended to be used as clients (it's in the name), but out of the box most of them are just a bit too lean and mean to run as a server, which is how I will be running it, so it has have its spec upped just a bit, to become in effect a mini PC. A typical thin client might have say 2GB RAM, which is enough RAM, see above, but only 2GB of storage memory, which isn't enough, see above. However, if that gets upped to 16GB, then it is enough.

I could by a thin client, and up the storage memory myself (what could possibly go wrong), but as it happens, there are plenty of sellers on ebay doing just this, upping the storage memory on thin clients so the thin client can operate as mini PCs, or even a server. Slave becomes master. The box stays lean and mean, some so lean and mean they can even be fanless, so totally silent and very low power consumption, but upping that storage memory means it has room to store, and so run, an operating system. They tend to be very reasonably priced for what they are, and are used for all sorts of things: mini desktop PCs (with screen and keyboard added), the back end for a home theatre PC, and, in this case, home automation/monitoring software. Because it is in effect a PC (they often come with an OS already installed), I can understand what it is conceptually, a box with a processor (CPU) running an OS using RAM and storage memory, with various ways of connecting (internet, USB) to other things.

Yesterday I bought a Grade A refurbished Dell Wyse thin client with upgraded 16GB storage memory and wifi, for £47 delivered. The particular one I have ended up with is marginally over spec. If you intend to do the same thing, make sure the box meets the minimum spec above: 1.5GHz 64bit CPU, 2GB RAM, 16GB storage memory (8GB might be enough, but I wasn't going to chance it), ethernet and wifi for internet connections (latter often needs to be added, and it is worth having both) and some USB ports. It doesn't need to be a Dell box, others can do eg HP, as long as it meets the spec. Lastly, make sure it comes with a power supply (PSU). A lot don't, and buying one separately can add significantly to the cost, as well as being a hassle.

Next post hopefully will detail how to get HA and its OS running on this box.

Running total costs to date in GBP (not hours of my time, which are a lot more): £47.00.     

Posted by: @cathoderay

↑

As it happens, HA is surprisingly undemanding when it comes to minimum hardware requirements.

It is until you discover things like Frigate and think, that would be nice, then find you need more disk and maybe even coral ai.

Also if you really get into esphome, you will appreciate fast compile times when deploying new or updated firmware to your devices.

I think your wyse terminal is a good first step with HA.

Good luck, looking really good so far.

Part 3(a): Setting up the Box with Home Assistant - getting ready

In my last full post on this thread, I discussed the various options for setting up Home Assistant to do the bulk of the monitoring work for my air source heat pump heating. I decided a Dell Wyse Thin Client, which in higher spec versions is in fact a mini PC, was the way to go, and placed an order. What could possibly go wrong?

Well, the Fedex goons can deliver parcels wherever they wish. My mini PC ended up at a random address over twenty miles from where I live. Go figure. Initially Fedex swore they had delivered it here, but their own delivery note gave the game away, the delivered to address on it was that over twenty miles distant one. The ebay seller very much to his credit promptly and courteously sent out a second unit, and it arrived here a few days ago. Time to add Home Assistant to my new mini PC!

The rest of this and the next post (it's really one post, split into two to make it more digestible) is going to be a recipe/set of instructions, with some forays into reasons why. Together, they should be sufficient to allow anyone with a modicum of computer understanding to stand a good chance of being able to replicate the recipe. By modicum, I mean you know about the different types of memory (RAM, SSDs etc), what an OS (operating system) is, what a file system is (FAT32, NTFS), the word BIOS doesn't fill you with terror, and you are familiar with image based disk backups. If you are not yet at that level, you will need to bone up on these things, but they are not difficult concepts. The key bit is to understand what am image based backup is, and how it is used, because that is at the heart of what we are going to do. If you understand that, you will necessarily already know about computer memory, operating systems, file systems and the BIOS.

The key thing about an image based backup of a disk is that it is, almost literally, a digital photograph of the entire raw contents of a disk. If file based back up — copying files to a backup location — is like photocopying a chapter from a book, strictly for personal study, of course, then an image based back up is like photographing the entire book, not just the chapters, but the index, the covers, everything, letter by letter, until you have an image that represents the entire book. Unlike the photocopy of a chapter, this image means you can replicate the entire book in another place, by copying the image to another blank book. As long as the blank book is big enough, you will end up with an exact image of the original book. The closest analogy is a photographic slide. If I take a slide of the Mona Lisa, and project it onto my wall at home, I have a copy of the Mona Lisa. It isn't a perfect analogy, because the disk image is truly a perfect copy of the disk, in a way that even the finest grained film can never achieve, but it is a close enough analogy to, I hope, make the essentials of the process clear.

An image of a disk can contain not just an operating system, but all the gubbins needed to make the operating system work, including the boot system that starts the operating system, and the file system that organises the files. This makes copying disk   images to a blank disk a very easy way of cloning an entire working operating system, just as a photographic slide of the Mona Lisa allows us to clone the Mona Lisa on the wall at home.

But there is a gotcha. The disk image is a passive entity, a perfect but fixed static image of an OS, rather than live running copy of an OS. That means we need another OS to actually do the work of copying the image to our blank disk. Luckily there is a very easy way to do this, by putting another bootable image of a suitable OS on a USB stick, and then copying that image not onto a disk, but into the computer's RAM, and then booting from that OS. Such a free-floating 'live' OS is perfect for our needs, because it allows us to do things that a normal OS can, strictly speaking, never do, operate on its own vital organs while it is itself running. It is like asking a cardiac surgeon to do his own heart transplant while he is awake, it can't be done. Instead, you need to put the patient to sleep, and have another live and hopefully awake surgeon perform the operation.    

We can now turn to the heart, or OS, that we are going to transplant. Home Assistant (HA) comes in various packages, the most complete version, which we are going to use, being so called HA OS. This is a complete all-in HA system embedded, or overlaid, onto it's own OS, though strictly speaking, this is a bit of a fudge, it is actually overlaid on Linux, another well established family of operating systems, like Windows or Mac OS. Linux is often seen as the geeky OS, but in fact we all use it all the time, we just don't realise it is Linux behind the scenes. Smart TVs and DVRs (digital video recorders), through Amazon Kindles, to NASA and the Large Hadron Collider (perhaps not so everyday, but then again, settings where you want a reliable OS) all run on Linux. Even the internet is largely Linux based. The bottom line is that using Linux is OK, its everywhere. Unless (until) we end up choosing to be geeky, we will never actually see or interact with the Linux OS, because we will interact with HA using a web browser, in just the same way that we interact with other Linux based online services through our web browser. The only difference with HA is that it is all local, all done over our LAN, or local area network.

Here, by way of a carrot, is what it will end up looking like. This is HA running on my mini PC, accessed over my LAN from my desktop browser, showing heat pump energy used and produced, ambient temp and 3 hour trailing COP over the last few days. Everything is done locally, none of this comes from the Midea cloud: 

Part 3(b) Setting up the Box with Home Assistant - doing the deed

With some background out of the way, let's move on to the actual process, and what you need to get HA up and running on your mini PC. This post is long because, hopefully, it includes all the details you, as a beginner, need. The ingredients are:

(1) your mini PC, plugged in and ready to go, with temporarily added monitor plus mouse and keyboard. The former can be your normal PC monitor, or a spare if you have one, just make sure the necessary connections/adapters are available (I used an old monitor, and a DVI to VGA adaptor). The mouse and keyboard are best done using wired USB connections, if you don't have these available, your nearest supermarket will almost certainly have cheap off the shelf versions available.

The monitor plus keyboard and mouse are just needed for the setup, once HA OS is up and running on your mini PC, they are no longer needed, because HA runs in so called headless mode (not the best terminology…), meaning you access it from another computer over a network (can be wired or wireless/wifi, I prefer the former).

(2) two USB sticks. These doesn't need to be fancy, 8GB is probably just enough, but I used 16GB. One needs to be formatted to NTFS, because one of the files, the HA OS image file, is over 6GB, and that is too much for a FAT32 formatted USB stick. The other USB stick will be used to make a bootable USB stick, our 'live' OS which we will use to copy the HA OS image to the mini PC (see (4) below et seq).

(3) the HA OS image. This can be downloaded from here (<=link). This version is the Generic x86-64 version (because that is the so called architecture of your mini PC) and it will have a file name along the lines of "haos_generic-x86-64-9.5.img.xz" telling us that it is version 9.5 (this changes from time to time) of the HA OS for generic x86 64 bit (mini) PCs in compressed (xz) image (img) format. Once you have downloaded it, only a few hundred MB as it is highly compressed, you need to decompress it. I used 7-Zip, downloadable from here if you don't already have it. Once decompressed, you will have a haos_generic-x86-64-9.5.img image file, of something over 6GB. Put this file on your NTFS formatted USB stick.

We also need a way of copying the img file from this USB stick onto the SSD (solid state drive - the main storage in your mini PC). Remember, this isn't a normal file copy type of copy, it is a 'letter by letter' copy of the entire book type copy, so we need software that can do this literal form of copying. There is a vast array of software that all claim they can do this, and if you look around, you may well come across something called Balena Etcher. It is, in my experience, a bloated turd, and, having tried it, I wouldn't touch it with a extended barge pole. Given the actual letter by letter copy is a very basic low level process, it only needs simple but effective software to do the task. I used the HDD Raw Copy Tool available here (<=link, download link is at bottom of the page, use the portable version). Don't worry it doesn't mention it works on Win 10, it does, just fine. Copy it to the same USB stick that has the HA OS image on it.    

(4) a way of making your second USB stick bootable, with a 'live' OS, so you can plug it into your mini PC (or any compatible PC) and boot the PC from it (the USB stick). This, as I mentioned before, can be done by adding a bootable image of another 'live' OS to this USB stick. Here you are rather spoilt for choice in what you add, and how you add it. You may already have such a USB stick, perhaps for making backups of your main PC (I do), but if you don't already have one, I suggest you use a Win PE image (these are Win Pre-Installation Environment images, trimmed down versions of Windows that can run from a USB stick, intended for just the sort of thing we are doing) and a small standalone program to copy (flash/burn/whatever, they all mean the same thing) the image to the USB stick. A well established, free and usable Win PE version can be downloaded here (<=link). Read the front page for peace of mind, then click on Download at the top of the page, and download the iso file (just another image format) at the bottom of that page. Next, go to the USB Booting tab at the top of the page, and follow the instructions on that page to create your bootable USB stick.

(5) a standard ethernet cable long enough to connect your mini PC to your router (I assume you have one, and that your main PC is also connected to this router). The router needs to have a free ethernet port for this ethernet cable, the other end goes into the network/ethernet connection on your mini PC. You now have your mini PC and desktop connected via your local area network, or LAN. Using a wired connection is simpler and more reliable than using wifi. Note you don't actually need this cable at this stage, but it is as well to have it to hand for the next stage.

(6) lastly, nerves of steel. If you haven't done this sort of thing before, it will feel somewhat nerve-wracking at times. Expect to have to concentrate, to double check, to read things again if still not sure. Lastly, you also need nerves of steel because, while I am happy to write and publish this guide, what ever you do to your PC is your responsibility. If you are not sure, you can always stop and come an ask a question here on the forum, preferably in this thread. There are many experts here, who generously make their expertise available to others.

You now have everything you need: a mini PC with temporary monitor keyboard and mouse, two USB sticks, one bootable with Win PE on it, the other with the HA OS image on it, along with a raw copying program, and lastly a short ethernet cable connecting your mini PC and your router. The process itself doesn't take long, maybe 30 minutes. Here are the steps I took:

(1) put the bootable USB stick in a USB port on the mini PC (don't put the other one in yet). Out of the box, your mini PC will almost certainly not boot to this, and will instead go straight to the OS present on your mini PC when you bought it (if it had one, if it didn't it will just fail to boot, probably with a 'no OS' message). In both cases, we need to tell it to boot from the USB stick, and we do this in the BIOS. While we are there, we can also set a few other necessary options.

(2) boot (turn on) the mini PC, and immediately start pressing the F2 key at 1 second intervals. This is one of a number of keys that can be used to boot into the BIOS. As the mini PC boots, keep an eye on the bottom of the screen, it may have a message saying something like 'press xxx to enter setup'. If it does, and it is not F2, press Control + Alt + Delete all at the same time to reboot, and press the xxx key at 1 second intervals. If there is no message, and F2 doesn't work, let the mini PC complete whatever boot process it has gone into, then reboot, this time pressing the delete key, again at one second intervals. If you still can't get into the BIOS, try googling possible BIOS keys for your particular mini PC.

(3) some BIOS screens are password protected. Being presented with a password box when you don't know the password can cause even nerves of steel to go twang. Luckily, the password will usually be the default one, and google will find it for you. Mine was 'Fireport' (case senstive), and I believe that is the default password for most if not all Dell Wyse boxes.

(4) the BIOS looks like a 1960s terminal screen. You get around it using the arrow keys, and change things with the +/- keys and enter key. Read the onscreen tips which will tell you what does what. The things you need to confirm/change are as follows. The wording may vary somewhat - if you find something that clearly means the same thing, that is what you need to confirm/change.

(a) on the Advanced tab, make sure:

(i) Power Loss Recovery is set to 'Always on'. This means your mini PC will automatically reboot after a power cut, which is what you want to happen

(ii) Boot From USB is set to Enabled (self-explanatory, it is what you are trying to do)

(iii) Boot Mode is set to either 'UEFI' or 'Both' (which includes UEFI). UEFI is what you need, so I set it to that, rather than 'Both', but either should work. Sometimes you might need Both (Legacy for the USB 'live' OS boot, but normally UEFI for HA OS).  

(b) now move to the 'Boot' tab (top of the screen). This lists the boot order, the order of things from which the PC will attempt to boot itself. It goes down the list, starting at the top, and boots from the first thing ('device') it finds that has a bootable OS on it. For normal operation, the top item will be the SSD in your PC (it may be called SATA something, something flash drive), which is where your normal HA OS will be located. Make a note of the current first entry, so you can return it to the top of the list later. The USB entries are lower down, so normally they never get a look in. To boot from your USB drive, you need to move the USB entries to the top of the list, normally done by selecting the entry (arrow keys) and then using the + key to move it up the list. If there is more than one USB entry, it may not be obvious which USB entry is your USB stick, so move all the USB entries to the top of the list. The USB stick with Win PE on it is the only USB thing in your set up that has a bootable OS on it, so it will become the de facto top of the list entry.

(c) lastly, go to the 'Exit' tab, and select 'Exit Saving Changes', or just press F10, it does the same thing, and confirm when asked. Your mini PC will then reboot, and if all is well will boot to Win PE on your USB stick. This can take a while, the first clue that this is happening as it should is often a message on the screen saying 'Loading Files…'. Eventually you will get to the Win PE desktop.

(5) In passing, remember from earlier that this Win PE OS is actually floating in the RAM (random access memory) on your mini PC. That is what 'Loading files' meant: it was loading them into RAM. It is now running, not off the USB stick (that was just the store holding the OS files, that get copied into RAM), but in 'thin air'. This is what makes it so very useful: it means we have free and complete access to the device (SSD) that holds the normal OS.

(6) we can now wipe and clean up the SSD that has a pre-installed OS on it (the OS that came with your mini PC; if it didn't come with an OS, you can omit this step). Start (via the Start Menu > Hard Disk Tools > Partition Tools, or wherever you can find it) AOMEI Partition Assistant. Once loaded, it will show you a pictorial representation of the drives on your PC. Some of the physical drives may be divided into partitions, divisions of a physical (ie you can pick it up) drive that appears as drives on your PC (drive letters attached to them), so that they appear like physical drives in the OS, but are in fact only parts (partitions) of real physical drive. You need to identify the main OS drive. Usually it will be the largest (largest number of GBs), but not always, and it will have partitions identified with status labels like 'System' or 'Boot'. You can also right click on a physical drive (not partition) and click 'Properties', where you can normally see the brand and model number of the drive. When you are sure you have identified your OS drive, click on each partition to select it, and then delete it (via right click or top menu). This is a two step process, clicking delete adds the deletion to a To Do list, you then have to click Apply to make it actually happen. You will get severe warning messages, because normally you do not want to delete system/OS partitions. Go ahead and do it, you are going to add a new HA operating system. You will end up with an empty disk.

(7) you can now add a single formatted partition to this disk, again a two step process: right/top menu click > 'Create partition', use all the available space and select FAT32 as the file system (may be under 'Advanced'), then again Apply once it is in the To Do list. This step may not be necessary, I don't know, all I do know is that I did it, and it didn't cause any problems. Exit AOMEI Partition Assistant.

(8) we are now ready to copy the HA OS onto this empty disk. Put the second USB stick, the one with the HA OS image on it, and start the HDD Raw Copy Tool you also placed on the second USB stick. Being portable, it will run as is. Follow the on screen instructions very carefully making sure you select the HA OS image file (accessed via the bottom entry in the source list) as the source, then click continue, and then select your Dell Wyse internal SDD as the target. When you are absolutely sure you have got everything right, go ahead and start the copying process by clicking continue, and confirm as necessary.

(9) you are now almost there! Remove both USB sticks, and reboot the mini PC. Again, you will need to go into the BIOS, using the method you used before, to change the boot order by placing the internal SDD with the HA OS at the top of the list. Find that note you made earlier about what was originally at the top of the list, before you put the USB entries there, and restore that entry to the top of the list, confirm/set the boot mode to UEFI, and then finally Exit, saving changes. If all has gone according to plan, your mini PC will boot into the HA OS. During the boot process, you will see a lot of Linux stuff flying across the screen, that is normal. Once it has done its thing, it will land you at the HA OS command prompt, a rather gloomy screen that looks something like this (this is a redacted image from online, yours will differ in minor detail):      

If that is what you see, congratulations! You have successfully installed HA OS on your mini PC. The next full post will cover how to access HA from your desktop PC using your browser over your LAN, meaning instead of staring at a blinking cursor, your interface to HA will instead look something like this:

Running total costs to date in GBP (not hours of my time, which are a lot more): £47.00 (plus a few quid more if you need to buy a USB stick or two).     

Posted by: @cathoderay

↑

.... not hours of my time, which are a lot more ...

Awesome piece of work, especially from [what I think was] a standing start.

Impressive CoP.

@william1066 - thank you! Yes, it was pretty much from a standing start, though I did know what a disk image is, and what it can be used for, so I already knew the backbone of what I was trying to do. The devil though was in the detail, which is so often missed out.

The COP is interesting, I think it may be a little high, but at the moment there is no way of fully verifying it independently. It is a simple calculation, the last three hours energy produced divided by the last three hours energy consumed, as reported by the Midea wired controller, so it is totally dependent on those reported figures being accurate. I already know the Midea reported energy consumed is a little on the high side, as compared to the independent external dedicated kWh meter connected in line with the heat meter, and I am inclined to think the latter is the more accurate reading (it consistently reads about 10% less than the Midea figure) but the odd thing is that if the meter is the correct value, that means the true COP is actually a bit higher, not lower!  There may also be some displacement errors, a kWh ends up in an adjacent 3 hour block to the one it should be in, but they will be minor errors, and the kWh doesn't get lost, it just ends up in the wrong place (may explain some of the volatility), and over longer time frames they will become even less of a distortion.

The energy produced is a true black box number, I have no idea how Midea calculate it. The wired controller does report LWT/RWT and flow rate, so it has what it needs to calculate it, but does it use that data, or some other formula/method? I will go into this again when I come to do the post on setting up the COP calculation.  

Posted by: @cathoderay

↑

The rest of this and the next post (it's really one post, split into two to make it more digestible) is going to be a recipe/set of instructions, with some forays into reasons why. Together, they should be sufficient to allow anyone with a modicum of computer understanding to stand a good chance of being able to replicate the recipe. By modicum, I mean you know about the different types of memory (RAM, SSDs etc), what an OS (operating system) is, what a file system is (FAT32, NTFS), the word BIOS doesn't fill you with terror, and you are familiar with image based disk backups. If you are not yet at that level, you will need to bone up on these things, but they are not difficult concepts. The key bit is to understand what am image based backup is, and how it is used, because that is at the heart of what we are going to do. If you understand that, you will necessarily already know about computer memory, operating systems, file systems and the BIOS.

@cathoderay  Paragraph three and that's me done and out already 🤣 

Posted by: @morgan

↑

Paragraph three and that's me done and out already

Don't give up so easily! I was merely setting the level you need to be at, and said if you are not there yet, all you need to do is "bone up on these things, but they are not difficult concepts". Honestly, they are not! I even give a run through on what disk images are all about later on in the post. But I am not trying to write an introductory text on computing here, just a beginner's guide to heat pump monitoring. The posts are already long enough, and to add the basics of computing here would make them needlessly long given the basics will already be known to many, and for those who don't know them, they are widely available elsewhere.

Memory - all your devices use memory, you just need to know the names for the different types, and where they fit in. 

OS - the software that runs your device, again you just need to know which is which.

File system - how your device stores and indexes stuff in memory, you just need to know there is more than one way, and specifically the file systems you will encounter and use.

BIOS - the lower level operating system on your device, where you can set various settings that alter how the device operates.

Disk imaging - read the post for some plain English explanations.

Honestly, it's not difficult!  

OK, I'm a bit lost, maybe having forgotten some reasoning from another discussion. Why is Home Assistant (a control system) being installed if what's wanted is monitoring? HA can be made to do some monitoring, but it's not its usual purpose. It's rather like getting a flock of sheep when you want milk: you can do it, but most people would probably prefer cows.

Posted by: @mjr

↑

Why is Home Assistant (a control system) being installed if what's wanted is monitoring? HA can be made to do some monitoring, but it's not its usual purpose.

They, monitoring and controlling, are interconnected, sometimes only one letter separates them (get LWT/set LWT). Sometimes you need to get (monitor) X to set (control) Y eg you might want to get the ambient or room temp so you can set a new LWT. That said, from what I've seen, many people do use HA primarily for monitoring. The History Explorer for example is solely for monitoring, it can't do any controlling at all, and it appears to be very popular (unsurprisingly, it is very good).

Why HA? I think, though I may be wrong, that it is actively developed, has a huge user base (so less likely to turn into a zombified development) and has the greatest range of add ons, eg I don't think other systems have the equivalent of the midea_ac_lan python custom component. And again, that component is a dual purpose component, it can both monitor and control a heat pump.

My ultimate goal, once I have good enough monitoring set up (which is of great use and interest in and of itself, is to add some controls. Specifically, I am after the holy grail of adaptive weather compensation, ie tweaking the curve automatically in response to demand, eg reheat boost after a set back. Midea units can't do this out of the box, but I think it may be possible via HA. Because most HA is written in python (or javascript), and I have a nodding acquaintance with both, I hope the learning curve won't be too steep.

Lastly, I like having my own local (no cloud involved) data collection and storage, so I can look at it and do things with it as and when I feel like it. The local setup also means it continues to collect data, even when the internet is down, as happened the other day.

Posted by: @mjr

↑

Why is Home Assistant (a control system) being installed if what's wanted is monitoring? HA can be made to do some monitoring, but it's not its usual purpose.

Home Assistant + ESP Home provides a very comprehensive platform for data collection, both predictive and real time, and most are literally 1 click away.  Both types of data potentially useful for enabling sophisticated control.

You can't control what you can't measure.

Predictive

1. Met office
2. Oracle Agile Tariffs
3. Future Solar production
4. etc

Real-time

1. Modbus tcp, modbus serial
2. Floor temperatures, humidity, air temp etc. [esphome, shelly etc]
3. Occupancy of house
4. Inverters, batteries, solar generation
5. etc

You can also easily send this data on to other systems if you want to

I can send this data to emoncms with only a couple lines of config. 

To InfluxDB with also only a couple lines of config

And others.

The biggest question I have here is what the issue is with involving the cloud in your data gathering.

Given almost every piece of renewable energy kit installed these days is designed to talk to the cloud and most are commissioned precisely doing that, it seems an over-complication to query each item on the local network rather than pull the data from the cloud. After all, does it really matter if the box talks to the cloud and you then get the data from there rather than you getting your hands on it first? I have no problem with deciding to move to local collection if querying the cloud doesn't give all the data you want, but this is supposed to be a beginner's guide so shouldn't we start with the basics first?

I would suggest this guide as laid out is an intermediate setup, not that it should make any apologies for being so. However, a beginner wanting to monitor their system will probably do well to start with understanding what's available already before graduating to on-premises monitoring solutions, and incorporating further sensors is probably a step later still. I think @cathoderay has done a great job with dramatically simplifying the explanation of how to install a relatively complex solution, but I'm concerned what newcomers to renewable energy will think if they read this thread and assume this is where beginners have to start.