@cathoderay there are different ways to physically do this: it depends on what is where.

you need something that is basically a PC (a large one, a small one, or a mini one) or a Raspberry PI, running Home Assistant Operating system. It doesn't matter which. If you use HA for other things like I do, you might tend towards a medium (I use a NUC) and some might run to a large if using it for e.g. home CCTV camera management. all are basically the same architecturally though.

If its physically feasible to run a wire from the HA machine to the device that has the modbus connections , then a USB to TTL converter plugged into the PC, wired connection to the modbus device, should work. That is simpler in that no microcontrollers are needed - but getting the wire there may be harder, depending on the physicals of the premises.

If there is intent to monitor multiple devices - perhaps a heat pump outdoor unit, a circulation pump, a flow or heat meter, and/or they are in locations that wiring back to the HA machine doesn't make sense, then proxying the monitoring through a microcontroller becomes sensible and more scalable. That can be an ESP8266, ESP32 (basically same but the 32 is higher performance) or now a PiPico. In each case, the microcontroller would be running ESPhome - which seamlessly integrates to home assistant. The comms between the microcontroller and HA is over your house WIFI - you DONT need wires (but you can if you want - but you have to buy an ESP board with wired ethernet which costs more. most don't bother). this is a bit harder work because you have to build the microcontroller both physically and logically. But there are prebuilt boards available that have the controller and the I/O on board. you then run wires from the microcontroller's modbus interface, to the modbus terminals of the device to be monitored.

microcontrollers usually use USB for initial setup (the first download of firmware), and then after that purely for power. but the power can be from any 5v (or 12v, with a board that has 12v->5v regulators) PSU. or a battery - if you have to put it somewhere awkward. there are lithium power packs designed for ESP boards that take 18650 cells . not usually AA's. but if its anywhere in a house, you should be able to get power to it and that's the better option.

I haven't yet come across an ESP board that has the TTL interface on-board, so I have some cheap interfaces on order from aliexpress , along with a modbus flowrate monitor.

Logical diagram:

@derek-m - based on this in \core\message.py:

class MessageType(IntEnum):
    set = 0x02,
    query = 0x03,
    notify1 = 0x04,
    notify2 = 0x05,
    exception = 0x06,
    querySN = 0x07,
    exception2 = 0x0A,
    querySubtype = 0xA0

I have added a few lines (class C3Notify2Mess... and the second elif) to \devices\c3\message.py to try to get another message string :

class C3Notify2MessageBody(MessageBody):
    def __init__(self, body, data_offset=0):
        super().__init__(body)
 
class MessageC3Response(MessageResponse):
    def __init__(self, message):
        super().__init__(message)
        body = message[self.HEADER_LENGTH: -1]
        if (self._message_type in [MessageType.notify1, MessageType.query] and self._body_type == 0x01) or \
                self._message_type == MessageType.notify2:
            self._body = C3MessageBody(body, data_offset=1)
        elif self._message_type == MessageType.notify1 and self._body_type == 0x04:
            self._body = C3Notify1MessageBody(body, data_offset=1)    
        elif self._message_type == MessageType.notify2 and self._body_type == 0x05:
            self._body = C3Notify2MessageBody(body, data_offset=1)    
        self.set_attr()

Will report back if/when anything appears in the logs... So far, nothing, but it hasn't broken the code either.

Class MessageType is quite interesting, it seems there are set messages (set a value?), query messages (explicit querying of a value?), two types of notify messages (wired controller notifies values without being asked?), two types of exception messages (errors?) and two other types of query messages, SN and Subtype.  

@marekk and @iancalderbank - thank you both very much, I am, and I am sure others will be, very grateful for your contributions.

I already have a mini PC (Dell Wyse, adequate spec) running HA OS set up, full details and more in my Beginners Guide to Heat Pump Monitoring thread.

One of the key things for what I want to do is reliability. Once I have set it up properly, I want it to run and run and run, and carry on running. Reliability will be even more important if/when I use HA to control the heat pump (set as well as get values) eg to load adjust the weather compensation curve. I tend to assume a wired connection is always more reliable than a wifi one (rodents attacks excepted!) and is more secure (do I really want to let drive by hooligans turn my LWT up to 90 degrees at 3am?) but maybe things have moved on. It does seem a wifi based approach is more flexible and scalable, and it definitely cuts out drilling holes in walls and ceilings, but I do worry about having enough electromagnetic radiation pulsing through my house (and me) to power a small electric motor.

I will careful go through the options you have described and see what I come up with, and then post it here for comment.

Thanks again for your help, it really is appreciated.        

@cathoderay esphome can run with encryption. default is non encrypted but its just a config option. 

wifi, I get it, it does depend on the quality of your home wifi network (that's what I do for a living). For context, I used to be the same (always wire it if you can), I laid Cat5e wiring all round my house 15 years ago. Now, with the quality of modern consumer grade wifi mesh networking kit, I wouldn't bother.

That said, ESP's will run on a really poor signal. the bitrate required is tiny. The RF power is also tiny. but everyone has the right to do things how they want in their own home of course - if you just don't want wifi nodes all over your house thats up to you - feel free to run the wires!

if the aforementioned hooligans are about, I doubt that giving you a warm house would be their top priority :-). Although again I get it (again, what I do for a living) you don't want to present network attack surfaces unnecessarily. But unless you consider yourself at risk for some specific reason , I wouldn't worry about it.

@cathoderay 

since May 2022 ESPHome encryption is by default

ESPhome changelog

quote: "API transport encryption is now enabled by default when you create a new device in ESPHome. This will autogenerate a random encryption key in the device YAML file that you will need to retrieve when you attempt to add the device to Home Assistant."

This happens when you create new device:

but in general: you need to design a solution that will be capable to run in standalone mode, I would not trust my life any open source or custom build solutions.

Power failures, software crashes, freezes or failed updates happens, but that should not prevent your relevant target from running (like heat pump in this example). Rather collect data, generate reports and change some settings from time to time, but not real-time control. There are industry standards capable doing it like QNX, but rather not dedicated for average human...

Posted by: @Anonymous

↑

@cathoderay 

since May 2022 ESPHome encryption is by default

good point, I've been using it since well before that and hadn't picked up on the change of default.

Posted by: @Anonymous

↑

but in general: you need to design a solution that will be capable to run in standalone mode

agree for heating - I'll be setting mine up to monitor and optimise with modbus controls from HA - but if that all dies I'd have a fall back to a simple "run" coming from an old school thermostat's boiler relay.

Posted by: @Anonymous

↑

but in general: you need to design a solution that will be capable to run in standalone mode, I would not trust my life any open source or custom build solutions.

Power failures, software crashes, freezes or failed updates happens, but that should not prevent your relevant target from running (like heat pump in this example). Rather collect data, generate reports and change some settings from time to time, but not real-time control. There are industry standards capable doing it like QNX, but rather not dedicated for average human...

Posted by: @iancalderbank

↑

agree for heating - I'll be setting mine up to monitor and optimise with modbus controls from HA - but if that all dies I'd have a fall back to a simple "run" coming from an old school thermostat's boiler relay.

Good points. I haven't given enough attention to this side of things, what happens when things go wrong. Power cuts are not uncommon here, it's rural and so many vulnerable overhead power lines. 

The main control as opposed to monitoring function I have in mind is to load adjust the weather compensation curve. By this I mean when the difference between the desired room temp and actual room temp is above a certain value (say dRT - aRT > 5), maybe after an overnight set back period, the weather comp curve gets shifted up a bit (say new set LWT = old set LWT + 5), until the difference is less than 1 degree, when the set LWT returns to normal. Some heat pumps (Ecodan?) appear to be able to do this natively (Auto Adaption?), but the Midea units can't. Not having the auto adpation is the main reason I don't have an overnight set back, which I am convinced (it has been discussed many times, and I am in this camp) will save me a fair amount of money. Last night, albeit a cold one, my heat pump used 4kWh per hour. A 6 hour set back from say 2100 to 0300 could save 24 kWh during the overnight period, a saving not to be sniffed at, even if the post set back boost costs more than normal running without a boost.

To my mind, that's not a lot of control tweakery. The set back itself will be controlled by the heat pump wired controller, the only thing HA will do is test the room temp against the desired temp, and boost the LWT if required. A power cut, or any thing else that renders HA out of action, will only mean the boost instruction won't happen, meaning, I think, the only malfunction, given a running heat pump and out of action HA, is the LWT adjustments won't happen, the heat pump will just run in normal weather comp mode. Both the heat pump and the mini PC auto restart after a power cut.

Posted by: @iancalderbank

↑

if the aforementioned hooligans are about, I doubt that giving you a warm house would be their top priority :-). 

I was thing more along the lines of a bit warmer than warm! It does sound as though security is better than it used to be, but also not exposing things unnecessarily is also a sound principle. I'm currently working on what I really want in the way of sensors, and then work from that, rather than monitor things just because I can. I also note it seems you have ordered one of those Chinese blue box non-invasive flow meters, it will be interesting to hear how that works. If it turns out to be good enough, it will be very useful as a way to monitor what's going on in my secondary radiator circuit, and even get a reasonably definitive answer of what the heat losses or rather inefficiencies are across my plate heat exchanger.  

@cathoderay

How do you keep your heat pump running but not your PC when you have a power cut?

Posted by: @derek-m

↑

How do you keep your heat pump running but not your PC when you have a power cut?

Magic. I connect it to trees, and use their energy.

Seriously, I was a bit too brief in what I wrote here (the bit I think you are getting at, and you are quite right to pick me up, I don't like sloppy writing):

"A power cut, or any thing else that renders HA out of action, will only mean the boost instruction won't happen, meaning, I think, the only malfunction, given a running heat pump and out of action HA, is the LWT adjustments won't happen, the heat pump will just run in normal weather comp mode. 

when I should have written

"A power cut, or any thing else that renders both the heat pump and HA out of action, will only mean the boost instruction won't happen. , meaning, I think, [T]he only potential malfunction, given a running heat pump and out of action HA for some other reason eg software crash, is the LWT adjustments won't happen, the heat pump will just run in normal weather comp mode." 

@cathoderay

I have had a little time to spare to have a further look at the structure of the messages, which I have put in the spreadsheet below.

Could you explain which of the midea_ac_lan routines you used to obtain the messages, particularly the type 4 showing power in and power out.

Looking at the structure of the sub-routines, with my limited knowledge of the syntax of Python, I think that it may be quite feasible to add the code to extract further data from the controller, assuming that it is present.

As you can see, I have proposed using a wired ethernet connection, with added PoE (Power over Ethernet). It solves the power supply problem, and was the robustness of wires (I am not sure how well those PCB trace antennae will get wifi through my solid stone walls, and it would be a real nuisance to buy the kit and have it not work).

The mini PC, router and Midea wired controller are already in place. The new additions are:

(1) PoE injector: about a tenner off ebay eg here (<=link)

(2) PoE ethernet enabled ESP32: these are more expensive, but they potentially solve a number of problems. Possible candidate here (<=link)

(3) I'm still working on details for the other three sensors (the ones in the left hand box will be later additions, once the basics are set up)

(4) Cables as shown

Any thoughts/comments very welcome, as ever!      

@derek-m - just seen your post as I posted mine, will get back to you tomorrow.