Maximising Home Automation: Integrating Octopus Energy & More

In this fourth and final article on home automation, I’m going to see where and how the system we’ve built so far can be extended to get all your kit working more cohesively and, hopefully, reap some real efficiency benefits as a result. I mentioned in the last article that this one would be a bit of a marathon since there’s a lot to get through, so you might want to take a break for a well-earned cuppa halfway through.

Recapping, we’ve already set up Home Assistant and installed several integrations for interacting with various things; in my case, we have a solar generation forecast service, a Melcloud integration allowing me to link with the Ecodan heat pump, and a Growatt integration for doing the same with the inverter and battery. The first two are pretty solid and performing as expected, while the web service that the Growatt integration tries to communicate with is less than dependable for various reasons. Moreover, the Growatt integration allows a reasonable range of monitoring but precious little ability to change settings.

This leaves us with some jobs still to do:

• We need to incorporate my Octopus Energy agile tariff information (import and export), but there is no out-of-the-box integration for that.
• We need a better way to manage the Growatt inverter and battery.
• We need to start configuring Home Assistant to make some automated decisions based on the data we’re gathering—after all, that is rather the point of this whole exercise!

Custom Integrations

As I touched on in the last article, each integration is just a small package of code allowing Home Assistant to talk with stuff. If the Home Assistant developers (or manufacturers wanting their kit to integrate easily) haven’t provided an integration for the kit you want to talk to, it’s perfectly possible to write your own integration and there is a VERY good chance someone has already done so. In fact, Home Assistant encourages this and provides what it calls the HACS (Home Assistant Community Store) as a place where you can find third-party-developed integrations and, if you’re techie enough, offer your own.

This is exactly what I need to get Home Assistant talking with Octopus Energy since they provide a web service—an API—but no in-house Home Assistant integration, meaning we’ll need to use a custom one. Going to the left-hand-side button bar and clicking on the HACS icon takes me to a screen where I can search for “Octopus”; I’m going to pick “Octopus Energy.” Clicking on it takes me to a page which starts to look worryingly techie, but in amongst it is the information about how to install it. Even better, it splits those instructions in two, the first part being how to install via HACS and that is little more than “click on the ‘Download’ button in the bottom right-hand corner of the screen.” The Download button gives me a quick popup telling me stuff and reminding me to restart Home Assistant after downloading, and I just follow the instructions. Once the download has finished, I am able to click on the Settings cog on the left-hand button bar, and the ensuing screen tells me a restart is required, so I click on that and it restarts Home Assistant.

Bear in mind that all this has done is update Home Assistant to know there is another integration that can be used, but it hasn’t actually told Home Assistant to use that integration. In effect, we’re at the starting point we’ve seen before for adding standard integrations, so we can now add the Octopus Energy one in the same way as we added the Melcloud one or the Solar Forecast one in the previous article. So now we’ve done all that HACS stuff, when we click “Add Integration” on the devices and integrations page, we can now search for and find the Octopus one. Obviously, it asks for a different set of details that are Octopus-specific (an account ID in the format of A-XXXXXXXX that you’ll see on your Octopus dashboard, an API key that you can find by burrowing into My Account/Personal Details/Developer settings (API access button) and perhaps a tweak or two of the odd optional setting but probably not), but you fill them in and Robert’s your mother’s brother. In my case, since I have two tariffs with Octopus, the integration discovers two devices (one import, one export) and a load of entities in each.

Now I could click into each device and add the various entities to my dashboard as we did in the last article, and that’s the way I’ll leave it for now. However, there’s a lot to take in with all that info, so I got a bit clever—or at least took advantage of someone else having done so—and also installed an Octopus Energy Rates Card. It does involve a bit of configuring that isn’t just clicking on buttons, but I think it’s worth it. Just bear in mind that it doesn’t affect the automation and decision-making that relies on the Octopus data, but the benefit it provides is easier reading of that data. If you want more information about how I did it, head over to the appendix for more detail, but for now, we’ll ignore it.

Now for that inverter. Perfectly solid bit of kit, not good cloud or app management capability. For those who have been regulars on the Renewable Heating Hub Forums, you’ll probably have already guessed the answer I came up with: modbus. This all feels very much outside the comfort zone of the uninitiated, and the process is an exercise in jargon overload interspersed with a heavy dose of acronyms, so I’ll deal with the process of modbus itself in the appendix later—especially since it’s a hardware topic rather than a home automation topic and I don’t want it to confuse matters. From a Home Assistant point of view, though, it just needs to know it can talk modbus with my inverter via one route or another, so our steps are as follows:

• Set up a way of talking modbus to my inverter as outlined in detail in the appendix.
• Add another HACS integration to my Home Assistant box, this time for SolaX Inverter Modbus.
• As we’ve seen before, I now need to go to my Devices and Integrations page and add the SolaX integration. This gives me a dialogue box asking for some pertinent details; specifically, I need to choose the interface to be TCP/Ethernet since I am using a modbus to ethernet converter (full details in the appendix), the modbus address of the inverter to be 1, and the inverter type to be Growatt. When I submit that, it asks me for the IP address of the modbus to ethernet converter and (big gotcha that had me head-scratching for quite a while) the modbus TCP variant needs to be set to Modbus TCP (not Modbus RTU over TCP, which you might have thought would be necessary having read the config stuff in the appendix). Hitting the second “submit” button adds the integration to the list we’ve been building up.

If the hardware stuff has been set up correctly and the converter is reachable on the network, then Home Assistant will now be using the SolaX integration to talk via modbus every 20 seconds with my Growatt inverter without having to talk to Growatt’s cloud services at all. In fact, now I have been running with this way of communicating with the inverter for several months, I have found it has not missed a beat. The Growatt hardware is plenty solid enough and does as it’s told reliably even when the Growatt web portal is suggesting the inverter is offline. Bliss.

Automations

At this point, I am now able to use Home Assistant to pick up a bit of forecasting, control my heat pump, get lots of Octopus info, and control my inverter. All the bits of the puzzle are now manageable in one central place, and that’s all well and good, but if I’ve just swapped several management websites for one single central one, then that’s not much benefit for quite a lot of time and effort setting all this up. In order to make the most of it, therefore, what I now want to do is start putting some automated decisions in place. Home Assistant calls those Automations. Automations are simply rules that can do things if and when things change. You may also have heard of the term IFTTT (If This Then That), and that is a surprisingly accurate explanation.

In my case, now I have a reliable and comprehensive enough ability to send my inverter instructions (thanks, modbus), and now I am receiving half-hourly pricing from Octopus, I want to get a bit more clever. What I would like to achieve is:

• If a half-hour has a negative price, then I want to tell the battery to charge from the grid.
• I also want to tell the battery to stop charging from the grid if the price goes positive.
• Just because I’m cautious and know the 4 pm to 7 pm stretch is always the expensive part of the day, I want to have a safety net of ensuring the battery is not charging from the grid once we get to 4 pm.

Clicking on the cog on the left-hand button bar, I get to the now familiar Settings page. This time I need to go to Automations and Scenes where I can click the “Create Automation” button in the bottom right-hand corner. This gives me a dialogue box where I can choose Create new automation and that brings me to a new page that is actually fairly self-explanatory. There are three sections; “When,” “And if,” and “Then do,” and each section is exactly as it says on the tin. “When” is where I can say what I want Home Assistant to look out for (i.e. what is going to change), “And if” is for extra things I want to check for, and “Then do” is what I want to happen if the thing I’m looking out for happens and the extra checks are true.

So for my first automation, I’m going to tackle the question of charging the battery when a plunge price comes along, and to do that I first need to understand how to tell the inverter to charge from the grid if I’m using the manufacturer’s own systems. The more complex the manufacturer makes it, the more settings my automation has to change, but at least I only have to do it once. In my case, the inverter has multiple “modes” to give load, battery, or grid the priority, and each operating mode has multiple time slots which control when a particular mode can be enabled. To tell the inverter to charge from the grid, I need one of those time slots to cover the time needed, I need to enable that time slot, and I need to put the inverter in “Battery first, charge from grid” mode. Phew!

So to automate this, in the “When” section of my new automation, I’m going to add a trigger and pick “Device” because I want to look out for when something about Octopus changes. I then choose “Electricity Meter” from the Device dropdown, and then, in the Trigger dropdown, I select Current Rate Electricity (my ID) money changes. In the “Below” box, I can type 0 because I only want the automation to trigger if the price has changed AND it has gone below zero.

Now I can move to the “And if” section, and here I’m going to check whether or not the battery is already charging from the grid (the plunge price might have changed from -1p to -2p, in which case I don’t want to give the battery an unnecessary instruction). So here I add a condition and select “Device.” The device I choose is Growatt, and the condition I choose is “Current Growatt Battery First Charge From Grid selected option.” The option is then set to “Disabled,” meaning I’m asking Home Assistant to find the inverter, check its “battery first charge from grid” setting and see if it’s disabled.

Finally, I can look at the “Then do” section. This is what I want Home Assistant to change and in what way when the trigger(s) in “When” are activated and everything I listed in “And if” is true. So I can:

1. Click “Add Action” and select “Device.” I can then pick Growatt as the device, and in the action select “Change Growatt Battery First Time 1 Begin option,” finally choosing the option value of “00:00.”

2. Repeat step 1 but with the action “Change Growatt Battery First Time 1 End option” and set it to “23:59.” Strictly speaking, both these first two only need to be set once, but it’s not a big deal.

3. Repeat step 1 and select the action “Change Growatt Battery First Time 1 option” and set the option to “Enabled.”

4. Repeat step 1 and select the action “Change Growatt Battery First Charge From Grid option” and set the option to “Enabled.”

Now I can hit the blue Save button in the bottom right-hand corner of the screen, give the automation a sensible name, and I’m done. Now, when the half-hourly price changes and the new price is below zero, and if the inverter’s “Battery First Charge From Grid” setting is disabled, then the automation will set the relevant time slot’s begin and end times to the earliest and latest possible times, it will enable that time slot, and then it will enable the option for the inverter to charge from the grid. A lot of changes for a very specific set of circumstances all wrapped up into one rule that I can now leave and largely forget about.

Obviously, I need to go through a similar set of steps for telling the inverter to stop charging from the grid if the price goes positive, and yet another rule for stopping it charging from the grid in any case when the time gets to 4 pm, but they are all understandably variations on the same theme. In fact, in my case, once the time slot’s begin and end times have been set, then they don’t need to be set back again; I can just enable/disable the time slot as I need. The only reason I included it in one of the automations is that it’s easier to set it here than navigate through Growatt’s app.

Wrapping it up

So here we have it. In this series of articles, I have walked you through choosing a home automation system, installing the one I chose, incorporating bits of kit to manage, and then using some automations to pull the whole lot together. It’s not a small task, especially when you consider how complex any one of your bits of kit is to manage, let alone all of them together, but if you want a cohesively controlled home, then it’s necessary either by you doing it or by paying someone to set it up for you.

Has it been worth it? That’s a difficult question to answer simply. If I had had to account for my time in monetary terms, then it would have been difficult to justify a return on investment. However, my choice of home automation system was free, and even if I had had to pay the sub £100 for a preinstalled box, then my only outlay would have been the modbus network converter that wasn’t strictly speaking a home automation issue. In any event, the system has given me a single place to see what’s going on, cut out a troublesome third-party app, enabled me to make decisions about bits of my system that are based on changes in completely different areas, and incidentally has allowed me to extend far beyond just energy generation and home heating. Now I have a solid platform, there are a world of other innovative ways I could set things up (e.g. if I put the heat pump into holiday mode, tell any smart lighting and CCTV monitoring to start doing their “no-one’s home” routines). Overall, I’d say the small actual outlay is a very small price for getting my home working as I want it to.

Hope you found this series of articles useful and, if so, keep an eye out in case I follow up with further related ones focusing on related spin-off topics.

Bonus Tips

Adding the Octopus Energy Rates Card

If you decided you wanted an easier way to view the Octopus data, here’s the way I used another extra for Home Assistant; the Octopus Energy Rates Card. To make it available, I’m going to go back to HACS and do another “Octopus” search, this time finding the Octopus Energy Rates Card. As before, I can hit the “Download” button and add this new integration as the page tells me to. Once I have done so, it is available for Home Assistant to use but still needs to be configured.

In this case, the instructions also tell me some steps I need to take for a bit of configuration and this does indeed require a bit more rolling up of sleeves, but once we’ve gone through it you’ll see it’s far less complicated than it first looks. The important thing, though, is that there are several lines of text that we need to keep handy for copying/pasting, and those are detailed in the configuration section of the page we’re on. However, I’ll provide it further down this page as well just in case you lose your place.

There is also a quick bit of preparation we’ll need to do and that is to find our “Octopus ID”. In fact, it’s a combination of two bits of info that you can find in your Octopus dashboard, so let’s do that first. If we open a browser window and log onto the Octopus Energy site as we normally would and go to “My Account”, we can scroll down the “Home” section to find our tariff(s). In the white box(es) we should see two fairly long numbers; first is our MPAN and that will be a 13 digit number. I have an import and an export tariff, so I see two white boxes and each white box has a different MPAN listed. The second important bit is the serial number of the smart meter, and that will be the same for any and all tariffs listed. My “Octopus ID”, therefore (as required for the card) is the meter’s serial number followed by an underscore character followed by the MPAN number. As you can see, I have a different “Octopus ID” for my import and export tariffs.

Now I’ve worked out what my IDs are, what the configuration asks me to do now is as follows (with added hints from me to remove some of the assumptions being made):

1. Go to my dashboard, look in the top right hand corner of the page and click on the little pencil icon. This will put the page into an “edit” mode.
2. In the bottom right hand corner of the page, click on the “Add card” button. Scroll down to the bottom of the dialogue box and you’ll see your new “Custom: Octopus Energy Rates Card” entry. Click on that.
3. The dialogue box will now show two boxes; the left-hand one is one you can type in, the right-hand one shows you what the result will be. Currently the right-hand one rather alarmingly shows a warning and a red background. Worry not. What you need to type (or copy/paste) is what was in the configuration part of the instructions we were given for the Energy Rates Card, but adding in your Octopus ID in where it asks for it.
   
   In case you didn’t make a note of it before, below is my version of the necessary text and I just need to replace (ID) with the import ID I calculated before:
   
   type: custom:octopus-energy-rates-card
   currentEntity: event.octopus_energy_electricity_(ID)_current_day_rates
   pastEntity: event.octopus_energy_electricity_(ID)_previous_day_rates
   futureEntity: event.octopus_energy_electricity_(ID)_next_day_rates
   cols: 2
   showday: true
   showpast: false
   hour12: false
   title: Agile import rates
   mediumlimit: 20
   highlimit: 35
   cheapest: true
   

4. As you can see, if I’ve done it correctly the right-hand box now looks much less alarming. Click “Save” and the card is now on your dashboard.

5. Since I have an export tariff too, I can repeat the steps to add another card but with some slightly different choices for the configuration text as below

type: custom:octopus-energy-rates-card
currentEntity:  event.octopus_energy_electricity_(ID)_export_current_day_rates
pastEntity: event.octopus_energy_electricity_(ID)_export_previous_day_rates
futureEntity: event.octopus_energy_electricity_(ID)_export_next_day_rates
cols: 2
title: Agile export rates
exportrates: true
showday: true
showpast: false
hour12: false
lowlimit: 0
mediumlimit: 8
highlimit: 15
cheapest: false

and instead of using my import ID I use my export ID (i.e. the one with the other MPAN).

With these cards added, I now have a pretty user-friendly representation of what’s happening with my agile half-hourly rates. Given the medium and high limits I chose, any half-hours with an import price above 35p will show red, any between 20p and 35p will show as orange and any below 20p will show as green. It will also highlight the cheapest half hour with a lighter green. If I see any plunge pricing (a negative price) then those slots will show as blue. Similarly, on the export card the same logic applies but is reversed so any slots above 15p will show as green, any between 8p and 15p will show as orange and any below 8p will show as red.

In both cases, it will show the current slot and all future ones, so I see a steadily diminishing list. At around 4pm (ish), when Home Assistant gets the next day’s half-hourly prices, the cards get longer and the whole cycle starts again. Easy to read some really useful information at a quick glance.

Setting up modbus

Most heat pumps, inverters and similar kit are designed to be controlled by a very low level protocol called modbus. Modbus is basically a set of instructions sent across two physical wires; one for sending and one for receiving.

Those two wires need to be plugged into the kit somewhere. Some bits of tech (my Ecodan ASHP, for example) require that you buy an extra little adapter that plugs into the box and then presents somewhere to plug the wires in. Other boxes like my inverter already come ready for the wires to be plugged in. Sometimes you’re required to attach the cables by inserting into holes and tightening a screw (a bit like wiring a 3-pin plug) whilst other times you may have a ready-made socket. Only your documentation will tell you which it is, but in my case Growatt provide a socket (that it has labelled as an RS485 port) that accepts an RJ45 plug (the same as you get on the end of a physical network cable). As a result, my first step iss to plug a network cable into that socket.

In the case of Growatt, I then have to follow some of the documentation to tell that RS485 port to operate in something called VPP mode. What’s that? No idea, but since the port was empty there is little to be lost in changing its mode.

Next, I need to connect the other end of the network cable to something and I have a choice to make. Either I can connect it to a serial or USB port on my Home Assistant server or I can connect it to a little box that sits on my home computer network. There is no right answer, but I have chosen the latter since that allows me to connect to the little networked modbus box from any of my computers at home, meaning I can troubleshoot from a familiar computer even at the same time that Home Assistant is talking to it. Some others on this forum will disagree with that choice and prefer the serial port route, but I haven’t personally tried it. Nonetheless, my particular choice of clever little box is called an RS485 to Ethernet converter and the one I’ve settled on is from WaveShare.

Bear in mind that even though a network cable such as I have used actually has eight wires in it, modbus only uses two of them and in any case the WaveShare box doesn’t have an RJ45 socket to plug the modbus cable into; it only has one of those sockets for plugging into the network. As a result, I now have to snip off the unattached end of the modbus cable plugged into the inverter, identifying wires 4 (blue) and 5 (blue/white) and attaching them to terminals RS485B and RS485A respectively on the WaveShare device.

Next is to configure the WaveShare box as a computer network device. WaveShare’s wiki is pretty reasonable here but it is still a bit of network configuring; you’ll know how confident you are or not with doing that once you’ve read their instructions. It involves downloading and installing a bit of software called VirCom so that, once the WaveShare box is on the network, you can find it, give it a network address (not mandatory but highly recommended so it doesn’t later change), set a password (once again not mandatory but highly recommended) and change a couple of settings if needed. Specifically for the Growatt inverter, I need to ensure the baud rate (the speed instructions can be sent across the wires) is brought down to 9600 and I need to make sure the transfer protocol is set to “Modbus TCP to RTU”.

Modbus is a tinkerer’s paradise so there’s lots to get confused about. However, at this point my inverter should be accessible on the network and ready for Home Assistant to start using.