Joining the Renewable Heating Hub forums is completely free and only takes a minute. By registering you’ll be able to ask questions, join discussions, follow topics you’re interested in, bookmark useful threads and receive notifications when someone replies. Non-registered members also do not have access to our AI features. When choosing your username, please note that it cannot be changed later, so we recommend avoiding brand or product names. Before registering, please take a moment to read the Forum Rules & Terms of Use so we can keep the community helpful, respectful and informative for everyone. Thanks for joining!
First plot done. Initially I missed the discrete minus sign before the 100 that changes the sign of the percentage differences, but it is inluded in this plot:
In fact, some of the differences are negative, at and above around 14.5°C OAT. Methinks that is because that is around about the heat loss break even point (no heating actually needed at and above this OAT).
Midea 14kW (for now...) ASHP heating both building and DHW
Meanwhile, I have done the residual plots for the analysis. These are somewhat technical, but in general terms, these plots indicate whether it is reasonable to use an analysis method that has been used. Here they are, for the two data sets, Setback and No Setback:
The left clustering in the Residuals vs Fitted plots is because there are more data points at higher OATs and so lower energy use. Otherwise, the plots are OK, there is reasonably equal spread about the Residual zero line. The Normal Q-Q plots are also OK, points are on or close to the diagonal dotted line. Conclusion: this suggests the method used was reasonable.
Midea 14kW (for now...) ASHP heating both building and DHW
And finally, the mean daily IAT against mean daily OAT plots. I have broken my 'Y axis must start at zero' rule because on this occasion is may make sense to do so.
Midea 14kW (for now...) ASHP heating both building and DHW
And finally, the mean daily IAT against mean daily OAT plots. I have broken my 'Y axis must start at zero' rule because on this occasion is may make sense to do so.
Thanks for the analysis! This last graph possibly shows the reason for the reduced energy usage at lower OATs. The no setback IAT is broadly flat as OAT drops below 10C, although IAT actually rises below 5C. While the setback IAT diverges noticeably downward, at OATs from 10C to 5C the difference (to the no setback IAT) approximately doubles to over 2C.
You have shown a 10% consumption difference at both 5C and 10C OAT, yet the mean IATs are different (and difference to the no setback IAT), this suggests that to achieve the 10% difference at lower OATs the mean IAT will need to be progressively lower.
The mean IAT with setback is only ~18.5C at 5C OAT and from the no setback values you seem to like an IAT of about 20.5C. If the lines are extrapolated to sub-zero OATs then the setback mean IAT will be 3-4C below the no setback IAT, how low does the IAT get during the setback period at these low OATs?
Also, can I ask how you derived the 1.18 correction factor to the energy consumption? Your independent electricity meter monitors the largest and most variable source of consumption, the other sources of electricity consumption, like water pump and electronics, will essentially be constant and not proportional to heat pump (mainly compressor) consumption.
@hcas thanks for your message saying that from a pure energy consumption (kWh) perspective, a setback makes sense and that you see the same in Havenwise data... could you please expand?
Before we take this any further a 2.5C 24hr average IAT difference is large - does the house really cool that much?
For now taking the answer to the above as 'yes' then the first question I want to answer is, can the first graph be explained by the difference between the figures in the second graph, applying a simple model of a monolithic fabric with heat loss proportional to IAT-OAT. If so then we are looking (to first order) at a simple thermodynamic effect (houses require less heating if the IAT is cooler), otherwise we need to look further.
If you have equations or just a few data points for the lines I would try to answer this as a plot which would likely be instructive. In the absence of that I will just present some manual calculations from eyeballed figures:
I would expect the calculated saving to differ from the actual saving rather more at higher OATs than at lower OATs due to the effect of waste heat in the house (which contributes to the heating), pumps etc. One way to correct for that is to use a 'Base temperature' which is a bit lower than the actual IAT (the base temperature being the OAT at which the house 'self heats', sometimes stated to be 15.5C but of course very house dependent). For now Im not doing that.
The 'calculated' figures are easily close enough to the measured ones at the lower OATs that I dont feel we need to look for another explanation at present. I would have said this even if they had been somewhat further apart. At higher OATs (particularly at 16) there is a serious divergence. However your figures suggest that setback isnt really of interest (in your case) at such IAT, so Im not so bothered at this early stage. If I were bothered then I would be looking to the COP penalty for part time heating for a possible explanation.
Thus far (and very much subject to further interrogation) its looking like nothing major and unexpected is going on at the temperatures of interest (say 10C or less). If we can confirm that then a huge sigh of relief!
If we can confirm that then, particularly given the quite large average IAT delta, I think we need to look at the 'comfort delta', ie the extent to which your setback has reduced the house temperature at the times we (or rather you) care about it. I don't know what times you designed for but, for many people, the times of interest are something like 7am-8.30am and 4.30pm-10.30pm. I would suggest that a larger comfort delta can be tolerated in the morning slot than during the evening slot. I would further suggest that if the evening slot temperature is reduced in setback case relative to non setback case, then the part of the reduction in loss that can be accounted for by this is a consequence of reducing the target evening house temperature not a consequence of setback (because the same comfort level could be achieved in the non setback case by reducing the target house temperature. Put another way the comparison needs to be between setback and non setback cases with (as a minimum condition) the same evening IAT.
So my next question for you is, what is the average IAT in the morning slot and the evening slot, for setback and non setback cases, again as a function of OAT.
Sorry to keep asking questions but this data is sufficiently valuable IMHO that its worth extracting as much as we can out of it!
Whilst you are looking at this question (if you are prepared to) I will look back at some of my modelling to see of the general trends in the figures I have analysed above are present.
This post was modified 3 weeks ago 2 times by JamesPa
4kW peak of solar PV since 2011; EV and a 1930s house which has been partially renovated to improve its efficiency. 7kW Vaillant heat pump.
You have shown a 10% consumption difference at both 5C and 10C OAT, yet the mean IATs are different (and difference to the no setback IAT), this suggests that to achieve the 10% difference at lower OATs the mean IAT will need to be progressively lower.
I do lean to the idea that the main way setbacks save energy is by lowering the daily mean IAT. With careful timing, my setbacks start at 2100 which is a bit before bedtime, but unnoticeable in comfort, and end at 0300, which allows time for recovery, plus my auto-adapt script which can boost recovery when needed, this reduction in daily mean IAT can happen without compromising comfort.
It is happenstance (when I happened to choose to run with a setback or no setback) which determined the periods included in the study, and they do not include the coldest months. This is a gap in the data which I can fill next winter, by running with a setback then, with this last winter (2025/2026) as the comparator, but it does mean we should be very careful about extrapolating from the current study to lower OATs. In particular, there are no periods of regular defrosts in the current data set. From previous discussions, we know that the effects of defrosts on energy use can be interesting and thought provoking.
That said, and going against what I have just said (don't extrapolate!), we can compare the mean energy use over a range of OATs and speculate on what might happen at lower OATs. Here is a plot showing the mean percentage savings over a wider range of OATs. Again, I emphasise the tails are whatiffery, but none the less they are interesting speculation, in particular the right hand end uptick. The middle range from say 4 to 12°C OAT is based on real observations, and suggests the savings are indeed in the 10% range, and they decline with increasing OAT:
The mean IAT with setback is only ~18.5C at 5C OAT and from the no setback values you seem to like an IAT of about 20.5C.
This is more happenstance again, rather than I like an OAT of 20.5°C. The design temp for the living rooms is 19°C, anything above that is fine, and I can tolerate IATs down to about 17.5 to 18°, below that I start to notice the chill. What this means in practice is that I am comfortable at anything at or over 19°C, and variations of a degree or so either way go unnoticed.
Also, can I ask how you derived the 1.18 correction factor to the energy consumption? Your independent electricity meter monitors the largest and most variable source of consumption, the other sources of electricity consumption, like water pump and electronics, will essentially be constant and not proportional to heat pump (mainly compressor) consumption.
It is based on a comparison between the Midea reported energy use, as calculated from the volts/amps in, and the energy use reported on a dedicated kWh meter that only supplies the heat pump. The latter is what the heat pump uses in total, but it is manual read only (there is a modbus version available, but I have never got round to getting one and fitting it), meaning the comparison periods are relatively long (days. weeks and more) which means the 1.18 correction factor is the long term average, even if over shorter time periods it does vary. For the sake on not setting up a not comparing like with like scenario, I have kept the correction factor at 1.18 throughout (it gets added early on in the calculations) so we are at least comparing like with like, which is what matters here because we are looking at relative differences between setback and no setback running, even if the absolute values may be out a bit.
Midea 14kW (for now...) ASHP heating both building and DHW
Before we take this any further a 2.5C 24hr average IAT difference is large - does the house really cool that much?
The mean (over the entire range of values) difference is only 20.57 - 19.28 = 1.29°C. At lower OATs this does indeed increase, as the drop in IAT during the setback becomes more pronounced, see this chart (this is the with setback period in the study) from earlier:
Compare the periods in early March, when the OAT dropped below zero, with the period in late April. The IAT drops are much more pronounced in early March.
If you have equations or just a few data points for the lines I would try to answer this as a plot which would likely be instructive. In the absence of that I will just present some manual calculations from eyeballed figures:
I do have the equations. For the regression lines in the IAT vs OAT plot, which I fitted as second order polynomials (maybe simple (straight) linear would be better?), they are now added to the plot:
The same numbers, to ridiculous precision, but easier to copy and paste, from the model summaries:
If we can confirm that then, particularly given the quite large average IAT delta, I think we need to look at the 'comfort delta', ie the extent to which your setback has reduced the house temperature at the times we (or rather you) care about it.
See my recent comments on what I am comfortable with. Setback starts at 2100, a bit before bedtime, but not long enough before for me to notice any chilling. Setback ends at 0300, and this allows sufficient recovery time before I get up in the morning on all but the coldest days. My design IAT for the living rooms and kitchen is 19°C, but I am OK if it ranges a degree or so either side, which is what it does in practice.
Put another way the comparison needs to be between setback and non setback cases with (as a minimum condition) the same evening IAT.
This is the old 'return to status quo within 24 hours' requirement. Generally it is met. See the first plot in this post, a bit scrunched up, but you can get a sense of what happened. When the OATs are very low, it may be marginal, but I can live with it.
So my next question for you is, what is the average IAT in the morning slot and the evening slot, for setback and non setback cases, again as a function of OAT.
I will need to work out how to get this. All the data in R at the moment is daily data, but I do of course have the hourly data that is behind the daily data in csv format. I think there will be ways I can pull out the daily IAT and OAT at say 0900 and 2100, and then plot the daily IATs and OATs against each other.
Sorry to keep asking questions but this data is sufficiently valuable IMHO that its worth extracting as much as we can out of it!
Not a problem at all! What is the point of collecting the data if we don't do anything with it? I'll look into getting those hourly IATs and OATs, probably later today.
Put another way the comparison needs to be between setback and non setback cases with (as a minimum condition) the same evening IAT.
This is the old 'return to status quo within 24 hours' requirement. Generally it is met. See the first plot in this post, a bit scrunched up, but you can get a sense of what happened. When the OATs are very low, it may be marginal, but I can live with it.
Indeed it is, albeit modified a bit to take into account human factors as best we can. The fact you can live with it is great, but in terms of the science necessary but not strictly sufficient. You could (presumably) equally live with the non setback case but the temperature reduced by a degree. For the science we need to compare like with like for at least some defined period. My suggestion of the evening session is because I think it would be reasonable to compare two situations namely
the non setback situation and
the setback situation where the evening temperatures are the same (or an adjustment is made)
We can then say that these are functionally 'equivalent' except for the 'penalty' that its a bit cold at breakfast (when it might matter) and during the day (when it doesn't matter to many).
Not a problem at all! What is the point of collecting the data if we don't do anything with it? I'll look into getting those hourly IATs and OATs, probably later today.
Brilliant. I do feel we are actually getting somewhere!
This post was modified 3 weeks ago 2 times by JamesPa
4kW peak of solar PV since 2011; EV and a 1930s house which has been partially renovated to improve its efficiency. 7kW Vaillant heat pump.
@jamespa — as a taster, which may in fact already have given us the answer, here is the plot for the whole dataset (no setback/no setback sub sets) for the hourly mean IATs vs OATs at 0900 and 2100 for all days in March and April in 2025 and 2026. The values being hourly means are the mean for the preceding hour eg 0900 is 0800 to 0900.
Most of the time the IAT is where it should be. It is only at very low OAT it drops a bit, on perhaps half a dozen occasions, if that.
Midea 14kW (for now...) ASHP heating both building and DHW
@jamespa — as a taster, which may in fact already have given us the answer, here is the plot for the whole dataset (no setback/no setback sub sets) for the hourly mean IATs vs OATs at 0900 and 2100 for all days in March and April in 2025 and 2026. The values being hourly means are the mean for the preceding hour eg 0900 is 0800 to 0900.
Most of the time the IAT is where it should be. It is only at very low OAT it drops a bit, on perhaps half a dozen occasions, if that.
Great. I dont think it yet gives us the answer, but it maybe tells us the direction of travel. I think the x axis on the plot should however be 24 hour mean OAT, same as previous plots.
Low OATs are interesting, the low 24 average OAT is 'suspiscious' and may indicate that there is in fact a material comfort penalty which, once corrected for, dilutes the savings. Perhaps the conclusion may turn out to be 'setbacks are good at mid OATs, bad for consumption at high OATs, and bad for comfort at low OATs. That could be a useful conclusion. But this is getting ahead of ourselves.
