Will come back to the above points, but for now, I've just taken a step back, to look again at the core of the contradiction we are trying to resolve here. As I see it, it is this:
(1) my 'observed vs expected' method strongly suggests running an overnight setback saves energy. My analysis of the 'two populations' hypothesis further suggests this is the case.
vs
(2) the conservation of energy principle says any savings must be modest (or strictly speaking non-existent, if overall a steady state is maintained), and furthermore any modest savings, if present, may be cancelled out by detrimental COP changes.
Some observations:
Since all the 'empirical' studies are observational studies, not controlled trials in lab conditions, there is a significant risk of bias.
My 'observed vs expected' method is totally dependent on the expected (predicted) values being correct.
The conservation of energy principle is beyond doubt, but are we applying it right in practice? Could there, for example, be another not obvious source of energy during the reheat period that we haven't accounted for? Or have we somehow over-estimated the actual loss during the setback?
Given the same (mean) OAT, if the mean IAT overall during a period of setback running is lower than that achieved during a period of no setback running, then there should be a saving. This has always be my supposed explanation of how the saving happens, but ever changing conditions and an overall shortage of data have to date made comparing like with like a rather sketchy business, there simply aren't enough aggregate data points over a wide range of conditions. If I try to match setback days to non setback days with similar OATs, things get very sparse (note the small number of data points overall, which means wide confidence intervals, and nothing below 4°C OAT):
Nonetheless, this is what my 'matched pairs two population' analysis is all about. If I can find a way to incorporate the mean IAT data (not the delta t, that will obscure what I want to look at, which is the actual mean IAT, to see whether it is less on setback days), then that may (or may not) throw some light on the question.
Midea 14kW (for now...) ASHP heating both building and DHW
I have managed to get the data points in a scatter plot to scale according to a variable in R / ggplot, in this case using the 24 hour mean IAT as the scaling variable. This means we can get an immediate visual sense of the 24 hour mean IAT from the plot. Each data point represents a day, setback days are orange, no setback days are blue:
Now, I dare say this might be a bit of a breakthrough. If you look at the full size version of the plot, I think it is fair to say that by and large for any mean OAT below say ten degrees, the orange (setback days) points are typically smaller than the blue (no setback) days, telling us that the mean IAT on setback days tended to be a bit lower. This reduction in 24 hour mean IAT comes mostly from the setback and the immediately following recovery period, during the day, the IAT is generally where it should be.
Could the generally lower 24 hour mean IAT on setback days be the explanation for lower energy use on setback days?
Caveats:
(1) still not really a lot of data to be doing this sort of thing with
(2) my heating system has to be set up just so to achieve this: the setback long enough to save some energy use, but the system must have enough in reserve to get the IAT back to where it should be by around 0900
(3) something else is going on when the mean OAT is above around 10 degrees which needs to be explained
(4) I haven't (yet?) found a way to analyse this numerically - this plot relies on the human brain to do the analysis and interpretation.
At some point I will do the same plot for previous periods and see what comes out in the wash, but they are before 'Big Bang' and so are not like for like compared to 2025.
Midea 14kW (for now...) ASHP heating both building and DHW
Could the generally lower 24 hour mean IAT on setback days be the explanation for lower energy use on setback days?
Er dont you expect the mean 24hr temp to be lower on setback days. Thats sort of the point. The mean of interest is surely the mean IAT during the hours when you 'want' it heated.
Or am I missing something!
4kW peak of solar PV since 2011; EV and a 1930s house which has been partially renovated to improve its efficiency. 7kW Vaillant heat pump.
Er dont you expect the mean 24hr temp to be lower on setback days. Thats sort of the point. The mean of interest is surely the mean IAT during the hours when you 'want' it heated.
Yes, but I haven't previously been able to visualise this. There was a clue in this previous table, where the spring months are setback months and the autumn months are no setback months but as they are monthly means, a lot of detail in the data has been lost:
The means are a tiny bit lower here, and are more obviously so in the more detailed scatter charts, but the daytime IAT can still be where it should be. Here's the chart for the 9/10th April 2025 setback that we looked at in detail earlier, showing the overnight IAT dip which lowers the overall 24 hour mean IAT, but the day time IAT is where it should be:
Midea 14kW (for now...) ASHP heating both building and DHW
And here is the matched pairs version. The matched pairs are a subset of the previous plot's data, where I have manually (read crudely) matched a setback day and no setback day based on mean OAT (within 0.1°C), in the hope of getting closer to comparing like with like. It severely cuts down the number of observations, meaning less 'information' and wider confidence intervals, but in other ways it is clearer, with the orange (setback) days being smaller ie they have lower 24 hour mean IAT values than the blue (no setback days) but only at mean OATs below around ten degrees. Above ten degrees mean OAT, this tendency is either far less obvious, or not present at all:
What I am getting at here is the idea that we can reconcile the empirical data (I do save energy with setbacks) with the conservation of energy principles if we accept that the setback days are in energy balance, but at a slightly lower 24 hour mean IAT, caused by the setback, while at the same time the daytime IAT is most if not all of the time in the correct range.
Midea 14kW (for now...) ASHP heating both building and DHW
Er dont you expect the mean 24hr temp to be lower on setback days. Thats sort of the point. The mean of interest is surely the mean IAT during the hours when you 'want' it heated.
Yes, but I haven't previously been able to visualise this. There was a clue in this previous table, where the spring months are setback months and the autumn months are no setback months but as they are monthly means, a lot of detail in the data has been lost:
OK but the lower average IAT is precisely why some saving might be expected. So the answer to the question you pose
Could the generally lower 24 hour mean IAT on setback days be the explanation for lower energy use on setback days?
is in theory yes.
Exactly. And at the same time, the house remains in overall energy balance / steady state, in that it there is a dip in IAT overnight, but it generally recovers by mid morning or earlier, and leaves the 24 period at the same IAT as at the beginning of the 24 hour period (see previous chart above), thus satisfying the conservation of energy principle.
More by luck than good judgement, I have managed to have the heating set up 'just so', the daytime IAT is where it should be, but the overnight dip means a slightly lower overall mean IAT, and that is what provides the saving.
Midea 14kW (for now...) ASHP heating both building and DHW
Exactly. And at the same time, the house remains in overall energy balance / steady state, in that it there is a dip in IAT overnight, but it generally recovers by mid morning or earlier, and leaves the 24 period at the same IAT as at the beginning of the 24 hour period (see previous chart above), thus satisfying the conservation of energy principle.
More by luck than good judgement, I have managed to have the heating set up 'just so', the daytime IAT is where it should be, but the overnight dip means a slightly lower overall mean IAT, and that is what provides the saving.
Well of course I agree. There is a small saving in energy that you need to deliver to the house as a result of setback, which reduces the average house temperature and thus the loss. But the magnitude of that saving is modest by comparison with the 10s of percent that have previously been mentioned, and small enough that the corresponding saving in energy to the heat pump could well be wiped out by the higher FT required than when there is no setback. That's sort of where we 'came in'.
4kW peak of solar PV since 2011; EV and a 1930s house which has been partially renovated to improve its efficiency. 7kW Vaillant heat pump.
@jamespa — this thread got reactivated a couple of weeks ago after an 8 month gap of no activity with a link to a 'Protons for Breakfast' post that concluded it was inconclusive as to whether setbacks saved energy, but included a possibility that they might actually use more energy. As it was a while since I had looked at my data, I said I would do an analysis of my most recent data, and then realised I would need to go back to the spring to find my most recent period of setback running. There then followed a wide ranging discussion of different ways of approaching the analysis, and a useful reiteration of the importance of not violating the conservation of energy principle, useful because while it may be obvious to those with a physics/engineering background, it is not obvious to most folks. We then effectively ended up at the usual impasse: on the one hand the conservation of energy theoretical principle says you can't save energy over the longer term with overnight setbacks while at the same time keeping the daytime IAT stable, and on the other hand my partly empirical observed vs expected method said you could save energy.
What my most recent plots show, in a way that I haven't done before, is how these two approaches can be reconciled, at least at lower OATs. While we may have had a hunch mean IAT might have had something to do with it, previously there wasn't really, so far as I am aware, a clear statement with some evidence that setbacks can save energy while at the same time energy is conserved and comfort is not impaired (ie IAT where it should be by day), and they do that by lowering the 24 hour mean IAT, because up until now we haven't explicitly included the 24 hour mean IAT in the analysis. It does require getting the heat pump and setback settings just so, but there is a sweet spot where the setback has enough bite to save energy, but the same time the heat pump also has enough bite to recover in good time.
My most recent analysis of the entire setback period this spring is that it saved 18% of the energy that would have been used had I used continuous running with no setbacks. This estimate is still subject at the vary least to the vagaries of my way of calculating the expected (predicted) energy use under continuous running (using the regression equation from a period of no setback running), and can only be considered as just that, an estimate (and it only applies to my house in the conditions at the time etc etc).
At some time in the future when I have more time on my hands than I know what to do with I may try and find some setback and no setback days where the OAT profiles are very similar, and compare them. I can even make a stab at identifying the concurrent weather, to exclude comparing a sunny day with a cold wet and windy one. That will provide an 100% empirical way of comparing the two, with no need to estimate the expected values. The thought of endless scrolling through my data trying to spot similar days somehow lacks appeal, so I have been trying to think of a way of automating the detection of days with similar OAT profiles, but so far have drawn a blank. If anyone knows of any way of doing this sort of thing in R or python, then now is the time to speak up!
Midea 14kW (for now...) ASHP heating both building and DHW
The thought of endless scrolling through my data trying to spot similar days somehow lacks appeal, so I have been trying to think of a way of automating the detection of days with similar OAT profiles, but so far have drawn a blank. If anyone knows of any way of doing this sort of thing in R or python, then now is the time to speak up!
The OpenWeatherMap API provides plenty of historical data and for any given day gives a weather code. That would be standardised enough to classify dates according to similar temperatures and similar weather. Then joining that with your local OAT/IAT data ought to achieve what you’re after.
105 m2 bungalow in South East England
Mitsubishi Ecodan 8.5 kW air source heat pump
18 x 360W solar panels
1 x 6 kW GroWatt battery and SPH5000 inverter
1 x Myenergi Zappi
1 x VW ID3
Raised beds for home-grown veg and chickens for eggs
"Semper in excretia; sumus solum profundum variat"
@majordennisbloodnok— thanks, useful to know, I will look into it.The pattern recognition problem, how to find days with similar OAT profiles, is more of a challenge. Given a profile like this, how do I find days with the same (or a very similar) profile?
Some form of mathematically derived 'signature'? Borrow something from audio or other signal processing? An interesting challenge, if you like this sort of challenge!
Midea 14kW (for now...) ASHP heating both building and DHW
@majordennisbloodnok— thanks, useful to know, I will look into it.The pattern recognition problem, how to find days with similar OAT profiles, is more of a challenge. Given a profile like this, how do I find days with the same (or a very similar) profile?
Some form of mathematically derived 'signature'? Borrow something from audio or other signal processing? An interesting challenge, if you like this sort of challenge!
My preference would be to keep it simple. I'd say that if two days share similar maximums, minimums and averages then it's a close enough profile. The peaks and troughs may have occurred at different times but the heat pump still had to deal with them during that day, and the average on top of that means, within those extremes, the curve was similar enough to be comparable. Add in the weather code from the API and that should give enough for a composite key to compare between days.
105 m2 bungalow in South East England
Mitsubishi Ecodan 8.5 kW air source heat pump
18 x 360W solar panels
1 x 6 kW GroWatt battery and SPH5000 inverter
1 x Myenergi Zappi
1 x VW ID3
Raised beds for home-grown veg and chickens for eggs
"Semper in excretia; sumus solum profundum variat"
