Make of this quick and dirty analysis what you will. Actually, it wasn't that quick, because it involved a lot of copying and pasting. First of all I scanned the whole data set plot for 7 day periods where the OAT was both reasonably stable and at the same time not extreme. It doesn't happen that often. I then selected four seven day (noon to noon) periods where these conditions were met for periods when I did have a nightly setback, and four periods when I didn't have a nightly setback. At this point I had eight data sets of real data, four from setback periods, and four from no-setback periods. All the data is real world, there are no predicted values.
I then averaged the OAT and IAT for each period, and summed the energy in, and tabulated the results. Again, this is real world data, not whiff of whatiffery in sight. This is what I ended up with:
The groupings are arbitrary: I chose to sort by average OAT and and groups similar values together. Because it comes from real world data, it is a bit all over the place, and I can't yet think of a way to present it visually in a way that draws out whatever it is telling us.
The first pair shows less energy use with the setback, but the OAT was a bit higher, and the IAT a bit lower - but enough to reduce the energy in by 32%?
The second pair again shows less energy use with the setback, but this time the OAT favours the no-setback period, and the IAT means are closer - yet the setback period still uses 20% less energy that the no-setback period.
The last four rows are less easy to compare, but I do see the no-setback periods both used more energy than the setback periods, despite the former having slightly higher OATs. But the IATs are also higher...
As ever, this is data from one house in one location, and it represent an investigation of the data, in an attempt to try and understand it better. It is absolutely not yet ready to be generalised to any house in any location.
Midea 14kW (for now...) ASHP heating both building and DHW
Back to the unanswerable question that imo cant be universally answered. I have tried many times and setbacks for my house 100% work for my comfort level. It would be great if continuous running was cheaper, i could set the heat pump and forget about it. We just do not get enough continuous cold weather to just set it and leave it. If you want exact same temperature of say 22deg IAT 24/7 then yes continuous running is only option but most people are happy enough to lose couple degree over night for me to 19/20deg and reheat again after set back and have nice warm house for when they are sitting down in the evening time. At current emps i can run for approx4-5 hours and cost me approx10kwh. Running 24/7 would cost at least13kwh, tried different scenarios last winter and for MY house and location setbacks and using fixed flow temp of 29degworks. I encourage all heat pump users to try setbacks and fixed flow temp. I know i have only my anecdotal evidence and not charts and graphs but i think last winter there was good few anecdotes from users about significant savings with setbacks.
This post was modified 6 months ago 2 times by newhouse87
Make of this quick and dirty analysis what you will. Actually, it wasn't that quick, because it involved a lot of copying and pasting. First of all I scanned the whole data set plot for 7 day periods where the OAT was both reasonably stable and at the same time not extreme. It doesn't happen that often. I then selected four seven day (noon to noon) periods where these conditions were met for periods when I did have a nightly setback, and four periods when I didn't have a nightly setback. At this point I had eight data sets of real data, four from setback periods, and four from no-setback periods. All the data is real world, there are no predicted values.
I then averaged the OAT and IAT for each period, and summed the energy in, and tabulated the results. Again, this is real world data, not whiff of whatiffery in sight. This is what I ended up with:
The groupings are arbitrary: I chose to sort by average OAT and and groups similar values together. Because it comes from real world data, it is a bit all over the place, and I can't yet think of a way to present it visually in a way that draws out whatever it is telling us.
The first pair shows less energy use with the setback, but the OAT was a bit higher, and the IAT a bit lower - but enough to reduce the energy in by 32%?
The second pair again shows less energy use with the setback, but this time the OAT favours the no-setback period, and the IAT means are closer - yet the setback period still uses 20% less energy that the no-setback period.
The last four rows are less easy to compare, but I do see the no-setback periods both used more energy than the setback periods, despite the former having slightly higher OATs. But the IATs are also higher...
As ever, this is data from one house in one location, and it represent an investigation of the data, in an attempt to try and understand it better. It is absolutely not yet ready to be generalised to any house in any location.
Interesting data.
Assuming a difference between the intake air sensor and the actual OAT of 2C, and that 600W is contributed from sources other than the heating system, both of which are reasonable assumptions given the figures you have presented, I cant find a meaningful correlation with SB/No SB given the very large scatter of the data. Even if you set both of these assumptions to zero, the best 'performance' (W/C) is the week of 12/3/2024, when there was no setback.
My suspicion is that the conditions preceding may matter, it might sensible to ignore the first two days if the periods were otherwise stable and of similar temps. Or there may be another factor at play. Its perhaps relevant that the season end figures are roughly consistent, and the centre season figures are roughly consistent, but the two groups are not consistent with each other.
Back to the unanswerable question that imo cant be universally answered. I have tried many times and setbacks for my house 100% work for my comfort level. It would be great if continuous running was cheaper, i could set the heat pump and forget about it. We just do not get enough continuous cold weather to just set it and leave it. If you want exact same temperature of say 22deg IAT 24/7 then yes continuous running is only option but most people are happy enough to lose couple degree over night for me to 19/20deg and reheat again after set back and have nice warm house for when they are sitting down in the evening time. At current emps i can run for approx4-5 hours and cost me approx10kwh. Running 24/7 would cost at least13kwh, tried different scenarios last winter and for MY house and location setbacks and using fixed flow temp of 29degworks. I encourage all heat pump users to try setbacks and fixed flow temp. I know i have only my anecdotal evidence and not charts and graphs but i think last winter there was good few anecdotes from users about significant savings with setbacks.
You have a house with an enviously low loss and able to run at an enviously low flow temperature (29C!). I wish mine was like that!
At a flow temperature this low the usual arguments against part-time operation weaken considerably. These arguments are based around the fact, which comes from the thermodynamics, that heat pumps operate more efficiently when run at a lower flow temperature. However once you get below 30 or thereabouts, the effect is much smaller because the increase in flow temperature needed to get the amount of energy into the house in a shorter period is relatively small, so the efficiency penalty is correspondingly small. The same argument means that WC is much less important at low flow temperatures than it is if you are operating at 40+.
Once the 'thermodynamic' effect becomes small then other factors, for example the load of the circulating pump and the (largely unquantified) effects associated with cycling, which of course will be worse if you run 24*7 than if you run part time.
Based on the combination of experimental results posted and theory, my gut feel (supported in very general terms by a combination of the theory and experiment) remains that for a fair proportion of houses, perhaps the majority, a modest set back where you don't have to raise the flow temperature materially to recover 'in time,' is very likely to save some money but nowhere near 30%, perhaps more like 10%. However it should only be attempted having first set the heat pump to run as efficiently as possible 24*7 and after tuning the WC curve. This takes time, weeks or a whole season. If, when setback is introduced, you don't have to jack up the WC curve to remain comfortable then its almost certainly going to save some money. If you do have to jack up the WC curve then you quite likely need to cut back the setback time.
There are definitely a fair number cases where setback may perform very differently however. many heat pumps which start off poorly set up (or which are controlled by thermostats not weather compensation) are likely in this category. More or less anything could happen, but what is quite possible in such cases is that you end up 'trapped' in a situation where setback does save money relative to the situation precedent, but more money could have been saved by better adjustment of the pump controls, however that is no longer accessible unless you first reverse the setback.
Another situation, of which I think yours is probably an example, is where the demand is sufficiently low that factors other than the raw efficiency of the pump itself dominate.
There are also at least three human factors in play:
Heating systems run ~24*7 at low flow temperature (whether from a fossil burner or a heat pump) are much more comfortable than the 'old' high temperature heating that most of us are used to. I have a feeling this may mean we can tolerate lower room temperatures (I'm pretty sure I do). Its entirely plausible that some set up their heat pump to run at the room temperature they are used to, then do a setback, and find that they are still comfortable even though the room is in fact colder. In this case they would most likely be better off (financially) reducing the flow temperature and reducing the setback.
Most of us do prefer it cooler at night, so a modest setback adds to comfort
Most of us want to save money, so can fall victim to confirmation bias
I cant find a meaningful correlation with SB/No SB given the very large scatter of the data
I wasn't looking for correlation, just gross eyeball differences between SB and no SB running. As I said, I can't think of any other way of presenting the data other than in a table, and, given the very large scatter of the data, I can't see that any meaningful statistical tests can be done to compare running modes. Nor am I sure that the concept of 'performance' has any meaning given the variety of conditions and spread of the data. Instead, I am asking a very simple question: given two similar-ish weeks, one with SB and one without, how does energy use compare?
The first problem is that similar-ish is inevitably rather loose. These are real world observations, and I have no control over what Mother Nature does, a problem I am only too familiar with as it happens all the time in epidemiology. Data from the real world contains random variation, measurement errors, and a host of factors and variables both known and unknown that influence the actual values recorded in the data.
Let's try another approach: adjusting one row to make it as if it was operating in the conditions in another row, thereby making them more comparable. We have established many times that energy in is very dependent on OAT, and that makes sense, because that is how a heat pump works when in weather compensation mode. We are also pretty sure that a building's heat loss is largely determined by the difference between IAT and OAT (delta t). How about we do a regression of energy in on IAT/OAT delta t and use the regression equation slope to adjust one of the rows in my previous table to make it as if the delta t was as in the other row? This what we get:
What I have done here is adjusted the SB row in each pair to make it as if it was running at the same IAT/OAT delta t as the No SB row (I stuck with the OAT as it is because adding an AIT correction makes no difference to the end result). The slope column is the regression slope for each row's data, energy in against IAT/OAT delta t. In the top pair, the heat pump had an easier time of it in the SB row (smaller IAT/OAT delta t), meaning we need to increase the kWh in for that row, to make it as if it was running at the No SB delta t. In the bottom pair, the opposite is the case, the heat pump had to work a bit harder in the SB row (delta is a bit larger), meaning we need to decrease the kWh in that row to make it as if it was running at the lower No SB row delta t. In both cases, even after adjusting the SB row to make it as if it was running in the same delta t conditions as the No SB row, the SB row uses less energy.
Incidentally, the slope column also gives an empirical estimate of the energy saving achieved in my setup for each degree by which the IAT/OAT delta t and so by implication the IAT is lowered: between 15 and 20%. Not trivial...
By the way, here is the latest 24 hour plot for my system. It is operating in marginal conditions, warm steady OAT, which is only marginally affected by the heat pump coming on for most of the time, though there was more variability between 0600 and 0900 this morning. In these conditions I tend to get suspiciously high 60min trailing COPs. The big spike on the left is the DHW heating coming on between 1300 and 1400.
Midea 14kW (for now...) ASHP heating both building and DHW
What I was hoping was that, if I consider the results (energy consumed) with no SB as one 'population', and the results with SB as a second 'population', that within any one population there was a scatter which is substantially smaller than the variation between the populations. If that were the case then it would suggest that they are in fact two separate populations and thus SB vs no SB can be shown by this data to make a difference.
Your method to compare a result with SB and a result without SB is sound so far as I can see, but (in the absence of a large number of samples) does depend on the scatter within each population being smaller than the difference between them. That is not the case in the raw data. I think its obvious why, namely that the key variables are insufficiently controlled, for the simple reason that we cant control them (or at least not all of them).
So the next step is either to control the important variables or to to correct as best we can for them (alternatively we can collect lots more data and see if the important variables are sufficiently random that we can use averaging to take out their effect, but that's rather tedious).
There are in fact many uncontrolled variables which might matter but the two obvious ones are IAT and OAT, and we know how to correct for those as energy loss is proportional to the difference. We also know that the sensor measures AIT not OAT, and that there are other energy sources in the house, so we can attempt to adjust for these as well. That's what my two tables do and what I was hoping for is similar figures in the final column for the members of any one population, and ideally a difference between the two. Unfortunately this si not what we find, there still are not two distinct populations according to sb vs no sb. So we still cant establish from this data what difference setback is making. Interestingly, as a possibly relevant aside, there does possibly appear to be two distinct populations based on end of season and mid season).
So my next step is to think about what other variables there are which we are not controlling. The key one which comes immediately to mind is the the energy stored within the fabric. This will be affected principally
the conditions precedent. If it was cold in the period immediately prior to any given experimental period then the amount of energy stored in the fabric at the start will be different to the amount stored if it was warm immediately prior to our 'experiment'
the IAT (and possibly OAT) at the beginning and end of each experimental period.
If these are not the same at the end of any given experimental period as they were at the beginning, then the energy use is distorted by this stored energy.
The conditions are known, the challenge is how to correct for them. We basically need to know the heat capacity of the house which can in principle be determined by observing the fall in temperature after the heating is turned off (albeit that the non-monolithic nature of the house means that any determination by this method is an approximation). This is why I asked about the conditions prior to the start of each period and should have asked about IAT and OAT at the beginning and end of each period.
In summary I don't think the data currently tells us about the difference between SB and no SB, but with some more information which is presumably available, it still might., as might thinking a bit about why mid season and end of season appear to be so different.
I hope that's a clearer explanation.
4kW peak of solar PV since 2011; EV and a 1930s house which has been partially renovated to improve its efficiency. 7kW Vaillant heat pump.
Thank you, it is. Your second paragraph also covers something I had been thinking about as well, the situation where a true but small inter-sample difference gets lost in intra-sample variation.
The basic problem we have is controlling for, or adjusting for, different OAT/IAT conditions. My latest effort is yet another example of the observed vs expected methodology I have used before: adjust one result as if it has the same conditions as the other, and then compare the adjusted (expected) result with the observed. Another option, had we more data, is stratification, in this case by OAT. As time goes by, I will have more samples, but that option is still going to take a long time.
One thing we definitely have in abundance is the rate of cooling of the house when the heating is turned off, because that is what happens every time there is a setback, albeit only six hours of cooling, but we may be able to get something from it that tells us about the buildings heat capacity. The fact the setbacks happen at night is a good thing, there are fewer disturbing factors in play, still house, sleeping occupants, apart from the first hour or so of the setback.
Antecedent IATs and OATs are also available for each observation period. Typically these are not wildly different. This chart (only two months to keep it readable) shows four of the periods I used. You can tell whether it was a SB or not period from the saw tooth pattern on the IAT (saw tooth = SB). Note the gradient of this fall is emphasied in this tightly packed 2 month long plot (see the plot for one day I posted earlier, which also makes reading the rate of cooling during a setback more easily read). The chart can also serve as a list of the variables I collect and so are available. Note that all the raw data are collected at minute intervals, but the energy values are plotted as trailing 60 minute values.
Of course, if the Met Office forecasts really were any good, all this would be a doddle. Just wait for a two week forecast of steady settled weather, then run with SB for the first week, then No SB for the second week. Unfortunately, Met Office medium term forecasts are still master classes in the art of waffling:
"Wednesday 13 Nov - Wednesday 27: Nov After a relatively settled start to November, around mid-month there will probably be a change toward more unsettled conditions for a time. This means an increased risk of periods of wet and windy weather for parts of the UK, perhaps more so in the south. However, there is low confidence whether unsettled, wetter weather or drier and more settled conditions will dominate by the end of the month. Temperatures will probably be close to average overall, although some colder interludes are possible. Updated: 15:00 (UTC) on Tue 29 Oct 2024"
Nonetheless, 'After a relatively settled start to November...' has not escaped my notice.
Midea 14kW (for now...) ASHP heating both building and DHW
A quick eyeballing of one of your earlier weekly graphs (and assuming that your house loss is ~8kW @-2C) suggests that the heat capacity of your house is ~10kWh/C, but this is an attempt to estimate slope from overlapping lines on a screen, so isn't likely to be very accurate! I looked back at some studies I did of your data in Dec 2023 and, by some curve fitting, got to 17kWh/C for IAT changes, and 2kWh/C for OAT changes (the presumption being that the temperature gradient through the fabric changes if either changes). Both figures were subject to very considerable uncertainty (+/-50%) and both are purely empirical as a way to get best fit between total energy to house and sum(IAT-OAT) over the same period.
All of these estimates of heat capacity are of the same order as the scatter between the figures in your table so its possible that it might explain in part some of it. I think that, since Dec 2023, some changes have been made to your analysis so I wouldn't want to rely on what I did back then other than as an indication of the magnitude of the heat capacity.
From data you posted back in 2023 I see that the AIT on 3rd December (which preceded the week at the high end of the energy scatter) was 6C until about 10am, when it started warming up to nearer the 8C average of the observation period, and the IAT during most of 3rd December was 17.2. Given that the mean IAT during the run starting 4th December was 19C, this difference (plus the OAT difference) could easily account for 30-40kWh (on top of the loss) that needed to be pumped into the house to get it to where it ended up. That's quite a significant proportion of the scatter, albeit not all of it. in summary what I think this tells us is that conditions precedent and changes in IAT/OAT from start to finish of any period may well matter in terms of our analysis, unless we are prepared to collect lots more data so that they 'average out'.
To be honest I wouldn't like to go any further without looking at actual numbers. Daily figures are sufficient, what we really need is IAT, OAT at the same time of day (preferably a time of day when not much is changing) for many weeks during the heating season (daily averages will probably do as an alternative), degree days for the day or average oat, total heating energy to the house for each 24hr period and total energy to the heat pump for each 24hr period, plus a setback/no setback flag, ideally all aligned to the same 'day ends'. Hourly figures would allow a more accurate estimate of sum(oat-iat) to be calculated which would be good, anything finer grained is too much data. I am a little doubtful about spending too much time on analysis whilst we are reliant on the AIT sensor as a proxy for OAT, as we know that it has a somewhat variable offset to OAT (and in particular the offset changes during the event we are interested in). I don't know if there is a weather station near you (and which is likely to be representative) from which both hourly temperatures and daily average temperatures are available, but if so then bringing in this data as well will help us understand whether we can reasonably rely on the AIT sensor to tell us about OAT to a sufficient accuracy. Of course there is absolutely no obligation to do any of this and it may well not tell us much anyway. Unfortunately I cant currently think of a way to shortcut compensating for the key variables that we know about (and know to be material), without collecting vastly more data than is comfortable.
This post was modified 6 months ago 2 times by JamesPa
4kW peak of solar PV since 2011; EV and a 1930s house which has been partially renovated to improve its efficiency. 7kW Vaillant heat pump.
Unfortunately I cant currently think of a way to shortcut compensating for the key variables that we know about (and know to be material), without collecting vastly more data than is comfortable.
I do already have a lot of data. Since late March 2023, I have been collecting data at minute intervals for all of the variables you mention, apart from the true OAT, but I think we can make a reasonable stab at estimating that from the AIT. The nearest weather station to me with available online data is about five miles away, but it is in a different terrain, and I am not at all convinced it will provide a better estimate of my OAT than adjusting my AIT. I can run some checks. The weather station data is also hour rather than minute based, and I need to check its continuity, I think it may have gaps in the data. It is also an amateur (WOW) station, not that that is a bad thing (I'm an amateur too!), but the nearest official Met Office station is over 12 miles away, and on the coast, definitely not representative of here. The only other thing of note is that prior to 19 Sep 2023 the IAT came from a stand alone hourly data logger in the kitchen, after that date it was collected at minute intervals over modbus from a sensor in the dining room. The kitchen and dining room are next door to each, and the IAT feels very similar, but comparing the two sensors when they were both running, the dining room sensor is usually a little warmer than the kitchen sensor (mean 0.64 degrees C warmer, range -1.4 to 2.0). I don't know whether that is actually true (both are correct), or measurement error, but it may be better to limit data analysis to after 19 Sep 2023. Even with that limit, we have the whole of the 2023/24 heating season, and the start of the current season.
All of the above data is collected by a single python script that is timed to collect the data at minute intervals. It is very reliable, and even restarts itself after a power cut. Another script then takes the minute data, and aggregates it (a) over the last hour and (b) over the last 24 hours, and then adds these aggregates to two other data files, one for the hourly data, the other for the 24 hour data. This aggregation happens at one minute past the hour, the aggregates being means for temperature variables and sums for the energy variables. The energy variables are calculated from minute values, amps x volts for energy in, and flow rate x LWT/RWT delta t x circulating fluid specific heat. Not all of the minute variables end up in the hour/24 hour data files, but I think all the important variables are available in the various files. All of the charts and tables that I post come one way or another from this data.
Finally, we have the wonderful 'q text as data' program that makes it possible to do sql like queries on csv files. This opens up all sorts of possibilities. Lets say I want daily (24 hour) values noon to noon for OAT and heating energy in. This line of code does the trick, and copies the results to the clipboard:
Z:\modbus>q -H -d , "select datetime,amb_24h_mean,htg_24h_kWh_in from midea_24h_data.csv where datetime like '%T12:%' order by datetime desc" | clip
In plainer but not necessarily plain English, collect the OAT and heating energy in from the 24 hour data file for noon every day (the datetime is in the format YYYY-MM-DDThh:mm:ss, thus the noon entries will have the substring T12 in them), order the rows by datetime descending, and copy them to the clipboard, all ready to paste into whatever program I want to use for further analysis. Using similar 'where' clauses I could get all the rows for Jan 2024 (where datetime like '%2024-01%') and so on. Very fast, and very useful.
All of which is to say you can have pretty much any data you want, as long as it is contained in the above. Yes, I know that is a bit of a 'any colour you like as long as it is black' statement, but the fact is there are a lot more colours than black in my data.
Midea 14kW (for now...) ASHP heating both building and DHW
To keep things simple for an analysis I think we should stick to months when heating was definitely on, which I presume are October to March. That said, certainly where I live, there are days in both October and March when heating isn't actually necessary so, depending on what its like around you November-February might be 'safer'
It would also be good to avoid having to deal with GMT/BST so, unless you record solely in GMT, I suggest that only the period between clock shifts are included.
Since you do hourly aggregates (and since the daily aggregates, assuming they are calendar days, will 'split' a setback period and thus cause unnecessary problems) lets work with the hourly aggregates.
What would be good would be the hourly:
AIT*, IAT, Energy to house, Energy to heat pump
from 30th October 2023 - 30th March 2024 (or whenever in spring 2024 constant heating stopped)
and, if possible, a setback flag, although that can presumably be deduced from the data
*lets work with AIT for now as its what we have. I wonder if the adjustment should be something like 2C when the heat pump is running and 0C when it isn't, which would not be difficult to build into excel.
At one point you applied a 18% (?) correction to one of the energy variables, is that resolved for the whole of the last heating season?
My first step will be to see if how well I can get the energy to house to correlate well with IAT-OAT, by adjusting an figure for heat capacity. If this doesn't correlate well, there is something else that is significant affecting the basic energy balance of the house (or it needs to be modelled as something more complex than a monolith), which needs to be worked out!
Its worth a shot. It may well fail but, by the sound of it, isn't too difficult for you and I don't think its too difficult for me as I have developed most of the required excel tools at some point in my journey.
4kW peak of solar PV since 2011; EV and a 1930s house which has been partially renovated to improve its efficiency. 7kW Vaillant heat pump.
Struggling to find a reliable heat pump installer? A poor installation can lead to inefficiencies and high running costs. We now connect homeowners with top-rated installers who deliver quality work and excellent service.
✅ Verified, trusted & experienced installers
✅ Nationwide coverage expanding
✅ Special offers available
