I cant believe I didnt do something similar at some time with your data.  Looking back I did, this is 19/9/23-3/12/23, the units will be degree-minutes (IAT-OAT) and kWh.

I tend to agree they are probably the same population.

However this does not (I think) mean that there is no (apparent) setback saving (which is likely why I didn't make a fuss about the finding at the time) - the whole point of setback is that the average IAT is lower.  However, what it potentially does mean, is that the setback saving is close(ish) to what one would expect solely because of the lower average IAT, rather than the lesser value one might expect because of the offsetting of this by the loss of thermodynamic (COP) efficiency. 

The more I think about this the more attracted I am to the thought that the theory predicting that setback will save very little or indeed increase consumption with a perfect heat pump is likely correct, but in practice all heat pumps cycle some of the time and a sizeable proportion cycle a lot or even all of the time (most particularly when people are likely to operate a setback).

We know that there is a thermal penalty due to cycling, and its also likely (I would say inevitable) that setback reduces cycling.  So in practice, in many cases, we are comparing a heat pump that is cycling (in the no setback scenario) with a heat pump that is cycling less or possibly not at all, in the setback scenario.  Both incur a thermodynamic COP penalty for the exact same reason, and it wouldn't be all that surprising if the penalty were about the same, or at least similar.  This then partially or completely negates the COP difference between the two cases, leaving (at least a part of) the difference in heat loss from the house as an actual saving in consumption.  

There is no way I will be even attempting to model this until someone else has created a model of cycling.

Slicing and dicing data can sometimes be fun, but it's often frustrating.

I often start by asking "what do I think i'm looking for".

So..."does set back save money?" - Yes.  You can see that in the data.

How about "does it save money all the time?" - not sure.

The pink dots are generally lower than the blue dots at the same IAT - OAT.  But the difference seems to get less as IAT - OAT increase - although there isn't much data to make that assessment.  I've drawn some lines in the data and posted below.  I think the lines cross about 15C.

What does that mean?  It could mean that when the IAT - OAT is large, running with a setback requires more energy to get the building back up to temperature.  Which is a theory that's been discussed before.

"The lines not straight".  Humm....Q = UA. (IAT - OAT), so Q / (IAT - OAT) should be a straight line, but it's not.  That means something else is affecting the heat flow out of the building.  It could be that UA is a function of IAT & OAT, which would make sense if you think about convection from hot surfaces.

I would add a trend line to each group of data and see if those lines cross.  Then you could differentiate the equation of the line to work out the gradient, which would actually be 1/UA.

I would also experiment with different types of trend line, if a polynomial line is the best fit that means UA is a function of IAT & OAT.

Regards

Bob

Posted by: @jamespa

↑

I tend to agree they are probably the same population.

It's possible the extra noise makes them appear to be from the same population (they're samples from population(s), not the entire population) when you include the IAT component but if you just use OAT (which isn't unreasonable) there is a hint of two sample from two different populations, setback and no setback (posting scatter plot again here for ease of reference): 

Still thinking more widely about methodology.  

Posted by: @bobtskutter

↑

"The lines not straight".  Humm....Q = UA. (IAT - OAT), so Q / (IAT - OAT) should be a straight line, but it's not.  That means something else is affecting the heat flow out of the building. 

Briefly for now as after a long day I need some supper - I find it all to easy to forget whether we are talking about energy in or energy out, because we talk about both. For empirical heat loss assessments, we want energy out, and that should be a straight line, but in this post, where we are looking at costs and potential savings, we are looking at energy in. My hunch is this isn't linear, because performance (efficiency) drops as OAT falls and LWT rises. Put simply, doubling the output will need more than double the energy in.  

The topic and data is beyond my ability based on LG controller. A couple of observations:

1. I used setbacks a little for a couple of years when I was on Octopus Tracker and feel it "worked" in terms of kwh and efficiency - I think I had a setback of 1 deg from 9pm until 2am. More setback took too long & worked the heat pump to heat back up.

But the real issue for me was moving to cheaper overnight ToU tariffs... last year Agile... now Intelligent Go. This completely changes the landscape.

2. To the extent that I've changed to using higher temps at night. What do we call that? Opposite of setback. Setup?! With 7p/kwh overnight even if colder temps and lower COP .. trying to heat the house +1deg overnight so that daytime demand is lower. During cold weather I temporarily raised the weather compensation curve overnight too for same reasoning.

Not sure if that's useful to this discussion!

@cathoderay at constant room temperature, energy in = energy out

Q = UA.DT tells you how much energy flows through an object, in this case a wall, with a temperature difference of DT.

If your room temperature stays the same, the energy lost through the walls is equal to the energy from the radiators, which is the energy the heat pump puts into the circulating water.

Bob

Posted by: @cathoderay

↑

here we are looking at costs and potential savings, we are looking at energy in. My hunch is this isn't linear, because performance (efficiency) drops as OAT falls and LWT rises. Put simply, doubling the output will need more than double the energy in.  

I agree.

Posted by: @bobtskutter

↑

If your room temperature stays the same, the energy lost through the walls is equal to the energy from the radiators, which is the energy the heat pump puts into the circulating water.

I agree (how could I disagree!) and its the basis of any of my arguments and the occasional attempt at simulation.

Posted by: @bobtskutter

↑

If your room temperature stays the same, the energy lost through the walls is equal to the energy from the radiators, which is the energy the heat pump puts into the circulating water.

Like @jamespa, I completely agree, it is the basis on which my empirical heat loss assessments are based, which have been covered in many posts. But in this thread we are not so much interested in the energy out (ie delivered to the house, and equal to the heat loss when the IAT stays the same), as the energy that needs to be put in to get that energy out, because it is the energy in that determines what we pay, and so directly affects whether setbacks save money, have no effect, or actually cost more per day. Setbacks obviously save money when in operation, it is what happens afterwards during the recovery period that matters, both in the increased energy use to achieve the recovery, and in whether the recovery is adequate. Without some sort of a boost, my heat pump can take 12 or more hours to get back to my desired IAT, so the question becomes how much extra energy do I need to put in during the recovery boost to maintain comfort levels.

At the moment my heat pump isn't doing too badly, using my auto-adapt script to boost the LWT when the actual IAT is below the desired IAT: 

My desired IAT is 19°C. At 2000 yesterday evening it was 19.4°C, and then fell during the setback to 16.5°C at 0300 (setback end). By 0700 this morning it had got back to 18.1°C, not bad. I have however modified my auto-adapt script to give an even bigger boost should the IAT fall 3°C or more below the desired IAT (in my context, fall below 16°C).

The other thing is energy in. Last night, before the setback, I was using about 2kWh per hour, which might suggest a saving of 12kWh during the setback. After the setback, the 2kWh doubled to about 4kWh, but at least some of that increase is due to the fact the IAT is lower. The challenge is to determine how much of that 4kWh is extra energy over and above what I would have used during the recovery hours, had I not had a setback. 

Methodological note: the fundamental problem is we can't do controlled experiments, ie put two identical houses in two identical glass boxes with identical conditions, and run one with a setback, and the other without. Even if we could, I am not sure how useful the result would be, because the controlled experiment is done in very abnormal conditions. This is something we recognise in medicine, when we talk about efficacy (results from a randomised controlled trial) and effectiveness (real world results), with the two often very different. Instead, we have to do so-called natural experiments, or perhaps more accurately observational studies, where we observe what happens in the field, and grab opportunities to compare results. The problem is the field has a mind of its own, and keeps on changing things - perhaps one night I had a salad for supper, then next I had a casserole which spent three hours in the oven, perhaps today it is sunny and the air is still, while yesterday was wet and windy.

Nonetheless, one branch of medicine, epidemiology, routinely has to deal with these problems (inability to control what happens) and has developed methods to manage them. Consider something as basic year on year mortality. How do you know if it is getting better or worse? Or two countries - which has the worse mortality, given one country has a young population, and the other an old population? The latter will have more deaths, but is that solely because there are more old people? Might the country with the older population (say a developed nation with good living standards and health care) even have lower mortality across all age bands than the younger population (say a developing nation with poor living standards and rudimentary health care), and yet still have a higher overall mortality, just because there are so many more old people? I have tried to apply some of these epidemiological methods, notably the idea of comparing observed values with expected values, to the setback question. The idea is disarmingly simple: compare the observed values (of energy in) which we have given a setback with the expected values (of energy in) had we not had a setback. The devil is in the detail: how to estimate the expected value. As it happens, the OAT is a pretty good predictor of energy in - hardly surprising, because that is weather compensation does - and so I can get a good enough estimate of the expected value, and the results (from such analyses done in the past) do suggest setbacks save energy/money, albeit to varying degrees, and perhaps more modestly than might at first appear to be the case.    

Posted by: @tim441

↑

But the real issue for me was moving to cheaper overnight ToU tariffs... last year Agile... now Intelligent Go. This completely changes the landscape.

The thing is the vast majority of homes don't use TOU tariffs - see this rather unreadable OFGEM chart from here (scroll down to 'Number of domestic electricity customer accounts by supplier (excluding pre-payment customers): Standard variable, fixed and other tariffs (GB)'): 

Around 80% of homes are on standard variable tariffs, and most of the remaining 20% are on fixed tariffs, with TOU tariffs being a tiny minority of the whole. I strongly suspect TOU tariff users are hugely over-represented on this forum, out there in the real world folks are stuck, whether by choice or necessity, with the same old unchanging landscape of standard variable tariffs with quarterly sharp blows to the back of the head price announcements by OFGEM.

Not everyone wants 'smart' tech just because it is there. I for example don't have a 'smart' meter (and I'm defo not in a hurry to get one, so no TOU tariff for me). In fact, I don't even have a 'smart' phone. Increasingly this defines me as a weirdo, and it raises some interesting questions about creating de facto second class citizens, for example, those who choose, for whatever reason, not to have a 'smart' phone. I don't think historically we (ie the UK) have ever made it a requirement to have a sinister gadget in your hand at all times to allow participation in society. For example, when most people had landlines, those that didn't could always use the postal service. Nowadays, increasingly, there are no alternatives to to the 'smart' phone route. I suspect we are already very close to that situation with border controls. There is a world of difference between a printed 'dumb' passport, and a 'smart' and at the same time sinister gadget that probably knows more about you than you do, and increasingly sends all that lovely data to who knows where. Some folk may want to live with their head in the cloud, in more ways than one, but I for one am not ready to join them.

Much as I dislike smart phones (and the list of reasons is long, and for me the disadvantages outweigh the advantages), I am absolutely not a technophobe. I am perfectly content to set up a DIY heat pump monitoring system using modbus communications and python scripts. But I do it on my terms, in my home, and it stays there.     

I'm still on the case, collecting data, but the last 48 hours has been 'interesting', given the wide variation in OAT, the significant drops in IAT during the setback period, and the energy used during the recovery. Please bear in mind I have my auto-adapt script running, which increases the LWT when the actual IAT is one or more degrees below the desired IAT (currently set at 1 degree boost for 1-1.99 degrees below, 3 degrees boost for 2-2.99 degrees below, 5 degrees boost for 3 or more degrees below, the latter being shall we say non-trivial). The lower chart also shows how much COP varies with OAT or perhaps more accurately with set LWT. @jamespa, you might want to note the changes in OAT (AIT) at the start and end of the setback. I do have the independent OAT logger running so we will also have the true OAT once I download the data.

I thought we’ve been here…. It’s fact isn’t it?

https://renewableheatinghub.co.uk/forums/postid/27514

Posted by: @sunandair

↑

It’s fact isn’t it?

Possibly. Or possibly not. We still don't have a cast iron empirical methodology that all can accept as giving a definitive answer, robust enough to trump theoretical objections. For sure, both your data and my data suggest savings are possible, but we are plagued with three main problems. Firstly, if we get the saving by merely by having a lower mean 25 hour IAT, how do we demonstrate that that lower IAT does not come with a penalty, ie IAT too cold for some of the day? Both of us have failed to do that. Secondly, how do we do a like for like comparison when by definition the two states (setback and no setback) are by definition not like for like? On paper, the answer is to have two identical homes sitting next door to each other, with identical heating setups, one running a setback, the other running continuously. But we can't do that, so we have to work out how to adjust our data so as to approximate a like for like comparison. My observed vs expected approach is probably the closest we will ever get to doing that. And thirdly, we have the n=1 problem, and even worse, even within n=1, things don't stay the same. Take my system over 24 hours, firstly on a mild steady OAT, and then with a wide OAT variation, including some below zero OATs, along with defrost cycles. Here is the first scenario (charts are noon to noon so setback sits in the middle):

Before the setback, I was using ~1.5kWh per hour. This suggests I may have saved 9kWh during the 6 hour setback. But use is higher during the recovery, about 2kWh per hour for the first 5-6 hours, suggesting the recovery boost (triggered by my auto-adapt script) uses an extra ~0.5kWh per hour, say 3kWh in total, net saving over 24 hours is 6kWh (say 30kWh per day rather than 36kWh per day). The IAT dropped to 17°C during the setback, but recovered to 18.3°C by 0700, a 'comfort pass' for me. This is all pretty straightforwayd, and I doubt many are going to object to this assessment.

Now take the second scenario, where the OAT changes substantially, and gets cold enough to trigger defrosts:

Here things are more complicated. For a start, how do I know what I would have used, had I not had the setback? I can in fact estimate this from the hourly OAT vs hourly energy use scatter plots, but there is a problem, my OAT isn't in fact the 'true' OAT, it is the heat pump AIT (air intake temperature). Notice how during the setback the OAT shows a step rise at the start and then fall at the end. During the setback, it probably is showing the 'true' OAT, because the heat pump isn't cooling the ambient air, but when it is running, it does cool the ambient air. But, at the same time, when the heat pump is running, it responds to the AIT. For this reason, I think it is reasonable to say the AIT during the setback was around 1°C, and from my scatter plots that means I would have expected to use about 3.5kWh per hour, or 21kWh during the six hour period, whereas I observe I actually used 0kWh, meaning a saving of 21kWh. 

Now comes the complicated bit. Visibly, I used more energy during the recovery boost, but the OAT was also lower, so I would expect to use more energy anyway. How to estimate the extra energy, to achieve the recovery boost? Well, I can estimate the expected energy use for the given OAT (-1°C) from the OAT vs energy use scatter plot, and it is around 4.5kWh per hour, or 27kWh over 6 hours, had I not had a recovery boost running. The actual (observed) energy use over those six hours was 27.2kWh, which suggests no extra energy use, over and above what i would have used, because of the lower OAT. Net saving for the 24 hour period: 21kWh.

Definitely not trivial, but is it credible?

Besides the TOU tariff issues, where we may save usage at one rate and then use more in recovery at a cheaper rate, I find an overnight setback is most beneficial on days where there is a large OAT swing. For example, if temps drop to 0C overnight during the setback and then rise substantially in the morning during the recovery period, we are saving usage when the heat pump is least efficient (it's cold outside and we may have defrost cycles) and are working harder for recovery when the temperatures are warmer, so more efficient. Compare this to days where the OAT is near constant, and such advantages are substantially diminished. I can see this in my daily COP figures, as the COP is higher as a result of not having to run for 8-10 hours when it's colder outside.

To illustrate this point, the manufacturers data for my heat pump shows a COP of 3.86 for a flow temp of 30C and OAT of 2C (e.g, overnight, heat pump left running but flow temp turned down) vs a COP of 4.53 for a flow temp of 35C at an OAT of 7C (e.g, heat pump in recovery during the day when it's warmer).

The wider the spread in OAT, the more pronounced the effect. This has been particularly evident for us over the last few days where we have had overnight temps just above freezing, and clear sunny days with OAT of ~10C. The savings have been further enhanced by solar, as we've saved using electricity during the night with a setback from midnight until 9-10am when we get up, and much of the recovery and subsequent daytime usage is then covered by solar affording even more financial advantage to a setback.

Posted by: @sunandair

↑

I thought we’ve been here…. It’s fact isn’t it?

I wish I thought we could say that with any certainty, but IMHO we cant.

The simple physics (the physics is simple, but unfortunately results an differential equations which can only be solved by numerical methods) suggests that there are very modest savings (a few %) to be had in some circumstances, most notably if one or more of the below are the case

• OAT is moderate (say 7C or more)
• House is very lossy, or very low loss so that fixed consumption due to pumps etc and fixed heating from sources other than the HP dominate
• Diurnal variation in OAT is large

In other circumstances the simple physics say that setoff will result in a higher consumption. 

Note this applies principally to a set-off ie switching the heat pump off entirely rather than just reducing the flow temperature.  It also ignores ToU tarifs which can easily dominate the cost.  Perhaps significantly it ignores the effect of cycling, and there is at least a plausible argument that the COP penalty from the set-off is offset by a reduction in the COP penalty from cycling, on the grounds that a heat pump that is heating part time should cycle less than one that is on 24*7.  Since many heat pumps are likely to be cycling at the temperatures where setback/setoff is most commonly applied, this might just matter.

Experimentally we have a few isolated claims that setback/set off reduces consumption, sometimes by a lot more than can be accounted for by simple physics.  However others tell us that they have tried set back and it does not, for them, reduce consumption.  

Some of the experiments reported from time to time are of just a few days measurement, which given the scatter seen in experimentally measured consumption, calls their validity into question.  However, even if we accept them, in the absence of an explanation (given that the simple physics does not appear to explain them, we cannot possibly know how to apply these experiments to systems other than the one on which the experiment was performed.

Finally we have to eliminate confirmation bias (which I think is broadly the physics equivalent of the placebo effect) which is almost certain to exist with the experiment and quite possibly also in the theory.

Taking all of this together I think the only thing we can be certain of is that we cant be certain.  There is a limited theoretical underpinning, based on a relatively crude model, for modest savings in some circumstances, but this is counterbalanced by (predicted) increases in consumption in other circumstances.  The experimental results, such as they are, tell a variable story, and as haven't yet been reconciled even with conservation of energy, which we can be absolutely certain applies, extrapolating from the results specific to one system to expectations for another would be irresponsible.  With sufficient results from only crudely controlled experiments from many different systems we might be able to draw a reliable conclusion, even though we don't understand it, but we certainly do not have that.

Well thats my take anyway.

@sunandair you referred to some experiments on your system and posted a table of the full results which I didnt understand.  I would be quite interested to look at this one in more detail, and so, if you were prepared to do so, I would be grateful if you you explain the table and post it in a way that the figures can be downloaded.  Of course you are under no obligation to do this!