@cathoderay

You still did not explain why, if your method is correct, the values that should have been the same on the 4th Nov. were not the same thanks to the formula you have produced.

There appears to be a great deal of assumptions being made, and dis-guarding of data that does not fit your perceived end result.

Posted by: @jamespa

↑

Thanks for the responses....

However see my post above which I suspect crossed yours and calls into question the explanation for the apparent difference between actual and expected, on the basis that it appears to be present from 19th October, not 4th November when setback was said to start.

@derek-m @cathoderay until we get to the bottom of this and either explain it or conclude that i have goofed in my analysis I really don't think its worth much more discussion/analysis.

Posted by: @cathoderay

↑

I am deliberately NOT modelling anything (apart from in effect doing a regression of energy in on OAT, which I concede in some people's minds might be a model). A core assumption is that all of the variables and their minutiae are baked into that relationship (equation), which, assuming a normal distribution for the energy in values for a given OAT, gives a meaningful average energy out for that given OAT. If that is true, then I don't need to account for all the other variables, they are already accounted for. If I then assume regression to the mean occurs, then over aggregate periods of time, the values will regress towards a mean that is a not too inaccurate representation of the true mean. 

For the avoidance of doubt I accept that this is (most probably) true but it does mean that it will be necessary to collect much more data in order to take out effects (noise) which could easily be taken out by simple arithmetic.   That's obviously a choice that the experimental scientist has to make driven by various factors including (for example and relevant to this case) how long they are prepared to maintain the system in any given state (because the assumption is not valid in the face of systematic as opposed to random, changes in one of the relevant variables)

@jamespa - I have other things to do today, as the sun is out, and so will return to it later. I also need to clear up whatever is goofing my spreadsheets (probably memory overload, I have had a few out of memory error messages).

Posted by: @derek-m

↑

There appears to be a great deal of assumptions being made, and dis-guarding of data that does not fit your perceived end result.

I agree with the first part, and have been at pains to point them out, so they can be critically examined, but not the second part. I do not do 'here are my perceived results, now where is the evidence'. Instead, I post my methods and results, and invite commentary. 

Posted by: @cathoderay

↑

Posted by: @derek-m

↑

There appears to be a great deal of assumptions being made, and dis-guarding of data that does not fit your perceived end result.

I agree with the first part, and have been at pains to point them out, so they can be critically examined, but not the second part. I do not do 'here are my perceived results, now where is the evidence'. Instead, I post my methods and results, and invite commentary. 

Maybe it would have been better if you had checked the quality of the data before posting.

Posted by: @cathoderay

↑

I am deliberately NOT modelling anything (apart from in effect doing a regression of energy in on OAT, which I concede in some people's minds might be a model). A core assumption is that all of the variables and their minutiae are baked into that relationship (equation), which, assuming a normal distribution for the energy in values for a given OAT, gives a meaningful average energy out for that given OAT. If that is true, then I don't need to account for all the other variables, they are already accounted for. If I then assume regression to the mean occurs, then over aggregate periods of time, the values will regress towards a mean that is a not too inaccurate representation of the true mean. 

Posted by: @jamespa

↑

For the avoidance of doubt I accept that this is (most probably) true but it does mean that it will be necessary to collect much more data in order to take out effects (noise) which could easily be taken out by simple arithmetic.   That's obviously a choice that the experimental scientist has to make driven by various factors including (for example and relevant to this case) how long they are prepared to maintain the system in any given state (because the assumption is not valid in the face of systematic as opposed to random, changes in one of the relevant variables)

Thinking more about the above, its all a matter of degree.  You choose to compensate for OAT by using a model, but you choose not to compensate for IAT or energy stored in the fabric.  These are both choices.  Its not necessary to compensate for OAT, you could simply collect several years worth of data (with the system kept constant throughout to avoid systematic changes) ignoring OAT altogether! 

However the choices have consequences for the experiment, or to turn it round...

• How long you are prepared to wait for answers
• how much data you are prepared to collect
• how long you are willing to keep the system free of systematic changes which you may otherwise want to make

drives how many/which of the known variables you must correct for in order to get results which isolate the phenomenon of interest to a sufficient extent that any conclusion is robust.

it really is a choice!

I worked for a while on a type of sensor.  The problem with many sensors is that they are sensitive to variables in addition to the one you wish to measure.  To overcome this you either correct the output to take account of those variables that you aren't interested in measuring, or keep them constant (which may not be possible).  How accurate you need the result to be drives the decision of which extraneous variables must be corrected for/kept constant.   Its exactly the same principle!

@jamespa

I ran the latest data, well the 4th of Nov bit, through my spreadsheet, which actually predicted electrical energy in to be lower than the original data, but with an IAT of 19C which is the required setpoint. If I had used the indicated IAT within the data, it would have probably been close to the original data values. Because of the underlying problems with the system, it does make correctly analysing the data a little more difficult.

Posted by: @jamespa

↑

Thinking more about the above, its all a matter of degree.  You choose to compensate for OAT by using a model, but you choose not to compensate for IAT or energy stored in the fabric.  These are both choices.  Its not necessary to compensate for OAT, you could simply collect several years worth of data (with the system kept constant throughout to avoid systematic changes) ignoring OAT altogether! 

I agree it's all about choice, and I am trying to choose a method which gets a good enough answer, without unnecessary effort, in a reasonable time frame. I have made it very clear the current method is a proposal, and I fully accept that it may not work, but then again it might, and if it does, it is relatively simple to implement.

I don't think I have underlined clearly enough how I am approaching this. Given the method, observed vs expected, I need to have both the observed value with a setback, and expected value without a setback. The observed is relatively easy to get now that I have automatic monitoring, I just let the data accumulate. The expected I am trying to estimate using what I am pretty sure is the major determinant of energy use, the OAT. I know there are many other variables that might affect the energy in, but I am assuming that in a steady state condition, with no net gain or loss in heat over longer time frames, that those variables work out in such a way that the actual energy in ends up being approximately normally distributed, and I can therefore use the mean energy in for each given OAT. If all this adds up to a viable method, then I don't need to explicitly incorporate other factors, because they are in effect baked in, ie the observed energy in over longer periods for each OAT includes the effects all those other variables, they just haven't been explicitly identified and individually dealt with. There may also be some mutual cancelling out, sometimes the variable goes one way, another time the other way, and they cancel out. The average 24 hour energy stored in the fabric is effectively constant when the system is running in steady state, otherwise the IAT would rise or fall over time (you lake analogy again works here. Because of the setback, the IAT will not be the same in the two scenarios. The lower average IAT caused by the setback is the 'price' of the setback, but it is easily paid because it happens mostly overnight when I am in bed.

I am not sure I can avoid using the OAT as the predictor variable. Even if I had decades of data, I still need to answer the what is the expected energy in given this OAT question (not the observed, I already have that). During a setback, that data (what would the energy in have been without the setback) is not in the data as such, which is why I need a way of predicting it.  

I will see if I can come up with an answer to what happened on or about 19th October, or whether something else explains the observations/findings. It does look as though something happened: 

During cold spell last week, i first started with setback, however the usage turned into 5/6kw per hour and house wasnt at ambient temp until late in the evening, i switched to running 24/7, much more comfortable and cheaper at approx. 1kw per hour after system reached set lwt. Today for instance, its mild so i ran from 1pm to 9pm, approx 15kwh used. If i was running today 24/7, it would have been bit more comfortable but would have been near24kwh. For me, it depends on the weather, when cold below 6deg OAT its beneficial to run 24/7 but when mild above 8, setback keeps us comfortable and we use less kwh. The fabric of the house im sure isn't was warm as it would be running 24/7 but i dont notice the fabric temp, i notice IAT and at the end of the day that's all that matters.

Also found fixed flow @29 even warmed hosue to 22 in minus weather but 27 wouldnt warm house to 22 in mild weather so think i have found sweet spot regards flow so no wc needed.

Posted by: @newhouse87

↑

During cold spell last week, i first started with setback, however the usage turned into 5/6kw per hour and house wasnt at ambient temp until late in the evening, i switched to running 24/7, much more comfortable and cheaper at approx. 1kw per hour after system reached set lwt. Today for instance, its mild so i ran from 1pm to 9pm, approx 15kwh used. If i was running today 24/7, it would have been bit more comfortable but would have been near24kwh. For me, it depends on the weather, when cold below 6deg OAT its beneficial to run 24/7 but when mild above 8, setback keeps us comfortable and we use less kwh. The fabric of the house im sure isn't was warm as it would be running 24/7 but i dont notice the fabric temp, i notice IAT and at the end of the day that's all that matters.

Also found fixed flow @29 even warmed hosue to 22 in minus weather but 27 wouldnt warm house to 22 in mild weather so think i have found sweet spot regards flow so no wc needed.

Can you confirm, was the heat pump running almost continuously at 29C LWT when cold and cycling when operating at 27C in milder weather?

Yes that's exactly it, even when cold though after maybe 8hours, the heat pump modulated down to 11.4l/min,approx 1/3 of capacity so energy usage dropped dramatically compared to running full tilt in that weather. Had alot of cycling atlwt27 and even see some at lwt29 yesterday so dont think i can run lower then29, tbh its big improvement on last year running high 30s,low 40s in colder weather. When running 24/7 the house is noticeably warmer throughout, even my colder gym room was up to 20, i had never seen that before for that room in cold weather, almost looking forward to running 24/7 again.

@jamespa - I think I might be getting somewhere.

(1) the current midea data file you have (and I have been using) does have a double 1.18 correction applied to the heating energy in column for most but not all of the rows (up until the point I put the correction into the python script in late November). I have NOT corrected this yet, having already made one mistake I don't want to compound it by making others - easy to do, as the first mistake demonstrates! 

(2) I have now managed to do a sql left join to get the independent RC4 data logger temperatures into the midea data file, meaning I now have an IAT record for the entire period. The modbus MD02 and the RC4 data logger values are similar, but the RC4 data tend to be around 0.6 degrees below the MD02 when the heating is on, most likely because of their respective placements, the MD02 is wall mounted not that far (about 5-6 ft) from a radiator, the RC4 data logger being on the opposite side of the room further from the radiators on a shelf. This is what the two look like when plotted together:

In the spring, the heating was on until the end of April, and it came back on again on 16th October. Both in the spring, and in the early part of this autumn, the IAT was on average about 1 -2 degrees above the desired IAT. On the 4th on November, I started the setback plus recovery boost regime and either at the same time, or very close to the same time, lowered the baseline WCC because the house had been running a little warmer than I wanted it. From then on, the IAT is on average pretty much where it should be. 

There are thus at least three IAT periods, spring over warm, early autumn over warm and mid autumn about where it should be.

(3) I then looked at the observed - expected data, looking for patterns, at this stage still using the double corrected heating data (OK-ish, because I am looking for patterns). By and large I think there is a pattern, different periods can be seen (middle is not the same as the right or left hand side for example), though the picture is blurred somewhat in the background noise:

In the spring, I consistently used more energy than expected, in the early autumn, before the setback period, I used a bit less, and then during the setback I used a lot less. The curious thing is this appears to persist after the setback ends, or maybe I just don't have a long enough run of data to see it setting back to pre setback levels. These periods (apart from the last post setback period) roughly tie in with the periods in the IAT record. Spring: warm IAT => more energy used than expected; early autumn: consistently a bit less, setback period: noticeably a lot less. No I can't explain the outliers. The visibly biggest one in the middle is on the day I put the heating on (16th Oct).  

The other key thing is that, because overall I still do not have that much data, five weeks in the spring, eight weeks from this autumn, totalling around 90 days, I used all the data points, including the setback days to derive the OAT-energy in equation. This as can be seen includes data from periods that were not the same, and so will add all sorts of noise into the overall summary equation. At some point I will do the equation again, using a relatively homogeneous and representative period, perhaps the period this autumn preceding the setback, bearing in mind that is not a lot of data points, and the IAT was a little bit above the desired IAT.

Bottom line is I don't think there is any harm in exploring the data further as it comes in.