Wow, thank you so much for this write up @transparent 

That explains everything, and also seems somewhat insane (although perhaps necessary) that the system was designed in this way with so many varying clocks.

I'll not waste your time with queries around the logic of particular design decisions, but that this is all already documented makes me think that action on it as a result of my intent is unlikely to be successful...

@transparent

So to minimise the possibility of being overcharged when supplying an EV or battery storage system, the switch on of such devices should be delayed by 30 minutes from the stated low tariff period start time.

By doing so it should ensure that the recorded energy usage will be within the lower tariff period and also the correct tariff is being applied.

To ensure that no energy usage is at the higher tariff rate it would be necessary to stop charging such devices at the end of the low tariff period. The usable low tariff charging period will therefore be reduced by 30 minutes.

One thing I don't understand is that if companies have the ability to accurately charge using the 30 minute Smart Meter data then why don't they? Octopus uses it and EDF told me they'd be switching to using it (at some point). 

A possible solution to prior overcharging would be to require all energy companies to use the 30 minute data going forward, then backwards compare historic usage and refund overcharging?

I downloaded my 30-minute smart meter data using the following instructions: https://kb.loop.homes/how-to-download-your-data

It allows 90-days worth of data to be downloaded at a time and enabled me to challenge EDF's billing.

Posted by: @korwraith

↑

One thing I don't understand is that if companies have the ability to accurately charge using the 30 minute Smart Meter data then why don't they? Octopus uses it and EDF told me they'd be switching to using it (at some point). 

Do you have any evidence that Octopus' billing system (Kraken) takes into account the Randomised Offset to the Tariff Switching Table?
I don't.

Although it's not directly related, do you think any Supplier offering a cheap-rate or ToU Tariff actually downloads those price points into the Tariff Matrix in your Smart Meter?
On many ESME units you would be able to see the price points by pressing the appropriate button to cycle through the data.
But I've not yet met anyone who's been able to view data in the ToU fields.

I think all Suppliers' billing systems are retrieving only the consumption data for the 48 periods of the day.
The pricing is applied later by the billing system software, regardless of the Randomised Offset.

Come to that... although I appreciate the need for the security features within most of the ESME specifications, I don't understand why a customer shouldn't know their own Randomised Offset.
We shouldn't need to try working it out, or manually delay our off-peak use, as suggested by @derek-m

I can be quite 'anal' at checking my bills.  With energy bills, it used to be very straightforward: read the electricity meter for kWh used since the last bill and similarly for gas (but calculate the energy from the calorific value stated on the bill).  Then do some maths. Now I have Octopus Agile and Tracker for gas, I can spend hours every month downloading meter data, daily calorific value details and historical TOU prices and yet still have difficulty reaching the same value as that on the bill.  The change to BST caught me out in March and I still need to identify the source of a regular approximate 1% difference in bill calculation between Octopus calculations and mine for gas (I calculate higher, so I’m not complaining). Strangely, the billing period start and end meter readings agree exactly.

I have received only my gas bill for this month so far, up to and including 8th May.  I went to my usual meter reading data source only to discover that the most recent data was for 6th May. I thought that I should be able to fill in the missing data from my Home Assistant meter data, but I found it impossible to correlate the two datasets because the half-hourly samples were not identical. As it happens, my gas boiler failed on 4th May so I was without heating for 4 days, using only 2.25kWh for cooking. However, my bill shows 200kWh used over the four days (but final readings tally!). I can't see the point in raising an issue as I think I'm already quids in on these tariffs.

Out of interest, I also tried to complete the missing data for my electricity meter using my Home Assistant records. This data has an approximate sample rate of two minutes, but correlating with the ‘official’ meter readings is complicated by the random offset mentioned by @Transparent.  Maybe I could determine the randomised offset for my meter (Arghh! Who put that idea in my head! @ Transparent?).  Considering that the offset can be as much as 1799 seconds, the idea of a half-hourly ToU tariff sounds somewhat foolish.  I believe that some people with Vs take it as a challenge to charge their batteries as cheaply as possible and take aim at the cheapest half-hour slots (notwithstanding the random hold-off time for smart chargers also mentioned by @Transparent).  A hopeless task.

Posted by: @korwraith

↑

One thing I don't understand is that if companies have the ability to accurately charge using the 30 minute Smart Meter data then why don't they?

They can't do it accurately.

Part 4 - How the Smart Meter specification facilitates correct use of low cost periods.

Energy Suppliers argue that they need to use the internet to communicate with customers' Smart Chargers because they require monitoring data at intervals shorter than 30-mins.

That argument fails to differentiate between 'retrieval of monitoring data' and 'sending commands'. Monitoring of charging data could still be facilitated in real-time via the internet, whilst commands to the charger were being transmitted across the secure/encrypted Smart Meter network.

Let's step back a moment to understand how the Smart Meter design team envisaged that home-based devices should operate.

Each Smart Meter system must provide at least 5 channels of Auxiliary Load Control Switching (ALCS).
The only instance in which ALCS has been deployed thus far (May 2024) is within 5-terminal meters.

The 5th terminal is powered when the ESME acts on a SMETS-Command which may have been sent minutes or hours earlier.
The command to Turn-on is only enacted as the Tariff-Switching Table moves to the next time period.

But there are several other scenarios by which the ALCS command can activate a home-based device.
Here's a diagram showing other options, taken from a Government Information Leaflet

Using ALCS is quite different from a Supplier sending an On-command across the internet to a 'Smart' Charger.
Even if they incorporate a delay of 2 minutes, to comply with Sect-11 of the EV Charging Regulations, that might still energise the EV Charger before the next tariff period starts:

The upper (green) section of the timeline shows an ALCS command being sent to a meter with a Randomised Offset of 7m40s,
whilst the lower section shows the effect of a Charger-ON command being sent via the internet.

In practice an EV Smart Charger will be operational for longer than 30mins of course, thus greatly reducing the overall percentage error.
The above example is merely illustrative.

The error levels are sufficiently low that few customers would notice an increased use of electricity in the half-hour period immediately before the charger is meant to be running.
But the accumulated errors are biased in favour of the Energy Suppliers.

Energy Suppliers are being permitted to over-charge customers with Smart EV Chargers, using a mechanism which prevents them being able to calculate the over-charge.
They cannot therefore be required to retrospectively re-pay that amount.

DESNZ have been made aware of the above billing issues. Their response reads:
"[….] smart metering software also incorporates a random number
generator, which it uses to delay tariff switching times in a process called Randomised
Offset but this only applies to meter register readings and not the half-hourly
consumption readings. In addition the energy supplier can read the Randomised
Offset value remotely."

But the Smart Meter Specifications do not refer to there being “Meter Register Readings” and “Consumption Readings”.
It is therefore not possible to understand the distinction, nor how Energy Suppliers are obtaining and using both sets of readings (if they are indeed both available).

The SM Specifications refer to only one set of registers at half-hour intervals.
Those are stored within the meter for 13 months.

The SM specification documentation does imply how the Randomised Offset can be read remotely. There is a SMETS-command to read it, but the specification does not state whether this command can be sent across the Wide Area Network, or the Home Area Network, or both.

If Energy Suppliers could retrieve the Randomised Offset number across the WAN, then this would need to be the basis of how Smart EV Chargers would be implemented. But it isn't. See EV Charging Regulations, Sections 10 & 11.

If DESNZ is correct, then Parliament has passed legislation which allows Energy Suppliers to ignore the Randomised Offset, and instead implement their own in-house ‘randomisation’ of charger on-times.

That places into the hands of those Suppliers, the capability to charge customers at a higher rate than that stipulated in their tariff, by switching on an EV charger remotely .

Why is that strategy acceptable?

Isn’t inaccurate billing a breach of electricity suppliers’ Licence Conditions?

DESNZ have responded to this as follows:

As set out in the ‘EV Smart Charge Point guidance’, randomised delay is a requirement of
smart charging to ensure and maintain grid stability. More information can be found online
at: www.gov.uk/guidance/regulations-electric-vehicle-smart-charge-points. The guidance
sets out that the ChargePoint must be configured to operate a randomised delay of
between 0 and 600 seconds (10 minutes) at each charging instance. If a smart meter is
applying a randomised offset, then randomised delay through the EV ChargePoint would
not also be applied, this is covered in the EV Smart Charge Point Regulations guidance.

Turning to [....] split/variable tariffs, it is accurate to say that under certain
circumstances, it could be possible for a short period of charging to occur outside of the
cheaper tariff period. Such scenarios might occur where billing is based on the smart
meter tariff registers and given other factors including type of tariff and charging control
method.

For example, where charging does not end on the hour, or finish before a tariff’s lower rate
period ends, there is a chance that charging could occur up to 600 seconds into a different
tariff. It is for individual suppliers and charge point operators to comment on how they
manage charging periods and the billing associated with them. Suppliers do however
have a legal obligation to charge their customers fairly.

DESNZ do not appear concerned at consumers being billed for consumption outside of the low-cost periods which they believe they are using.

Nor do they have an objection to Energy Suppliers instigating the charging of EVs at a higher price without their customers’ knowledge or consent.

Posted by: @transparent

↑

Do you have any evidence that Octopus' billing system (Kraken) takes into account the Randomised Offset to the Tariff Switching Table?

EDF provided me with this in the course of our interaction:

https://docs.google.com/spreadsheets/d/1BEgQe6y_frDcB0jU7i_gT3kRwCPw8he1LesO8wn7sbA/edit?usp=sharing

I've no idea how Octopus works things (or doesn't, certainly they say your IHD will not show correct tariff data on TOU tariffs) in relation to the random offset, but what I do know is that they bill customers on Octopus Go based on the 30 Minute Interval Reads in the lower section of that spreadsheet. The times/usage data on those reads match up 1:1 with UTC and the times consumers would configure their EVs/batteries to charge. (Note that time format is backwards looking, so 2023-11-14 00:30 = power used between 00:00 and 00:30). 

EDF (and possibly other non-Octopus suppliers) currently bills TOU customers based on the "Cumulative Register Reads" displayed at the top of the spreadsheet. These are the two counters I referenced earlier, one runs during peak times, the other during off peak. Those counters do not match the 30 minute interval reads when tallied up, presumably due to the random offset feature, thus generating erroneous bills.

I feel bad for those poor suckers at the long tail of the random offset graph, potentially getting charged a full 30 minutes of peak rate during what they believe to be their off peak window 😤 

@transparent

If my understanding is correct, the randomised offset is designed to try to prevent large surges in demand from too many consumers all switching on power hungry devices at the same time. This would work fine if the switch on is being controlled via the smart meter, but I doubt this is the case.

I don't know the full details, but if the timing of EV chargers and battery storage systems are being set by the consumers to coincide with the commencement of a low tariff period, then I doubt that the desired timing offset is being achieved.

I fully agree that if each consumer knew the randomised offset applied to their own electricity meter, then they would be much more inclined to delay the switch on of higher power devices to ensure correct billing, and also help prevent surges in demand.

@derek-m Indeed! Not to mention washing machines/dishwashers, 99% of which most certainly don't have smart meter random offset scheduling integration 😆 

Energy companies would be better advertising their tariffs as "X hours off peak electricity, approx 1am to 5am, your exact off peak window will be confirmed when registering"

Posted by: @korwraith

↑

poor suckers at the long tail of the random offset graph, potentially getting charged a full 30 minutes of peak rate during what they believe to be their off peak window

We don't yet know if that's what is happening.

For example, DCC might've realised the problem and sent commands to all Smart Meters which reduces the Randomised Offset Limit from 1799 to some lower figure(s).

I suggested to DESZN that they write to DCC to ask them for information on the randomised offset figures.

Indeed it's possible that DCC could calculate the discrepancies between sudden surges in demand and the commencement of Tariff Time Periods, although they could only do so based on a random sample because they can't tell where each Smart Meter is located.

But DESNZ instead decided to write to the Energy Suppliers... who have a vested interest in not investigating any such discrepancies, let alone telling DESNZ or Ofgem!

Posted by: @derek-m

↑

if the timing of EV chargers and battery storage systems are being set by the consumers to coincide with the commencement of a low tariff period, then I doubt that the desired timing offset is being achieved.

The largest proportion of electricity which is being billed incorrectly isn't likely to be instances when consumers are switching devices manually.

It's most likely that the highest billing discrepancies are those where the Energy Supplier initiates the Charger-ON commands.
That's because it is a legal requirement that all chargers installed under the government subsidised scheme must be 'smart' to comply with the regulations.

Parliament has passed the EV Smart Charging Regulations (2021) whilst they are in conflict with the SMETS2 Specifications, which they approved in 2013.
MPs should not have been placed in that position.
They cannot be expected to understand the technicalities of randomised offsets and the effect of surges on the grid.

Posted by: @korwraith

↑

Not to mention washing machines/dishwashers ...

Actually the amount of electricity consumed by those devices in particular are so low that they are unlikely to be discernible within the range of analytical tools available to us consumers.

There was a report by the Exeter-based Regen group several years ago when the tabloid press carried stories of 'smart washing machines' costing more than the energy which could be saved over their lifetime. I don't think any white-goods manufacturer had seriously suggested such a smart washing machine.

We need to use common sense in pursuing the truth of how much consumers are being over-charged.

Smart EV chargers are very high users of electricity, and that means there should be billing errors which can be detected.