Creative Connectivity - the site for eHealth, telematics and wireless

When Smart Meters go wrong

July 26th, 2019 |  Published in Smart Energy  |  7 Comments

Most people don’t think much about firmware – the embedded software which runs the microcontrollers in all of the devices we have around us.  We’re aware of the frustration when they don’t do what they’re meant to, at which point we realise that “smart” may not have been the best adjective to use to promote the product, but even when they do go wrong, turning them off and on again, or taking the battery out generally clears the problem.  They almost always go wrong because the design process didn’t include enough testing, or not enough time was given over to thinking about the “edge cases” – those unexpected combinations of events which result in things not working the way they should.  Most of the time it’s just a short-term annoyance; if it’s worse than that we’ll probably send it back, or throw it out and buy a new one.

However, we do expect safety critical devices like cars and planes and national infrastructure to be a lot better designed than this.  Your boiler turning off because it thinks there’s a flow problem when there isn’t is annoying (time for a firmware upgrade please, Vailant), but it’s not life threatening.  In contrast, a self-driving car that runs over a cyclist is not something the public is generally happy about.  Nor is a plane falling out of the sky.  But where would you put a smart meter in the scale of things that might affect your life?  Last week we found out, and it’s not a happy answer.

At this point it’s worth considering how important a smart meter is?  I’ve tried to have that conversation with various parts of the industry and it’s always a question which has raised eyebrows, because they don’t seem to understand why I’m asking it.  The current line from Smart Energy GB is that “Smart meters can’t solve climate change on their own, but with the smarter, more energy efficient grid they help to create, they are a start”.  At which point they regale us with photos of polar bears and cute animals.  Look closely at the photos and you’ll see that smart meters have also managed to get rid of plastic straws, as there aren’t any of those in the adverts.  Clever chaps down there in Bloomsbury, ensuring the area maintains its reputation for fiction.  Away from the glare of the publicity lighthouse, the Energy Minister of the day tends to trot out the same line that they’re leading to a more efficient energy grid, whilst our energy suppliers enthuse about the fact we’ll no longer get estimated bills.  Or meter readers.

That dichotomy between smart grid and accurate bills is key to understanding the GB smart metering programme and why it’s gone wrong.  The key reason for installing smart meters should be to provide data to make the grid more efficient.  To make the grid efficient, you need to be able to react to demand, which means real-time information and the knowledge of how to use it.  However, in Britain, we have let the meter design be driven by the energy suppliers.  They have no real interest in real-time data; as for billing they only need it on a monthly basis.  Instead, they compromised and designed meters which upload data once a day.  The whole of the rest of the smart metering infrastructure, from the DCC through to the cellular contracts for uploading the meter data, has been designed and costed on the same basis, which means that the £20 billion or so we’re spending on the program will not help us get a smarter, more efficient energy grid.

To be fair, we’re not alone.  Spain has garnered acclaim for rolling out smart meters to every household by the end of 2017.  They made some good decisions, such as not making the specification overly complex and letting the network install them instead of the utilities.  Both decisions helped them to keep their deployment on track at a fraction of the cost of the GB programme.  It will also help them keep their bills down, as the cost of smart meters ends up being paid for by consumers.  However, a recent set of interviews with senior managers involved with the project shows a lot of them wondering whether there will be an overall cost benefit and whether they are getting the right data to help the grid.  As a result, they’re already looking at a second generation of smart meters, despite the fact that the current meters were meant to have a 15 year life.  It means that even Spain’s low-cost smart metering deployment could end up proving to be a very expensive experiment.

Coming back to the question of how important a smart meter is, there’s an important fact that people need to grasp.  If all it does is send your energy usage to your supplier, it’s not very important.  If it goes wrong the energy supplier can go back to estimating bills or ask you to send in readings.  But that isn’t all a GB smart meter does.  Our smart meters have something else in them.  It’s an OFF switch, which can disconnect your gas or electricity.  That’s a little extra which the utilities added to save them the inconvenience of sending someone out to connect or disconnect you.  It’s so much easier if someone in their call centre, somewhere in the world, can just press a button.

At this point I need to do something I rarely do, which is to quote Josef Stalin, who is credited with saying that “A single death is a tragedy; a million deaths is a statistic”.  Once a smart meter has an OFF switch you need to turn that around: “A single smart meter is irrelevant; a million smart meters is a tragedy”.  Because if a million smart meters were to disconnect their users in one go, we could probably say goodbye to our energy infrastructure.  I don’t believe that this concept has ever got through to those involved in the smart metering roll-out.  Whenever I’ve raised the point about the consequences of someone hacking smart meters as a composite grid component, the only response I’ve had is “why would anyone do that?”  Which is akin to wondering why anyone would fly a plane into a building.  But they probably wouldn’t contemplate that, as it’s not in their Health and Safety training.

What is worrying is that this might not need a hacker – it could happen by accident.  The most high profile example of this is the recent crashes of Boeing’s 737 Max aircraft.  It appears that both were caused by firmware not being able to cope with the edge case of failed sensor, which no-one had expected to happen.  After the first crash everyone tried to convince themselves it was not a problem.  It took a second crash to see the aircraft grounded.  The very obvious lesson is that when a safety critical system starts to go wrong, you need to take it seriously.

With smart meters I am concerned that the level of software process, testing and understanding is nowhere where it needs to be to ensure the safety of the grid.  Smart meters are treated as individual devices with no concept of the damage which could be done if millions of them malfunctioned.  Security discussion within the smart metering programme is largely confined to data privacy and tampering, as the industry is still fixated on consumer fraud.  It probably wouldn’t be difficult for a determined extremist with computing expertise to get employed by one of the meter manufacturers and sabotage them, but that’s probably fanciful.  (Although I would hope there is some vetting of those working on the firmware).  What is more likely is a simple mistake.  The critical thing is how to deal with any such mistakes when they surface.

Last month we learnt what that response is.  Many customers of Bulb energy reported that their smart meters (although they probably meant their In Home Displays) suddenly changed from displaying English to showing Welsh.  The correct response to that would be to stop any further installations whilst conducting an immediate software review.  Instead this was treated as a joke, with the supplier’s spokesman responding to the failures by quipping that “We think Welsh is a great language”.  They might have acted differently if it had changed to Chinese and displayed the message that “Our Government is listening to you”, but they’d probably have made a joke of that as well.

It illustrates how ill-equipped the industry is for the magnitude of what it is trying to do.  Fifty million smart meters with a remote disconnect capability are not just individual billing units, but together become critical national infrastructure.  As such, that needs to be recognised at the design stage, through software verification and testing and monitoring of their performance.  Any unexpected behaviour needs to be flagged up as a cause of concern, with processes in place to determine what has gone wrong.  It will probably not be malicious; rather than being a Black Mirror plot, the cause is more likely to be a simple programming mistake that testing didn’t pick up, not least because of the insane pressure on developers and testers to get meters out of the door and shipped.  But that doesn’t mean we can laugh it off as a funny story for a quiet news day.  Each mistake like that has the potential to cripple the grid.

What is sad is that we have opened up this Pandora’s Box at immense expense, without including the features which would make it useful.  Instead we have exposed the grid to all of the risks and obtained virtually none of the benefits.  If the worst should happen, is there a contingency plan to reconnect a million or more homes which have lost power?  Our utilities are good at individual interventions like fallen powerlines and other utilities cutting through cables, but how would they cope with the need to replace or reset a million meters?  Have they even thought about that scenario?

It could happen.  Bulb have rather sheepishly reported that almost 30% of their smart meters are not working; another energy supplier told the press that 20% required a second visit from an installer.  Every time a smart meter exhibits a problem it should be logged centrally and investigated.  The saga with Whirlpool dryers shows the consequences of not doing so.  Once again, those are individual tragedies, not the infrastructure meltdown that could come from malfunctioning smart meters.

As more of these problems come to light, the industry needs to up its game in terms of security and take these reports seriously.  There is more to do than laugh as the world ends, whichever language you do it in.

7 comments ↓

#1 Jake Maverick on 07.27.19 at 2:10 pm

The whole point of these nasty devices is so that the state and anybody else with the right technology (you can buy an app for your smart phone) is to see and record everything that goes on in your home in full 3d holoporn. Everything else is just propaganda…

Then there are also the health problems…I happen to be particularly sensitive. One was forced upon us few weeks ago, ever since I’ve had major headaches, blurred vision, lot more muscle cramping. Somedays I can barely function and it’s all since the damn thing was installed 🙁 Can’t even get away from it. I am a prisoner here.

#2 Dave Ward on 07.28.19 at 5:37 pm

“But even when they do go wrong, turning them off and on again, or taking the battery out generally clears the problem”

It’s not just Boeing who have software problems:

“Airbus A350 software bug forces airlines to turn planes off and on every 149 hours”

https://www.theregister.co.uk/2019/07/25/a350_power_cycle_software_bug_149_hours/

“If the worst should happen, is there a contingency plan to reconnect a million or more homes which have lost power?”

That’s assuming the grid didn’t fall over big time when they all disconnected simultaneously…

#3 Nick on 07.28.19 at 7:27 pm

I’m pretty sure that the grid would fall over, but it’s surprisingly difficult to get anyone in the industry to understand that, let alone any minister or anyone at BEIS. So I’ve reverted to the simpler problem of visiting a million homes and installing new meters, or just bypassing them. That seems to be just about within the scope of their understanding. But as you know, there’s unlikely to be a grid behind it when they do reconnect.

Thanks for the Airbus link – I’d missed that. A bit worrying to see that they’re promoting the XWB variant as “capable of flying over 20 hours non-stop”. Reminds me of the old joke about why you wouldn’t want Windows running anything on the flight deck.

#4 Nick on 07.28.19 at 7:30 pm

You can’t be forced to have a smart meter – they are not compulsory. If someone did force one on you, or attempts to do so, lodge a complaint with OFGEM.

#5 A barber on 07.31.19 at 7:29 am

People seem to think that installing a smart meter will instantly cut their bills .there are only two people in our house we don’t leave lights on and apart from daily living fail to see how I’m going to save money unless I stop watching tv. to stop boiling kettle . stop cooking meals . Stop running shower .and sit on sofa in the dark all night drinking a water.wake up everyone the only people going to save is the energy companies .and if my house burns down they will not give a shit

#6 Joe Patel on 08.03.19 at 3:30 pm

Nick – would be interested in your take on this:

https://www.ncsc.gov.uk/information/the-smart-security-behind-the-gb-smart-metering-system

I think you should contact them!!!

#7 Richard on 08.06.19 at 8:06 am

In this article and in his previously well argued papers Nick has addressed the essential shortcomings of this misguided programme. From the merely silly (patently unSMART headline) to the more subtle and insidious aspects of the choice and application of the base technology – 2G networking as an example, and downright lies by Government and the suppliers. The real problem remains that no-one, except the few like Nick who are prepared to question the logic and premise of such projects, seems to be giving a damn. As I have pointed out myself on a number of forums, it will take balls of steel to cancel or downgrade this programme when there are so many vested interests and so much sunk capital. Perhaps we should spare a thought to how the behemoth that has been created can be stopped or modified to provide real benefits to consumers and the suppliers alike. At the moment the benefits are all asymetrical despite propanda to the contrary.

Leave a Comment

About Creative Connectivity

Creative Connectivity is Nick Hunn's blog on aspects and applications of wireless connectivity. Having worked with wireless for over twenty years I've seen the best and worst of it and despair at how little of its potential is exploited.

I hope that's about to change, as the demands of healthcare, energy and transport apply pressure to use wireless more intelligently for consumer health devices, smart metering and telematics. These are my views on the subject - please let me know yours.

You can Subscribe via RSS »




%d bloggers like this: