Can utilities have their multi-layered cake and eat it too?

Utilities are increasingly in the eye of the storm these days.  Whether it’s having to deal with hurricanes in the Gulf Coast over the last few months or wildfires on the West Coast, utilities have had to put more sensors out in the field to keep abreast of changing weather conditions and potential risks to their power grids.  The increasing demands for utilities to show that they are carbon-free is also changing the way they generate and distribute energy.  The one common denominator that utilities have is more data to collect and backhaul from their power grids, which is driving increasing demand on their communications networks.

Many utilities may not realize it, but recent advancements have resulted in several bandwidth-intensive applications and processes driving up demand on their networks:

1. Video Surveillance
   Security continues to be top of mind for utilities and security surveillance in the past has been more “after the fact”; where video surveillance is stored locally at the substation and only accessed after a security breach.  Today’s approach is to backhaul all security video footage to a centralized data center and apply artificial intelligence (AI) techniques to proactively determine if a security breach is in the process of occurring.  In those cases, security personnel can be dispatched on site in near-real time.  Each video camera at a substation can generate 9 Gigabytes of data per day and a typical substation could have a dozen video cameras to surveil.
2. Synchrophasors
   Prior to the big power outage of 2003 in the Northeast United States (where 50 million households lost power for two days), sensors on the power grid using SCADA (Supervisory Control and  Data Acquisition) would sample the state of the grid about once every four seconds.  This significant outage could have been avoided had the grid been sampling data more frequently.  To address this, a device called a synchrophasor (not the Star Trek type!) was introduced, which would sample the state of the grid 30 to 60 times per second.  This has allowed the grid to be more reliable but produces significantly more data to backhaul and process.  Each synchrophasor PMU (Performance Measurement Unit) can generate 15 Gigabytes of data per day and all of that must be backhauled to a central data center for analysis.
3. Smart Meters
   In the US, over 50% of households are now serviced by a smart meter that measures your household’s power consumption every 15 minutes. Beyond their billing function, they help utilities track power consumption hotspots during peak usage.  For a utility of 1 million households, which would be the middle range for most US Investor-owned Utilities (IOUs), this can generate 1 terabyte of data per day that needs to be backhauled to a central data center for processing.
4. Internet of Things (IoT) devices
   These include what we mentioned earlier: weather sensors and sensors on power equipment to proactively identify issues.  Smart thermostats in homes is another growing trend which utilities are using to offer smart “on-demand” billing plans where you allow the utility to raise your thermostat during periods of peak usage during the hot summer months in exchange for a lower cents per kWh price.

For the first three categories we mentioned above, a utility of 1 million households would result in a daily requirement for data backhaul of 6 to 8 terabytes.  With this amount of data to backhaul and process, it is no wonder utilities are exhausting the available capacity of their legacy communications networks.

The Information Technology (IT) group in a utility is tasked with managing many of these new applications associated with a smarter grid.  Some utilities have been leasing copper-based TDM services for many years from service providers for smart grid, IT and substation traffic.  The cost of this approach has been onerous and only gets more expensive as service providers are migrating their networks away from copper to fiber and wireless options.

Many utilities have been actively modernizing their networks to a private packet-optical fiber architecture that could scale to support all these new smart grid applications and would allow utilities to capitalize their communication costs upfront versus the OpEx approach as they had been doing in the past.

How?

Ciena has and continues to work with a large IOUs to deploy an innovative 3-layer Adaptive Network^TM architecture – a 3-layer cake, if you will – leveraging Ciena’s Adaptive IP^TM approach to meet all their communication needs.  The layers of the network and the cake are described below:

Substation Access Layer

Let’s start at the bottom layer of our cake:  The substation access layer requires small footprint hardened devices that aggregate and transport mission critical Operational Traffic (OT) and IT services between substations and back to control centers.  This layer connects to the aggregation layer (next layer of the cake) to maximize network efficiency and provide test and troubleshooting capabilities. This layer also needs to manage the low-latency requirements for OT applications, such as teleprotection, as well as the growth of high-bandwidth video surveillance, synchrophasor, and aggregated smart meter traffic.  Security is always top of mind for utilities and Layer 2 MACsec encryption is becoming the de facto standard for securing this part of the network.  Ciena’s Adaptive IP-powered 3926 Platform is one of the key ingredients for this layer of the cake.

Aggregation Layer

As bandwidth requirements in the substation access layer grow, so do the scalability requirements of the middle layer of the cake.  Utilities require universal aggregation of OT and IT traffic from the substation access layer and can also include broadband traffic from utility substations as some utilities are deploying middle mile networks for rural broadband services. This is driving a change in the mix of connections and services, from 1GbE aggregation to 10GbE, and 10GbE aggregation to 100GbE.  The key ingredient of this cake layer is Ciena’s Adaptive IP-powered 5171 Platform, which provides high-density aggregation of 10GbE into efficient 100/200 Gb/s WaveLogic 5 Nano coherent 
optics. In addition, its support for an extended temperature range broadens the application space into uncontrolled environments such as utility substations for outdoor aggregation of 1/10/25/100GbE, which provides high-capacity at the outdoor network edge.

Optical Core DWDM Layer

Finally, the top (and icing) of the cake is the optical core layer, which provides a DWDM superhighway that reduces operational complexity, ensures investment protection while providing maximum scale, flexibility, and reliability.  As all traffic from the substation access and aggregation layers flows into the optical core, this layer must provide 100Gb/s, or higher, transport capabilities to support the aggregate of all of the core IT and OT traffic from substations back to the utility data centers, control centers, and backbone network sites.

Ciena’s 6500 Packet-Optical Platform and or Waveserver are key network platforms for utilities to build out the optical core layer.

A critical part of Ciena’s Adaptive IP approach is Ciena MCP and Adaptive IP Apps, which fill a fundamental IP/MPLS management gap. While SNMP, Syslog, NetFlow, Deep Packet Inspection (DPI), Application Performance Monitoring (APM), and other tools are ubiquitous, only Adaptive IP Apps provides real-time visibility into how routing behavior affects mission-critical services such as teleprotection. Adaptive IP Apps captures real-time telemetry from network devices, as well as from domain controllers to provide network forensic capabilities. This visibility into the IP/MPLS network control plane allows utilities to see precisely how specific traffic traverses the Layer 3 IP network, and where performance can be optimized on an ongoing basis.

These features are something we’re actively working on with several utility operators  Check out what the Chief Network Architect at one of the US top IOUs had to say about their journey to an Adaptive IP powered network:

Investor owned utility journey to Adaptive IP

Overall, the 3 layered-cake approach enables utilities to shift their communication costs from a leasing (OpEx) to a capital (CapEx) financial model, which is more attractive for Utility CFOs.  The network can scale to meet all their future communications needs without the need to increase OpEx and support the utility’s shift to renewable energy and a carbon free future.

So the answer is, YES, utilities can have their cake and eat it too!