Optimizing electric grid operations requires utilities to manage their communication networks with the same priority and application of technology as their power generation, transmission and distribution infrastructure. Improving efficiency and achieving higher levels of customer satisfaction is increasing the volume of operational and customer-related data exponentially. Grid Modernization requires highly available mission critical networking equipment.
Events which occurred over the last few years have demonstrated the power industry’s need to provide plentiful, reliable electric power under increasingly demanding conditions. Power utilities are ever more reliant on Industrial Ethernet as the foundation of their critical networking solutions. The choice of using Ethernet within power utilities was made to leverage the cost effective and high speed Ethernet technologies so commonly used in the computer networking industries. Current and emerging standards such as IEC 61850, DNP3, and Modbus-TCP, to name a few, specify Ethernet as the fundamental enabler for their protocols. In addition, Ethernet is common enough to allow the use of familiar visibility tools for all devices in the Ethernet ecosystem, such as routers and switches, which already exist within utility business networks.
One of the most common implementations in Substations is the Ethernet based IEC 61850 standard that has a Station Bus and a Process Bus. Both these buses require hardened and redundant Ethernet equipment to allow non-interrupted data transfer for each device on the bus. The devices which interact with the process equipment (aka primary equipment) sit on the Process Bus. Data on the process bus would include high speed sampled values which provide current and voltage information to respective protective relays. This information is transmitted at such high rates that any loss would be detrimental to the operation of the substation and could compromise the overall protection scheme in place. Hence the increased importance of reliability in such networks.
Figure 1. Below shows an example of the IEC 61850 based logical topology.
Modular versions of Ethernet switches that provide a mix of noise-immune fiber and convenient copper ports as well as 10M bps, 100M bps, 1000M bps, and 10G bps speeds allow Ethernet LANs to be designed into substations where control and instrumentation equipment and connection techniques are evolving and changing rapidly. Highly reliable Ethernet products make it possible to maintain high availability of electric power to the public.
While the migration to IP-enabled devices are driving the implementation of Ethernet in Sub-Stations and Generating Stations, the case for migration from low speed, serial, analog, asynchronous circuits and onto TCP/IP can be justified many ways. Here are a few examples:
- Eliminating Leased Circuits: business cases for conversion to IP can be built completely around the cost savings of eliminating leased circuits. Perhaps the greatest potential for financial benefit may lie here, as leased protection circuits might be in place now and upgrading/consolidating with a digital circuit may provide some significant operational cost reductions.
- Increased Data Speeds: Without getting into specifics on existing costs that utilities are paying for their frame relay circuits many utilities find that even sub-rate circuits are charged as a full T-1 from the customer premises to the Telco central office. Then there are typically additional charges for the WAN portion. If utilities where to move to Ethernet they would likely find that the costs will be the same or lower, but with much higher data speeds. The typical low tier IP service is 5 or 10 Mbps. This provides additional bandwidth to support the Smart Grid applications.
- Frame Relay Migration: Carriers may increase the rates on frame relay circuits to motivate customers to move to IP.
- Improved Maintainability of the Network: Maintainability is improved by migrating to TCP/IP. With modern packet based systems remote network monitoring and troubleshooting is vastly improved versus a serial data circuit. SNMP protocol is incorporated into most CPE equipment.
- Increased reliability and decreased time to repair TCP/IP versus leased lines. Most Telcos are moving to Metro Ethernet and MPLS for their private line offerings. Point-to-point analog, T-1, and frame relay are moving to the “legacy” offerings status. Utilities will likely not be able to order new connections with these legacy circuit types in the near future. The Telco may even request that Utilities replace the existing circuits with current service offerings.Staying with leased legacy circuits may result in decreased reliability and increased time to repair when troubles occur. This is usually the result when the Telcos reduce the support and training of legacy services. Simply put, the Telcos eventually find that they don’t have anyone who knows how to work on the older circuits.
- More bandwidth for less Cost:
- Operational data:
- The need for more operational data to improve visibility and operational awareness at the control center has been traditionally limited due to low bandwidth channels implementing a serial protocol.
- Taking advantage of new systems, like EMS and OMS, may require more data from the field devices. Implementing new or upgrading system is an opportune time to upgrade old protocols and slow connections.
- Non-operational data is an increasing trend to provide more data and files that are not required for normal operations. Moving to IP facilitates sending data directly where it needs to go, allowing operations to see only the data they require while the utility enterprise also has access to the data and files they require to improve system performance and operation.
- Remote engineering access to substation IEDs (relays, meters, RTUs, data concentrators, HMI, etc). This was typically done as separate dial up connections, or perhaps several of them, that may now be removed because of NERC CIP. Convergence not only supports the security requirements, but it reduces circuit costs and the costs associated with driving to substations to:
- Manage IED configurations to meet applicable security requirements
- Review and update device settings
- Manage assets
- Additional Bandwidth makes is cost-effective to remotely monitor support equipment: Battery maintenance and other asset management applications (relays, transformers, breakers, reactors, cap banks, etc) can be securely supported by the additional bandwidth without impacting the control center and reducing the need to be at the substations.
- Disturbance recording – access to dedicated DFRs, GFIs or protective relays with DFR-like capabilities to download oscillography to meet NERC PRC requirements without impacting the control center.
- Corporate network access – this is something valued highly by field workers, not only providing access to corporate email, but also secure file servers for configurations, drawings, documentation, etc; however, justifying these soft costs may not be easy to accomplish.
- Operational data:
- Security is another concern that is reduced by going to TCP/IP, if implemented properly. Most customer premises equipment (CPE) would attach to the Telco network using encryption capabilitiesthat support securing these Smart Grid applications to enable NERC CIP, NIST, and/or best practices. But security is more than just encryption; it is adding other security applications that also require bandwidth: monitoring, logging, and reporting on the cyber security and physical security.