Resilience, redundancy and security to ensure accurate timing and synchronization of critical infrastructure

Making the right architectural choices is critical to ensure sustained performance. A comprehensive network engineering design study should include where the master clock unit needs to be deployed and its performance and accuracy requirements. These steps will guide you on what type of precise timing and synchronization equipment to choose.

By Eric Colard, Head of Emerging Products, Frequency & Time Systems Business Unit

Critical infrastructure services such as telecommunications, utilities, transportation and defense are of national strategic importance. The U.S. Cybersecurity and Infrastructure Security Agency (CISA) lists 16 such departments deemed critical to security. Presidential Policy Directive 21 (PPD-21): Critical Infrastructure Security and Resilience sets out a national policy to enhance and maintain the security, functioning, and resiliency of critical infrastructure.

Positioning, Navigation and Timing (PNT) together constitute a necessary condition for the proper functioning of the nation’s critical infrastructure. However, widespread adoption of the Global Positioning System (GPS) as the primary source of PNT information introduces vulnerabilities. CISA works with government and industry partners through the National Risk Management Center to enhance the security and resiliency of the U.S. national PNT ecosystem. Executive Order (EO) 13905 (EO), Enhancing Nation’s Resilience Through Responsible Use of Positioning, Navigation, and Timing (PNT) Services, signed in early 2020, promotes policy promotion to enhance responsible use of PNT services by governments and infrastructure operators .

The following outlines cost considerations and explores three key elements of critical infrastructure that help enhance PNT with an emphasis on synchronization and precise timing: redundancy, resiliency, and security.

Assess cost and location

It is often difficult for operators to justify the costs associated with deploying resiliency, redundancy and security at each layer of the architecture. New timing and synchronization solutions and design choices contribute to a reasonable cost structure and provide a robust and reliable solution.

The issue between cost and solution type is often related to the deployment location being considered. As technology evolves, such as the migration of SDH/TDM to Ethernet and the development of mobile LTE/4G and 5G, the number of cluster offices (especially network access points located at the edge) has proliferated. This necessarily results in smaller devices (usually 1-U rack-mountable devices) and costs in line with small form factor edge base stations (small cells and gNodeBs).

Operators are faced with the question: what is the best way to provide redundancy, resiliency and security in this environment? There are two core levels to consider – the architectural level and the design level.

Explore Redundancy

Architecture-level redundancy can be designed by deploying core functions at both ends (east/west), directional redundancy with dual paths, and high-performance functions for long-distance, high-efficiency, high-accuracy time transfer for cost-effective distribution. The Virtual Primary Reference Clock (vPRTC) architecture is one such architecture-level solution.

The equipment itself can also be considered for redundancy. At this point, the choice of design is crucial. Small appliances can’t really be cost-effectively designed with modular hardware redundancy. The innovation here is to provide software redundancy in order to deploy low-cost, high-efficiency, and high-performance distributed solutions. Hardware modules are usually expensive for two reasons: one is cost, and two is that redundant modules take up space in another module (usually for input and output ports).

Hardware module redundancy often results in a trade-off between increased redundancy and loss of functionality, e.g. between 10G Ethernet (GE) support or multi-band Global Navigation Satellite System (GNSS) if redundancy is supported choose or make other compromises. On the other hand, with software redundancy, there are no tradeoffs to make. This means that redundancy can be introduced while retaining all existing functionality, without having to remove inputs or outputs, and without having to deprecate multi-band GNSS functionality. Redundancy is introduced through a software upgrade, so no hardware is removed. However, hardware redundancy means that an existing module is duplicated inside the device with a similar module; the new module occupies the slot of the existing module, and the existing module loses function when removed from the unit.

Figure 1 shows a commonly deployed redundancy use case involving two aggregation routers using the Virtual Router Redundancy Protocol (VRRP).

Resilience, redundancy and security to ensure accurate timing and synchronization of critical infrastructure
Figure 1: Example of redundant connections between working and standby units

Software redundancy is a two-unit approach based on two reasonably priced devices, with one unit active and the other standby. This approach is more cost-effective: firstly, it does not involve expensive equipment design containing expensive hardware modules; secondly, each unit (both inactive and active) retains all its functions, whereas hardware redundancy design involves in the equipment Duplicating modules, by making room for redundant modules, is likely to reduce existing functionality. Also, since the working unit and the standby unit are the same, the software redundancy is the total redundancy of the entire plant. All functions are redundant, including oscillators, GNSS receivers, ports, and I/O, while hardware modules are redundant only in their own functions (not the rest of the unit).

Take advantage of flexibility

Architectural level resiliency is key to network design to ensure that the grandmasters in deployments can be connected to each other. Some master clocks connect to GNSS and use it as a source of time and frequency. It is important to link these systems to other 1588 master clocks to implement Auxiliary Partial Time Support (APTS) and take advantage of key innovations such as Automatic Asymmetry Correction (AAC). AAC is a key (patented) advantage in resiliency design, the ability to calibrate the different paths to/from the upstream grandmaster that may be used by the PTP flow, allowing for backup in the event of GNSS failure at the grandmaster location. The backup path of the upstream master clock ensures uninterrupted precise timing and phase operation. This architecture ensures that GNSS can be backed up by the IEEE 1588 Precision Timing Protocol (PTP) in the event of an outage and utilizes the best path.

Another architectural choice is virtual PRTC (vPRTC), which enables operators to achieve high accuracy over long distances (usually over optical networks) by leveraging redundancy and resiliency using PTP’s high-performance boundary clock chain; this architecture reduces It relies on GNSS and uses PTP as its primary time and phase source.

Figure 2 shows an optical network deployment with a dedicated optical timing channel (OTC) that enables high-precision phase distribution over longer distances.

Figure 2: Optical network deployment with OTC

Device-level resiliency starts with the right choice of oscillators (from OCXOs to atomic clocks (rubidium)), depending on the location, use case, and corresponding timekeeping performance requirements. In addition, the choice of GNSS receiver is critical because some receivers typically support a single frequency, but ionospheric phenomena can cause considerable delays during periodic events such as solar storms; to reduce such delays, it is necessary to Use a multi-band GNSS receiver.

Figure 3 compares single-band and multi-band delays due to ionospheric effects and shows how multi-band significantly reduces time error (highlighted in red).

Figure 3: Comparison of ionospheric phenomena. Source:…/Galileo-OS-SDD.pdf

GNSS satellites transmit time information in multiple frequency bands. The difference in delay between different frequency signals provides information on the effect of the ionosphere on the absolute delay. Using this information, multi-band GNSS receivers can compensate for delay differences in the radio signal sent from the satellite to the receiver. Embedding a multi-band receiver reduces such delays, which are critical for applications that require a 40 ns Class B primary reference clock (PRTC-B) and a 30 ns enhanced PRTC (ePRTC).

The choice of these device designs is equally important. GNSS receivers can either be embedded into a unit on the motherboard or offered as a hardware module, but the latter usually comes at an additional cost and may require the removal and replacement of existing modules. It is preferable to take a multi-band receiver-enabled unit and license the multi-band capability, rather than provide the multi-band option on a hardware module, as the latter approach requires trade-offs with other important features.

Assess security

Safety is paramount. Authentication and authorization through standard mechanisms such as Terminal Access Controller Access Control System+ (TACACS+) and Remote Authentication Dial-In User Service (RADIUS) provide the benefits of a standard security framework. Additionally, two-factor authentication (2FA) is an extra layer of protection that goes beyond just securing accounts with usernames and passwords.

Also, be sure to provide different levels of security profiles for the Secure Shell (SSH) extension to provide more granularity for user types and associated access rights and restrictions. Provides a high security profile to define and enforce the strictest system access rules.

Scripting vulnerabilities and related Common Vulnerabilities and Exposures (CVE®) issues need to be addressed to ensure that all potential security vulnerabilities are reviewed and addressed.

Additionally, evolving jamming and spoofing threats need to be part of a precise timing security strategy, enabled by signal monitoring, consistency checks, and remediation. Automatic gain control (AGC) and other metrics can be used to provide thresholds and explain the corresponding results and mitigations when they occur.

final decision

Making the right architectural choices is critical to ensure sustained performance. A comprehensive network engineering design study should include where the master clock unit needs to be deployed and its performance and accuracy requirements. These steps will guide you on what type of precise timing and synchronization equipment to choose.

Additionally, network planners and synchronization engineers should pay particular attention to design choices such as fanless versus fan-required devices, modular hardware redundancy versus software redundancy, cost and trade-offs, and similar concerns about embedded or modular choice of GNSS.

These choices can lead critical infrastructure operators to deploy redundancy, resiliency, and security at all deployment levels.

The Links:   BSM50GB120DLC LA084X01-SL01