Implementing Network Time Security at the Hardware Level

At Netnod, we have been busy with Network Time Security (NTS) for some years now. We worked on the IETF proposed standard published in 2020 as RFC8915 and introduced an NTS-enabled time service in 2019 with NTS deployed in software. The next step was to implement NTS at a hardware level. This post will look at some of the challenges involved in this hardware deployment and how it helps provide even more secure time services.

What is NTS?

Many devices connected to the Internet rely on the Network Time Protocol (NTP). However, NTP exposes networks to security issues. As the protocol was created in 1985, and not designed with security in mind, it is vulnerable to attack. This is especially problematic given that accurate and secure time is crucial for some of the key security technologies upon which networks and society as a whole depend. NTS has been designed to fix this.

How does NTS work?

NTS uses modern cryptography to add an important layer of security to NTP services. It prevents spoofing and MITM attacks by using authenticated packets. Amplification attacks are prevented by ensuring that request and response packets are always the same size.

NTS is really two protocols: a key establishment protocol, and NTP with some new Extension Fields. The reason for using two protocols is separation of concerns:

1. The seldom used key establishment on top of standard Transport Layer Security (TLS), and;
2. The (already existing) low latency UDP-based time synchronisation path.

This means that the existing NTP functionality is the same as before, but the time data can now be authenticated.

The authentication process consists of a key establishment and a timestamp request. The key establishment server typically runs on an ordinary computer, but the slim NTS-enabled NTP server is UDP-based and stateless. It can be served from anycast addresses and can be implemented at the hardware level. The NTP server’s state about each client is kept in a cookie provided by the client itself with each request. As there can potentially be hundreds of millions of clients, this is crucial for the smooth operation of a large-scale NTS service. If you want to find out more about how the NTS key establishment and timestamping process works, see Netnod’s NTS white paper from 2020.

Since the cryptographic operations in the NTS path are symmetric it is both easier to implement them in hardware and possible to make them use constant time. This increases the accuracy of the time synchronisation and keeps the slower key establishment outside of the time synchronisation path.

NTS uses Authenticated Encryption using the Advanced Encryption Standard (AES), more specifically what is known as Synthetic Initialisation Vector (RFC5297). This is a block cipher mode of operation providing nonce-based, misuse-resistant authenticated encryption. Using AES-SIV enables the encryption processes to add integrity and origin authentication. The consequences for this at a hardware level will be discussed below.

Implementing NTS at a hardware level

The benefits

Netnod saw three main benefits in implementing NTS at a hardware level.

• Firstly, it enables more deterministic processing time. This improves both the quality of the time synchronisation provided and the security of the service by reducing the ability for side-channel attacks on the keys. Side-channel attacks exploit the physical effects caused by processing: for example the variance in power consumption or the variance in how long it takes to perform a given operation.
   
• Secondly, it makes the service much more robust and secure. An attacker able to inject or modify code (for example by specially crafted network packets) in the computer on which a software implementation is running could alter the behaviour of the implementation. A hardware implementation significantly reduces the possibility of such attacks.
   
• Finally, it provides improved efficiency and scalability. NTS processing relies heavily on cryptographic operations more suitable for a hardware rather than software implementation. The former enables an efficient, compact implementation of the NTS core functionality while also allowing for scalability. As opposed to having multiple servers with a load balancer, the hardware implementation allows for everything to be done internally using a smaller amount of equipment and consuming less power. These efficiencies lead to a much lower operational cost compared to using a cluster of servers.

The challenges

Due to the complex nature of the NTS timestamp request, the packets must be read and modified in multiple passes. Furthermore, the security mechanism selected for NTS (AES-SIV, see RFC5297) does not readily allow for streaming without significant duplication of FPGA resources. Instead, multiple passes over different parts of the packet are needed to perform an encryption or decryption operation such as decrypting a cookie or verifying the packet. Each pass requires multiple AES operations.

This means that an NTS packet needs to be stored. It also means that tens to hundreds of cryptographic operations are needed for a single NTS packet. These operations can not be performed in parallel without much unnecessary complexity and duplication of resources.

The end result is that at 10 Gbps the processing time required for a packet is longer than the minimum time between incoming packets. We realised a single NTS engine would not be able to handle all traffic for a port and that we needed multiple engines working in parallel.

This meant that the architecture of the NTP FPGA design had to be changed. For more details on how we redesigned the architecture, see Netnod’s NTS white paper from January 2022.

The deployment

The first version of Netnod’s FPGA implementation has been publicly available for tests since August 2020.

We have ported our FPGA implementation of an NTP/NTS server to the Arista 7130-48LB FPGA switch. The source code is available on github here: https://github.com/Netnod/FPGA_NTP_SERVER/tree/arista

Since February 2022, the Arista FPGA implementation has been live in production on sth1.nts.netnod.se and sth2.nts.netnod.se. This is where you should connect if you need a production service. You can also use nts.netnod.se which will resolve to all of Netnod's production NTS servers.

We also have our test servers running on 1ab1.nts.netnod.se and lab2.nts.netnod.se. which are where we will test any future improvements. The lab machines can break in interesting ways without any warning, so please do not use them for anything important. But we appreciate it if you test your clients with the lab machines and report any issues with them to us.

If you have any questions or suggestions about Netnod’s NTS work, please feel free to contact me.

Further information

• For more details on NTS background and how the NTS key establishment and timestamping process works, see Netnod’s NTS white paper from 2020.
   
• If you are interested in the background to our FPGA implementation of NTS, you can find more details in Netnod’s NTS whitepaper from January 2022.
   
• Netnod’s NTP/NTS services are freely available to anyone. For our best practise on connecting to NTP servers, see my RIPE Labs article. The same advice is relevant for NTS as it is an extension of NTP.
   
• While NTS helps secure time distribution over the public internet, networks requiring the highest levels of accuracy and reliability often use Precision Time Protocol (PTP) services. When used over a dedicated connection, PTP can provide better accuracy and reliability than NTS services. For more information about PTP and its use cases, see the Netnod blog post here and the Q&A focused on the energy sector here.

This post is adapted from an article I originally published on RIPE Labs.