Distributed control systems based on high accurate timing synchronization

Abstract

This thesis is focused on high accurate timing synchronization technologies and their utilization in different kind of applications from scientific facilities to industrial infrastructures. This aims to develop systems which includes new generation System-on-Chip (SoC) devices combining high data bandwidth capabilities together with high accuracy timing synchronization protocols over the same network. In the initial sections, a full revision of the state of the art is presented in regards of timing synchronization technologies taking into consideration their advantages and drawbacks. Some of them are based on timing signals as 10 MHz square/sine wave or 1 Pulse Per Second (PPS) ones. Other alternatives deploy Global Navigation Satellite System (GNSS) [1] with many receivers creating a synchronization network via wireless links. In contrast to the timing signal solutions, there are standard packet-based protocols as Network Time Protocol (NTP) [2] [3] or Precise Time Protocol (PTP) [4] that are widely used in Ethernet-based networks. However, synchronization accuracy of these are limited to millisecond and microsecond scale respectively. While it is true that are metrology solutions, that can provide accuracies up to few picoseconds, they require very specific and expensive equipment and they are not easily adapted to be used in many applications. Under this context, a new standard packet-based solution is raised to overcome time synchronization performance issue: White Rabbit (WR) [5] [6] technology. It is based on Precise Time Protocol version 2 (PTPv2) but includes some enhancements to reach a time synchronization accuracy in the sub-nanosecond scale with a precision of picoseconds. Currently, WR technology has been successfully integrated in many scientific applications specially in the context of High-Energy Physics (HEP) and astrophysics facilities as Square Kilometer Array (SKA) [7] and Cherenkov Telescope Array (CTA) [8]. Then, the design and development of a new family of devices for WR systems based on new generation programmable SoC devices have been accomplished in a partnership with industrial partners. Under this context, a new platform has been used to implement a enhanced WR node taking advantage of WR synchronization performance and, at the same time, offering advanced software capabilities. Due to the goodness of this solution, it has been proposed to be used in SKA, concretely for the PPS distribution system. In addition to the development, some tests have been performed for characterizing the system in terms of timing performance, scalability and the effects derived from temperature variations. Such results guarantee that the proposed system fulfills the SKA needs. Apart from the timing synchronization topic, high data bandwidth networks have been studied and, specially, its application in Data ACQuisition (DACQ) systems. They are usually composed of many distributed sensors which generate data that are typically processed by a central server. Consequently, data aggregation mechanisms must be implemented to join several network connections from sensors to a single channel in order to reach the central server. On the other hand, DACQ systems also require a fully configurable and flexible routing communication channel between the central server and sensors for control/monitor tasks. These requirements are very specific for conventional high data bandwidth networks that can not be applied successfully for many DACQ systems. In this regard, a novel and generic asymmetric network architecture has been proposed to overcome this issue providing aggregation and routing capabilities. Due to the flexiblity and optimized design of this solution, it has been selected to be used in the CTA infrastructure, concretely inside the Compact High Energy Camera (CHEC). In this scenario, several tests have been performed to verify that this solution fulfills the CTA requirements obtaining results that outperform them. Furthermore, the proposed solution has been integrated properly in the CHEC at Deutsches Elektronen-Synchrotron (DESY) [9] in a collaborative framework with a CTA partner. In collaboration with an industrial partner, other important thesis contribution has consisted on the update of the WR technology to work with high data bandwidth networks as 10 Gigabit Ethernet (10G) ones. This development is required in order to overcome WR limitations in terms of data bandwidth and interoperability, enabling its utilization in other applications in which it has not been a feasible option for the timing synchronization system up to now. In this regard, a fully modular solution has been developed taking into consideration the learned concepts from the SoC developments and the asymmetric network design given as a result an unified architecture for data and synchronization purposes. This is able to provide high accurate timing synchronization together with high data bandwidth transfer services. Moreover, this solution has been validated obtaining a timing performance comparable or even better in some aspects than standard WR devices. Additionally, a system characterization has been performed measuring the data bandwidth, latency and the interoperability with commercial 10G devices obtaining satisfactory results. Finally, it is important to remark that for first time in the literature, a timing system able to provide high accurate timing synchronization and 10G data distribution has been presented, avoiding the deployment of different separated networks for data transfer and synchronization purposes.