Tracing the Evolution and Solutions to Finding the Best Data
It Isn’t the Best Source You Want, but the Best Data You Need
Tim Gatton
Product Line Manager,
Telemetry and Data Systems
Wyle
ABSTRACT
In today’s wide ranging and long duration test environments, aircraft typically traverse many different geographical areas. The RF coverage of each area is the responsibility of any number of managerial departments and/or architectural approaches and capabilities. Yesterday’s environment allowed postmission processing to bring data sets together under one file set, but today’s resources and mission demands require a real-time solution because there is just no time to postprocess. The need is NOW!
Best Data SelectorThe Evolution of Solutions
Over the past years, Best Source Selection (BSS) technology has evolved tremendously. In earlier days, the man-machine interface was a human observing indicators and moving patch cables from one source to another. Technology then evolved to a level of automation by switching to sources based on receiver automatic gain control (AGC). This was better, but it still offered large data gaps.
The third generation of automation was a switch to sources based on decommutator (decom) status. Certainly, this third level of BSS advanced the goal of “perfect data.” However, this still induced data gaps and even erroneous data due to the inherent latency that exists between streams coming from different physical locations and from vastly different architectures. With today’s differing architectures and hardware solutions, the user may find one source arriving via direct RF link, a second arriving from a remote location with data buffered through ATM or IP connections, and a third directly arriving via fiber optics.
This is a difficult concept to grasp, so let’s provide an example. First, look at the data structures (both input and output) in figure 1. In this example as is typically the case, the output is to be processed at the central data center for classic real-time display and archiving. With a third generation “decom-only” approach to best source selection, the following example and its impact is offered. Frame 1 is shaded for ease of identification.
The starting conditions assume that all four input sources are in lock and that data is being output. The output, shown at the top of both figures 1 and 2 for ease of diagramming, is outputting data from Source 1 as it is in lock, but the analysis that follows is applicable for any condition.
In figure 1, we assume that Stream 1 is in lock, and the output (at the top) matches Stream 1 (just under the OUTPUT link) because it is in lock and making 100 percent of the output contribution.
Figure 1
Now refer to figure 2. At time T0, stream 1 drops out of lock and stops contributing to the decom-only BSS function. The decom-only BSS then switches to steam 2, and its output continues uninterrupted. However, the output result is that Frame -3 is put into the Frame 2 position, Frame -2 is put into the Frame 3 position, and so on.
Figure 2
Then at T1, stream 2 drops out of lock and the decom-only BSS switches to stream 3. Now, Frame 4 is put into Frame 5’s position and so on. The corruption continues and dropouts occur whenever latency exists between data sources as can be seen in Source 4. Compounding this corruption are the natural bit errors that occur in real-time telemetry. In this example, if at T1 the data from Source 3, Frame 4 has bit errors in the data set, it is passed through as “perfect data” (because this is just a best source selector!) even though the same data set (Frame 4) in Source 4 could be perfect; therefore, the true good data is tossed away.
Therefore, third generation decom-only BSS solutions promote poor data integrity by allowing:
- Corruption of the output data set by not addressing time skew
- Data gaps that become enlarged due to the “resync” data loss with each dropout
- Ignoring of perfect data that can exist in another noncontributing data source will be overlooked
- Possesses no ability to handle either static or dynamic time skews of data sources
The Ultimate Need
The ideal BSS is not a best source selector because it is not the best “source” you need but the best “data.” Therefore, the solution is actually the Best Data Selector which, as shown in figure 3, provides:
- Traditional front-end decommutation
- Dynamic buffering with delay variability to align, in time, the incoming data sources
- Algorithms to watch across “N” data sources and on a “bit-by-bit” or “word-by- word” basis and to provide output data that is truly best data—not just best source
Figure 3
Wyle has developed the next generation product—Best Data Selector—to correct the deficiencies of the decom-only BSS.
Decommutation
Wyle has been building decommutation hardware and software for many years. The first stage of the Best Data Selector uses that decom technology and supports:
- Operation to 30 Mbps
- Sync patterns to 64 bits with programmable masks
- No restrictions on word sizes
- Frame sizes to 33,554,432 bits
This technology has been available to users for many years, but it is now part of a larger solution.
Dynamic Buffering
With Wyle's, Telemetry and Data Systems’ Best Data Selector solution, the next stage after decommutation is to implement a dynamic buffer that first aligns by frame sync (a time skew correlator) and then examines all of the word values to find the proper alignment between data sources. As an example, take the following four streams of data as shown is figure 4.
Figure 4
Figure 5 (assuming all streams are in lock) shows that data is coming in but with offsets in the time relationships caused by factors such as satellite links and telecommunications delays. The Best Data Selector brings in all of this data (up to ten buffers worth) and searches the data to find the best data alignment. In Source 1, is the first frame sync aligned with the first frame sync of Source 2, or is it aligned with the second frame sync? In real time, the Best Data Selector examines the word contents of each stream and quickly finds the best alignment as shown in figure 5.
Figure 5
Figure 5 also shows that alignment has been found between the streams and that each time offset has been realized. Once found, however, it does not stop. The Best Data Selector time skew correlator function is dynamic, independently monitoring each incoming data stream and watching for total word alignment so that any change in path delay will be instantly addressed.
Not Just Best Source but Best Data
When the time skew is factored out, the Best Data Selector has the ability to select the preferred output based on individual needs. Preference can be given to a particular stream number, to a stream that has been in lock the longest, to a stream that was in lock last, or to get the best data selected from all of the streams on a word-by-word or bit-by-bit basis.
Figure 6
When selection is bit-by-bit, the Best Data Selector takes the time-aligned data sets, strips across all words, finds the most common word and bit values, and outputs the result that occurs the most often, thereby dynamically shifting from word correlation to bit correlation depending on the level of data corruption that it must address.
In figure 6, the first word (word 1 after the frame sync) has candidate values of A and B (A and B represent a sequence of bits and are not real values). Since A occurs three times (streams 1, 3, and 4) out of the four possibilities, the most common bit values are output, which would be A. Words 2, 3, and 4 have no dispute because they are all the same in all time-aligned data sources. The same “voting” occurs again in time slot position 4. The bit value of D occurs more often and the value of D is output.
This continues through all data sets, words, and streams. The data example is shown in figure 7 with its output resultant table.
This voting mechanism occurs for each bit and word in the defined frame (typically a minor frame but within definitions it can be defined to be a major frame) and outputs the bits that are present the most often in each particular time slot.
The Best Data Selector Product
The Best Data Selector from Wyle provides multiple iterations of the best data selector time correlation and bit-voting applications within each product such that multiple input-to-output pairings are available for implementation. The ability to run multiple iterations of the best data selector allows for a configuration of any number of input and output sources. As an example, with six input ports and three output ports to run every possible combination and configuration—6 to 1, 4 to 1, 2 to 1, three sets of 2 to 1, dual 3 to 1’s, etc.—all without any impact or dependencies between the iterations.
The Main Menu
This menu is the top level GUI for a four-stream Best Data Selector. It shows the status of each incoming stream, the frequency of selection, the amount of correlation obtained within the stream data, and the amount of time correlation that was required. Pop-up menus reflect system set-up parameters from frame sync pattern to data selection criteria. See figure 8.
