FPGA debug using high-bandwidth, mixed-signal oscilloscopes
How do you test and debug a device with hundreds of thousands of internal logic cells and transceiver speeds up to 28 Gbps? Such is the challenge facing designers of today’s industry leading FPGAs. From the perspective of digital debug, the biggest challenges arise from the inaccessibility of critical logic nodes and a limitation on the number of available physical pins. Innovations such as internal signal muxing, JTAG communication, and internal logic analyzers have helped alleviate these challenges. However, all of these techniques offer tradeoffs and require the correct test and measurement equipment.
Digital debug of FPGAs has traditionally been the domain of logic analyzers. Offering from 32 to hundreds or even thousands of digital channels, synchronous and asynchronous acquisition, and complex state triggering conditions, the logic analyzer is, by design, a powerful tool for analysis and debug of digital signals. However, for many applicationslogic analyzers are not the best tool for the job. The mixed signal oscilloscope (MSO) is an extremely powerful tool for applications that require both digital and analog measurements. First, having analog channels allows designers to make critical analog measurements on their digital devices; for instance, testing the transceiver on an FPGA. Further, if something in the logic appears incorrect the designer has correlated analog channels readily available for deeper investigation. MSOs offer broad analog and digital triggering capabilities and deep memory in a familiar and easy-to-use interface.While the MSO is not without its tradeoffs (e.g. typically limited to max channel count of 20 and only capable of asynchronous acquisition based on internal sampling clock), it is a critical tool for designers of mixed signal systems such as an FPGA.
The MSO has traditionally been found on low and medium bandwidth scopes in order to address the heart of the mixed signal market. However, new applications have demanded ever higher data rates, as evidenced by the 28 Gbps transceiver speeds available on today’s FPGAs, driving the need for high bandwidth mixed signal oscilloscopes that can handle both logic analysis and the critical signal integrity challenges of high speed serial measurements. In this article we address these challenges in FPGA debug and how a high bandwidth MSOcan help accomplish these demanding tests.
Given the challenges due to a lack of internal visibility, different approaches have emerged in FPGA debug. The most common approaches to FPGA debug are: direct routing from logic nodes to pins, muxing out signals to pins, and internal logic analyzers.
1. Direct routing to pins
The simplest way to access internal nodes in an FPGA is to leverage the programmability of the device to route these signals out to physical pins where they can be probed by the digital channels of a mixed signal oscilloscope. A simplified diagram of this approach is shown in figure 1. This method, while effective, comes with significant limitations. First, in many cases designers are limited by the number of physical pins available on the FPGA package. This approach requires the designer make a tradeoff between the number of physical pins available and the number of internal nodes available to probe for test and debug. Further, it is often difficult to predict which nodes will need to be observed while debugging the FPGA logic. This challenge becomes aggravated when directly routing nodes to pins and only 8 or 16 pins are available to dedicate to debug. If new signals need to be probed, the FPGA must be redesigned to route these signals out to the physical pins. This process of manually managing design and node-to-pin routing results in equal (and relatively long) time investment between iterations. Inefficiencies aside, this tried and true debug technique is simple and provides both state and timing modes for thorough analysis of the probed signals.
2. Muxing out signals
A variation on direct routing to pins, signals can be muxed out to physical pins on the FPGA, as shown in figure 2. This approach offers many critical benefits. Primarily, the designer is no longer so constrained by physical pins, as the number of internal nodes available to probe is many times the number of physical pins. Using the example shown in figure 2, let’s assume the designer has dedicated 16 pins to logic debug. A 16:1 mux allows the designer to route 256 internal nodes to the multiplexer, and observe them all using only 16 physical pins. In most implementations, and as show in figure 2, the mux selection is controlled using the JTAG interface on the FPGA. This flexibility dramatically reduces the need to redesign the FPGA to observe additional nodes and improves the time between iterations. Further, as signals are still being directly observed at the physical pins, both state and timing modes remain available.
3. Internal logic analyzer
In many instances, FPGA vendors provide internal logic analyzers (ILAs) built into their FPGAs to aid debug. The ILA features trigger circuitry and uses the internal memory to store traces. JTAG communication between the FPGA and a PC is used to configure the ILA and read the logic signals it outputs. This setup is shown in figure 3. The convenience of this setup is that no incremental physical pins are needed and only a PC is required for basic logic analysis.
However, there are many limitations to this technique. The ILA can be a resource hog, monopolizing FPGA slices and internal memory needed for the working logic. Further, only state mode is available using an ILA, timing mode, which allows designers to observe signals relative to one another and measure asynchronous events, is not supported.
That being said, ILAs are common and many vendors offer hybrid debug solutions where both an ILA and muxing can be employed for maximum flexibility. In order to remain a truly multipurpose tool, mixed signal oscilloscopesoffer JTAG decode software so that logic signals output by an ILA can be analyzed directly on the scope.
There are certainly many instances where analog measurements can be invaluable to a digital designer. For instance, being able to see the underlying analog signal can clarify anomalous behaviors in the logic. That being said, state of the art FPGA testing represents a class of truly mixed signal applications where the analog and digital challenges can be equally critical to device performance. The latest FPGAs offer transceiver speeds up to 28 Gbps.
Research and development into ever faster ethernet speeds, a critical market supported by FPGA makers, has driven the need for these bleeding edge transceiver speeds. Current work on 100 Gb Ethernet has focused on 10 lanes X 10 Gbps implemented in CFP modules. A second generation of 100 GbE is being developed employing a 4 X 25 Gbps architecture to be integrated into CFP2 and ultimately CFP4 modules. The advantages of increasing serial data rates and reducing parallelism are a significant reduction in power dissipation and module size enabling higher densities. In order to support the 25 Gbps serial data lanes demanded by the standard, FPGAs have pushed transceiver speeds out to 28 Gbps. Peering another generation into the future, 400 GbE calls for 16 x 25 Gbps.
Designing and measuring a serial data stream at 28 Gbps is very much an analog problem. Insertion loss, reflections, cross talk, and other analog challenges that can be safely ignored at data rates less than 1 Gbps can be catastrophic at 28 Gbps, often resulting in completely closed eye diagrams. Real-time oscilloscopes, including MSOs, offer signal integrity software that can dramatically improve the quality of analog measurements at these data rates. For example, a lossy channel can be de-embedded from the signal path. This allows the user to observe the signal as it appeared prior to propagation through the channel.
The old adage of needing to see at least the 3rd harmonic would require a measurement system with a minimum of 42 GHz bandwidth to measure a 28 Gbps signal. However, this is often not the case. In figure 4 a 28 Gbps PRBS^7 signal is measured at both 33 GHz and 63 GHz of acquisition bandwidth. There is very little difference in these measurements, aside from the unavoidable high frequency noise in the 63 GHz acquisition. An FFT of this signal shows the third harmonic is 30 dB below the fundamental and almost entirely negligible in real world measurements. Still, only recently have MSOs expanded into analog bandwidths high enough to capture this 28 Gbps signal. The industry’s highest analog bandwidth MSOs have achieved 20, and most recently 33 GHz.
There are critical challenges in both digital and analog test and debug of state of the art FPGAs. Digital signals are difficult to access and physical pin count is limited. Meanwhile, transceiver speeds reaching 28 Gbps bring analog non-idealities to the forefront. Making measurements in this challenging environment calls for a high bandwidth mixed signal oscilloscope combining >30 GHz analog bandwidth, superior signal integrity, and 16 digital channels into one integrated instrument. Until recently, such an instrument didn’t exist. However, the latest class of high bandwidth MSOs are uniquely positioned to support these challenging measurements.
Daniel Ruebusch manages strategic marketing of high performance oscilloscopes at Agilent Technologies. Daniel joined Agilent in 2011. He has past experience in semiconductor device physics and processing and consumer sales and marketing. Daniel holds a B.S. from Cornell University in both Electrical Engineering and Materials Science and an M.S. in Electrical Engineering from U.C. Berkeley. He is a published technical author.