SSD Storage

U.2 NVMe SSDs are the types of drives that are commonly used in enterprise servers and data centers. Compared to M.2 NVMe SSDs, which are more commonly found in consumer devices, U.2 drives are physically bigger, using the same form factor as the 2.5 inch SATA SSDs. The bigger form factor allows U.2 drives to consume more power and thus deliver higher sustained performance than their equivalent M.2 versions. This performance advantage is the reason why U.2 SSDs are becoming more popular in FPGA applications.

Hands down, the best solution for non-volatile storage on Versal designs, is NVMe solid-state drives. One of the main reasons why they are such a good fit is because they interface with the host device over PCI Express. Versal Prime, AI Core and AI Edge devices all have integrated PCIe blocks, and most of them are PCIe Gen5 compliant. This means that there are no extra IP costs to connect an NVMe SSD to the Versal devices because everything you need is already built into the device.

The FPGA Mezzanine Card standard (VITA 57.1) has significantly enhanced the FPGA ecosystem by decoupling the FPGA board from the input/output components. Years ago, if you wanted to process samples from an ADC, you would buy an FPGA board with an on-board ADC. If you wanted to add a DAC, or upgrade the ADC, you would replace the entire board! The FMC innovation provided developers with choice, so that they could select the right FPGA for their specific I/O requirements. They could also swap-out their I/O cards as their needs changed or as upgrades or improvements became available.

High-capacity non-volatile storage is pretty handy in the intensive computing applications that the Versal ACAP adaptive SoCs get employed in. NVMe SSDs are a perfect way to provide that storage because they can directly interface with the Versal’s integrated blocks for PCIe. Those integrated blocks are Gen4 compliant which makes for an extremely high bandwidth connection between the FPGA fabric and the storage medium.

Over the past couple of weeks, my team and I have been bringing up an NVMe SSD on the Versal AI Core VCK190 Evaluation kit using the FPGA Drive FMC Gen4 adapter. We’ve built a new reference design from our work and shared it on Github. In this post we’ll look at how you can build and run the example design, and then we’ll partition and write a file to one of the SSDs. For those who are trying to do this from scratch using the QDMA Subsystem for PCIe IP, Vivado and PetaLinux, the last half of this post will be about the things that we found to be important in getting everything working correctly.

Update 2020-02-07: Missing Link Electronics has released their NVMe Streamer product for NVMe offload to the FPGA, maximum SSD performance, and they have an example design that works with FPGA Drive FMC!

Probably the most common question that I receive about our SSD-to-FPGA solution is: what are the maximum achievable read/write speeds? A complete answer to this question would require a whole other post, but instead for today I’m going to show you what speeds we can get with a simple but highly flexible setup that doesn’t use any paid IP. I’ve run some simple Linux scripts on this hardware to measure the read/write speeds of two Samsung 970 EVO M.2 NVMe SSDs. If you have our FPGA Drive FMC and a ZCU106 board, you will be able to download the boot files and the scripts and run this on your own hardware. Let’s jump first to the results.

Early this year IntelliProp released a demo video of their NVMe Host Accelerator IP core running on the Intel Arria 10 GX FPGA Development board. As you can see in the video, they are using Opsero’s FPGA Drive product with the PCIe slot connector to interface the NVMe SSD to the FPGA board. They measured an impressive performance of around 2300MBps sequential write speed and 3200MBps sequential read speed. The FPGA Drive adapter was designed to fully handle Gen3 speeds precisely because these high throughputs are only possible with a Gen3 interface (note that M.2 SSDs have a 4-lane PCIe interface).

I’ve just done a video to demo Intelliprop’s NVMe Host Accelerator IP core on the Xilinx Kintex Ultrascale KCU105 dev board and the Samsung 950 Pro M.2 NVMe SSD. To connect them together I’ve used the FPGA Drive FMC plugged into the HPC connector to give us a 4-lane PCIe Gen3 interface with the SSD. The read/write speeds I got are simply incredible and line up very well with the numbers I wrote about in an earlier post. So here they are:

Just released a video showing how to connect an M.2 SSD to the FPGA Drive FMC.

I’ve been totally overloaded with projects in the last couple months but I’m back with some really exciting news today. A few months back a company called IntelliProp, based in Colorado, released a NVMe Host Accelerator IP core for interfacing FPGAs with NVMe SSDs. This IP core allows reads and writes to be performed directly from the FPGA fabric, without the latency overhead of an operating system (read about the NVMe speed tests I did under PetaLinux). IntelliProp has tested their IP core with an FPGA Drive FMC loaded with a Samsung 950 Pro 256GB SSD and here are the results:

Orders can now be placed for the FPGA Drive products on the Opsero website. Both the PCIe and FMC versions allow you to connect an M.2 PCIe solid-state drive to an FPGA development board and both can be purchased at the same price of $249 USD (solid-state drive not included).

The PCIe version has an 8-lane PCIe edge connector for interfacing with the PCIe blade (aka. goldfingers) of an FPGA development board. The board is powered by 12VDC so it comes with a power cable which allows you to power the FPGA Drive from the same power adapter that supplies power to the FPGA board.