Measuring code coverage with GHDL

I have recently switched to using GHDL as my VHDL simulator of choice. GHDL (currently) supports three different backends: GCC, LLVM and a builtin code generator.

The GCC backend got me thinking… since my VHDL test benches are now native ELF binaries compiled with GCC, I should be able to instrument them with Gcov support – and measure code coverage for each unit under test (UUT).

Turns out to be pretty simple – just a matter of using the right compiler and linker flags, just like building a Gcov-instrumented binary from C code.

Take for instance the full adder example from the GHDL documentation – once the adder.vhdl and adder_tb.vhdl files are saved to disk, it is just a matter of modifying the GHDL commands a bit to instrument the final binary with Gcov support.

First analyze the design files with -fprofile-arcs and -ftest-coverage added to CFLAGS for all the VHDL files, that should be instrumented. Here, I have only instrumented the UUT, not the test bench:

$ ghdl -a  -Wc,-fprofile-arcs -Wc,-ftest-coverage adder.vhdl
$ ghdl -a  adder_tb.vhdl  

Next, elaborate the test bench and link the binary against libgcov:

$ ghdl -e -Wl,-lgcov adder_tb

Run the test bench as usual:

$ ghdl -r adder_tb
adder_tb.vhdl:53:7:@8ns:(assertion note): end of test

Generate a code coverage report using the gcov(1) tool:

$ gcov -s $PWD adder.vhdl 
File 'adder.vhdl'
Lines executed:100.00% of 4
Creating 'adder.vhdl.gcov'

The Gcov code coverage details can be found in the adder.vhdl.gcov file. Details on the file format can be found in the gcov(1) man page:

$ cat adder.vhdl.gcov 
        -:    0:Source:adder.vhdl
        -:    0:Graph:adder.gcno
        -:    0:Data:adder.gcda
        -:    0:Runs:1
        -:    0:Programs:1
       13:    1:entity adder is
        -:    2:  -- `i0`, `i1` and the carry-in `ci` are inputs of the adder.
        -:    3:  -- `s` is the sum output, `co` is the carry-out.
        -:    4:  port (i0, i1 : in bit; ci : in bit; s : out bit; co : out bit);
        -:    5:end adder;
        -:    6:
        2:    7:architecture rtl of adder is
        -:    8:begin
        -:    9:   --  This full-adder architecture contains two concurrent assignment.
        -:   10:   --  Compute the sum.
       17:   11:   s <= i0 xor i1 xor ci;
        -:   12:   --  Compute the carry.
       10:   13:   co <= (i0 and i1) or (i0 and ci) or (i1 and ci);
        -:   14:end rtl;

Another option is to use a tool like Lcov, which can produce nice looking HTML pages with the Gcov code coverage details:

$ lcov -c -d . -o adder_tb.info
Capturing coverage data from .
Found gcov version: 4.9.4
Scanning . for .gcda files ...
Found 1 data files in .
Processing adder.gcda
Finished .info-file creation
$ genhtml -o html adder_tb.info 
Reading data file adder_tb.info
Found 1 entries.
Found common filename prefix "/home/brix/tmp"
Writing .css and .png files.
Generating output.
Processing file adder/adder.vhdl
Writing directory view page.
Overall coverage rate:
  lines......: 100.0% (4 of 4 lines)
  functions..: 100.0% (7 of 7 functions)

The Lcov code coverage details can be found in the html/index.html file:

Linux on the Arty, Part 0: Establishing a build environment

I have long wanted to explore the endless possibilities of soft core CPUs like the Xilinx Microblaze. A few months back, I stumbled across the announcement of the Arty by Digilent/Avnet/Xilinx – an affordable evaluation/development board clearly built for makers and perfect for getting my feet wet with a few Microblaze-based projects.

Part0-Arty-0

I initially wanted to get GNU/Linux up and running on a Microblaze CPU on the Arty to establish a familiar software environment for further tinkering. This proved to be quite a task in itself – a task which required reading through lots of product guides, manuals, blog posts and howtos online. Lots of documentation has been written on the Microblaze subject, but none of it was quite to the point or simple for me to follow.

In the following series of blog posts I will describe my way of accomplishing this goal; Linux on the Arty, with all on-board peripherals up and running and exposed to userland.

In the first posts, post 0, we will establish a build environment for both the hardware and the software. Later posts will span from establishing a bare-minimal, Linux-ready hardware design to getting each on-board peripheral of the Arty included in the hardware design, supported by the Linux kernel and exposed to the GNU/Linux userland.

My main development environment is a Macbook running OS X. I constantly use VMware Fusion for running various other (typically either FreeBSD or GNU/Linux) virtual machines to fulfill my needs for other development platforms. For developing the hardware and building the software for the Arty, I have installed CentOS 7 in a VM. I then use ssh to establish a local connection from OS X to the CentOS VM and forward the X display to OS X, thus allowing my to see e.g. the Xilinx Vivado windows on my OS X desktop through XQuartz.

You could of course use any PC capable of running Xilinx Vivado for the hardware part, but note that you will need a GNU/Linux host for building the Linux kernel and GNU/Linux userland. We will be focusing on Vivado 2015.4 and buildroot 2016.02 for this blog series.

If you plan on using a virtual CentOS 7 for the build environment, as I do, I recommend going with at least 50 GB HDD. The Xilinx Vivado installation itself will take up at least 20 GB and the hardware/software projects will take up another 5 GB. You will need a few packages and customizations on top of the Minimal 64 Bit installation image. Also, make sure you create an administrator user account during the installation.

Part0-CentOS-0

First off, almost all CentOS command-line tools will complain about missing locale settings. Fix this by logging in and setting the locale to e.g. U.S. English with UTF-8 encoding by issuing the following command:
sudo localectl set-locale LANG=en_US.utf-8
Remember to log out and back in for the locale settings to take effect.

Next up, install and start the Open VM Tools (skip this if not running under VMware):
sudo yum -y install open-vm-tools
sudo systemctl enable vmtoolsd
sudo systemctl start vmtoolsd

Before we install the Xilinx Vivado suite, we will need to install a few dependencies and tools:
sudo yum -y update
sudo yum -y groupinstall 'Development Tools'
sudo yum -y groupinstall 'X Window System'
sudo yum -y groupinstall Fonts
sudo yum -y install glibc.i686 libstdc++.i686 fontconfig.i686 libXext.i686 libXrender.i686 glib2.i686 libpng12.i686 libSM.i686
sudo yum -y install evince wget ncurses-devel bc screen
sudo reboot

The *.i686 packages are needed by the Xilinx Vivado Documentation Navigator, which for some reason is still shipped as a 32 bit binary. Evince is a PDF viewer, which we will be using from within the Documentation Navigator.

Download the Vivado HLx Web Install Client for Linux and install it using the following commands:
sudo mkdir /opt/Xilinx
sudo chown $USER:$USER /opt/Xilinx
chmod +x Xilinx_Vivado_SDK_2015.4_1118_2_Lin64.bin
./Xilinx_Vivado_SDK_2015.4_1118_2_Lin64.bin

If you get an error about no X11 DISPLAY variable being set, make sure you’re logged on to the CentOS host using ssh -Y .... The -Y option will enable trusted X11 display forwarding.

Make sure to enable installation of the Software Development Kit, as this will be needed later on. You propably already aquired a license for Xilinx Vivado using the voucher included with the Arty, so you can skip acquiring a licence during the installation.

Part0-Vivado-0

The Xilinx Platform Cable USB Driver/Digilent JTAG Driver are a separate install requiring root access. Install them using the following command:
cd /opt/Xilinx/Vivado/2015.4/data/xicom/cable_drivers/lin64/install_script/install_drivers
sudo ./install_drivers

We also need to make sure that our user account has access to the USB JTAG and serial port devices exposed by the Arty. These will be owned by the dialout group, so add our user to that group:
sudo usermod -G wheel,dialout $USER

To be able to launch the various Xilinx Vivado tools, we also need to set up our shell accordingly.
echo 'source /opt/Xilinx/Vivado/2015.4/settings64.sh' >> ~/.bashrc
source /opt/Xilinx/Vivado/2015.4/settings64.sh

Finally, install the license for Xilinx Vivado obtained by using the voucher included with the Arty:
mkdir -p ~/.Xilinx
mv Xilinx.lic ~/.Xilinx/Xilinx.lic

Finally, launch docnav, open the Settings Dialog and change the PDF Viewer Path to evince.

Part0-Vivado-1

This concludes the first part of this blog series. The next part will describe how to do the basic, Linux-ready hardware design.