Testing LAVA with DummySys

jeu. 19 janvier 2017 by Rémi Duraffort

For more than 3 years now, I have been working on LAVA. Looking at the documentation:

LAVA is an automated validation architecture primarily aimed at testing deployments of systems based around the Linux kernel on ARM devices, specifically ARMv7 and later.

In order to test LAVA, you need:

  • a wide range of board types
  • boards that are stable (always the same behavior)

It's also recommended to have multiple boards of each type to stress test LAVA by running many jobs in parallel.

Boards are usually expensive and take a lot of space on the desk. That's the reason why I created DummySys.

DummySys genesis

DummySys is just a set of libraries and binaries to create dummy systems.

The idea is to create a program that will interact with the outside world like the original system. The dummy system should accept the same inputs and provide the same outputs.

For instance for a board flashed with u-boot and booting with tftp and nfs, a dummy system should:

  • Provide excepted logs over a serial connection
  • Load the right files with tftp
  • Mount the NFS directory and grab some files

If implemented correctly, LAVA (or any similar programs) won't notice the difference between the real system and the dummy one.

Using DummySys with LAVA

To use DummySys, just grab the sources from Framagit or GitHub:

$ git clone https://framagit.org/ivoire/DummySys.git

The LAVA board configurations are available in share/conf/lava/.

Each configuration does match a given board and a given boot strategy.

For instance, to emulate a Beaglebone-Black board that is flashed with u-boot and booting with tftp and NFS, you should use share/conf/lava/uboot-bbb-cmd.yaml.

To start the DummySys for this Beaglebone-black, run:

$ python3 lava-board --commands share/conf/lava/uboot-bbb-cmd.yaml
Trying ::1...
Connected to localhost.
Escape character is '^]'.

U-Boot SPL 2014.04-00014-g47880f5 (Apr 22 2014 - 13:23:54)
reading args
spl_load_image_fat_os: error reading image args, err - -1
reading u-boot.img
reading u-boot.img

U-Boot 2014.04-00014-g47880f5 (Apr 22 2014 - 13:23:54)
I2C:   ready
DRAM:  512 MiB
NAND:  0 MiB
*** Warning - readenv() failed, using default environment
Net:   <ethaddr> not set. Validating first E-fuse MAC
cpsw, usb_ether
Hit any key to stop autoboot:  1

In LAVA, create a Device Type for the Beaglebone-Black and add a Device:

$ lava-server manage add-device-type beaglebone-black
$ lava-server manage add-device --device-type beaglebone-black --worker $(worker) bbb-01

As for each v2 devices, you have to add a device dictionary:

$ lava-server manage device-dictionary --hostname bbb-01 --import bbb-01.yaml

The device dictionary should be:

{% set power_off_command = '/bin/true off' %}
{% set power_on_command = '/bin/true on' %}
{% set hard_reset_command = '/bin/true reboot' %}
{% set connection_command = 'python3 lava-board --commands uboot-bbb-cmd.yaml --context uboot-bbb-ctx.yaml' %}

We are creating a dummy board: the goal is to make LAVA believe that it's talking to a real board::

  • power on/off command should do nothing and return 0
  • the connection_command should be a call to DummySys with the right configuration files

Then submit a classical health-check for your dummy Beaglebone-Black. LAVA will do exactly the same work as if it was connected to a real board.

Current status

This is a fairly simple approach but that does provide good results.

For the moment, only three board types are provided:

  • ARM Juno (uboot, tftp, nfs, secondary media)
  • Beaglebone-Black (uboot, tftp, nfs)
  • ST b2260 (uboot, tftp, nfs)
  • more to come...

As usual, if you have any issues or comments, add a bug report on the GitHub or Framagit trackers.