Skip to content
This repository has been archived by the owner on Aug 5, 2022. It is now read-only.

intel/DAFT

BranchesTags

Repository files navigation

Distributed Automatic Flasher Tester (DAFT)

Contents

  1. Overview
  2. Creating DAFT setup
  3. DAFT and AFT settings and commandline interface
  4. Creating BeagleBone Black filesystem for NFS
  5. Creating support image for DUT
  6. Troubleshooting

DAFT hardware images

1. Overview

DAFT hardware setup.

DAFT (Distributed Automatic Flasher Tester) is a tool for automated flashing and testing of an OS image on a real device. Its goal is to both standardize and simplify the testing on a variety of devices, while keeping minimal the cost and the complexity of the setup. In particular, it strives to use only inexpensive off-the shelf hardware components and to keep one single configuration that can be used by both large testing farms and individual testers/developers. This guide explains how to setup and use DAFT. Note that this guide was written for Ubuntu so you might have to do additional steps in other distributions.

1.1 DAFT setup

DAFT setup consists of a PC, Beaglebone black(s) (BBB) and device(s) under test (DUT). Role of the BBB is to control the DUT so flashing and testing can be done. Every DUT in the setup needs one BBB to control it. BBB has the ability to emulate keyboard, mass storage and ethernet device at the same time through one USB OTG port. It also has different kinds of input/output pins like GPIO and I2C which enables lots of possibilities with testing. For example at the time of writing GPIO pins are used to control a relay that is used to turn the DUT on/off. Role of the PC is to control the BBBs and to provide them the image to be flashed and tested. PC also provides BBBs the filesystem that they boot from using NFS and TFTP.

DAFT is also split code wise with testing_harness including the code BBB uses and pc_host including the code the PC uses. The program that BBB uses is called AFT (Automatic Flasher Tester) and the program that the PC uses is DAFT.

1.2 How it works

At the highest level DAFT is called from the command line with the device type and the image file to be tested as its arguments:

daft joule os-image.img

DAFT will then procede to flash the image to the device. After that has been done successfully the new image is booted and tests are executed. Tests aren't strictly part of DAFT but rather DAFT includes tools and libraries to interact with the device so own test can be written or existing test suites can be integrated.

2. Creating DAFT setup

Fewest mandatory steps to create DAFT setup with one device under test (DUT) and BeagleBone Black (BBB):

When the setup has been completed check the section on adding more devices.

2.1 Required hardware

1. USB to USB-mini cable 2. BeagleBone Black 3. Micro SD card 4. USB-Serial cable 5. Ethernet cable 6. BBB power supply 7. DUT power supply 8. Power jack extension cable 9. Relay 10. Dupont cables 11. DUT (Minnowboard MAX) 12. Micro SD card 13. Secondary network interface 14. Ubuntu PC

List of hardware needed for DAFT setup:

  • PC preferably with Ubuntu
    • Other distros should work also but this guide is written using Ubuntu
  • Secondary network interface for PC if it doesn't have one
    • Easiest option is USB ethernet adapter, cheapest is PCI network card
  • BeagleBone Black
  • Micro SD card for BBB U-Boot (optional)
  • Ethernet cable
  • USB to USB-mini cable
    • BeagleBone Black should come with one
  • USB-serial cable (recommended)
    • Used for debugging BBB/DUT or for recording serial data from DUT
  • Device to test with: Joule or Minnowboard MAX/Turbot
  • If DUT is Minnowboard (it has no internal storage), micro SD card or USB stick is required so testable image can be flashed to it
  • Power supplies for BBB and DUT
  • GPIO controllable relay
    • BBB GPIO pin output is 3.3V and max 6mA, VDD pins are 5V/3.3V 250mA, so relay has to be operable within those limits
    • Relay with and amplifier in the input would be preferred so BBB GPIO pin doesn't have to drive the coil
  • 3x Dupont cables to connect relay to BBB
  • DC plug extension cable (recommended)

At the time of writing only Joule and Minnowboard MAX/Turbot are supported as DUT but any PC like device with BIOS should work with small tweaks. Older version of the setup was also tested to work with Gigabyte NUC, Intel Edison, BeagleBone Black and Intel Galileo V2 but they might require some work on the BeagleBone filesystem to make them work.

2.2 BBB booting overview

This section will give you an overview of the BBB booting components, boot sequence and how it relates to the PC Host. This should give you a good understanding of why we need to setup various different services and what is their role.

BeagleBone Black booting with the PC Host.

The above diagram shows the general steps when setting up the BBB and booting it:

  1. Connect the PC Host to the BBB via serial.
  2. Get an IP adress for the BBB from the DHCP server.
  3. Get the BBB bootsystem kernel via TFTP. The BBB will fetch the bootsystem kernel from the PC Host via TFTP.
  4. BBB executes the kernel, which is the bootsystem core and is needed to be able to boot the rest of the system via NFS.
  5. Boot the rest of the system via NFS.

2.3 Setting up BeagleBone Black and DUT filesystems on the host PC

Here we will set up the filesystems for BBB and DUT support image. The filesystems will be made available via NFS, which we will set up in sections 2.4 to 2.5.

2.3.1 Setting up BeagleBone Black filesystem on NFS

If you don't have a BBB filesystem for NFS you can make a BBB filesystem for NFS from scratch. If you have a ready made one you can use it with the following commands

mkdir /daft
cd /daft
sudo wget address-to-bbb-fs
sudo tar -xzf bbb_fs.tar.gz

If your bbb_fs folder has a different name than bbb_fs it is helpful to rename it to bbb_fs.

2.3.2 Setting up DUT support image

If you don't have a DUT support image yet you can make one by navigating to Creating support image for DUT. If you have a DUT support image you can simply move the support_image.img to /daft/support_img/ folder on your host PC.

mkdir /daft/support_img
sudo mv address-to-image/support_image.img /daft/support_img/

2.4 Configuring host PC

In this section, we set up the necessary services to enable the devices to communicate with and access files on the host PC. The phases here correspond to BBB booting overview steps 2-5 but are solely to enable the required services on host PC. In terms of services there are different ways to do everything but the end goal is to have a static IP address on the secondary network interface (192.168.30.1) and to have DHCP, TFTP and NFS servers running on that network.

It is very important that you are careful especially with the DHCP server so that you don't accidentally configure it to the wrong interface (you should never configure the DHCP server to the interface from which your PC Host gets its internet, unless you know exactly what you are doing). This guide has been written using Ubuntu so on a different distro you might have to do things differently.

2.4.1 Secondary network interface

Configure the secondary network interface to have static 192.168.30.1 IP address. Use ifconfig to determine which one is the secondary network interface. If you have desktop Ubuntu you can use the graphical Network Manager to change the IP address: edit -> IPv4 Settings, change Method to Manual, Address to 192.168.30.1 and Netmask to 255.255.255.0. Other way is to use terminal: remove all the settings in /etc/network/interfaces for the interface and add following lines to it (change eth1 to previously determined one):

sudo vim /etc/network/interfaces
auto eth1
iface eth1 inet static
   address 192.168.30.1
   netmask 255.255.255.0
   network 192.168.0.0

Restart the network interface (change eth1 to previously determined one):

sudo ifdown eth1 && sudo ifup eth1

2.4.2 DHCP server

DHCP (Dynamic Host Configuration Protocol) server is used to allow devices such as BBB to obtain the required IP configuration. A service called dnsmasq has the required dhcp server. Dnsmasq can be installed with

sudo apt install dnsmasq

Configure dnsmasq by adding interface=eth1 and dhcp-range=192.168.30.2,192.168.30.254,10m lines to /etc/dnsmasq.conf (again instead of eth1 use your secondary network interface name):

sudo sh -c 'echo "interface=eth1" >> /etc/dnsmasq.conf'
sudo sh -c 'echo "dhcp-range=192.168.30.2,192.168.30.254,10m" >> /etc/dnsmasq.conf'

Restart and enable dnsmasq so it automatically starts after reboot:

sudo systemctl restart dnsmasq; sudo systemctl enable dnsmasq

Refer to DHCP problems if you are having issues with this.

2.4.3 TFTP server

TFTP (Trivial File Transfer Protocol) server is a server from which we can get the bootsystem kernel files. With the kernel executed it is possible to start using NFS and get rest of the system.

Install the required packages for TFTP:

sudo apt install xinetd tftpd tftp

Create a new file /etc/xinetd.d/tftp and add TFTP configuration:

sudo vim /etc/xinetd.d/tftp

service tftp
{
protocol        = udp
port            = 69
socket_type     = dgram
wait            = yes
user            = nobody
server          = /usr/sbin/in.tftpd
server_args     = /daft
disable         = no
}

Restart and enable xinetd service:

sudo systemctl restart xinetd; sudo systemctl enable xinetd

Note here that not all distros have xinetd service. In that case you might have to install something else. Refer to TFTP problems if there are problems.

2.4.4 NFS server

NFS (Network File System) server is used to give the devices access to a specified set of files on the computer. The files will be treated in a manner similar to local storage.

First install nfs-kernel-server:

sudo apt install nfs-kernel-server

Choose a user that DAFT will be used with and find it's ID (uid) and group ID numbers (gid):

id <user>

Choose a workspace directory that BBB will use to get the image to flash and write logs to. DAFT can only be used inside it and its directories. It could be your users home directory or some other one which the user has write permission. Add following lines to /etc/exports but change /your/workspace and values of anonuid and anongid to match your workspace and user choices:

sudo vim /etc/exports

/your/workspace 192.168.30.0/24(crossmnt,rw,root_squash,anonuid=1001,anongid=100,sync,no_subtree_check)
/daft/support_img 192.168.30.0/24(crossmnt,rw,root_squash,anonuid=1001,anongid=100,sync,no_subtree_check)
/daft/bbb_fs 192.168.30.0/24(crossmnt,ro,root_squash,anonuid=1001,anongid=100,sync,no_subtree_check)

Put changes in effect and restart and enable nfs-kernel-server:

sudo exportfs -ra
sudo systemctl restart nfs-kernel-server; sudo systemctl enable nfs-kernel-server

Refer to NFS problems if you have issues with NFS.

2.4.5 NTP server

This step is partially optional, if you don't care that the BBBs time is incorrect or you have another method for correcting it, feel free to skip this section. The problem with BBBs is that they don't have an internal battery. Therefore, each time they reboot, their clock resets. This causes the time in BBBs to be wrong, if they are not corrected manually. Installing the NTP server removes the problem by providing correct time to BBBs each time they boot. It works as following:

  1. PCs NTP server corrects its own time from a NTP server higher in hiararchy, e.g. company NTP server or some open NTP server pool.
  2. When BBB boots, it sends a request to the PCs NTP server.
  3. The server replies to NTP request.
  4. BBBs NTP client receives the reply and corrects its time.

Let's install the NTP server to PC.

First install the NTP Daemon and ntpdate:

sudo apt install ntp ntpdate

Now let's configure the NTP server correctly. Open /etc/ntp.conf and modify the file.

vim /etc/ntp.conf

If you want to use your company's NTP server or have another NTP server from where your PC synchronizes its time, replace the default servers 0.pool.ntp.org with your own, e.g.

server 0.pool.ntp.org iburst

Now add the following lines to allow BBBs to poll the PCs NTP server:

#Allow LAN machines to synchronize with this ntp server
restrict 192.168.30.0 mask 255.255.255.0 nomodify notrap

#Broadcast the time to your local subnet, just incase
broadcast 192.168.30.0

Finally restart the NTP service to apply the changes.

sudo systemctl restart ntp

2.4.6 Changing BBB filesystem permissions

Previously on the NFS chapter we chose a user that DAFT will be used with. We will need to change the owner of /daft/bbb_fs and /daft/support_img to that user and users group. It's done with:

sudo chown -R user:group /daft/*
sudo chown root:root /daft/bbb_fs/root/.ssh/config

2.4.7 Changing NFS workspace that BBB and DUT support image will mount

On the NFS chapter we chose a workspace for BBB to use. We need to change /home/tester in /daft/bbb_fs/etc/fstab to that one:

sudo vim /daft/bbb_fs/etc/fstab

192.168.30.1:/home/tester/ /root/workspace nfs rsize=8192,wsize=8192,timeo=14,intr,nolock,auto

Then change DUT support image to also use it. Use fdisk to check partitions and use the values from it for the mount offset value (multiply unit size and end block eg. 512*1050624):

sudo fdisk -l /daft/support_img/support.img

Disk support.img: 7,2 GiB, 7746879488 bytes, 15130624 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 891A46A3-0ACF-42D6-8009-9024012A0EAD

Device                 Start     End Sectors  Size Type
working_support.img1    2048 1050623 1048576  512M EFI System
working_support.img2 1050624 6983679 5933056  2,8G Linux filesystem

sudo mkdir /tmp_daft
sudo mount -o loop,offset=537919488 /daft/support_img/support.img /tmp_daft
sudo vim /tmp_daft/etc/fstab

192.168.30.1:/home/tester/ /mnt/img_data_nfs nfs rsize=8192,wsize=8192,timeo=14,intr,nolock,auto

Unmount the image and remove the temporary folder:

sudo umount /tmp_daft; sudo rmdir /tmp_daft

2.4.8 Install and configure DAFT software

Clone DAFT github repository:

cd; git clone https://github.com/01org/DAFT.git

Install Python3 setuptools and the server side software:

sudo apt install python3-setuptools
cd DAFT/pc_host; sudo python3 setup.py install

Change DAFT configuration file to use your previously chosen workspace directory instead of /home/tester/:

sudo vim /etc/daft/daft.cfg

[daft]
workspace_nfs_path = /home/tester/
bbb_fs_path = /daft/bbb_fs/

Now you can update both DAFT on the PC and AFT on the BBB filesystem with:

cd ~/DAFT; sudo daft --update

Finally change /etc/daft/lockfiles owner to your DAFT user:

sudo chown user:group /etc/daft/lockfiles

2.5 Setting BeagleBone Black to boot from NFS

This next step can be a little confusing, if you have not worked with BBB or similar systems before. We need to configure the BBB to automatically boot from the NFS instead of internal memory or SD card. To do this we modify a file called uEnv.txt. Each time BBB boots it goes through this file. We can set the BBB to boot from NFS by writing a boot command into the /uEnv.txt file. Note that this means the uEnv.txt file in the BBB itself and not in the NFS files. Sometimes there is no uEnv.txt in the root folder, in this case you need to create one. You can use the default factory image inside the BBB or flash a working BBB image with U-Boot to SD card.

2.5.1 Getting the BeagleBone Black image

A working BBB image can be downloaded from here. To flash the image to the SD card first connect the SD card to the PC and check lsblk and/or dmesg to see the path to the SD card. Flashing the downloaded file to SD card can be done with these commands (change bone-debian to match the downloaded image and sdX to match your SD card path):

cd ~/Downloads
unxz bone-debian.img.xz
sudo dd if=bone-debian.img of=/dev/sdX bs=8M status=progress

If you want to use a SD card you have to first boot BBB with the factory image and use following command on it to destroy its U-Boot files so it won't boot from those:

dd if=/dev/zero of=/dev/mmcblk0 bs=4M count=10; sync

Refer to PC to BBB serial connection on how to use serial to communicate with the BBB (in case you don't have a monitor and a keyboard).

Alternatively, you could flash the image in the SD card to the internal memory. This can be done by inserting the SD card to the BBB and booting it while holding down the boot from SD card button located in the BBB near SD card. However, this doesn't always do the trick, in which case you want to boot to the SD card, then go to /boot/uEnv.txt

vi /boot/uEnv.txt

and uncomment this line in the bottom of the file:

cmdline=init=/opt/scripts/tools/eMMC/init-eMMC-flasher-v3.sh

This is a flasher script, which will automatically flash the image in the SD card to the internal memory of the BBB. After uncommenting the line, simply save and reboot the device, it should start flashing. After the flashing is done, you can remove the SD card and boot the device.

2.5.2 Changing the uEnv.txt file

When you have a working BBB image in the devices internal memory (or have destroyed the internal U-Boot and have the image in SD card), boot it up. Use a serial connection or SSH to communicate with the device PC to BBB serial connection.

Once the BBB has been booted, edit the uEnv.txt file located in root.

vi /uEnv.txt

Include the following lines to the uEnv.txt file:

bootcmd=setenv autoload no; dhcp; setenv bootargs console=ttyO0,115200n8, root=/dev/nfs nfsroot=${serverip}:/daft/bbb_fs,vers=3 ro ip=${ipaddr}; tftp 0x81000000 bbb_fs/boot/vmlinuz-4.4.9-ti-r25; tftp 0x80000000 bbb_fs/boot/dtbs/4.4.9-ti-r25/am335x-boneblack.dtb; bootz 0x81000000 - 0x80000000
uenvcmd=boot

Make sure that the highlighted files match the ones you have in your NFS.

Now to make sure that the BBB boots from the NFS and not the internal memory, go to another uEnv.txt file, which is in /boot folder.

vi /boot/uEnv.txt

Comment the following line in the top of the file:

uname <something here>

Save and reboot the BBB and it should boot from the NFS, if everything was done correctly. In case something goes wrong, you can always flash a new image to the BBB and try again.

2.6 Setting up relay

Relay with extension cable hooked up.

First cut the DC plug extension cable in half from the middle (or power supplys one if you don't mind cutting it). At least some cables had trouble if they weren't cut short so it's recommended to shorten the cables first. Then solder the ground lines back together and use tape or heat shrink tube to cover it. Then solder or screw the DC line cables to the relay. If the relay has screw holes it's recommended to solder small tin beads to the cable ends and flatten them with pliers to get better contact. On default it's expected that the relay is normally closed (DUT is powered on) but this can be changed by changing gpio_cutter_on and gpio_cutter_off values:

sudo vim /daft/bbb_fs/etc/aft/devices/platform.cfg

[PC]
leases_file_name = /var/lib/misc/dnsmasq.leases
keyboard_emulator = GadgetKeyboard
cutter_type = GpioCutter
gpio_pin = gpio60
gpio_cutter_on = 1
gpio_cutter_off = 0
pem_port = /dev/hidg0
serial_bauds = 115200

2.7 Connecting hardware

Wiring diagram for hardware.

How to connect the relay to the BBB.

After everything else has been done, hardware can be connected together according to the pictures above and BBB can be booted from NFS. If there is a problem with booting it's most likely due to dnsmasq, TFTP, NFS or file permissions. Check troubleshooting section for more help if needed.

2.8 Setting up DUT

On default it's expected that the DUT has specific boot order set in BIOS so with keyboard emulation we can choose the right device to boot from. The first boot option should be the device that the image is going to be flashed to eg. SD card or USB stick. Minnowboard MAX/Turbot should have the BBB USB stick emulation as the last option and Joule should have it as second option (BIOS in Joule seemed to reset the order this way so it should be the most stable option). Use USB-serial cable or keyboard and monitor to change BIOS settings. Other option is to change the keyboard sequence that boots from the BBB USB stick emulation:

sudo vim /daft/bbb_fs/root/kbsequences/joule/boot_usb
sudo vim /daft/bbb_fs/root/kbsequences/minnowboard/boot_usb

Or you can create new keyboard sequence files and change AFT settings for BBB:

sudo vim /daft/bbb_fs/etc/aft/devices/catalog.cfg

[Joule]
platform = PC
test_plan = iot_qatest
target_device = /dev/mmcblk0
service_mode = Ubuntu
boot_usb_keystrokes = /root/kbsequences/joule/boot_usb
boot_internal_keystrokes = /root/kbsequences/empty
serial_port = /dev/ttyUSB0

[Minnowboard]
platform = PC
serial_port = /dev/ttyUSB0
test_plan = iot_qatest
target_device = /dev/mmcblk0
service_mode = Ubuntu
boot_usb_keystrokes = /root/kbsequences/minnowboard/boot_usb
boot_internal_keystrokes = /root/kbsequences/empty

2.9 Using DAFT

After BBB has been booted successfully check dnsmasq.leases file to see BBB IP address:

cat /var/lib/misc/dnsmasq.leases

In some distros the location of this file might be different. In that case you can use arp -n in the terminal to do the same thing.

1481817223 b0:d5:cc:f4:8c:41 192.168.30.61 DAFT_BBB *

Copy that IP address to /etc/daft/devices.cfg:

sudo vim /etc/daft/devices.cfg

[Minnowboard1]
bb_ip = 192.168.30.2

[Joule1]
bb_ip = 192.168.30.3

Now everything should be set to use daft. Basic DAFT command that flashes and tests an image:

daft minnowboard yocto-image.dsk

Flash and test an image while recording serial output:

daft minnowboard yocto-image.dsk --record

Next, we'll take a look at what happens when we run DAFT. This is a high level overview. For more detailed DAFT run refer to Detailed DAFT run

Running daft.

  1. Start daft by executing the following command in the PC Host: daft DUT PATH/TO/IMG.img. Replace DUT with the desired DUT e.g. minnowboard or joule and PATH/TO/IMG.img with the image you want to flash. You can also run the command directly in BBB but then it won't lock the DUTs. Running daft in BBB is done with the command aft DUT PATH/TO/IMG.img. In case you want to only flash or test or boot the DUT refer to DAFT commandline interface to add correct options for DAFT.
  2. DAFT then checks if the device is blacklisted or locked in the lockfiles. If not, then it locks it so other simultaneous DAFT runs won't use the same DUT. Or if you call DAFT explicitly e.g. daft joule1, it will only look for that device.
  3. BBB reboots the DUT and proceeds to start the support image in the DUT. The support image is needed so that the target image can be flashed to the SD card or to another memory storage. BBB emulates itself as multiple components to enable automated and simple function:
    1. As a keyboard (to get through DUT BIOS and to select the correct device to boot from).
    2. As a USB drive (so that the DUT can boot the support image).
    3. As an ethernet interface (ethernet is needed so the BBB can connect to the DUT and run commands on it required when flashing or testing).
  4. When the support image boots, it gets an IP address from the BBBs DHCP server (NOT THE PC HOSTS). If you want to, you can SSH to the DUT from the PC Host by first doing a SSH to the BBB and then another SSH to the DUT.
  5. Flash the target image fetched from the PC Host via NFS to the DUTs memory storage. The DUT doesn't connect directly to the PC Host but instead through the BBB, which forwards the connection both ways.
  6. Boot the flashed image and run the tests on DUT.

For more info about using DAFT and its settings check DAFT and AFT settings and commandline interface section. If flashing fails DAFT will blacklist the BBB used by locking the device until /etc/daft/lockfiles/<device> is emptied or removed.

2.10 Adding more devices

Setup with three DUTs.

To add more DUTs and BBBs to the setup an ethernet switch is needed between the PC and BBBs. Nothing else special is needed. Just configure new BBBs to boot from the same NFS directory as previously and check /var/lib/misc/dnsmasq.leases file after they have booted for their IP address. Then add new devices to /etc/daft/devices.cfg with different number:

vim /etc/daft/devices.cfg

[Minnowboard1]
bb_ip = 192.168.30.2

[Minnowboard2]
bb_ip = 192.168.30.4

[Joule1]
bb_ip = 192.168.30.3

[Joule2]
bb_ip = 192.168.30.5

After these are configured, you call them the same way you'd call a single device. DAFT automatically checks which devices are in use and which are not by locking and unlocking them in

/etc/daft/lockfiles/

So you don't have to call the DUTs explicitly daft minnowboard1 PATH/TO/DUT.img but instead you can just call daft minnowboard PATH/TO/DUT.img.

3. DAFT and AFT settings and commandline interface

This section includes information about DAFT and AFT config files, CLI and also an example DAFT run where all the steps that DAFT takes are explained.

3.1 DAFT settings

DAFT settings are located in /etc/daft/daft.cfg and on default are:

[daft]
workspace_nfs_path = /home/tester/
bbb_fs_path = /daft/bbb_fs/
bbb_aft_path = /usr/local/lib/python3.4/dist-packages/aft-1.0.0-py3.4.egg/aft

Settings:

  • workspace_nfs_path: Path to the workspace directory that is shared with NFS to BBB and DUT.
  • bbb_fs_path: Path to the directory that contains the filesystem from which BBB boots from.
  • bbb_aft_path: Path to the directory that contains AFT code.

DAFT device settings are located in /etc/daft/devices.cfg and on default are:

[Minnowboard1]
bb_ip = 192.168.30.2

[Joule1]
bb_ip = 192.168.30.3

Settings:

  • bb_ip: IP address to the BBB handling the specific DUT.

3.2 DAFT commandline interface

Basic DAFT command:

daft <dut> <image_file>

DAFT commandline interface options:

  • dut: Type of DUT or sepcific DUT to flash. Should be one from the /etc/daft/devices.cfg.
  • image_file: Image file to flash to the DUT.
  • --record: Record serial output from DUT.
  • --update: Update AFT on the BBB filesystem and DAFT on the PC. This should be ran while being on the root DAFT repository directory and with root permission.
  • --setout: Flash DUT and reboot it in test mode without running test stuff
  • --noflash: Skip flashing of DUT.
  • --notest: Skip testing of DUT.
  • --noblacklisting: If flashing or testing fails don't blacklist the device.
  • --emulateusb: Use testing harness USB emulation to boot the image instead of flashing it. You can use --notest to only boot the image without running automatic tests.
  • --testplan: Specify a test plan to use from bbb_fs/etc/aft/test_plan/. Use the test plan name without .cfg extension. On default the test plan for the device in AFT device settings is used.

3.3 AFT settings

AFT settings are located on the BBB filesystem in /etc/aft/aft.cfg and on default are:

[aft]
lock_file = /var/lock/
serial_log_name = serial.log
aft_log_name = aft.log
nfs_folder = /root/workspace

Settings:

  • lock_file: Path to directory that lock files are saved to. Lock file is used so multiple instances of AFT can't flash/test a DUT at the same time.
  • serial_log_name: Name for the serial log file.
  • aft_log_name: Name for the aft log file.
  • nfs_folder: Path to the directory that the workspace NFS is mounted to.

AFT device settings are located in two files on the BBB filesystem in /etc/aft/devices/. The files are platform.cfg and catalog.cfg. The configuration of a device is combined out of these files. The platform.cfg is the highest level configuration file. It is intended to store settings which are shared between all high-level device-types,ie. PC-devices or gadget-devices. The catalog.cfg is the configuration file describing each device type. Settings on these files:

  • leases_file_name: Path to the dnsmasq.leases file that contains IP addresses of connected devices.
  • keyboard_emulator: Keyboard emulator type. On default GadgetKeyboard which is the BBB keyboard emulation.
  • cutter_type: DUT power supply cutter/relay type. On default GpioCutter which is a relay controlled with BBB GPIO pins.
  • gpio_pin: GPIO pin that controls the cutter/relay if using GpioCutter.
  • gpio_cutter_on: GPIO pin value when relay should be closed. On default 1.
  • gpio_cutter_off: GPIO pin value when relay should be open. On default 0.
  • pem_port: Path to the keyboard emulator port.
  • serial_port: Path to the serial cable port.
  • serial_bauds: Bauds for serial recording of DUT.
  • test_plan: Test plan used with the device.
  • target_device: Path to the target device that the image to be tested is flashed to.
  • service_mode: String included in /proc/version when DUT is booted with support image.
  • boot_usb_keystrokes: Path to the keystrokes which boot DUT to service mode.
  • boot_internal_keystrokes: Path to the keystrokes which boot DUT to test mode.
  • platform: On catalog.cfg file option to inherit settings from specific platform.

3.4 AFT commandline interface

AFT can be used on the BBB. Basic AFT command:

aft <dut> <image_file>

AFT commandline interface options:

  • dut: Type of DUT to flash. Should be one from the /etc/aft/devices/catalog.cfg.
  • image_file: Image file to flash to the DUT.
  • --emulateusb: Use testing harness USB emulation to boot the image instead of flashing it. You can use --notest to only boot the image without running automatic tests.
  • --record: Record serial output from DUT.
  • --flash_retries: Change how many times flashing will be tried. On default 2.
  • --noflash: Skip flashing image to the device eg. 'aft joule --noflash' would run only tests.
  • --notest: Skip testing image.
  • --nopoweroff: After aft run, don't turn off device.
  • --boot: Boot device to specific mode. Options are 'service_mode' and 'test_mode'. For example: 'aft joule --noflash --notest --boot=test_mode' would boot device to the flashed image.
  • --testplan: Specify a test plan to use from bbb_fs/etc/aft/test_plan/. Use the test plan name without .cfg extension. On default the test plan for the device in AFT device settings is used.
  • --verbose: Increase aft run verbosity.
  • --debug: Change aft logging level to 'debug'.

3.5 Detailed DAFT run

More detailed example of a DAFT run:

host PC:

  • run daft joule image.wic --record on the DAFT workspace

host PC DAFT:

  • Parse DAFT config file /etc/daft/daft.cfg
  • Parse devices config file /etc/daft/devices.cfg
  • Look for 'joule' devices and try to find one that isn't reserved in /etc/daft/lockfiles/
  • If a free joule is found, lock it by writing to a file in /etc/daft/lockfiles/
  • Use the bb_ip from the devices.cfg to ssh to the correct BBB and run aft joule image.wic --record --notest on it to flash the image

BBB testing harness AFT:

  • Parse AFT config file from /etc/aft/aft.cfg
  • Parse devices config files from /etc/aft/devices/
  • Find and use 'joule' settings
  • Reboot the device by turning off the relay with GPIO pin, wait a while, turn on the relay
  • Start sending keystrokes determined by boot_usb_keystrokes in device settings
  • After the keystrokes has been sent, try connecting to the device with the IP address found in /var/lib/misc/dnsmasq.leases
  • When there is a connection, use ssh to run commands on the DUT support image

DUT support image:

  • Mount the NFS in /etc/fstab, this is the NFS from host PC that BBB has forwarded with iptables
  • Flash the image file determined in the arguments to the path determined by target_device in the AFT device config files
  • Find the root partition of the flashed image and add ssh-keys to the image

BBB testing harness AFT:

  • After the commands has been run, flashing should be successful and relay can be turned off
  • Return to DAFT on the host PC

host PC DAFT:

  • Rename all the log files with 'flash_' prefix
  • Using ssh run aft joule --record --noflash on the BBB to test the flashed image

BBB testing harness AFT:

  • Parse AFT config file from /etc/aft/aft.cfg
  • Parse devices config files from /etc/aft/devices/
  • Find and use 'joule' settings
  • Reboot the device by turning off the relay with GPIO pin, wait a while, turn on the relay
  • Start sending keystrokes determined by boot_internal_keystrokes in device settings
  • After the keystrokes has been sent, try connecting to the device with the IP address found in /var/lib/misc/dnsmasq.leases
  • When there is a connection, run the tests determined by test_plan in the device config files
  • After the tests has been run, relay can be turned off
  • Return to DAFT on the host PC

host PC DAFT:

  • Rename all the log files with 'test_' prefix
  • Release the 'joule' device by emptying the lockfile in /etc/daft/lockfiles/
  • DAFT run is done

4. Creating BeagleBone Black filesystem for NFS

Download BBB image from here (bone-debian-8.4-lxqt-4gb-armhf-2016-05-13-4gb.img was used when writing this guide). Use dd to copy it to USB stick. SD card could also be used but working with USB stick is faster (change /dev/sdX to match your USB stick):

sudo dd if=bone-debian-8.4-lxqt-4gb-armhf-2016-05-13-4gb.img of=/dev/sdX bs=8M status=progress
sync

After copying it reseize the partition so it has more space (change /dev/sdX to match yours):

sudo fdisk /dev/sdX
d
n
press enter
press enter
8192
a
w
sudo e2fsck -f /dev/sdX1
sudo resize2fs /dev/sdX1

Connect the USB stick or SD card to BBB and boot it. Use USB-serial cable or monitor and keyboard and stop the U-Boot from doing default boot by pressing space. Use this command to boot from your storage device and change 4.4.9-ti-r25 to match your version number:

Boot settings for USB stick

setenv bootargs 'console=ttyO0,115200n8, root=/dev/sda1 rootfstype=ext4 rw'
usb start
ext4load usb 0:1 0x81000000 /boot/vmlinuz-4.4.9-ti-r25
ext4load usb 0:1 0x80000000 /boot/dtbs/4.4.9-ti-r25/am335x-boneblack.dtb; bootz 0x81000000 - 0x80000000

Boot settings for SD card

setenv bootargs 'console=ttyO0,115200n8, root=/dev/mmcblk1p1 rootfstype=ext4 rw'
ext4load mmc 0:1 0x81000000 /boot/vmlinuz-4.4.9-ti-r25
ext4load mmc 0:1 0x80000000 /boot/dtbs/4.4.9-ti-r25/am335x-boneblack.dtb; bootz 0x81000000 - 0x80000000

When BBB has booted do all of these commands to make all the changes needed for DAFT (notice the lines that make proxy settings, change them or don't use them if you don't use proxy):

dmesg -n 1
cd /opt/scripts/boot/
rm am335x_evm.sh autoconfigure_usb0.sh capemgr.sh
cd
rm /etc/systemd/system/multi-user.target.wants/connman.service
vim /etc/apt/apt.conf.d/03-proxy-https
    Acquire::http::Proxy "http://yourproxyaddress:proxyport"; # Add this line to the file
apt update
apt install nfs-common dnsmasq python3-setuptools python3-pip expect ntp ntpdate
pip3 --proxy http://yourproxyaddress:proxyport install pyserial netifaces unittest-xml-reporting
git config --global https.proxy http://yourproxyaddress:proxyport
git config --global http.proxy http://yourproxyaddress:proxyport
git clone https://github.com/01org/DAFT.git
cd DAFT/testing_harness
python3 setup.py install
cd
cp DAFT/testing_harness_image_extras/initialize_testing_harness/stop_libcomposite /usr/bin/
cp DAFT/testing_harness_image_extras/initialize_testing_harness/start_libcomposite /usr/bin/
cp DAFT/testing_harness_image_extras/initialize_testing_harness/initialize_testing_harness /usr/bin/
cp DAFT/testing_harness_image_extras/initialize_testing_harness/initialize_testing_harness.service /etc/systemd/system/
ln -s /etc/systemd/system/initialize_testing_harness.service  /etc/systemd/system/multi-user.target.wants/
cp DAFT/testing_harness_image_extras/initialize_testing_harness/libcomposite.service /etc/systemd/system/
ln -s /etc/systemd/system/libcomposite.service  /etc/systemd/system/multi-user.target.wants/
cp -r DAFT/testing_harness_image_extras/kbsequences /root/
mkdir support_image /ramdisk /config .ssh workspace
iptables -t nat -A POSTROUTING --out-interface eth0 -j MASQUERADE
iptables -A FORWARD --in-interface usb0 -j ACCEPT
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 2233 -j DNAT --to 192.168.7.2:22
iptables-save > /etc/iptables.rules
cp -r /var/ /_var
vim /etc/sysctl.conf
    net.ipv4.ip_forward = 1 # Add this line to the file
vim /etc/network/interfaces
    # Remove everything but have these lines
    auto lo
    iface lo inet loopback
    auto eth0
    iface eth0 inet dhcp
vim /etc/dnsmasq.conf
    # Remove everything but have these lines
    dhcp-range=192.168.7.2,192.168.7.2,10m
    interface=usb0
vim /etc/fstab
    # Remove everything but have these lines
    192.168.30.1:/daft/support_img /root/support_image nfs rsize=8192,wsize=8192,timeo=14,intr,nolock,auto
    192.168.30.1:/home/tester/ /root/workspace nfs rsize=8192,wsize=8192,timeo=14,intr,nolock,auto
    tmpfs /ramdisk tmpfs defaults,size=100M 0 0
vim /root/.ssh/config
    # Add these lines
    Host *
        IdentityFile ~/.ssh/id_rsa_testing_harness
vim /etc/ntp.conf
   #Remove these lines
   server 0.pool.ntp.org iburst
   server 1.pool.ntp.org iburst
   server 2.pool.ntp.org iburst
   server 3.pool.ntp.org iburst

   #Add these lines
   server 192.168.30.1 iburst
   disable auth
   broadcastclient
vim /etc/default/ntpdate
   #Remove this line
   NTPSERVERS="some_address"

   #Add these lines
   NTPSERVERS="192.168.30.1"
   UPDATE_HWCLOCK="yes"
rm /etc/localtime
#Remember to select the locale /Europe/Helsinki to your own
ln -s /usr/share/zoneinfo/Europe/Helsinki /etc/localtime

Now make id_rsa_testing_harness and id_rsa_testing_harness.pub SSH-keys with ssh-keygen -t dsa or copy some other keys to /root/.ssh/ but use the previous names for them. The id_rsa_testing_harness.pub SSH-key should be added to /root/.ssh/authorized_keys and also to DUT support images authorized_keys file. The authorized_keys file from support image will also be copied to authorized_keys on the images that are flashed. After this the image should work with DAFT. Poweroff the BBB and connect the USB stick to PC. Mount it and copy the filesystem from it (change /dev/sdX1 to match your USB stick):

mkdir tmp_mnt bbb_fs
sudo mount /dev/sdX1 tmp_mnt
sudo cp -r tmp_mnt/* bbb_fs/
sync
sudo umount tmp_mnt
sudo rmdir tmp_mnt

Now you should have a working BBB filesystem that BBB can use to boot from with NFS. You should be able to move/copy the folder to /daft/ and boot from it with this U-Boot command:

setenv autoload no
dhcp
setenv bootargs console=ttyO0,115200n8, root=/dev/nfs nfsroot=${serverip}:/daft/bbb_fs,vers=3 ro ip=${ipaddr}
tftp 0x81000000 bbb_fs/boot/vmlinuz-4.4.9-ti-r25
tftp 0x80000000 bbb_fs/boot/dtbs/4.4.9-ti-r25/am335x-boneblack.dtb; bootz 0x81000000 - 0x80000000

5. Creating support image for DUT

5.1 Ubuntu for PC like devices

Support image made using this guide should work for any PC like devices but has only been tested with Minnowboard Turbot and Joule. Two USB-sticks, USB hub, keyboard, USB-ethernet dongle, monitor and Joule was used when writing this guide. First download Ubuntu server 16.04.2 LTS. Ubuntu server is used as it's basically normal Ubuntu without graphical interface. Copy it to a USB-stick with dd (change X to match your USB-stick):

sudo dd if=ubuntu-16.04.2-server-amd64.iso of=/dev/sdX bs=8M status=progress
sync

Connect USB hub, USB-sticks, keyboard and monitor to the DUT. Boot the DUT using the USB-stick with the Ubuntu server. Select install Ubuntu server. When installing it, you can choose whatever region, keyboard layout, user and password you like. Don't encrypt home directory. Select manual partitioning when you get to the partitioning section. Select your other USB-stick and create empty partition table on it. Then create new bootable EFI partition with size of 512 MB. Then create EXT4 root partition with size of 2.5 GB. Then continue with the installation and ignore the warning about no swap partition. When asked you can leave proxy setting empty for now and also select no automatic updates. When choosing what software to install, only select OpenSSH server (note that space selects software and enter continues the install). Then you should be done with the install.

Boot the newly installed Ubuntu, login and change /etc/default/grub to stop network interfaces getting renamed:

sudo su
vim /etc/default/grub

# Change this line
GRUB_CMDLINE_LINUX="net.ifnames=0"

update-grub

Now use reboot to reboot the Ubuntu. After reboot set up proxy settings if you need to:

export http_proxy=http://your.proxy.com:port
export https_proxy=http://your.proxy.com:port

Change to root and use dhclient to connect to internet:

sudo su
dhclient eth0

Then install bmap-tools, nfs-common and parted:

apt update
apt install bmap-tools nfs-common parted

Configure usb0 in /etc/network/interfaces:

vim /etc/network/interfaces

# Add these lines
auto usb0
allow-hotplug usb0
iface usb0 inet dhcp

Edit sshd settings:

vim /etc/ssh/sshd_config

# Change this line
PermitRootLogin yes
# Add this line
UseDNS no

Make some directories and files:

mkdir -p /mnt/img_data_nfs /mnt/super_target_root /mnt/target_root
touch /root/.ssh/authorized_keys

Add your SSH key to /root/.ssh/authorized_keys with ssh-copy-id, scp, ssh or some other way you prefer. The key should be the same that BBB uses so it can log in with SSH.

Change /boot/efi/EFI directory as Minnowboard on default looks for boot files from /boot/efi/EFI/BOOT:

cp -r /boot/efi/EFI/ubuntu /boot/efi/EFI/BOOT
mv /boot/efi/EFI/BOOT/grubx64.efi /boot/efi/EFI/BOOT/bootx64.efi

Add nfs mount option to /etc/fstab and mount root as read-only:

vim /etc/fstab

# Change this line to include ro
UUID=0b225574-0443-481b-ba4c-0f098441364a / ext4 ro,errors=remount-ro 0 1
# Add this line
192.168.30.1:/home/tester /mnt/img_data_nfs nfs rsize=8192,wsize=8192,timeo=14,intr,nolock,auto

Now the image should be ready and you can poweroff the device and remove the USB-stick. Then connect the USB-stick to your PC and use fdisk to check the partitions (change sdX to match your USB-stick):

sudo fdisk -l dev/sdX

Disk /dev/sdg: 14.9 GiB, 16022241280 bytes, 31293440 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 98662738-D145-45FE-82BB-CA80AE5073A2

Device      Start     End Sectors  Size Type
/dev/sdg1    2048  999423  997376  487M EFI System
/dev/sdg2  999424 5881855 4882432  2.3G Linux filesystem

Notice the unit size and end block of the last partition. Then use dd to copy the image from the USB-stick. Use the previous numbers but add 100 to end block number so we get some extra space for rewriting backup GPT partition table:

sudo dd if=/dev/sdX of=ubuntu_support.img bs=512 count=5881955

Then fix the image by rewriting the backup GPT partition table:

sudo gdisk ubuntu_support.img

Command (? for help): x
Expert command (? for help): w
Do you want to proceed? (Y/N): Y

Now you should have a nice small working support image to use with DAFT. The support image should be placed to /daft/support_img/ and named as support.img.

6. Troubleshooting

6.1 PC to BBB serial connection

Find out the serial port connected to BBB with

dmesg | grep tty

Install screen

sudo apt install screen

Open a "Screen" from your terminal to the serial (usually its something like ttyUSB0). You can use the following command:

sudo screen /dev/ttyUSB0 115200 -ixon,-ixof

but remember to replace the /dev/ttyUSB0 with your own path. Now boot the BBB by supplying it power with a 5V charger. If you are having problems when booting BBB, refer to the troubleshooting section Beaglebone Black booting

6.2 Beaglebone Black booting

Instead of allowing BBB to boot normally you can interrupt it by pressing space while booting. It will then enter U-Boot terminal. Use USB-serial cable or monitor and keyboard to do it. Then you can manually try to boot from nfs with these U-Boot commands:

setenv autoload no
dhcp
setenv bootargs console=ttyO0,115200n8, root=/dev/nfs nfsroot=${serverip}:/daft/bbb_fs,vers=3 rw ip=${ipaddr}
tftp 0x81000000 bbb_fs/boot/vmlinuz-4.4.9-ti-r25
tftp 0x80000000 bbb_fs/boot/dtbs/4.4.9-ti-r25/am335x-boneblack.dtb
bootz 0x81000000 - 0x80000000

If it gets stuck on dhcp it's dnsmasq or network configuration problem. Check with ifconfig if the network interfaces IP is 192.168.30.1 and with systemctl status dnsmasq to see if dnsmasq is running. If it gets stuck to tftp commands it's a problem with TFTP or with the pathnames. If you have changed the location of the BBB filesystem or NFS/TFTP folder paths you might have to change /daft/bbb_fs and bbb_fs/boot/* paths in the boot commands. U-Boot is quite clunky with the pathnames so try different paths like /bbb_fs/boot/*, daft/bbb_fs/boot/* or /daft/bbb_fs/boot/*. If it starts booting but gets stuck it's probably a problem with NFS or file permissions. For file permissions take a look at Changing BBB filesystem permissions and for NFS take a look at NFS problems.

If you get an error when you try to tftp the files, check that the kernel files vmlinuz and dtbs e.g. vmlinuz-4.4.9-ti-r25 match the ones you have.

6.3 DHCP problems

If you are having problems with the DHCP server, you can try the following steps:

  1. Check that your dnsmasq service is on. This is done by executing the following command systemctl status dnsmasq. If it is not, you need to start it with systemctl start dnsmasq. If it still isn't working, check that you have completed all the steps when configuring the DHCP server.
  2. Make sure your firewall isn't blocking the connection. DHCP uses UDP and the ports 67 and 68, so make sure they are open. (e.g. if you have iptables firewall you can check the situation with sudo iptables -L.)

6.4 TFTP problems

If you are having problems with TFTP, you can try the following steps:

  1. Check that your TFTP server is on. systemctl status xinetd Then check that you have completed all the steps in TFTP server. Note that if you are not using Ubuntu, you might have to install and setup something else (e.g. in Fedora you have to install a different tftp server because there is no xinetd).
  2. Again check that PC Hosts firewall isn't blocking the connection.

6.5 NFS problems

If you are having problems with NFS, you can try the following steps:

  1. Check that your NFS server is on. systemctl status nfs-kernel-server Then check that you have completed all the steps in Refer to NFS server. Note that if you are not using Ubuntu, you might have to configure some additional things. Search for help in the web for your distro.
  2. Again check that PC Hosts firewall isn't blocking the connection.

Note that it's not possible to export anything via NFS in an encrypted file system volume.

We have noticed that in some other distros different than Ubuntu getting the BBB to boot from NFS can be very problematic. If you cannot get it to work, you can always boot the BBB from USB drive. Refer to Creating BeagleBone Black filesystem for more info.

DAFT hardware images

USB-serial

USB-serial cable.

Relays

Examples of the types of relays that work with BBB though the left one is better as it includes optocoupler which reduces the current needed from GPIO.

About

Automated Flasher Tester

Resources

Readme

Security policy

Security policy
Activity
Custom properties

Stars

35 stars

Watchers

17 watching

Forks

18 forks
Report repository

Releases

No releases published

Packages

No packages published

Contributors 5

Languages