QA Testing

QA testing

Panel plugins talk to servers, so most of what can go wrong needs a server to go wrong against. The repository carries a sandbox for that: Docker scenarios that build mc from the working tree and run it against a host serving the same archives over sftp, ssh, ftp and samba.

Everything below lives in tests/misc/docker.

Getting one running

tests/misc/docker/sandbox.sh arcmc up

That builds two images, starts the server and builds mc — a few minutes the first time, seconds afterwards. Then:

tests/misc/docker/sandbox.sh arcmc mc

mc opens in the container, built from your working tree. Nothing is written into the tree itself: the sources are mounted read-only, copied inside, and the build lives in a Docker volume.

The scenario name may be left out. arcmc is the default, so the short form works:

tests/misc/docker/sandbox.sh mc

After editing the sources, rebuild without leaving the sandbox:

tests/misc/docker/sandbox.sh build

The rest of the commands:

command	what it does
up	images, server and mc, from nothing
mc	run mc against the scenario
build	rebuild mc, keeping the object files
check	ask every protocol for a listing, without a terminal
shell	a shell next to mc: ssh, curl, smbclient are there
remote	a shell on the server
logs	what the server has to say
down	stop the containers
clean	stop them and throw the build away
list	the scenarios there are

Telling a broken plugin from a broken sandbox

Before blaming the code, ask the sandbox whether it is alive:

$ tests/misc/docker/sandbox.sh arcmc check
  sftp         ok
  ftp          ok
  ssh          ok
  smb          ok
  mc           ok
  plugins:     arcmc docker ftp git hello k8s panelize s3 samba sftp shell-link systemd

This needs no terminal, so it also works from a script. A protocol that says FAILED here is a server that never came up, not a plugin that misbehaves.

Where to connect from inside mc

One host, one directory, four ways in. The user is mc and the password is mc everywhere.

panel	host / UNC	port	remote path
SFTP	remote	22	/home/mc/archives
Shell link (FISH)	remote	22	empty, or the same
FTP	remote	21	/archives
SMB	//remote/archives	—	—

localhost will not do: mc runs in its own container and the server is the one next to it, reachable by the name remote.

The same tree is in /work/local inside the mc container, for whatever needs no server at all.

What is in there to press keys on

One directory per situation, and in each a cases.tsv saying what its files are for — read it as a checklist:

$ cat 03-nested/cases.tsv
file                      key     expect                  why
outer.tar                 Enter   archive panel           then Enter on small.zip inside it
outer.tar                 ..      the panel it came from  twice: inner archive, outer archive, then sftp or ftp
zip-in-zip.zip/uzip://    cd      extfs panel             utar:// is gone, uzip:// is the filesystem left to try
small.zip inside uzip://  Enter   archive panel           a file inside an mc filesystem is left to mc.ext.ini

directory	what it is for
01-formats	tar, zip and 7z, including one past libarchive's buffer
02-content	archives with no extension, and plain text named as one
03-nested	an archive inside an archive, and one inside uzip://
04-non-ascii	Cyrillic and spaces in names, inside the archives and out

The columns are meant to be walked by something automated later; for now they are a checklist a person reads.

What the same file is supposed to do in different places

The interesting part is not the file, it is which panel you are standing in when you press the key.

sftp and shell link hand the archive over as a stream, so it opens without being downloaded first. 01-formats/big.7z is the case that only works because that stream can seek: a 7z keeps its directory at the end of the file. 02-content/noext opens too — the format comes from the content, not from the name.

ftp and samba cannot supply a stream yet, so mc fetches a local copy first and you will see the pause. 02-content/noext does not open there: with no stream, the decision is made by the name, and that name says nothing.

A local panel in /work/local covers the same ground with no server. 03-nested/zip-in-zip.zip is the odd one: cd zip-in-zip.zip/uzip:// puts the panel inside an mc filesystem, where a file is not something open(2) can reach, so Enter on the archive inside it goes to mc.ext.ini rather than to a plugin.

Anywhere, 02-content/notanarchive.tar.gz must do nothing at all: the name promises an archive, the content does not, and the plugin turns it down. Doing nothing is right; an error is not, and neither is opening a panel.

Adding a scenario

The scenario that exists is a comfortable environment: current Debian, UTF-8, zsh, every tool installed. Its point is that plugins work at all. Other things worth catching — an older distribution, a machine without 7z or unzip, a server that drops the connection halfway — belong in scenarios of their own.

Add a directory under scenarios/ with a docker-compose.yml in it. That is the whole of it: sandbox.sh finds scenarios by looking for that file, and each one is a separate compose project with its own network, containers and build volume. An existing scenario is never edited, never rebuilt, and never fights another one for a port.

A scenario that only changes the environment is a Dockerfile.mc with a different FROM and the same three COPY lines from common/:

FROM debian:bullseye-slim

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        build-essential autoconf automake libtool pkg-config gettext autopoint \
        libglib2.0-dev libncurses-dev libssh2-1-dev libarchive-dev libsmbclient-dev \
        libcurl4-openssl-dev libmagic-dev libsqlite3-dev \
        check rsync file ca-certificates \
    && rm -rf /var/lib/apt/lists/*

COPY common/build-mc.sh common/check-remote.sh common/fixtures.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/build-mc.sh /usr/local/bin/check-remote.sh /usr/local/bin/fixtures.sh

WORKDIR /work

build-mc.sh does not care which distribution it is on, which is what makes a scenario for checking that a release still builds on an older one cheap to write. rsync and check are what it needs itself; the rest is what mc is being built against.

This one is worth writing: on Debian 11 every plugin builds except s3, whose libcurl is older than the 7.75 it wants. That is the kind of answer such a scenario exists to give, and it takes one up to get it.

The build context is the sandbox root, which is why those paths start with common/. Copy scenarios/arcmc/docker-compose.yml, change the project name and the dockerfile: line, and the new scenario is ready:

name: mc-sandbox-bullseye

services:
  remote:
    build:
      context: ../..
      dockerfile: common/remote/Dockerfile.remote
    hostname: remote

  mc:
    build:
      context: ../..
      dockerfile: scenarios/bullseye/Dockerfile.mc
    volumes:
      - ../../../../..:/src:ro
      - work:/work
    tty: true
    stdin_open: true

volumes:
  work:

Then:

tests/misc/docker/sandbox.sh bullseye up

Poking at it by hand

tests/misc/docker/sandbox.sh arcmc remote   # a shell on the server
tests/misc/docker/sandbox.sh arcmc shell    # a shell next to mc
tests/misc/docker/sandbox.sh arcmc clean    # containers and build gone

The unit tests run in there as well, against the same build:

tests/misc/docker/sandbox.sh arcmc shell
cd /work/build/tests && make check