Add info about end-to-end testing

[ondrej-fabry]

Ligato Docs is missing info about our e2e tests. Executing them is very similar to Integration Tests (e.g. make e2e-tests)

Testing code is located here: https://github.com/ligato/vpp-agent/tree/master/tests/e2e

[chrismetz09]

Want to run make integration-tests on mac as baseline before hacking on e2e testing docs. Does the following error indicate problem with make command execution or is the test working and something in the code is broke?

vpp-agent % make integration-tests --debug
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0
Reading makefiles...
Updating goal targets....
 File `integration-tests' does not exist.
Must remake target `integration-tests'.
# running integration tests
VPP_IMG=ligato/vpp-base:20.05 ./tests/integration/vpp_integration.sh
# go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin/linuxcalls
plugins/linux/nsplugin/linuxcalls/namespace_api.go:95:9: cannot use &namedNetNsHandler literal (type *namedNetNsHandler) as type NamedNetNsAPI in return argument:
	*namedNetNsHandler does not implement NamedNetNsAPI (missing CreateNamedNetNs method)
plugins/linux/nsplugin/linuxcalls/system_linuxcalls.go:55:9: undefined: syscall.Mount
make: *** [integration-tests] Error 2

[ondrej-fabry]

Weird, are you sure you have the latest master version?

[chrismetz09]

yes, replaced make command with remake command for more visibility. Output:

➜  vpp-agent git:(master) remake integration-tests --debug --verbosity=full
GNU Make 4.3+dbg-1.5
Built for x86_64-apple-darwin19.5.0
Copyright (C) 1988-2020 Free Software Foundation, Inc.
Copyright (C) 2015, 2017 Rocky Bernstein.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
Reading makefile 'Makefile'...
Reading makefile 'vpp.env' (search path) (no ~ expansion)...
Reading makefile 'scripts/make/buf.make' (search path) (don't care) (no ~ expansion)...
Updating makefiles...
Updating goal targets...
Considering target file 'integration-tests'.
/Users/christophermetz/vpp-agent/Makefile:180 integration-tests
 File 'integration-tests' does not exist.
 Finished prerequisites of target file 'integration-tests'.
Must remake target 'integration-tests'.
# running integration tests
VPP_IMG=ligato/vpp-base:20.05 ./tests/integration/vpp_integration.sh
# go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin/linuxcalls
plugins/linux/nsplugin/linuxcalls/namespace_api.go:95:9: cannot use &namedNetNsHandler literal (type *namedNetNsHandler) as type NamedNetNsAPI in return argument:
	*namedNetNsHandler does not implement NamedNetNsAPI (missing CreateNamedNetNs method)
plugins/linux/nsplugin/linuxcalls/system_linuxcalls.go:55:9: undefined: syscall.Mount
Makefile:181: *** [integration-tests] error 2

#0  integration-tests at /Users/christophermetz/vpp-agent/Makefile:180

[rastislavs]

I guess Chris might be trying to run this on Mac? That would explain the missing syscalls.

[chrismetz09]

Yes, running on a mac. I'll mention the "make running on mac" limitation and proceed with documenting the e2e tests section. No further action needed at this time.

-cm