Connection closed during setup while trying to use the remote infrastructure via a script #808

initBasti · 2021-07-30T06:58:21Z

initBasti
Jul 30, 2021

Hello,

I am currently trying to figure out how to interact with my DUT with a script from my host device (laptop) over an intermediate device running the exporter + coordinator. I have searched extensively in the documentation but was currently unable to find a simple example for my scenario.

The general idea looks like this:
Host (Laptop with environment configuration file and script)
-> NanoPC-T4 (Small device running the exporter and the coordinator)
-> Raspberry Pi (DUT, set up with a special test user and connected to NanoPC-T4 via serial)

Here is the configuration on the host:

targets:
    example:
        resources:
            - RemotePlace:
            name: rpi_serial
        drivers:
            SerialDriver: {}
            ShellDriver:
                prompt: 'testlab_user@\w+:[^ ]+ '
                login_prompt: ' login: '
                username: 'testlab_user'
options:
    crossbar_url: 'ws://192.168.178.70:20408'

The Remote place rpi_serial was created using labgrid-client -x ws://192.168.178.70:20408/ws --place rpi_serial create
and matches were added for the resources:

(labgrid-venv) basti@basti-TUXEDO-Book-XA1510:~/$ labgrid-client -x ws://192.168.178.70:20408/ws resources
nanopct4/Raspberry_Pi_libcamera/NetworkPowerPort
nanopct4/Raspberry_Pi_libcamera/SerialDriver
nanopct4/Raspberry_Pi_libcamera/ShellDriver
nanopct4/Raspberry_Pi_libcamera/ShellStrategy
nanopct4/Raspberry_Pi_libcamera/NetworkSerialPort

(labgrid-venv) basti@basti-TUXEDO-Book-XA1510:~/$ labgrid-client -x ws://192.168.178.70:20408/ws -p rpi_serial show
Place 'rpi_serial':
  matches:
    nanopct4/Raspberry_Pi_libcamera/NetworkPowerPort
    nanopct4/Raspberry_Pi_libcamera/NetworkSerialPort
    nanopct4/Raspberry_Pi_libcamera/SerialDriver
    nanopct4/Raspberry_Pi_libcamera/ShellDriver
    nanopct4/Raspberry_Pi_libcamera/ShellStrategy
  acquired: None
  acquired resources:
  created: 2021-07-15 08:05:32.721400
  changed: 2021-07-25 18:15:56.988193
Matching resource 'NetworkPowerPort' (nanopct4/Raspberry_Pi_libcamera/NetworkPowerPort/NetworkPowerPort):
  {'acquired': None,
   'avail': True,
   'cls': 'NetworkPowerPort',
   'params': {'host': '192.168.178.29', 'index': 1, 'model': 'eg_pms2_network'}}
Matching resource 'SerialDriver' (nanopct4/Raspberry_Pi_libcamera/SerialDriver/SerialDriver):
  {'acquired': None, 'avail': True, 'cls': 'SerialDriver', 'params': {}}
Matching resource 'ShellDriver' (nanopct4/Raspberry_Pi_libcamera/ShellDriver/ShellDriver):
  {'acquired': None,
   'avail': True,
   'cls': 'ShellDriver',
   'params': {'login_prompt': ' login: ',
              'prompt': 'testlab_user@\\w+:[^ ]+ ',
              'username': 'testlab_user'}}
Matching resource 'ShellStrategy' (nanopct4/Raspberry_Pi_libcamera/ShellStrategy/ShellStrategy):
  {'acquired': None, 'avail': True, 'cls': 'ShellStrategy', 'params': {}}
Matching resource 'USBSerialPort' (nanopct4/Raspberry_Pi_libcamera/NetworkSerialPort/USBSerialPort):
  {'acquired': None,
   'avail': True,
   'cls': 'NetworkSerialPort',
   'params': {'extra': {'path': '/dev/ttyUSB0',
                        'proxy': 'localhost',
                        'proxy_required': False},
              'host': 'nanopct4',
              'port': None,
              'speed': 115200}}

I created those resources using the configuration for the exporter (I collected most of the info from multiple examples found in the documentation not sure if this is correct):

Raspberry_Pi_libcamera:
  location: Testbox_1
  USBSerialPort:
      match:
        '@ID_PATH': platform-xhci-hcd.1.auto-usb-0:1.1
      speed: 115200
  SerialDriver: {}
  NetworkPowerPort:
      model: eg_pms2_network
      host: 192.168.178.29
      index: 1
  ShellDriver:
    prompt: 'testlab_user@\w+:[^ ]+ '
    login_prompt: ' login: '
    username: 'testlab_user'
  ShellStrategy: {}

And my exporter and coordinator are set up as systems services:
Coordinator

[Unit]
Description=Labgrid Coordinator
After=network.target

[Service]
Type=simple
User=basti
Group=basti
StandardInput=null
StandardOutput=journal
StandardError=journal
Environment="MYVAR1=foobar"
ExecStart=/home/basti/.local/bin/crossbar start --cbdir=/opt/crossbar
ExecStop=/home/basti/.local/bin/crossbar stop --cbdir=/opt/crossbar
Restart=on-abort

[Install]
WantedBy=multi-user.target

Exporter

[Unit]
Description=Labgrid Exporter
After=network.target

[Service]
ExecStart=/home/basti/start_default_exporter.sh
Restart=on-failure
RestartSec=30s
User=labgrid
Group=labgrid
SupplementaryGroups=dialout
CacheDirectory=labgrid
CacheDirectoryMode=1775
Environment="LG_CROSSBAR=ws://127.0.0.1:20408/ws"
After=labgrid-coordinator

[Install]
WantedBy=multi-user.target

basti@nanopct4:~$ sudo systemctl status labgrid-exporter
● labgrid-exporter.service - Labgrid Exporter
     Loaded: loaded (/lib/systemd/system/labgrid-exporter.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2021-07-27 18:18:46 CEST; 20min ago
   Main PID: 2427 (start_default_e)
      Tasks: 3 (limit: 4467)
     Memory: 33.8M
        CPU: 5.758s
     CGroup: /system.slice/labgrid-exporter.service
             ├─2427 /bin/bash /home/basti/start_default_exporter.sh
             └─2428 /usr/bin/python3 /usr/local/bin/labgrid-exporter /home/basti/current_export.yaml

Jul 27 18:18:46 nanopct4 systemd[1]: Started Labgrid Exporter.
basti@nanopct4:~$ sudo systemctl status labgrid-coordinator
● labgrid-coordinator.service - Labgrid Coordinator
     Loaded: loaded (/lib/systemd/system/labgrid-coordinator.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2021-07-27 08:18:53 CEST; 10h ago
   Main PID: 1068 (crossbar-contro)
      Tasks: 9 (limit: 4467)
     Memory: 198.3M
        CPU: 13.725s
     CGroup: /system.slice/labgrid-coordinator.service
             ├─1068 crossbar-controller
             ├─2238 crossbar-worker [crossbar.worker.router.RouterController]
             └─2245 /usr/bin/python3 -mlabgrid.remote.coordinator

Jul 27 18:35:27 nanopct4 crossbar[1068]: 2021-07-27T18:35:21+0200 [Router       2238] detached session 3596293211679488 from realm "realm1" (authid="client/basti-TUXEDO-Book-XA1510/ba>
Jul 27 18:35:27 nanopct4 crossbar[1068]: 2021-07-27T18:35:21+0200 [Guest        2245] leave (3596293211679488)
Jul 27 18:35:27 nanopct4 crossbar[1068]: 2021-07-27T18:35:26+0200 [Router       2238] {'authextra': None,
Jul 27 18:35:27 nanopct4 crossbar[1068]: 2021-07-27T18:35:26+0200 [Router       2238]  'authmethod': 'ticket',
Jul 27 18:35:27 nanopct4 crossbar[1068]: 2021-07-27T18:35:26+0200 [Router       2238]  'session': 6564583069106653,
Jul 27 18:35:27 nanopct4 crossbar[1068]: 2021-07-27T18:35:26+0200 [Router       2238]  'ticket': 'dummy-ticket',
Jul 27 18:35:27 nanopct4 crossbar[1068]: 2021-07-27T18:35:26+0200 [Router       2238]  'transport': {'cbtid': None,
Jul 27 18:35:27 nanopct4 crossbar[1068]: 2021-07-27T18:35:26+0200 [Router       2238]                'channel_id': '0000000000000000000000000000000000000000000000000000000000000000',
Jul 27 18:35:27 nanopct4 crossbar[1068]: 2021-07-27T18:35:26+0200 [Router       2238]                'http_headers_received': {'cache-control': 'no-cache',
Jul 27 18:35:27 nanopct4 crossbar[1068]: 2021-07-27T18:35:26+0200 [Router       2238]                                          'connection': 'Upgrade',

I try to run a simple script:

from labgrid import Environment
from labgrid.protocol import ConsoleProtocol


env = Environment('example_conf.yaml')
rpi = env.get_target('example')

cp = rpi.get_driver(ConsoleProtocol)
print(cp)
cp.write(b'test')

but I am greeted with a message about a connection that closed during setup:

(labgrid-venv) basti@basti-TUXEDO-Book-XA1510:~/$ labgrid-client -v -x ws://192.168.178.70:20408/ws -p rpi_serial acquire
acquired place rpi_serial
(labgrid-venv) basti@basti-TUXEDO-Book-XA1510:~/$ python3 rpi_labgrid.py
dropping connection to peer tcp4:192.168.178.70:20408 with abort=True: WebSocket connection upgrade failed (404 - NotFound)
Traceback (most recent call last):
  File "rpi_labgrid.py", line 9, in <module>
    rpi = env.get_target('example')
  File "/home/basti//labgrid-venv/lib/python3.8/site-packages/labgrid/environment.py", line 48, in get_target
    target = target_factory.make_target(role, config, env=self)
  File "/home/basti//labgrid-venv/lib/python3.8/site-packages/labgrid/factory.py", line 152, in make_target
    self.make_resource(target, resource, name, args)
  File "/home/basti//labgrid-venv/lib/python3.8/site-packages/labgrid/factory.py", line 123, in make_resource
    r = cls(target, name, **args)
  File "<attrs generated init labgrid.resource.remote.RemotePlace>", line 11, in __init__
  File "/home/basti//labgrid-venv/lib/python3.8/site-packages/labgrid/resource/remote.py", line 101, in __attrs_post_init__
    super().__attrs_post_init__()
  File "/home/basti//labgrid-venv/lib/python3.8/site-packages/labgrid/resource/common.py", line 123, in __attrs_post_init__
    self.manager._add_resource(self)
  File "/home/basti//labgrid-venv/lib/python3.8/site-packages/labgrid/resource/common.py", line 99, in _add_resource
    self.on_resource_added(resource)
  File "/home/basti//labgrid-venv/lib/python3.8/site-packages/labgrid/resource/remote.py", line 50, in on_resource_added
    self._start()
  File "/home/basti//labgrid-venv/lib/python3.8/site-packages/labgrid/resource/remote.py", line 27, in _start
    self.session = start_session(self.url, self.realm, {'env': self.env})
  File "/home/basti//labgrid-venv/lib/python3.8/site-packages/labgrid/remote/client.py", line 1388, in start_session
    raise Error("connection closed during setup")
labgrid.remote.client.Error: connection closed during setup

So, I tried to access the DUT through CLI, but here I am told that the SerialDriver is unknown:

(labgrid-venv) basti@basti-TUXEDO-Book-XA1510:~/$ labgrid-client -v -x ws://192.168.178.70:20408/ws -p rpi_serial acquire
acquired place rpi_serial
(labgrid-venv) basti@basti-TUXEDO-Book-XA1510:~/$ labgrid-client -v -x ws://192.168.178.70:20408/ws -p rpi_serial con
labgrid-client: error: unknown resource class SerialDriver
This is likely caused by an error in the environment configuration or invalid
resource information provided by the coordinator.

And when I try to use the configuration directly:

(labgrid-venv) basti@basti-TUXEDO-Book-XA1510:~/$ labgrid-client -c example_conf.yaml console
Selected role example and place rpi_serial from configuration file
Could not connect to coordinator: [Errno 111] Connect call failed ('127.0.0.1', 20408)

Why does it try to access the coordinator over 127.0.0.1:20408 when I explicitly provide a different IP for the coordinator?

I have been stuck on this problem for a while now and I hope that someone can discover what I am doing wrong.

Greetings Sebastian

Answered by Emantor

Jul 30, 2021

Hi @initBasti,

TLDR: Your crossbar_url is missing a final /ws.

Your host configuration yaml won't work:

[…]
options:
    crossbar_url: 'ws://192.168.178.70:20408'

crossbar_url should be ws://192.168.178.70:20408/ws. The string here should be exactly the same as the argument to the -x parameter for labgrid client.

Also:

(labgrid-venv) basti@basti-TUXEDO-Book-XA1510:~/$ labgrid-client -c example_conf.yaml console
Selected role example and place rpi_serial from configuration file
Could not connect to coordinator: [Errno 111] Connect call failed ('127.0.0.1', 20408)

does not work since this call is missing the -x parameter for the explicit coordinator.

Instead of always supplying the Labgrid …

View full answer

Emantor · 2021-07-30T10:26:07Z

Emantor
Jul 30, 2021
Maintainer

Hi @initBasti,

TLDR: Your crossbar_url is missing a final /ws.

Your host configuration yaml won't work:

[…]
options:
    crossbar_url: 'ws://192.168.178.70:20408'

crossbar_url should be ws://192.168.178.70:20408/ws. The string here should be exactly the same as the argument to the -x parameter for labgrid client.

Also:

(labgrid-venv) basti@basti-TUXEDO-Book-XA1510:~/$ labgrid-client -c example_conf.yaml console
Selected role example and place rpi_serial from configuration file
Could not connect to coordinator: [Errno 111] Connect call failed ('127.0.0.1', 20408)

does not work since this call is missing the -x parameter for the explicit coordinator.

Instead of always supplying the Labgrid coordinator URL to all commands, you can also set the LG_CROSSBAR environment variable via:

$ export LG_CROSSBAR="ws://192.168.178.70:20408/ws"

I hope this fixes your problems.

4 replies

initBasti Jul 30, 2021
Author

Thank you @Emantor !
This solves the connection problem, but sadly the unknown resource class problem prevails:

(labgrid-venv) basti@basti-TUXEDO-Book-XA1510:~/$ python3 rpi_labgrid.py 
Traceback (most recent call last):
  File "rpi_labgrid.py", line 9, in <module>
    rpi = env.get_target('example')
  File "/home/basti/Documents/Programming_Projects/PyProjects/labgrid-venv/lib/python3.8/site-packages/labgrid/environment.py", line 48, in get_target
    target = target_factory.make_target(role, config, env=self)
  File "/home/basti/Documents/Programming_Projects/PyProjects/labgrid-venv/lib/python3.8/site-packages/labgrid/factory.py", line 152, in make_target
    self.make_resource(target, resource, name, args)
  File "/home/basti/Documents/Programming_Projects/PyProjects/labgrid-venv/lib/python3.8/site-packages/labgrid/factory.py", line 123, in make_resource
    r = cls(target, name, **args)
  File "<attrs generated init labgrid.resource.remote.RemotePlace>", line 11, in __init__
  File "/home/basti/Documents/Programming_Projects/PyProjects/labgrid-venv/lib/python3.8/site-packages/labgrid/resource/remote.py", line 101, in __attrs_post_init__
    super().__attrs_post_init__()
  File "/home/basti/Documents/Programming_Projects/PyProjects/labgrid-venv/lib/python3.8/site-packages/labgrid/resource/common.py", line 123, in __attrs_post_init__
    self.manager._add_resource(self)
  File "/home/basti/Documents/Programming_Projects/PyProjects/labgrid-venv/lib/python3.8/site-packages/labgrid/resource/common.py", line 99, in _add_resource
    self.on_resource_added(resource)
  File "/home/basti/Documents/Programming_Projects/PyProjects/labgrid-venv/lib/python3.8/site-packages/labgrid/resource/remote.py", line 55, in on_resource_added
    new = target_factory.make_resource(
  File "/home/basti/Documents/Programming_Projects/PyProjects/labgrid-venv/lib/python3.8/site-packages/labgrid/factory.py", line 119, in make_resource
    raise InvalidConfigError("unknown resource class {}".format(resource))
labgrid.exceptions.InvalidConfigError: unknown resource class SerialDriver

I have declared the Serial driver within the exporter without attributes and created a match within the place and I declared it within my configuration. Am I missing something obvious?

Greetings,
Sebastian

Emantor Aug 2, 2021
Maintainer

Your exporter configuration:

Raspberry_Pi_libcamera:
  location: Testbox_1
  USBSerialPort:
      match:
        '@ID_PATH': platform-xhci-hcd.1.auto-usb-0:1.1
      speed: 115200
  SerialDriver: {}
  NetworkPowerPort:
      model: eg_pms2_network
      host: 192.168.178.29
      index: 1
  ShellDriver:
    prompt: 'testlab_user@\w+:[^ ]+ '
    login_prompt: ' login: '
    username: 'testlab_user'
  ShellStrategy: {}

should not contain either the SerialDriver, ShellDriver or ShellStrategy. These are drivers, not resources. Labgrid tries to expand the RemotePlace to the defined resources in the place and can't since the drivers aren't resources. Consequently you should also remove the Drivers from the place matches.

initBasti Aug 2, 2021
Author

@Emantor
Ah thank you, that was the missing puzzle piece. I can now successfully connect to the console from the host. I thought that the exporter defines a place with all resources and accessories, but now I understand that it simply declares the devices that I can use and the main configuration is done on the host, where we combine those resources with drivers and strategies.
And with that knowledge, the error message makes perfect sense because a driver is not a resource :).
Thank you for enlightening me, one quick question, I would love to create a complete example case from my setup because I searched for something like that while discovering labgrid. Would you like me to add that example to the documentation (and obviously maintain it), so that other newbies can learn the concepts on a live example? I found that the examples within the example folder were too simple and didn't provide the full picture, which made them feel quite abstract.

Greetings,
Sebastian

Emantor Aug 2, 2021
Maintainer

Thank you for enlightening me, one quick question, I would love to create a complete example case from my setup because I searched for something like that while discovering labgrid. Would you like me to add that example to the documentation (and obviously maintain it), so that other newbies can learn the concepts on a live example? I found that the examples within the example folder were too simple and didn't provide the full picture, which made them feel quite abstract.

Greetings,
Sebastian

Sure, up to date/more complex examples would be great.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Connection closed during setup while trying to use the remote infrastructure via a script #808

{{title}}

Replies: 1 comment 4 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Connection closed during setup while trying to use the remote infrastructure via a script #808

initBasti Jul 30, 2021

Replies: 1 comment · 4 replies

Emantor Jul 30, 2021 Maintainer

initBasti Jul 30, 2021 Author

Emantor Aug 2, 2021 Maintainer

initBasti Aug 2, 2021 Author

Emantor Aug 2, 2021 Maintainer

initBasti
Jul 30, 2021

Replies: 1 comment 4 replies

Emantor
Jul 30, 2021
Maintainer

initBasti Jul 30, 2021
Author

Emantor Aug 2, 2021
Maintainer

initBasti Aug 2, 2021
Author

Emantor Aug 2, 2021
Maintainer