Multiple Simultaneous Applets

[gregdavill]

For a hardware project I'm working on now I'm using both the Glasgow as a UART pty and a separate basic FTDI breakout for JTAG.

I see potential for having both a JTAG and UART interface when debugging a target.
Or having two UARTs one for a CLI and another for Embedded Logic analyses in SoC designs.
From a hardware perspective there isn't any reason why Glasgow couldn't theoretically handle this.

But I'm not sure what that would look like from the software side, and the command line side.

I understand this feature may be impractical to support in a general sense, but do you think it may be possible for some subset of cases? Like Applet.* + UART?

[whitequark]

I understand this feature may be impractical to support in a general sense, but do you think it may be possible for some subset of cases?

I think it is, in principle, practical to support in general, and in fact, I have aimed to do so from the very beginning. It's been one of the design goals!

But I'm not sure what that would look like from the software side,

This one is easy: you simply build/run multiple applets. I've never actually tried it, but none of the applet APIs assume that there is only one applet; there can be any amount of submodules in the target, and asyncio takes care of concurrency inherent to multiple applets.

and the command line side.

Here... I'm not sure either, which is precisely why it isn't yet implemented.

Let's brainstorm CLI ideas in this issue?

[gregdavill]

Brainstorming sounds like a good idea.

One idea, which I think could work in practice, but may be not very elegant.
The initial command which creates the multiple applets, also creates a server, each applet gets a socket or pipe interface.

In separate terminals each applet can run in a "client" mode which connects to the server to interact with the hardware. Kind of like how litex_server operates.

[whitequark]

One problem is that currently applet code assumes that build/run state is one and same, and that's pretty well baked in. So these "clients" would have to be thin clients, perhaps reduced to passing the terminal fds and cwd through that socket.

But I think I could do that.

[whitequark]

That still leaves open a question of how to specify multiple applets on the CLI invocation of the server.

[gregdavill]

You could just specify run multiple times?

glasgow run uart --pin-rx 1 --pin-tx 0 -V 3.3 -b 115200 pty run uart --pin-rx 2 --pin-tx 3 -b 115200 pty

[whitequark]

I think you can't do that with argparse. Imagine what would happen if pty took a filename.

[gregdavill]

Turn each command into a string?

glasgow run "uart --pin-rx 1 --pin-tx 0 -V 3.3 -b 115200 pty" "uart --pin-rx 2 --pin-tx 3 -b 115200 pty"

So argparse just sees two strings? This gets ugly when filenames are required and strings need to be escaped.

Or adding a new command:

glasgow run-multiple

That drops you into a REPL or other interactive console that enabled you to add interfaces then run. If this is the path this feature heads down, it would be possible to show recently used interface combinations.

[whitequark]

I think a guard argument can be done:

glasgow run-multiple uart --pin-rx 1 --pin-tx 0 -V 3.3 -b 115200 pty -- uart --pin-rx 2 --pin-tx 3 -b 115200 pty

Still kind of ugly though.

[alexhude]

+1
run-multiple with -- guard argument is pretty straight forward, I vote for that :)

[andresmanelli]

Hello, what about a config file? I see it like a "glasgow compose" file

glasgow run -f config.?

[whitequark]

what about a config file?

I don't know what a good syntax would be, and worse, currently the frontend is very closely tied to argparse.

[RX14]

Would it be possible to have the interface be to run glasgow multiple times in multiple terminals and have that be handled gracefully? Then the server/daemon would be configured over some form of IPC?

[whitequark]

Would it be possible to have the interface be to run glasgow multiple times in multiple terminals and have that be handled gracefully?

Not really, since there's no partial reconfiguration possible on iCE40.

[alexhude]

I think config file should be an alternative to command arguments, not a replacement. Therefore run-multiple with guard or something similar should also exist.

[fabianfreyer]

What's the status of this. Is this blocked on replacing argparse in #234? Is there something that can be done to help with this? If it's really just the CLI layer that's problematic here, would making a prototype PR to play around with the ergonomics of the CLI be helpful?

Regarding more brainstorming:

what about a config file?

I don't know what a good syntax would be

one run command per line, with a finishing token. Take it from stdin by default, unless -f given.

$ glasgow run-multiple --port A -V 3.3
run uart --pin-rx 1 --pin-tx 0 -b 115200 pty
run uart --pin-rx 2 --pin-tx 3 -b 115200 pty
done // implicit if EOF
< further interaction >

$ cat glasgow-session
run uart --pin-rx 1 --pin-tx 0 -b 115200 pty
run uart --pin-rx 2 --pin-tx 3 -b 115200 pty
$ glasgow run-multiple --port A -V 3.3 -f glasgow-session
<further interaction>

This has the advantage of being able to still use argparse on each of those lines, I guess.

I think multiple guards as an alternative are probably the best tradeoff:

$ glasgow run-multiple --port A -V 3.3 -- \
    run uart --pin-rx 1 --pin-tx 0 -b 115200 pty -- \
    run uart --pin-rx 2 --pin-tx 3 -b 115200 pty

[alexhude]

I would agree even with dirty ugly hack for personal use currently 😁
If someone knows a way to push several applets into Glasgow, please post here 🙏

[ChuckM]

Random thought on this somewhat dated thread ...

One might consider both a config file and arguments. In many ways glasgow appears to be like a "hardware interpreter" in the same sense the python is a software interpreter. A glasgow "program" would consist of the applet(s) to run. It would specify at its entry point what command line arguments it would accept and how it would use them.

I'm going to have to noodle on some syntax ideas but basically think shell script where instead of shell commands you have applets.

[whitequark]

I'm going to have to noodle on some syntax ideas but basically think shell script where instead of shell commands you have applets.

We do not need new syntax. We have perfectly good Python right here. We even have a shell-style interpreter! (glasgow repl)

[fabianfreyer]

We do not need new syntax. We have perfectly good Python right here. We even have a shell-style interpreter! (glasgow repl)

@whitequark does this mean that the ability to run multiple applets via the CLI is no longer a goal? Above, you mentioned that this is blocked on the command-line side because it's not clear how this should look. There seem to be two, not necessarily mutually exclusive, approaches:

• Shoehorn this into a command-line (run-multiple, etc.), which seems difficult, if not impossible using argparse.
• Specify what applets to run in some config file. This was blocked earlier by the question about a good syntax (Multiple Simultaneous Applets #180 (comment)). Does this mean this is no longer an issue, and people should just use the script / real subcommand instead? In this case, it's not really clear to me right now how one would do that easily, without e.g. mocking argparse and instantiating each frontend, which seems cumbersome.

[whitequark]

@fabianfreyer Sorry, I was unclear earlier. This thread is so old I nearly forgot what I was thinking about back then...

• Shoehorn this into a command-line (run-multiple, etc.), which seems difficult, if not impossible using argparse.

You could do this with argparse--it is a bit hacky though. The problem is sometimes more with applets themselves that try to do things like read from the console, or take exclusive use of limited FPGA resources. The UI is also very awkward.

People will probably want this no matter what.

• Does this mean this is no longer an issue, and people should just use the script / real subcommand instead?

We still need a configuration file, but the surface syntax of the configuration file should just be Python.

The deep syntax, i.e. the meaning of the objects and functions used within, still needs to be defined.

Ultimately the current argparse syntax should be reduced to a special case of whatever we use for the configuration file. For that however the applets need to use some sort of language (again, Python-based) to declare the options they take.

[fabianfreyer]

FWIW, here's my quick-and-very-dirty script I just used to do things, if someone wants to clean that up somehow, I could imagine it to be somewhat useful.
https://gist.github.com/fabianfreyer/b6371761fa521d14b48c602e738006b9

[alexhude]

@fabianfreyer Thanks heaps! I hope I will be able to use that as reference to get JTAG+UART applet combo :)

[whitequark]

@fabianfreyer This is essentially the best way to achieve what you want right now. Good job!

[alexhude]

OK, as I understand maximum amount of applets is 2, because otherwise there is an error cannot claim pipe: out of pipes?

[whitequark]

For now yes, though there is an open RFC to change that.

[urjaman]

Thank you a bunch for that dual UART example - I updated and modified it a bit and made an UART (on the tty) + spi-flashrom (TCP) dual "applet". (Oh btw, it called uart0_parser twice to provide the args for uart0 and uart1 :P)

https://gist.github.com/urjaman/0e48bb2fc31bfe355cdd18a5388705a7

[whitequark]

@urjaman I'd like to prototype a much better interface for achieving the same sometime soon. Would you like to participate?

[urjaman]

@whitequark I'd be happy to participate - feel free to poke me on discord or IRC when it's relevant because I don't actively "poll" github.