[NOTES] Running BOLSIG+ and other tools

[daanboer]

In GitLab by @lxcat on Sep 29, 2022, 13:01

Background

• We want to run standalone tools like BOLSIG+, loki, etc from LXCat, and compare results.
• Since the tools are independent, it may not be always possible to modify the tool.

Possible architectures

• We can directly call into the tool as a library, run the calculation, and retrieve the results
  • webassembly (serverless)
  • a thin wrapper around the library (requires a server), the wrapper would need to translate the input/output to and from what the library understands (Fortran in case BOLSIG)
• If it's not possible to refactor the tool into a "library + exe", we can run a shell job
  • in a temporary directory, but that might require managing a working directory and configuration files
  • this can be circumvented (and remove any disk I/O) by using a ramdisk/named pipes
  • using a pipe would require that we can modify the tool to r/w its input/output to a stream (stdin/stdout)

Constraints

Application uses BLAS libraries like dgesv.

Action points

• Try to build a toy Fortran program that compiles to webassembly as POC
• Survey different ways to wrap and call a function in the tool's shared library (need access to tool source code)
  • Fortran ISO C binding
• Explore options how to deploy a ramdisk/named pipe based solution

[daanboer]

In GitLab by @lxcat on Sep 29, 2022, 17:14

@wgraef: summary from our discussion today with Gijs, any comments would be helpful.

@sverhoeven: I hope I didn't miss anything.

By Suvayu Ali on 2022-09-29T15:14:23 (imported from GitLab)

By LXCat Project on 2022-09-29T15:14:23 (imported from GitLab)

[daanboer]

In GitLab by @lxcat on Sep 29, 2022, 21:06

This looks like a good summary to me.

Regarding calling fortran code on as a library, while there are libraries to do everything in fortran, a wrapper is indeed a much better option. The third party tools required are much more mature.

The named pipe option I have never considered. It's definitely interesting. Any information on that would be very welcome.

In the case of a wrapped library and named pipe, some modifications will be required on the fortran side. For the named it would be enough if data from a stream can be processed right?

For the webassembly case, could that work in essence the same as the standalone program now, with the data read and written being located in some kind of virtual filesystem?

By LXCat Project on 2022-09-29T19:06:53 (imported from GitLab)

By LXCat Project on 2022-09-29T19:06:53 (imported from GitLab)

[daanboer]

In GitLab by @lxcat on Sep 30, 2022, 24:23

For the named pipe option, it depends a bit on how the configuration/input file is used currently. It's not necessarily simply reading/writing to a stream. For example, maybe different sections of the file is read before different stages of the simulation, or maybe it is read multiple times, then refactoring could be very difficult.

For the webassembly case, @sverhoeven would have to correct me, it would be like calling a function from a library/module but on the client side (in the browser). Basically the same code base is being compiled to both a native binary (for standalone use) and a webassembly module (for use on LXCat).

By Suvayu Ali on 2022-09-29T22:24:23 (imported from GitLab)

By LXCat Project on 2022-09-29T22:24:23 (imported from GitLab)

[daanboer]

In GitLab by @lxcat on Sep 30, 2022, 11:31

You are correct. The JavaScript in the browser needs to call a function inside the Web Assembly module (WASM). So the source (C/C++ wrapper around Fortran code) of the WASM also needs to expose/export a function.

By Stefan Verhoeven on 2022-09-30T10:09:29 (imported from GitLab)

By LXCat Project on 2022-10-11T21:04:27 (imported from GitLab)