-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
prolog on the browser #43
Comments
As part of my Masters Thesis I implemented both a JIT and AOT compiler for Constraint Handling Rules in JavaScript. Have a look at http://chrjs.net to get an insight. (Related repositories) I am really interested in a way to run Prolog in JavaScript, both in the browser and node.js - mainly because I completely disagree on Anne's intention:
I love both Prolog and JavaScript and see many advantages in use on in another. But motivations are different :) I would declare the following targets when implementing Prolog in JavaScript:
Recently I stumbled upon the article Solving riddles with Prolog and ES6 generators which really motivates me to have a deeper look into some Prolog-to-JavaScript transpilation. |
I have recently experimented with compiling SWI to WebAssembly. I managed to compile a core of a relatively old version (5.6). There are 2 issues that prevent it:
Atom table is generated using a script in older versions and I was able to compile 5.6 but because of missing boot.prc I only get error: This is related to #34 which is also made difficult due to cross-compilation issues. |
I'd go for a new version. As for the problems, they are the same as for iOS. To deal with the atom table, the Makefile/configure configures two C compilers: one that generates native code and one that may cross-compile. The first is used for the helper tools, the latter for the real target. At some point this was moved from shell script to C to simplify porting ... The simplest way to create boot.prc is to copy it from a locally installed native machine. The only requirements is that the word size (32/64 bit) matches. Alternatively you'd need something that can run WebAssembly on the host. I would assume that exists or will exist soon. Perhaps we should allow the system to start from the Prolog source in I welcome this very much. I'm glad to help with advice, little things and figuring out a way to get the changes into the main source repo. |
@JanWielemaker I'm still experimenting with 5.6 as I can build it without any patching. Conf tools and multiple compilers is still lots of magic for me. Anyway, I have found out:
I was able to compile the .prc file through Node.js-wrapped SWI WebAssembly binary using the boot directory with the I have verified with strace that it opens the .prc file I generated through the same thing. The .prc file looks like it contains something, it's not empty. Hex editor shows Signals are required for threading support? I disabled threading in the configure options, full command:
I also edited Starting without a .prc file sounds an option but for browsers we would still need some compact way to make those files available to the binary. |
Still, going for an old system is not a very good idea. Current versions rely far less on non-portable features and have lots of stuff fixed. They also have improved cross-compilation support as the Windows version is cross-compiled. See Signals were indeed used for threading. Current version doesn't need signals for threading. It does Why the state is wrong is a harder question. That probably requires comparing the output of the boot compilation with a working version and/or look in pl-wic.c to see what triggers this message. |
I have done some debugging for state loading. I got a 32-bit native state file for the same version and compared it with the wasm-based one. I noticed some minor differences but I think the issue is not in the file. The file seems to be opened successfully and read-in wholly during rc_open_archive but subsequent rc_read calls fail to read anything (end of stream reached). |
AFAIK, the old implementation uses memory mapping if it can. Possibly the backup scenario using ordinary I/O fails as it has not been tested for ages? It surely does require repositioning in the stream. How does web assembly handle additional files? Can it perform memory mapping? Note that 7.7 uses a completely different resource format based on |
There is no concept of filesystem in WebAssembly. The memory is just a linear space. There is no memory mapping (https://github.com/WebAssembly/design/blob/master/FAQ.md#what-about-mmap). It is possible to bind external functions into WebAssembly and that's only way to communicate with things outside. That's how fread and other POSIX functions have been supplied by Emscripten compiler. Here is more detailed description: https://github.com/WebAssembly/design/blob/master/Semantics.md I will look into compiling newer a version once I figure out how to effectively tweak the configure file. |
As a note, 7.7.14 does not allow to build with --disable-mt (without threads) as missing mutex support in pl-zip.c gives compilation error:
|
Another note, the configure script assumes that compiler creates certain types of object files, and greps it to detect endianness:
Configure script: https://github.com/SWI-Prolog/swipl-devel/blob/9fc9a97b6e0e6d4dcaf25a77602d23d89a1c9016/src/ac/ax_c_float_words_bigendian.m4 |
I have temporarily removed the endianness check by removing code from the m4 file and replacing it with ax_cv_c_float_words_bigendian=no (WebAssembly is little-endian).
Trying without
I also have some issues with modules in subdirectories. For example, object code for files in src/os/ and src/minizip/ is put directly under src/ but the linker is not expecting it. |
I just pushed a number of patches that makes the current git version (
Say |
P.s. do I understand webassembly does implement threads? |
WebAssembly has no threads on VM level. The Emscripten compiler provides pthread implementation using WebWorkers and shared memory. Shared memory is disabled in browsers for now (Spectre mitigation) which means no threads at the moment but I'm sure that one day they come back. More information: https://kripken.github.io/emscripten-site/docs/porting/pthreads.html |
Thanks! The git version resolves lots of issues above but at the moment I'm still stuck with wrong object file locations:
A compiler command (no output argument):
This is for the native version (-o arguments are present here):
I do not know where such discrepancy comes between generated Makefiles. |
Seems an issue with the Makefiles or configure not picking up things properly. Do you have a simple recipe to get me where you are now? It is probably easier for me to figure it out myself than to send zillions of log files and edited files around trying to guess what is wrong ... One of my dev machines runs Ubuntu 18.04 and there I have an |
The steps to get there (assuming the last git version, adjust paths):
zlib build result will just reside in this directory, there is no Build SWI wasm version (asjust zlib path):
The result of last step is still LLVM IR bitcode.
This will likely require native swipl binary copied from other location or some other magic. Once we have steps 11 and 12 executed, we should have enough code to generate the boot file by running the wasm binary through Node.js. It did work like that with SWI 5.6.
This last command produces files |
I looked that Ubuntu 18.04 contains old Emscripten package. It might be a better idea to install it using the official instructions: https://kripken.github.io/emscripten-site/docs/getting_started/downloads.html It is self-contained and installs into a single directory without spilling its guts all sround the system and possibly screwing up the system compilers. |
Thanks. I'll give it a try. |
Fixed one. It seems
Now we get a binary :) Next step is running it ... |
Need to add
Running the result fails:
One of the pittfals is that some functions are present, but stubs. E.g., it claims to have |
Hmmm. In recent versions |
I got it building on my system too. I'm actually surprised it gets this far. WebAssembly and Emscripten compiler are still very experimental. It's hard to give definite answer about mmap. Seems like some of it is there indeed. Needs more research. I compiled minigzip (zlib's test app) to WebAssembly and this passes:
There could be more things wrong, lots of moving parts with this setup. |
Good to know. Might be an issue configuring minizip code that is linked into Prolog. Now documenting my steps and compiling debugging into Prolog (add The fact that it manages the boot compilation is promising: it is already running quite a bit of Prolog to do that! Tried the swipl.prc from the 32-bit windows version, but that is not the first problem: same errors. |
The first problem is indeed that it tries to |
That is not what is used though. This is the compressor. Good that that works. What is used in Prolog is in |
There is indeed a bug inemscripten
Run using
The zero block should have been written with data2, but instead, data2 is at the end. Versions (on Ubuntu 18.04):
|
Thanks! This will likely also explain missing fseek behavior with 5.6. I'm going to try to make Emscripten fseek behave correctly. |
Yip. Both minizip and the old 5.6 resource management library use
Seems you suggest you can get this fixed in Emscripten? How quickly would this be distributed? |
I made a writeup with a bit saner instructions. The console example and cleaner build instructions are now here: https://github.com/rla/swi-prolog-wasm in case someone else wants to experiment with. |
Great! Merged your pull request. Considering the relevance, what about moving this repo to the GitHub SWI-Prolog organization and get a member thereof? |
How do I run the queens example here. It doesn't startup in Was using: |
There’s a button below the program area for reconsulting the program.
…Sent from my iPad
On 26 Jun 2018, at 20:04, Jekejeke Prolog ***@***.***> wrote:
How do I run the queens example here. It doesn't startup in
Seamonkey. And I get the following error in Chrome, but
did not yet try Edge. Always Windows 10 platform:
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
|
@jburse, according to the available information at https://www.seamonkey-project.org/releases/seamonkey2.49.3/, the latest SeaMonkey 2.49.3 is based on Firefox 52 ESR. That Firefox version has WebAssembly disabled. The console example was put together very hastily and currently requires input to stdin to be provided before SWI calls C's read() function, otherwise it locks up by waiting for input forever. A better console is probably coming soon. |
Enhancing the demo console: SWI-Prolog/swipl-wasm#2 |
👍 🥇 Thank you Raivo and Jan for this excellent work! I believe this is a powerful tool that will open up new doors for Prolog and the Web in general. |
SWI-Prolog switched recently to CMake-based builds and I have been trying to get it building with Emscripten again. The basic approach is similar than with autotools-based build:
(this ignores thread/mmap issues for now but I guess that those can be disabled through simple set()-based options inside cmake file itself) This of course gets stuck in running defatom step. It seems like there is a way to do this in CMake through separate build that does not go through cross-compilation toolchain setup. Links to potential solutions: However, it does seem like such use-case is not well addressed in CMake. The even bigger issue is generating the boot file where we have to generate host-runnable thing that still uses target platform compile settings to produce target-compatible boot file. There is also an issue with CMake 3.7.2 on Debian Stretch. Running CMake fails with its internal assertion failure:
It only happens in cross-compile setup when running through emcmake. Running with CMake 3.9.5 from Stretch backports seems to be ok. |
Thanks for looking into this. SWI-Prolog is not yet switched, but in due time this is indeed almost surely going to happen. I'd be grateful if you propose a solution as a pull request. If not, it will happen in due time as well. Possibly the easiest way is to use |
Comes out that the Emscripten CMake toolchain files does exactly that and attempts to run wasm binaries through Node. So I debugged the exact defatom issue and saw it was just having a segfault-equivalence of WebAssembly. It could be that when some file it reads does not exist, it will crash. I got both mkvmi and defatom working, it compiles and links libswipl but does not link swipl so it passes the wasm validator. I thought it does not pass because some thread primitives are not available. I went to disable threads and got the following error:
I think that File
I have my CMake file modifications in this branch: And I build using the command:
The toolchain file location depends on installed Emscripten version. |
Looks promising! I'll checkout disabling threads as there are still people in the rest of the world that do not believe in them either. |
Pushed a fix that allows single-threaded compilation. I guess getting the thread headers also pulled in |
Thank you! I also noticed that CMake is a bit easier to work with than autoconf. I have now built again a working wasm binary. While running it I had one more issue:
It seems like this predicate does not consider a case when no packages are built. Right now I have modified CMake files by adding a bunch of if()/else() around various things. I wonder if there is a better alternative. My last diff: rla/swipl-devel@19fe42b I'm currently porting my changes from my last configure-based branch and then proceed with a PR. |
Pushed a fix for this. Had a look at your diff. Looks fairly neat, I'd say. I'm not going to merge it as yet as with the planned changes there should be no more reason to run the installed Prolog system for documentation and index generation which will remove one of the Other pans that should materialize in the next couple of days are to make sure the source tree can be read-only and allow building a shadow home in the cmake build directory that allows running the full system identical to what will be installed from the build directory. That should make playing around with different versions a lot easier. |
@rla I guess you updated such that the emscripted builds with the latest version? I'd like to merge that. If you do not have that I'll assemble it from the branch above. Best thing would we can now simply have a native and WASM build from the same source tree at the same time. |
@JanWielemaker I decided to wait until cmake-based build files become a bit more stable and I have not worked on this recently. The set of changes in my comment from month ago do not include some important changes I made to the original autotools-based build. It is missing the actual build output that runs on browsers. Give me some time to pick them up and implement for the cmake-based build. I will let you know once I have done that. |
Was a good idea to wait. I now consider the overall structure of the cmake based build stable. Surely some small stuff will change. Seems the WASM build can easily be setup in a nice integrated way so I can test it regularly and possibly setup automatic testing. |
It would be very nice if changes based on this commit get into the SWI tree: rla/swipl-devel@a35a599 before it rots away again. I did not create formal PR as some things need review. CMake files are again a nest of ifs. Build command (adjust Emscripten version in the toolchain file path):
I got the following issues: Signals. Some data structures in Popen in swipl-ld. Wasm build does not use swipl-ld and I'm not sure if it ever will. It contains popen function that fails the build. Potential workaround is better way to disable building it than nest of conditionals or just use Minor issue with File The test is not fully automated yet. I added manual test with Also noticed that Utils.cmake has I noticed that Curses could use a similar switch as GMP and threads but I was not sure how much changes I would need to add it. There is now I think if we could figure out what to do with the nest of ifs in CMake files the current build scheme would be quite stable and I can update build and usage guides. |
Thanks for the list. I'll try to pick it up from here, probably somewhere next week. Then we probably go through a few iterations before it is a good primary citizen. |
I was wondering why the library index was not generated. Appears a bug in opendir(). Filed |
I have pushed the WASM stuff to the master branch. Mostly moved some stuff around and provide general options for e.g., disabling all signal handling. Also fixed some stuff the avoids the need for conditional expressions in Prolog and cmake files. I think it is now ready for incremental improvement. There should also be an easy to find file pointing where to find the several bits and pieces. What about adding a file One thing to consider is to create a Prolog resource file (boot32.prc) that contains the library and is embedded into swipl-web. That avoids the need for a data file and I see that the content of the data file is not compressed, while the Prolog resource file is. |
Thank you for accepting the changes! I'm now going to play around a bit and write WASM.md. I thought about adding the library to the boot file. However, I did not like embedding it into the |
I'm now able to run some the tests. I pushed a fix to avoid float->integer traps. Biggest problem testing the stuff is the above mentioned bug in opendir(). This breaks creating the library index and except for a limited number of old tests the tests are executed by scanning the test directory which doesn't work. Of course we can hack around that, but that seems a bad idea. We also need to properly set the optimization flags. These are currently simply |
I wait and see what happens to the readdir bug. At some point, in the old autoconf-based build, I used nodefs instead instead of noderawfs, which mounts to a subdirectory in the virtual filesystem but requires path translation during invocation through Node. I could port it over again as nodefs seems to be a lot less buggier. |
This issue has been mentioned on SWI-Prolog. There might be relevant details there: https://swi-prolog.discourse.group/t/what-is-the-status-of-swi-prolog-ported-to-webassembly/2339/1 |
I know this issue is a little bit old, but I did some experiments and mentioned also on SWI-Prolog: https://swi-prolog.discourse.group/t/clojure-clojurescript-with-swi-prolog/5399 |
Prolog is an ideal candidate for something ike ClojureScript - a Prolog that runs on the browser.
Such a system could make javascript disappear as a language - something we'd all applaud, I think.
There have been some hobby project level attempts to build Prolog in javascript, but a recent development, WebAssembly, means we could deploy to a sane VM.
WebAssembly
https://medium.com/javascript-scene/what-is-webassembly-the-dawn-of-a-new-era-61256ec5a8f6#.facwrlc6p
https://brendaneich.com/2015/06/from-asm-js-to-webassembly/
https://www.w3.org/community/webassembly/
Is this something (and I'm asking, not advocating) we could get ahead on, and have something head and shoulders better than anyone else ready when the browsers get there?
The text was updated successfully, but these errors were encountered: