Is there a GLIB emscripten port available ? #11066

o-micron · 2020-05-02T14:46:51Z

Hi, I had some code depending on glib and I went to build glib from source to find out it is a rabbit hole.
glib is using meson build system, no cmake or makefiles out there ...
6 hours later, it turns out the problem is building libffi for emscripten ...
Am I missing something ? Maybe there is a glib port available ?

sbc100 · 2020-05-02T16:25:15Z

I'm not aware of any glib port I'm afraid. I imagine porting libffi might be tricky as IIRC it involves assembly code. Perhaps there is way to remove that dependency.

Feel free to add a port to https://github.com/emscripten-ports if you manage to get it working.

kleisauke · 2020-05-02T16:25:41Z

I had a similar problem while porting libvips to Emscripten (see libvips/libvips#192 for details). GLib and libffi currently require patches to successfully build them with Emscripten. Could you try these?:
https://gist.github.com/kleisauke/acfa1c09522705efa5eb0541d2d00887

These patches were copied from my local Git repository and tested with libffi 3.3 and GLib 2.64.2. Most of the libffi work has already been done by @Brion (thanks!), see: https://github.com/brion/libffi/commits/emscripten-work

I just made it compatible with the latest Emscripten (by swapping Module['usingWasm'] with Module['wasmMemory'] and converting the EM_ASM_ to EM_JS).

Anyways, I'll eventually try to incorporate these patches upstream.

kripken · 2020-05-02T17:37:20Z

That sounds great @kleisauke !

If there are issues getting them upstream, adding a port in emscripten-ports is also an option.

o-micron · 2020-05-05T04:13:54Z

thanks @kleisauke for your patches

I tried them on ubuntu 20.04 with :

meson 0.54.1
GLib 2.64.2
libffi (the meson subproject master branch), not the brion's branch, because brion's branch doesn't contain a meson.build file which is a dependency for the main glib meson build script

Things look a bit better but I keep getting the error
ERROR: Problem encountered: Unsupported pair: system "emscripten", cpu family "wasm32"

I have applied your emscripten cross file --cross-file ./emscripten-crossfile.txt which is also what meson suggests for cross compiling as stated here starting a cross build

wasm32 is the right cpu family and not x86 of course ... but wasm32 seems unsupported
Meson is updated and so that shouldn't be the problem ?

I am exploring multiple issues at the same time I guess 😊

kleisauke · 2020-05-05T12:00:50Z

@o-micron I've updated the gist with an example build script. Let me know if that works for you.

o-micron · 2020-05-05T15:48:00Z

thanks @kleisauke that definitely works ! 💯

Also for anyone coming from the future for the same issue, here are some packages needed as well

# required tools ...
sudo apt install autoconf libtool m4 automake

# install meson from pip3
pip3 install meson
export PATH=~/.local/bin/:$PATH

# make sure you have emcc and wasm-ld in path
cd emsdk
./emsdk install latest
./emsdk activate latest
source ./emsdk_env.sh
export PATH=$(pwd)/upstream/bin/:$PATH

# download the gist https://gist.github.com/kleisauke/acfa1c09522705efa5eb0541d2d00887
# extract gist .zip and cd to that folder 
# now run ./build.sh and you should be good to go

kripken · 2020-05-05T17:34:44Z

Great!

If someone is interested to add this to emscripten-ports let me know - I can create a repo there and give you access. Would be good to do that since this has come up more than once.

o-micron · 2020-05-05T21:48:44Z

@kripken I would definitely help adding that to emscripten-ports but I would rather let @kleisauke do it and take the credit for his work

I have added above the simple steps I took to successfully build it on ubuntu

o-micron · 2020-05-06T13:04:24Z

@kripken Here is a script which does everything really ... could replace the whole port repository

Tested on Ubuntu 20.04

# required tools ...
sudo apt install autoconf libtool m4 automake

# install meson from pip3 because glib uses meson ..
pip3 install meson
export PATH=~/.local/bin/:$PATH

# Get the emsdk repo
git clone https://github.com/emscripten-core/emsdk.git

# Enter that directory
cd emsdk
# Fetch the latest version of the emsdk (not needed the first time you clone)
git pull

# Download and install the latest SDK tools.
./emsdk install latest

# Make the "latest" SDK "active" for the current user. (writes ~/.emscripten file)
./emsdk activate latest

# Activate PATH and other environment variables in the current terminal
. ./emsdk_env.sh

# Make sure we have mainly wasm-ld and other tools in path as well
export PATH=$(pwd)/upstream/bin/:$PATH

cd ..

# Get @kleisauke patches
git clone https://gist.github.com/acfa1c09522705efa5eb0541d2d00887.git
cd acfa1c09522705efa5eb0541d2d00887

chmod +x ./build.sh
./build.sh

kripken · 2020-05-07T22:17:04Z

Nice @o-micron, yeah, if we don't need a separate repo then this is quite easy. That script could be converted into a python script and put under tools/ports/glib.py basically, and integrated into ports. I think that would be great to do!

sbc100 · 2020-05-07T22:33:29Z

For a long time we (I) have been trying to come up with a place to put "contrib" ports like this. Simply adding everything and the kitchen sink to "tools/ports" doesn't scale very well. I don't have an answer right now.. only the question.

kleisauke · 2020-05-08T11:58:34Z

For a long time we (I) have been trying to come up with a place to put "contrib" ports like this. Simply adding everything and the kitchen sink to "tools/ports" doesn't scale very well.

The meson build system solves this problem with the Wrap dependency system.
https://mesonbuild.com/Wrap-dependency-system-manual.html
https://github.com/mesonbuild/meson/blob/master/mesonbuild/wrap/wrap.py

Perhaps we could do something similar with Emscripten ports? For example, port files could look like this:

Details

[port-file]
directory = glib-2.64.2

source_url = https://download.gnome.org/sources/glib/2.64/glib-2.64.2.tar.xz
source_filename = glib-2.64.2.tar.xz
source_hash = 9a2f21ed8f13b9303399de13a0252b7cbcede593d26971378ec6cb90e87f2277

patch_url = https://gist.github.com/kleisauke/acfa1c09522705efa5eb0541d2d00887/raw/37e3787a16cef2b8a1b577d469406ebbb80147d7/glib-emscripten.patch
patch_filename = glib-emscripten.patch
patch_hash = bb46e867ce457d4c34551968303ffde520b5f17a54df69653e11677b7df9cfc5

[port-git]
url = https://github.com/brion/libffi.git
revision = emscripten-work

^{(This is heavily inspired from Meson Wrap files)}

The advantage of this is that updating ports can then be done without much effort, since you don't have to embed an entire project into the source tree (i.e. tests/third_party) or to maintain separate forks (i.e. the emscripten-ports organization). Any Emscripten specific patches can then be easier integrated upstream.

This is only for the source part, I'm not sure how the actual build of these ports might look like (because it can be very specific to users' needs).

sbc100 · 2020-05-08T17:41:39Z

The immediate problem I'd like to solve is how to add ports without adding:

New settings such as USE_ZLIB
Load on the CI system (its fine if we have a separate builder that build the universe but we don't want to block evey change on this).

There are lots good ports systems out there. Its not clear if we want to block this work on choosing one and switching to it, or sticking with our existing one. Part of me would like to do all of this out-of-tree using something like pacman (from arch) the freebsd ports system: https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ports-using.html. But as an incremental step I think we could just use tools/ports/contrib with out existing ports system. I think the plan was to have a generic -s USE_PORT=foo setting to avoid adding new settings for these. Also, embuilder would not build these by default. I think is convered by an existing bug but I can't seem to find it right now.

sbc100 · 2020-05-08T17:42:50Z

At least one related bug: #10048

kripken · 2020-05-08T18:25:42Z

The immediate problem I'd like to solve is how to add ports without adding:

New settings such as USE_ZLIB

I think we can get rid of that boilerplate by automatically adding a setting for each port. So adding a port would mean just adding a single file under tools/ports/ but no changes to src/settings.js for example. Does that address your concern, or even so would you be unhappy?

Load on the CI system (its fine if we have a separate builder that build the universe but we don't want to block evey change on this).

I think we don't need to test every thing in ports/ on our CI. Newly added ports can be done with no new testing.

sbc100 · 2020-05-08T18:31:23Z

Yes I pretty much agree with that. I do think that using a subdirectory would useful to indicate to two distinct and separate types of ports. I like tools/ports/contrib myself.

sbc100 · 2020-05-08T18:32:09Z

Also the current ports system isn't quite plug and play enough, you currently also need to add to the list in __init__.py when you add one. We should make it purely file driven, but I've run into issues doing that in the past.

sbc100 · 2020-05-08T18:32:29Z

For now, adding to __init__.py is not a blocker IMHO

kleisauke · 2020-05-08T18:52:03Z

For the record, I would like to check in glib and libffi as port, but my main concern with https://github.com/emscripten-ports is that it's difficult to maintain when a new upstream version is available, especially if the Emscripten parts are also integrated upstream. Downloading tarballs and applying custom patches (if necessary) seems more common and a lot easier.

I think is convered by an existing bug but I can't seem to find it right now.

Looks like you're describing #9353.

sbc100 · 2020-05-08T20:33:44Z

emscripten-ports is basically just a way to store the result of applying patches to an upstream release. if you don't have any patches then you don't need to use it you can just use the upstream github release directly.

We can argue the benefits of using tarball + patche file vs using a git repo, but changing the way we do things would be much bigger change and if you just want to get glib working in emscripten than using emscripten-ports to hold your patches is the easiest way to go today.

If we do want to get into the weeds of git vs patch patch files, I think I would argue that maintaining a series of patches on top of an upstream set of code is pretty much exactly what git is designed to do. A system you like describe, such as quilt (https://linux.die.net/man/1/quilt) where the user must manually the parch series as a set of individual files, just seems like doing everything git does but with a lot more work and lot more error prone. I imagine there are some advantages to keeping the patches outside of git, but storing them as gists seems pretty odd, when you have git, which has things like history. I guess each gist has its own history? But then are you are basically maintaining each patch its own git repo? Anyways, as you can see, I don't think its clear cut argument.

kleisauke · 2020-05-08T22:59:26Z

emscripten-ports is basically just a way to store the result of applying patches to an upstream release. if you don't have any patches then you don't need to use it you can just use the upstream github release directly.

Is there a reason why some of the emscripten-ports repositories are simply forked without patches? Or is it just for the issue tracker? For example; zlib, Ogg, Vorbis, Asio (after commit chriskohlhoff/asio@607f99b), FreeType, boost (see emscripten-ports/boost#2), bzip2 and libpng could point to upstream instead. That's why I thought Emscripten ports didn't prefer upstream tarballs / Git releases.

I imagine there are some advantages to keeping the patches outside of git, but storing them as gists seems pretty odd, when you have git, which has things like history. I guess each gist has its own history? But then are you are basically maintaining each patch its own git repo? Anyways, as you can see, I don't think its clear cut argument.

Indeed, storing them in gists is odd. It was just to resolve this issue as the libvips WASM port (which contains the same patches) is not released yet.

sbc100 · 2020-05-09T00:25:22Z

If there is a historical reason I don't know it. I would have thought the point of creation of repository there is the point at which we need a patch. If no patch is needed no repository should be needed.

o-micron · 2020-05-09T00:40:24Z

I agree, I think it should be as simple as a single or couple of patching files.
I think consistency is very important here.
Also we can create some sort of template folder that anyone can clone and basically use it in his/her own repository, this way the problem is simply solved by allowing every repository owner to write a custom emscripten build and at the same time the user would see the exact same experience across all ports ... nobody knows how to build a project more than the one who actually created it, so we can delegate that ..

I like how cmake works, whenever I open a new project I search for CMakeLists.txt and simply create a build folder and write almost the same thing cmake .. and make

This type of consistency solves a lot of problems, it's like you can almost guess how to build the project and with 90% chance you will build it with no issues unless the author adds more complicated options and that's what makes cmake sometimes not so good ...

Emscripten could have same model but without the options and complexity, just a single command to build and everyone is happy .. that's not easy to do, but I think that's how it should be done

wurc · 2020-05-22T08:42:09Z

@kripken Here is a script which does everything really ... could replace the whole port repository

Tested on Ubuntu 20.04

# required tools ...
sudo apt install autoconf libtool m4 automake

# install meson from pip3 because glib uses meson ..
pip3 install meson
export PATH=~/.local/bin/:$PATH

# Get the emsdk repo
git clone https://github.com/emscripten-core/emsdk.git

# Enter that directory
cd emsdk
# Fetch the latest version of the emsdk (not needed the first time you clone)
git pull

# Download and install the latest SDK tools.
./emsdk install latest

# Make the "latest" SDK "active" for the current user. (writes ~/.emscripten file)
./emsdk activate latest

# Activate PATH and other environment variables in the current terminal
. ./emsdk_env.sh

# Make sure we have mainly wasm-ld and other tools in path as well
export PATH=$(pwd)/upstream/bin/:$PATH

cd ..

# Get @kleisauke patches
git clone https://gist.github.com/acfa1c09522705efa5eb0541d2d00887.git
cd acfa1c09522705efa5eb0541d2d00887

chmod +x ./build.sh
./build.sh

@o-micron I did according to above steps, but below error pop up
configure: error: in /home/wrc/wasm_study/acfa1c09522705efa5eb0541d2d00887/deps/ffi': configure: error: C compiler cannot create executables See config.log' for more details
...and I checkes the config.log, below is the info
configure:3877: /home/wrc/emsdk/upstream/emscripten/emcc -O3 -L/home/wrc/wasm_study/acfa1c09522705efa5eb0541d2d00887/target/lib -O3 conftest.c >&5
cache:INFO: generating system asset: generated_struct_info.json... (this will be cached in "/home/wrc/emsdk/upstream/emscripten/cache/wasm/generated_struct_info.json" for subsequent builds)
requested a shared WebAssembly.Memory but the returned buffer is not a SharedArrayBuffer, indicating that while the browser has SharedArrayBuffer it does not have WebAssembly threads support - you may need to set a flag
/tmp/tmpt2kyses0.js:171
throw ex;
^

Error: bad memory
at Object. (/tmp/tmpt2kyses0.js:1533:13)
at Module._compile (internal/modules/cjs/loader.js:936:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:947:10)
at Module.load (internal/modules/cjs/loader.js:790:32)
at Function.Module._load (internal/modules/cjs/loader.js:703:12)
at Function.Module.runMain (internal/modules/cjs/loader.js:999:10)
at internal/main/run_main_module.js:17:11
FAIL: Running the generated program failed!

kleisauke · 2020-05-22T09:02:12Z

@wurc Could you try this?:

# The struct_info file must be built without modifications to EMMAKEN_CFLAGS
EMMAKEN_CFLAGS= embuilder.py build struct_info

(I just added this to the gist)

wurc · 2020-05-22T09:22:56Z

@kleisauke, it works :-)
Thank you ~

wurc · 2020-05-22T11:09:34Z

@kleisauke one more question,
when we "#include<glib.h>"
and use emcc to build, below error will pop up
"/home/wrc/wasm_study/glib_wasm/include/glib-2.0/glib/gtypes.h:32:10: fatal error: 'glibconfig.h' file not found
#include <glibconfig.h>
^~~~~~~~~~~~~~
1 error generated.
"
and I do check, there is really no glibconfig.h

o-micron · 2020-05-22T11:16:40Z

@wurc you should find glibconfig.h in your target/lib/glib-2.0/include .
Just add that folder to the include headers as well

kripken · 2020-05-26T21:28:13Z

I think there's no reason to create a forked directory unnecessarily. Perhaps we created some historically when we thought we would need patches, but ended up not needing them? PRs to simplify that would be welcome, and we can delete those repos.

wurc · 2020-05-27T12:02:04Z

@kripken when I use the generated glib, below error comes, do you know why this issue happened?
wasm-ld: error: /home/wrc/wasm_study/glib_wasm/lib/libgio-2.0.a(gthreadedresolver.c.o): undefined symbol: res_query
wasm-ld: error: /home/wrc/wasm_study/glib_wasm/lib/libgobject-2.0.a(gclosure.c.o): undefined symbol: ffi_call
wasm-ld: error: /home/wrc/wasm_study/glib_wasm/lib/libgobject-2.0.a(gclosure.c.o): undefined symbol: ffi_call

kripken · 2020-05-27T16:54:47Z

Those functions are not being linked in, it seems? To investigate it, find where they are defined, and then you can use llvm-nm to verify through the build process where the symbols are, and that they reach the final link etc.

sbc100 · 2020-05-28T01:16:01Z

res_query is used to looking up hostnames in DNS. You should probably find a what to remove that call from gthreadedresolver.c to completely remove gthreadedresolver.c from the build since emscripten doesn't support networking by defaults.

ffi_call is most likely part of libffi which has not been ported to emscripten and porting it would most likely be a bunch of work. Can you find a way to disable the libffi support or disable the gclosure.c support completely?

sbc100 · 2020-05-28T01:17:18Z

Sorry .. see the above discussion about libffi and had forgotten we had already discussed it and it looks like the a solution already.

wurc · 2020-05-28T06:18:11Z

Yes I just use gsocket related apis. It is impossible to use those apis?
@kleisauke can you help me with this issue?
g_inet_socket_address_new_from_string
g_socket_address_get_family
g_socket_set_blocking
g_socket_bind
...

kleisauke · 2020-05-28T10:08:47Z

@wurc You could try to adapt this patch:
https://gist.github.com/kleisauke/acfa1c09522705efa5eb0541d2d00887#file-glib-emscripten-patch-L183-L203

I created that patch because res_query is not available in WASM and would otherwise cause a build error:

Checking if "res_query()" links: NO 
Checking if "res_query() in -lresolv" links: NO 
Checking if "res_query() in -lbind" links: NO 

gio/meson.build:60:6: ERROR: Problem encountered: Could not find res_query()

Perhaps it's only needed to just disable res_query? It might be possible that one of the other functions listed here are still available:
https://gitlab.gnome.org/GNOME/glib/-/blob/11e9c5a9ce96f7e8ffbdd588f1c8f8df1e51761d/gio/meson.build#L40-155

@kripken That's great, thanks for clarifying! I'll take a look to prefer the upstream repositories if no patches are needed. This may also facilitate port maintenance and speed up delivering improvements from new versions to end users.

wurc · 2020-05-29T03:24:36Z

@kleisauke it means currently it is impossible to use gsocket apis which depends on network libs, right? Actually we use the glib socket apis to implement muti-cast function, and we want to transfer it to wasm module

sbc100 · 2020-05-29T06:21:22Z

The socket I/O APIs in emscripten are based on websockets, which means you need to proxy all your socket data back through a websocket to your server and then out from there. I doubt very much that things like mutli-cast will work, but perhaps its possible. Sadly, sockets and web are not currently a good match and you will have to go through these extra hoops.

I would advise looking onto how the websocket bridge works and running some experiments to see if you use case can be made to work on the web.

wurc · 2020-06-02T09:51:39Z

Hi I use below method to use muti-cast, but when calling start_muticast_client in C++, it can't works
while the same code can be executed in node env
EM_JS(void, start_muticast_client, (), {
var dgram = require('dgram');
var socket = dgram.createSocket({ type: 'udp4', reuseAddr: true });
var multicastAddress = '224.0.0.251';
var multicastPort = 5353;
socket.on('listening', function(){
var address = socket.address();
console.log('UDP Client listening on ' + address.address + ":" + address.port);
socket.addMembership(multicastAddress);
socket.setMulticastLoopback(false);
});
socket.on('error', (err) => {
console.log('socket error:\n${err.stack}');
socket.close();
});
socket.on("message", function ( message, remote ) {
console.log("Message received from ", remote.address, " : ", message.toString());
});
socket.bind(multicastPort);
});

kleisauke · 2020-09-01T13:40:27Z

A first release for the above mentioned WebAssembly bindings for libvips has now been published.
https://libvips.github.io/libvips/2020/09/01/libvips-for-webassembly.html

I'll try to integrate the necessary patches of this project upstream (annotated with Upstream-Status: Pending in the commit message). The "In progress" column at https://github.com/kleisauke/wasm-vips/projects/1 shows the PRs that have already been submitted/accepted.

justin-hackin · 2020-11-30T02:28:41Z

I'm currently working to port some code from https://gitlab.com/inkscape/lib2geom to wasm and glib is the last dependency I need to tackle. Thanks a lot for working to sort this out folks.

I tried carefully and repeatedly to get a glib build using the instructions on Ubuntu 20.04 but failed. Here's the most relevant part of the log from the build log:

 Compiler stderr:
 emcc: error: STANDALONE_WASM does not support pthreads yet

Checking for size of "wchar_t" : -1

meson.build:1232:2: ERROR: Problem encountered: Compiler provides no native 16-bit integer type

My meson version was ahead of a tested version mentioned so I also tried downgrading from 0.56.0 to 0.54.1 and got the same results.

full console log
full meson log

Note: I renamed the hash folder to glib-container.

EDIT: I found this issue which seems related to this one I'm having.

justin-hackin · 2020-11-30T02:55:37Z

@kleisauke do you think the glib and ffi patches are the only ones we need to fix the glib build as instructed above?

kleisauke · 2020-11-30T11:12:00Z

@justin-hackin I've just updated the gist with a tested Dockerfile and necessary patches for glib and libffi. Let me know if that works for you.

justin-hackin · 2020-11-30T14:03:38Z

@kleisauke that's kind of you to share, thanks a lot, it works! You made my day :)

hoodmane · 2021-06-23T14:44:55Z

I did some work on libffi because it is a blocker for ctypes support in Pyodide. I added support for long double, structs, and closures. It now passes 56/56 libffi.call tests and 84/94 libffi.closures tests.

I have my changes here:
https://github.com/hoodmane/libffi-emscripten

Hopefully I can figure out how to set up a CI test runner for this.

kleisauke · 2021-06-25T14:50:57Z

@hoodmane Great, thanks for doing this! I made some changes to my fork:
hoodmane/libffi-emscripten@master...kleisauke:master

Let me know if you want a PR for that. With those changes, the libffi test suite almost passes with 20 failures (and 26 failures when building without WASM_BIGINT). Somehow the libffi.closures tests are detected as unsupported.

Details

Build with -s WASM_BIGINT:

Running ./libffi.bhaible/bhaible.exp ...
FAIL: libffi.bhaible/test-call.c -W -Wall -DDGTEST=68 -Wno-unused-variable -Wno-unused-parameter -Wno-uninitialized -O0 execution test
FAIL: libffi.bhaible/test-call.c -W -Wall -DDGTEST=68 -Wno-unused-variable -Wno-unused-parameter -Wno-uninitialized -O2 execution test
FAIL: libffi.bhaible/test-call.c -W -Wall -DDGTEST=69 -Wno-unused-variable -Wno-unused-parameter -Wno-uninitialized -O0 execution test
FAIL: libffi.bhaible/test-call.c -W -Wall -DDGTEST=69 -Wno-unused-variable -Wno-unused-parameter -Wno-uninitialized -O2 execution test
Running ./libffi.call/call.exp ...
FAIL: libffi.call/float_va.c -W -Wall -O0 output pattern test
FAIL: libffi.call/float_va.c -W -Wall -O2 output pattern test
FAIL: libffi.call/negint.c -W -Wall -O0 execution test
FAIL: libffi.call/negint.c -W -Wall -O2 execution test
FAIL: libffi.call/nested_struct12.c -W -Wall -O0 execution test
FAIL: libffi.call/nested_struct12.c -W -Wall -O2 execution test
FAIL: libffi.call/promotion.c -W -Wall -O0 execution test
FAIL: libffi.call/promotion.c -W -Wall -O2 execution test
WARNING: program timed out.
FAIL: libffi.call/va_2.c -W -Wall -O0 execution test
WARNING: program timed out.
FAIL: libffi.call/va_2.c -W -Wall -O2 execution test
WARNING: program timed out.
FAIL: libffi.call/va_struct1.c -W -Wall -O0 execution test
WARNING: program timed out.
FAIL: libffi.call/va_struct1.c -W -Wall -O2 execution test
WARNING: program timed out.
FAIL: libffi.call/va_struct2.c -W -Wall -O0 execution test
WARNING: program timed out.
FAIL: libffi.call/va_struct2.c -W -Wall -O2 execution test
WARNING: program timed out.
FAIL: libffi.call/va_struct3.c -W -Wall -O0 execution test
WARNING: program timed out.
FAIL: libffi.call/va_struct3.c -W -Wall -O2 execution test
Running ./libffi.closures/closure.exp ...
Running ./libffi.complex/complex.exp ...
Running ./libffi.go/go.exp ...

		=== libffi Summary ===

# of expected passes		524
# of unexpected failures	20
# of unsupported tests		205

Build without -s WASM_BIGINT:

Running ./libffi.bhaible/bhaible.exp ...
FAIL: libffi.bhaible/test-call.c -W -Wall -DDGTEST=68 -Wno-unused-variable -Wno-unused-parameter -Wno-uninitialized -O0 execution test
FAIL: libffi.bhaible/test-call.c -W -Wall -DDGTEST=68 -Wno-unused-variable -Wno-unused-parameter -Wno-uninitialized -O2 execution test
FAIL: libffi.bhaible/test-call.c -W -Wall -DDGTEST=69 -Wno-unused-variable -Wno-unused-parameter -Wno-uninitialized -O0 execution test
FAIL: libffi.bhaible/test-call.c -W -Wall -DDGTEST=69 -Wno-unused-variable -Wno-unused-parameter -Wno-uninitialized -O2 execution test
Running ./libffi.call/call.exp ...
FAIL: libffi.call/float.c -W -Wall -O0 execution test
FAIL: libffi.call/float.c -W -Wall -O2 execution test
FAIL: libffi.call/float3.c -W -Wall -O0 execution test
FAIL: libffi.call/float3.c -W -Wall -O2 execution test
FAIL: libffi.call/float_va.c -W -Wall -O0 execution test
FAIL: libffi.call/float_va.c -W -Wall -O2 execution test
FAIL: libffi.call/negint.c -W -Wall -O0 execution test
FAIL: libffi.call/negint.c -W -Wall -O2 execution test
FAIL: libffi.call/nested_struct12.c -W -Wall -O0 execution test
FAIL: libffi.call/nested_struct12.c -W -Wall -O2 execution test
FAIL: libffi.call/promotion.c -W -Wall -O0 execution test
FAIL: libffi.call/promotion.c -W -Wall -O2 execution test
FAIL: libffi.call/return_ldl.c -W -Wall -O0 execution test
FAIL: libffi.call/return_ldl.c -W -Wall -O2 execution test
FAIL: libffi.call/va_2.c -W -Wall -O0 execution test
FAIL: libffi.call/va_2.c -W -Wall -O2 execution test
FAIL: libffi.call/va_struct1.c -W -Wall -O0 execution test
FAIL: libffi.call/va_struct1.c -W -Wall -O2 execution test
FAIL: libffi.call/va_struct2.c -W -Wall -O0 execution test
FAIL: libffi.call/va_struct2.c -W -Wall -O2 execution test
FAIL: libffi.call/va_struct3.c -W -Wall -O0 execution test
FAIL: libffi.call/va_struct3.c -W -Wall -O2 execution test
Running ./libffi.closures/closure.exp ...
Running ./libffi.complex/complex.exp ...
Running ./libffi.go/go.exp ...

		=== libffi Summary ===

# of expected passes		516
# of unexpected failures	26
# of unsupported tests		205

hoodmane · 2021-06-25T15:25:50Z

Let me know if you want a PR for that.

Please! Thanks for working with me on this.

Somehow the libffi.closures tests are detected as unsupported

Yeah it's strange. The only relevant conditional seems to be:

if { [libffi_feature_test "#if FFI_CLOSURES"] } {

and we certainly defined FFI_CLOSURES in ffi_target.h. I'm still working on the selenium tests which will include those.

kleisauke · 2022-11-08T09:27:25Z

I'll try to integrate the necessary patches of this project upstream (annotated with Upstream-Status: Pending in the commit message). The "In progress" column at https://github.com/users/kleisauke/projects/1 shows the PRs that have already been submitted/accepted.

FWIW, after commit kleisauke/wasm-vips@10dd7e0 patches are kept in forks to facilitate upstreaming, for example:
GNOME/glib@2.74.1...kleisauke:wasm-vips

Unfortunally, the GLib patches might not be appropriate to upstream, as they currently have a hard requirement regarding function pointer conversions - see:
https://gitlab.gnome.org/GNOME/glib/-/blob/main/docs/toolchain-requirements.md#function-pointer-conversions
https://stackoverflow.com/a/14044244

So the work on that is stalled, it needs to be discussed upstream first. I'm not sure if GLib would drop that hard requirement, from an ABI/API perspective.

hoodmane · 2022-11-08T11:48:15Z

the ability to convert a function pointer to a void*
and back again losslessly

This emscripten can do though. It sounds to me like they are trying to rule out fat function pointers? If they need more flexible casts that change the number or type of arguments one can always call them with a trampoline.

kleisauke · 2022-11-08T14:54:04Z

TIL, about fat function pointers. I think you're right, that seems to only relate to the -Wpedantic warnings. Though, GLib also deliberately ignores the -Wcast-function-type and -Wbad-function-cast warnings - see:
https://gitlab.gnome.org/GNOME/glib/-/blob/2.74.1/meson.build#L510-512
https://gitlab.gnome.org/GNOME/glib/-/blob/2.74.1/meson.build#L491-493

I fixed most of these issues with commit kleisauke/glib@013ba3b (e.g. grep for void (*)(void)), but it sometimes also needs fixes downstream (e.g. https://github.com/libvips/libvips/blob/v8.13.3/libvips/conversion/extract.c#L287-L291).

FWIW, I'm aware that GLib currently also fails when running with -fsanitize=cfi-icall and that this also causes issues for downstream users who depend on GLib (harfbuzz/harfbuzz@60c6b77), so this is not only related to Wasm and its ecosystem.

These are the original set of changes very kindly organized and released in a patch by https://github.com/kleisauke. The details of this patch are in this thread: emscripten-core/emscripten#11066.

bferguson3 · 2023-01-18T19:09:19Z

Thanks to @kleisauke et al for the progress in this thread. I have been trying to build using the docker gist, with modifications, on Mac M1.

I had to add -pthread and -fPIC to the meson CFLAGS variable to get this far. I can only assume that all libs can be linked fine with these flags, emcc seems to complain without them.

Still stuck linking with emcc though. Everything from the gist compiles without error, but some functions, like file system stuff (g_unix_mount_is_readonly etc) are unavailable.

Is there a specific flag that might not be adding those functions to the compile list? They exist in the source, so I don't see offhand why they wouldn't be included, but others in the libraries are found just fine.

Make line:

emcc \
		-sMAIN_MODULE=1 \
		-fPIC \
		-I../../../glib-emscripten/target/include/ \
		-I../../../glib-emscripten/target/include/glib-2.0/ \
		-I../../../glib-emscripten/target/lib/glib-2.0/include/ \
		-v \
		./build/main.c \
		-o build/index.html \
		-L../../../glib-emscripten/target/lib/ -lffi -lgobject-2.0 -lgio-2.0 -lglib-2.0 -lz \
		-pthread

If anyone has made any further progress or has successfully linked glib let me know.

VitoVan · 2023-04-16T06:37:05Z

Based on @kleisauke 's work, I built Pango and Cairo with GLib.

In case anyone has interests, here is the gist:

https://gist.github.com/VitoVan/92ba4f2b68fec31cda803119686295e5

kripken added good first bug help wanted labels May 5, 2020

kripken mentioned this issue May 8, 2020

Generate port settings in ports #11110

Closed

sbc100 mentioned this issue Jul 17, 2020

Update harfbuzz port #11668

Closed

hoodmane mentioned this issue Jun 19, 2021

ctypes support pyodide/pyodide#728

Closed

kleisauke mentioned this issue Jun 24, 2021

Terser "cannot read property print of undefined" #14525

Closed

Ghabry mentioned this issue Jul 18, 2021

Emscripten: Fix CMake issues FluidSynth/fluidsynth#942

Merged

Siirko mentioned this issue Jun 16, 2024

Emscripten build openslide/openslide-bin#262

Open

Is there a GLIB emscripten port available ? #11066

Is there a GLIB emscripten port available ? #11066

Comments

o-micron commented May 2, 2020

sbc100 commented May 2, 2020

kleisauke commented May 2, 2020 • edited Loading

kripken commented May 2, 2020

o-micron commented May 5, 2020

kleisauke commented May 5, 2020

o-micron commented May 5, 2020 • edited Loading

kripken commented May 5, 2020

o-micron commented May 5, 2020 • edited Loading

o-micron commented May 6, 2020 • edited Loading

kripken commented May 7, 2020

sbc100 commented May 7, 2020

kleisauke commented May 8, 2020 • edited Loading

sbc100 commented May 8, 2020

sbc100 commented May 8, 2020

kripken commented May 8, 2020

sbc100 commented May 8, 2020

sbc100 commented May 8, 2020

sbc100 commented May 8, 2020

kleisauke commented May 8, 2020

sbc100 commented May 8, 2020

kleisauke commented May 8, 2020

sbc100 commented May 9, 2020 • edited Loading

o-micron commented May 9, 2020 • edited Loading

wurc commented May 22, 2020 • edited Loading

kleisauke commented May 22, 2020

wurc commented May 22, 2020

wurc commented May 22, 2020

o-micron commented May 22, 2020

kripken commented May 26, 2020

wurc commented May 27, 2020

kripken commented May 27, 2020

sbc100 commented May 28, 2020

sbc100 commented May 28, 2020

wurc commented May 28, 2020

kleisauke commented May 28, 2020

wurc commented May 29, 2020 • edited Loading

sbc100 commented May 29, 2020

wurc commented Jun 2, 2020

kleisauke commented Sep 1, 2020

justin-hackin commented Nov 30, 2020 • edited Loading

justin-hackin commented Nov 30, 2020

kleisauke commented Nov 30, 2020

justin-hackin commented Nov 30, 2020

hoodmane commented Jun 23, 2021 • edited Loading

kleisauke commented Jun 25, 2021

hoodmane commented Jun 25, 2021 • edited Loading

kleisauke commented Nov 8, 2022

hoodmane commented Nov 8, 2022

kleisauke commented Nov 8, 2022

bferguson3 commented Jan 18, 2023

VitoVan commented Apr 16, 2023

kleisauke commented May 2, 2020 •

edited

Loading

o-micron commented May 5, 2020 •

edited

Loading

o-micron commented May 5, 2020 •

edited

Loading

o-micron commented May 6, 2020 •

edited

Loading

kleisauke commented May 8, 2020 •

edited

Loading

sbc100 commented May 9, 2020 •

edited

Loading

o-micron commented May 9, 2020 •

edited

Loading

wurc commented May 22, 2020 •

edited

Loading

wurc commented May 29, 2020 •

edited

Loading

justin-hackin commented Nov 30, 2020 •

edited

Loading

hoodmane commented Jun 23, 2021 •

edited

Loading

hoodmane commented Jun 25, 2021 •

edited

Loading