New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fetch API with pthreads has missing symbols #7490
Comments
I've just updated to v1.38.30 and this is still occurring, is there anything I can do to help with this? |
We can definitely use help here. There basically isn't a full-time maintainer of the Fetch API. For this issue, I think rewriting the fetch worker to be just a normal pthread would be a good way to do, @juj proposed that. |
Do you have a test case for the failing behavior? |
I kind of have a test case (I just threw it together): #include <stdio.h>
#include <string.h>
#include <pthread.h>
#include <emscripten/fetch.h>
void* threadFunc(void *userData)
{
for (int i = 0; i < 1000; i++)
{
emscripten_fetch_attr_t attr;
emscripten_fetch_attr_init(&attr);
strcpy(attr.requestMethod, "GET");
attr.attributes = EMSCRIPTEN_FETCH_LOAD_TO_MEMORY | EMSCRIPTEN_FETCH_SYNCHRONOUS;
emscripten_fetch_t *fetch = emscripten_fetch(&attr, "file.dat"); // Blocks here until the operation is complete.
if (fetch->status == 200)
{
printf("Finished downloading %llu bytes from URL %s.\n", fetch->numBytes, fetch->url);
// The data is now available at fetch->data[0] through fetch->data[fetch->numBytes-1];
}
else
{
printf("Downloading %s failed, HTTP failure status code: %d.\n", fetch->url, fetch->status);
}
emscripten_fetch_close(fetch);
}
return NULL;
}
int main(int argc, char *argv[])
{
pthread_t threads[8];
for (int i = 0; i < 8; i++)
{
printf("Starting up thread #%d\n", i + 1);
pthread_create(&threads[i], NULL, threadFunc, NULL);
}
return 0;
} I'm really not sure what the VS plugin does, but I had to modify
It might take awhile, but it will eventually trigger the error, |
@samsinsane Short term solution would be to add the missing functions to |
Btw @kripken @samsinsane I don't mind doing the grunt work of implementing the pthread code to replace |
How did you handle |
That would be great! @juj knows best, but in general, I think the idea is just to remove all the |
Sadly my other diff seems like it will take up more time than i thought so I would not be able to do this this week. |
Hey @samsinsane, I was able to get your test case working by adding '___pthread_mutex_timedlock', '___pthread_mutex_trylock' to |
Yes, I mentioned this in my first post but I'm not sure I was very clear about that.
Yes, the quick fix doesn't actually fix the project at work because there are specific branches internally to those missing functions that end up calling I'm sure you can appreciate that I can't just provide the source code to the work project, and that trying to simplify a non-trivial system doesn't always provide the same results. The work project triggers the error in less than a second without the 10 or so missing functions being added, and with those functions added it can run for several minutes before triggering the error. It's really just a race condition and I can't give an exact sample for that.
Agreed, unfortunately I don't really understand the system enough to know how that would work. I attempted to write my own cut down XHR wrapper before posting this issue but sadly, there's a very clear gap in my understanding. |
Just to set expectations, I probably won't be able to get to this any time soon anymore |
Possibly related to #8242? (Or the fix might be the same) |
@VirtualTim Yeah, making it a real pthread is the fix - or not calling any C functions especially |
This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 7 days. Feel free to re-open at any time if this issue is still relevant. |
I don't believe this is fixed. |
This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 30 days. Feel free to re-open at any time if this issue is still relevant. |
I don't think anything has changed here. |
I've been attempting to port some of our projects to support Emscripten (v1.38.16), and I've started to encounter
Uncaught ReferenceError
s when both the worker thread and another thread are allocating at the same time. If a thread is already allocating when the Fetch worker attempts to allocate, it will fail the second conditional here: https://github.com/kripken/emscripten/blob/4bba062e95fb319ae9ba89797c88a0eef3dea1be/system/lib/libc/musl/src/thread/pthread_mutex_lock.c#L7-L9The Fetch worker will then attempt to call
__pthread_mutex_timedlock()
which will then result in this errorUncaught ReferenceError: ___pthread_mutex_timedlock is not defined
. I found thetools/shared.py
script which generates thefetch-worker.js
file and began adding all of the missing functions (as the reference errors came up) and ended up with:Unfortunately,
_pthread_self()
requires__pthread_ptr
which I believe requires the usage ofpthread_create()
? It seems like the solution is to make the worker thread more like a pthread as suggested in #7024?The text was updated successfully, but these errors were encountered: