Is there a chance of getting threads.h?

[a-p-jo]

Forgive my ignorance if this is an obvious question, but is there any way to supply C11 threads.h in w64devkit? I know that pthreads.h is a better choice realistically and that itself is provided here, but I am able to use C11 threads on all other platforms, for instance gcc on linux or msvc on windows. It would be nice to have full c11 support if its not too much trouble, regardless of how awful WG14's work with the standard threading library may have been.

[Peter0x44]

I agree it would be useful, but mingw-w64 currently doesn't offer this. If you contribute it, we will get it.

[Fierelier]

It might be best to look into headers that implement threads.h with pthreads, like this one: https://github.com/jtsiomb/c11threads

[skeeto]

Yup, like @Peter0x44 said this piece falls into Mingw-w64's part of the toolchain. Though building that implementation on winpthreads would create a dependency loop. Besides that, anything built on pthreads would need -pthread as a kind of abstration leak. It needs a different foundation like Win32 threads, though per @Fierelier's link that's a lot of redundancy, especially if it's going to support Windows XP, which lacks good threading primitives. libstdc++ threads are a layer up and get to build on GCC gthread.

The main problem with C11 threads is that they're less popular and worse than pthreads. Virtually any place you can use C11 threads you'd be better off with pthreads.

[a-p-jo]

Virtually any place you can use C11 threads you'd be better off with pthreads.

Absolutely, I don't argue to the contrary. It would just be nice to have, perhaps the same reason it exists on msvc (which is particularly surprising) and gcc.

How much nicer would it be? Not enough to justify mucking around with mingw-w64 directly for sure; from what little taste I've had of the GNU build systems and source code I wouldn't wish them on anyone.

On the other hand, and perhaps this has been discussed elsewhere, but is there a core reason this project chose the GNU toolchain and not Clang? Smaller size, perhaps? In my experience, clang is the best-behaved compiler on windows, although the whole LLVM toolchain is very, very large. I also doubt it could support older targets like Windows XP. Nevermind, I just read the Known Issues section of the llvm-mingw repo.

On that note, I'm very grateful that you accepted my suggestion for XP support almost 5 years ago, and still maintain that feature! That was issue 11, this is issue 388. That was me in high school, this is me nearing the end of university. W64devkit has been fantastic throughout, thank you for the good work :)

[Peter0x44]

Clang and llvm-mingw are quite good. But llvm-mingw did not exist when w64devkit was created (it came a few months after) and likely llvm still had quite a lot of maturing to do. I keep it on my PATH alongside w64devkit always, it's useful.

It doesn't have c11 threads either. So it won't change anything regarding that.

[Peter0x44]

The toolchain uses Windows native TLS support, which doesn't work properly until Windows Vista. This has no effect on code not using thread local variables.

Hmm. I wonder if this means TLS is broken for the i686 build (on XP). Perhaps we should use emulated TLS there?

[a-p-jo]

It doesn't have c11 threads either. So it won't change anything regarding that.

From what I recall, the version of clang/llvm that Visual Studio Build Tools (and presumably regular Visual Studio) install can do it. I guess mingw-w64 didn't like C11 threads :)

[Peter0x44]

yeah, a default build of llvm from llvm.org or the llvm releases page requires visual studio installed to work. llvm for mingw is a separate target/"thing".

[a-p-jo]

A full build of LLVM also needs Visual Studio? That would easily be 10+ GB for just a C compiler toolchain. Grateful as ever to have 0.5GB w64devkit to rely on instead.

[CyanoHao]

Hmm. I wonder if this means TLS is broken for the i686 build (on XP). Perhaps we should use emulated TLS there?

The actual problems are (1) LoadLibrary only allocates TLS slots for calling thread, not all threads; (2) LoadLibrary doesn’t copy initial value for calling thread. (Load-time linking, or implicit linking, is not affected.) MWE:

lib.cc:

#include <iostream>
#include <thread>

thread_local int value = 42;
thread_local int values[1024];

extern "C" void check_tls(void) {
  auto tid = std::this_thread::get_id();
  std::cout << tid << " [read tls] " << value << ' ' << values[1023]
            << std::endl;
}

main.cc:

#include <iostream>
#include <semaphore>
#include <thread>

#include <libloaderapi.h>

std::binary_semaphore s_thread(0), s_loader(0);
void (*check_tls_fn)(void) = nullptr;

void f1() {
  auto tid = std::this_thread::get_id();
  std::cout << tid << " [start]" << std::endl;
  s_thread.release();
  s_loader.acquire();
  check_tls_fn();
  std::cout << tid << " [end]" << std::endl;
}

void f2() {
  auto tid = std::this_thread::get_id();
  std::cout << tid << " [start]" << std::endl;
  check_tls_fn();
  std::cout << tid << " [end]" << std::endl;
}

int main() {
  std::thread t1(f1);
  s_thread.acquire();

  HMODULE h = LoadLibraryW(L"lib.dll");
  check_tls_fn = (void (*)())GetProcAddress(h, "check_tls");

  check_tls_fn();

  s_loader.release();
  t1.join();

  std::thread t2(f2);
  t2.join();

  return 0;
}

expected output (TID 1 - main, 2 - t1, 3 - t2):

2 [start]
1 [read tls] 42 0
2 [read tls] 42 0
2 [end]
3 [start]
3 [read tls] 42 0
3 [end]

on Windows XP:

2 [start]
1 [read tls] 0 0
2 [read tls] SEGV

I also tried llvm-mingw’s test case. The result is worse than segment fault: the program runs with exit code 0, but some destructors are not called.

Other possible “fixes”:

• Document it, and do nothing. This is how Microsoft “fixed” it: Dynamic-Link Library Data.
• Patch DllMainCRTStartup to allocate TLS slots for all threads. YY-Thunks.

[skeeto]

Thanks for the detailed information, @CyanoHao! This is very helpful.

Document it, and do nothing.

I'm leaning towards this option, adding native TLS to the growing list of new features unavailable on XP. (Along with unicode, cmake, ccache, etc.)

[skeeto]

I was thinking more about this, noticing that by accepting some constraints like not supporting XP or timed locks this would not be too difficult. I have a tentative implementation in 4e5de18. You can simply include <threads.h> as intended, no special linkage or flags required.

Actually getting my hands dirty with C11 threads, it's so obvious it never saw a production-grade implementation before it went into the standard. I suspect it wasn't even proofread. Nearly everything is underspecified, unclear, or in some places contradictory. For example, if mtx_lock "blocks until it locks the mutex pointed to by mtx" then what does that mean if it returns an error? It's only supposed to return if the mutex is locked, so should it be unlocked despite the error? This part of the specification has many such open questions that still haven't been addressed after 15 years because nobody's seriously using C11 threads.

[Peter0x44]

I suggest you get an agent to write you some tests and compare it with msvc's /experimental:c11threads
C11 threads is the only C threading API you can use that's portable across both, so that's really what it's important to be compatible with for w64devkit.

[skeeto]

I'm already spending more time on this than it deserves, but it's such a fascinating puzzle. I took a closer look at the Visual Studio threads.h, and now that I've written an implementation I can mostly guess from the declarations how their implementation works internally. Where there are multiple possibilities, I can do a little science. Should I desire, I could now build an ABI-compatible C11 threads with the vcruntime implementation. That's right, it's in vcruntime, not the CRT. As far as I can tell, the Visual Studio C11 threads API is entirely undocumented. The closest you get is the standard itself.

cnd_t is a straight wrapper around SRW CVs. No surprise because it's an exact match. Zero-initialization works and destruction is a no-op.

thrd_t is a handle and a thread ID. The latter is used for comparisons, the former is used to act upon the thread. thrd_current() returns a dummy thrd_t with only a thread ID that can only be used for comparisons. Trying to join or detach such a thrd_t is UB. The standard does not allow this so this is a bug. That behavior isn't surprising: Threads cannot recover their original handle, which may not even exist anymore. At best they can mint a new one, but the lifetime wouldn't fit the threads API. In w64dk threads I use a thread_local to get around this so that thrd_current() results are first class. (Though it's tempting to match Visual Studio and drop that thread_local hack, making w64dk threads available in CRT-free programs as well.)

I wondered if they handled integer overflow converting struct timespec to Win32 time values. Sort of. If the result won't fit in a 32-bit millisecond count it returns -2 from the function. This does not match any of their threads.h enums, and so is not up to spec, making this another bug. The threshold where this overflows is undocumented, so users may find that thrd_sleep sometimes just doesn't work. In w64dk threads, requesting a huge timeout upgrades to an infinite timeout, which is practically what it would be, making it more foolproof, though it could be more precise in edge cases.

mtx_t is (type mask, SRW mutex, SRW cv, thread id, counter). Zero-initialization works (plain lock) and destruction is a no-op. Timed locks use the CV both on lock and unlock in a straightforward way.

Recursive locks use an unsigned 32-bit counter that silently wraps. That is, if you recursively lock enough then the lock will suddenly become unlocked and bad things happen. The C standard doesn't say how recursive locks are supposed to work, and doesn't anticipate this overflow, so maybe this is up to spec? Though programs using recursive locks are broken anyway, making this just another way for them to break.

I'm guessing tss and once_flag work exactly the same way as in w64dk threads, as these map directly onto Win32. I didn't check carefully.