Make the library threadsafe #109

RReverser · 2021-03-11T04:06:07Z

As mentioned in #81, currently the library is not thread-safe, that is, it can't even be used on multiple separate images in parallel due to the shared global mutable variables.

There are a couple of ways to fix this. I've mostly done refactoring in a separate branch to pass everything around via params, but that kind of a change is rather huge and harder to review for correctness.

So I took a different approach I mentioned in #81 (comment), and instead went through all the statics and changed them depending on usage:

Into local function variables where they are initialized every time anyway - those clearly don't have to be static.
Into const static if it was a static that's been always initialised to the same constant value - there was just initinfo that fell into this category.
Into _Atomic if it's a global config - in particular sub_object_limit and extract_pixstack params. I believe there is no point in configuring those per-thread, so just left them as globals, but _Atomic ensures safe complete initialization in case of a race.
Finally, made all the other mutable statics _Thread_local, which ensures that none of the threads will override each others' data while storing temporary information about the image being processed.

Then, I used clang-tidy to:

Find and auto-mark all places where pointers can be const - this allows consumer to more easily reason about which data can be shared across images and/or threads, and which will be mutated internally by SEP and so needs to have separate copies per run.
Detect all non-thread-safe functions (clang-tidy can do this starting from LLVM 13). Such function was only rand, so I added a _Thread_local seed variable randseed, and switched those callsites to rand_r(&randseed), making those thread-safe too.

Note that due to trimming of unnecessary spacing by clang-tidy and editors the diffs can look a bit more bloated, so I suggest to review the diff using "ignore whitespace changes" mode here: https://github.com/kbarbary/sep/compare/master...RReverser:simple-threadsafe?expand=1&w=1

Overall I think this is the least invasive change that should be relatively easy to review for correctness (and, of course, I checked that all the tests are passing), while at the same time ensures thread-safety throughout the codebase.

Fixes #81.

RReverser · 2021-03-11T04:07:45Z

ctest/test_image.c

  result = (float*)malloc(n*sizeof(float));
  for (i=0; i<n; i++)
    result[i] = a + (b-a) * rand() / ((double)RAND_MAX);

  return result;
 }

-double gaussrand()


Ah yes, I also removed this because while looking for statics in the codebase that need to be updated for thread-safety, I've noticed that those functions are unused altogether, so just removed them.

RReverser · 2021-03-11T04:08:47Z

src/sepcore.h

@@ -33,7 +33,7 @@
 #define UNKNOWN_NOISE_TYPE  10

 #define	BIG 1e+30  /* a huge number (< biggest value a float can store) */
-#define	PI  3.1415926535898
+#define	PI  M_PI


This is something that clang-tidy also noticed during its runs and thought it's worth the minor cleanup too.

RReverser · 2021-03-12T16:50:40Z

I can split out const pointer and such changes separately into a different PR btw, it just was easier to fix them together, and it's semi-related to thread safety as well.

RReverser · 2021-03-12T16:55:31Z

Also I still think it's a good idea to make some refactorings to isolate data further, but I think it's worth implementing the minimal / least invasive change first, that already guarantees thread-safety at least for independent inputs and outputs isolated to their own threads, and then it's easier to do further improvements on top of it.

RReverser · 2021-03-28T02:09:01Z

@kbarbary Hey! I suspect you're busy, but just in case it got miseed - does this approach make sense in general?

kbarbary · 2021-03-28T17:58:56Z

Sorry for the delay, this is really impressive work! It looks good to me. It might be worth adding a note to the unreleased version in CHANGES.md explaining the implications. I think you'd be more qualified to write it than me. Want to add that and I'll merge it?

RReverser · 2021-03-28T18:40:24Z

@kbarbary Happy to do so, but I'll probably need clarification on versioning (#111) before I can add an entry to CHANGES.md.

RReverser · 2021-03-29T18:13:48Z

@kbarbary I've pushed commit with description of changes in CHANGES.md, but left version line as vTODO - please substitute with whatever is deemed appropriate depending on resolution of #111.

RReverser · 2021-03-29T18:16:19Z

Ah I see you used unreleased line upstream, will add there during rebase (UPD: done).

As mentioned in kbarbary#81, currently the library is not thread-safe, that is, it can't even be used on multiple separate images in parallel due to the shared global mutable variables. There are a couple of ways to fix this. I've mostly done refactoring in a separate branch to pass everything around via params, but that kind of a change is rather huge and harder to review for correctness. So I took a different approach I mentioned in kbarbary#81 (comment), and instead went through all the statics and changed them depending on usage: 1. Into local function variables where they are initialized every time anyway - those clearly don't have to be `static`. 2. Into `const static` if it was a static that's been always initialised to the same constant value - there was just `initinfo` that fell into this category. 3. Into `_Atomic` if it's a global config - in particular `sub_object_limit` and `extract_pixstack` params. I believe there is no point in configuring those per-thread, so just left them as globals, but `_Atomic` ensures safe complete initialization in case of a race. 4. Finally, made all the other mutable statics `_Thread_local`, which ensures that none of the threads will override each others' data while storing temporary information about the image being processed. Then, I used `clang-tidy` to: 1. Find and auto-mark all places where pointers can be `const` - this allows consumer to more easily reason about which data can be shared across images and/or threads, and which will be mutated internally by SEP and so needs to have separate copies per run. 2. Detect all non-thread-safe functions (clang-tidy can do this starting from LLVM 13). Such function was only `rand`, so I added a `_Thread_local` seed variable `randseed`, and switched those callsites to `rand_r(&randseed)`, making those thread-safe too. Note that due to trimming of unnecessary spacing by clang-tidy and editors the diffs can look a bit more bloated, so I suggest to review the diff using "ignore whitespace changes" mode here: https://github.com/kbarbary/sep/compare/master...RReverser:simple-threadsafe?expand=1&w=1 Overall I think this is the least invasive change that should be relatively easy to review for correctness (and, of course, I checked that all the tests are passing), while at the same time ensures thread-safety throughout the codebase. Fixes kbarbary#81.

RReverser · 2021-03-30T14:03:07Z

There is a lot more in terms of pointers that could be constified, but they all need manual review, and I'll probably leave them for a separate PR.

RReverser · 2021-03-31T15:45:18Z

@kbarbary Thanks!

kbarbary · 2021-03-31T15:45:45Z

Thank you! Again, great work.

kbarbary · 2021-03-31T15:46:14Z

Should I do a version release?

RReverser · 2021-03-31T15:59:26Z

@kbarbary I think tagging 1.2.0 and updating sep_version_string is reasonable at this point, yeah. Any further improvements can always be released under patch version bumps.

RReverser · 2021-04-01T21:38:45Z

@kbarbary I think tagging 1.2.0 and updating sep_version_string is reasonable at this point, yeah. Any further improvements can always be released under patch version bumps.

Assuming you haven't done that yet, I have a follow-up that could be also merged as part of 1.2.0 release tag :) #114

RReverser force-pushed the simple-threadsafe branch from 6c14c9b to 6ca3c15 Compare March 11, 2021 04:06

RReverser commented Mar 11, 2021

View reviewed changes

RReverser mentioned this pull request Mar 11, 2021

Threading problem of c library. #81

Closed

RReverser mentioned this pull request Mar 28, 2021

sep_version_string mismatch #111

Closed

RReverser force-pushed the simple-threadsafe branch 2 times, most recently from 6ea79e9 to 3c957b4 Compare March 29, 2021 18:20

RReverser added 2 commits March 29, 2021 19:20

Add myself to authors

65fae20

RReverser force-pushed the simple-threadsafe branch 2 times, most recently from 5807e09 to 5545ddc Compare March 30, 2021 13:46

Mark few more pointers as const

6b9f5f8

RReverser force-pushed the simple-threadsafe branch from 5545ddc to 6b9f5f8 Compare March 30, 2021 13:46

kbarbary merged commit aa40e6d into kbarbary:master Mar 31, 2021

RReverser deleted the simple-threadsafe branch March 31, 2021 15:45

RReverser mentioned this pull request Apr 1, 2021

Use const pointers in more places + MSVC fixes #114

Merged

RReverser mentioned this pull request Apr 1, 2021

Reduce number of global thread-locals #115

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make the library threadsafe #109

Make the library threadsafe #109

RReverser commented Mar 11, 2021

RReverser Mar 11, 2021

RReverser Mar 11, 2021

RReverser commented Mar 12, 2021 •

edited

Loading

RReverser commented Mar 12, 2021

RReverser commented Mar 28, 2021

kbarbary commented Mar 28, 2021

RReverser commented Mar 28, 2021

RReverser commented Mar 29, 2021

RReverser commented Mar 29, 2021 •

edited

Loading

RReverser commented Mar 30, 2021

RReverser commented Mar 31, 2021

kbarbary commented Mar 31, 2021

kbarbary commented Mar 31, 2021

RReverser commented Mar 31, 2021

RReverser commented Apr 1, 2021 •

edited

Loading

Make the library threadsafe #109

Make the library threadsafe #109

Conversation

RReverser commented Mar 11, 2021

RReverser Mar 11, 2021

Choose a reason for hiding this comment

RReverser Mar 11, 2021

Choose a reason for hiding this comment

RReverser commented Mar 12, 2021 • edited Loading

RReverser commented Mar 12, 2021

RReverser commented Mar 28, 2021

kbarbary commented Mar 28, 2021

RReverser commented Mar 28, 2021

RReverser commented Mar 29, 2021

RReverser commented Mar 29, 2021 • edited Loading

RReverser commented Mar 30, 2021

RReverser commented Mar 31, 2021

kbarbary commented Mar 31, 2021

kbarbary commented Mar 31, 2021

RReverser commented Mar 31, 2021

RReverser commented Apr 1, 2021 • edited Loading

RReverser commented Mar 12, 2021 •

edited

Loading

RReverser commented Mar 29, 2021 •

edited

Loading

RReverser commented Apr 1, 2021 •

edited

Loading