[24.11] libblake3: 1.5.5 -> 1.7.0

[silvanshade]

This PR is a backport of libblake3 version 1.7.0 to release 24.11 from #390458

This PR depends on another backport PR for tbb_2022_0: #391413

The backport is needed for NixOS/nix#12676

@Ericson2314

[wolfgangwalther]

Please do the following when manually backporting changes:

• mention in the PR description which other PR you are backporting
• cherry-pick with -x, which will put the commit reference in the new commit message and allow the "check cherry-picks" job to work.

This is also documented: https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#manually-backporting-changes

[wolfgangwalther]

Please rebase after the merge of #391413.

Edit: Although there doesn't seem to be a merge conflict.. so let's try merging this way...

[wolfgangwalther]

Hm, unfortunately realized too late, that this breaks the build for libblake3 on x86_64-darwin (not aarch64-darwin, though). It fails with:

In file included from /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:4:
In file included from /nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:23:
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/detail/_task.h:285:13: error: aligned deallocation function of type 'void (void *, std::align_val_t) noexcept' is only available on macOS 10.13 or newer
    virtual ~task() = default;
            ^
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/detail/_task_handle.h:49:5: note: in defaulted destructor for 'tbb::detail::d1::task' first required here
    task_handle_task(d1::wait_tree_vertex_interface* vertex, d1::task_group_context& ctx, d1::small_object_allocator& alloc)
    ^
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/detail/_task.h:285:13: note: if you supply your own aligned allocation functions, use -faligned-allocation to silence this diagnostic
    virtual ~task() = default;
            ^
In file included from /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:4:
In file included from /nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:27:
In file included from /nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/task_group.h:29:
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/detail/_task_handle.h:57:5: error: aligned deallocation function of type 'void (void *, std::align_val_t) noexcept' is only available on macOS 10.13 or newer
    ~task_handle_task() override {
    ^
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/detail/_task_handle.h:57:5: note: if you supply your own aligned allocation functions, use -faligned-allocation to silence this diagnostic
In file included from /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:4:
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:39:8: error: aligned deallocation function of type 'void (void *, std::align_val_t) noexcept' is only available on macOS 10.13 or newer
struct function_invoker : public task {
       ^
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:39:8: note: in implicit destructor for 'tbb::detail::d1::function_invoker<(lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:29:7), tbb::detail::d1::invoke_root_task>' first required here
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:149:44: note: in instantiation of template class 'tbb::detail::d1::function_invoker<(lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:29:7), tbb::detail::d1::invoke_root_task>' requested here
    function_invoker<F1, invoke_root_task> invoker1(f1, root);
                                           ^
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:194:5: note: in instantiation of function template specialization 'tbb::detail::d1::invoke_recursive_separation<(lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:29:7), (lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:33:7)>' requested here
    invoke_recursive_separation(root_wait_ctx, context, fs..., f1);
    ^
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:207:9: note: in instantiation of function template specialization 'tbb::detail::d1::parallel_invoke_impl<(lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:33:7), (lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:29:7)>' requested here
        parallel_invoke_impl(std::forward<T>(t), std::forward<Fs>(args)...);
        ^
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:216:5: note: in instantiation of member function 'tbb::detail::d1::invoke_helper<std::tuple<(lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:29:7)>, (lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:33:7)>::operator()' requested here
    invoke_helper<std::tuple<>, Fs...>()(std::forward<Fs>(fs)...);
    ^
/tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:28:16: note: in instantiation of function template specialization 'tbb::detail::d1::parallel_invoke<(lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:29:7), (lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:33:7)>' requested here
  oneapi::tbb::parallel_invoke(
               ^
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:39:8: note: if you supply your own aligned allocation functions, use -faligned-allocation to silence this diagnostic
struct function_invoker : public task {
       ^
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:39:8: error: aligned deallocation function of type 'void (void *, std::align_val_t) noexcept' is only available on macOS 10.13 or newer
struct function_invoker : public task {
       ^
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:39:8: note: in implicit destructor for 'tbb::detail::d1::function_invoker<(lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:33:7), tbb::detail::d1::invoke_root_task>' first required here
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:150:44: note: in instantiation of template class 'tbb::detail::d1::function_invoker<(lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:33:7), tbb::detail::d1::invoke_root_task>' requested here
    function_invoker<F2, invoke_root_task> invoker2(f2, root);
                                           ^
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:194:5: note: in instantiation of function template specialization 'tbb::detail::d1::invoke_recursive_separation<(lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:29:7), (lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:33:7)>' requested here
    invoke_recursive_separation(root_wait_ctx, context, fs..., f1);
    ^
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:207:9: note: in instantiation of function template specialization 'tbb::detail::d1::parallel_invoke_impl<(lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:33:7), (lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:29:7)>' requested here
        parallel_invoke_impl(std::forward<T>(t), std::forward<Fs>(args)...);
        ^
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:216:5: note: in instantiation of member function 'tbb::detail::d1::invoke_helper<std::tuple<(lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:29:7)>, (lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:33:7)>::operator()' requested here
    invoke_helper<std::tuple<>, Fs...>()(std::forward<Fs>(fs)...);
    ^
/tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:28:16: note: in instantiation of function template specialization 'tbb::detail::d1::parallel_invoke<(lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:29:7), (lambda at /tmp/nix-build-libblake3-1.7.0.drv-0/source/c/blake3_tbb.cpp:33:7)>' requested here
  oneapi::tbb::parallel_invoke(
               ^
/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/parallel_invoke.h:39:8: note: if you supply your own aligned allocation functions, use -faligned-allocation to silence this diagnostic
struct function_invoker : public task {
       ^
4 errors generated.
make[2]: *** [CMakeFiles/blake3.dir/build.make:170: CMakeFiles/blake3.dir/blake3_tbb.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:115: CMakeFiles/blake3.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

This does not happen on master, though.

[silvanshade]

@wolfgangwalther Thanks for merging these.

I don't have an x86_64-darwin system to test with unfortunately but did some investigation.

The x86_64-darwin system seems to be compiling with a macOS SDK that is too old based on the error:

/nix/store/kllykddp7hb8lcy24nf5mbg54a937qn0-tbb-2022.0.0-dev/include/oneapi/tbb/detail/_task.h:285:13: error: aligned deallocation function of type 'void (void *, std::align_val_t) noexcept' is only available on macOS 10.13 or newer

I think the minimum macOS available for aarch64 is 11.00 so it not being an issue there would make sense.

The fix seems like it would be related to this PR: #346043

There they mention this:

This PR adds a new SDK pattern for Darwin that provides a single SDK package for each version. A default SDK is provided as an extraBuildInput in the Darwin stdenv. To use a newer SDK version, add it as a buildInput. A hook is used to pick which SDK of those available is used (currently always the newest); and it is possible to have a different SDK for build, for host, and for target platforms. Most importantly, it is never possible to have more than one SDK active. Weird errors due to semicolons or other punctuation should be a thing of the past.

I think the solution then would be to make this change:

buildInputs = lib.optionals useTBB [ tbb_2022_0 ]
  ++ lib.optional stdenv.hostPlatform.isDarwin apple-sdk_11

If that fixes it I can submit another PR (although not sure how I should format that since it's not a backport technically) or maybe you can just submit a quick follow up?

[wolfgangwalther]

Thanks for looking into it. Your analysis was 100% spot on, your solution was close. Instead of another SDK version, I had to use the darwinMinVersionHook to make it build. See #392846.