Skip to content

simdutf: update to 8.0.0 #685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Lysxia merged 2 commits into from
Jan 17, 2026
Merged

simdutf: update to 8.0.0 #685

Lysxia merged 2 commits into from
Jan 17, 2026
+73,306 −34,440

Conversation

@TerrorJack
Copy link
Contributor

@TerrorJack TerrorJack commented Jan 15, 2026

  • simdutf: update to 8.0.0 and use upstream c api
  • ci: add simdutf-is-genuine ci job to check if vendored simdutf is genuine

@TerrorJack TerrorJack force-pushed the wip/simdutf_8_0_0 branch 2 times, most recently from d67eccf to 88b644d Compare January 15, 2026 03:00
@Bodigrim
Copy link
Contributor

Bodigrim commented Jan 16, 2026

Waiting for nomeata/inspection-testing#97...

There seems to be a genuine build failure at https://github.com/haskell/text/actions/runs/21018119841/job/60427258761?pr=685#step:3:705:

   cbits/hs_simdutf.c:1:10: error:
       fatal error: simdutf_c.h: No such file or directory
          1 | #include "simdutf_c.h"
            |          ^~~~~~~~~~~~~
    |
  1 | #include "simdutf_c.h"
    |          ^

@TerrorJack
Copy link
Contributor Author

TerrorJack commented Jan 16, 2026

i'm confused...the build works locally and in haskell-ci jobs, and the haskell-ci jobs have simdutf flag enabled by default right? i don't understand why there's a header not found error.

it can easily be worked around by just pasting the prototype and avoiding including simdutf_c.h, but maybe it's something more cursed going on here.

@Bodigrim
Copy link
Contributor

Bodigrim commented Jan 16, 2026

Emulated CI jobs are cursed indeed, I suppose one needs to adjust the incantation to pass GHC with all necessary files there.

@Lysxia
Copy link
Contributor

Lysxia commented Jan 17, 2026

It seems that the emulated jobs don't use simdutf. It builds using ghc --make, ignoring the cabal file. There is an argument cbits/*.c so it also tries to build the new hs_simdutf.c wrapper.

A simple fix may be to put hs_simdutf.c into a subdirectory.

@TerrorJack
Copy link
Contributor Author

TerrorJack commented Jan 17, 2026

nice catch! i've moved hs_simdutf.c to simdutf to avoid it being built by non-simdutf mode.

@Lysxia
Copy link
Contributor

Lysxia commented Jan 17, 2026

I'm looking into the inspection-testing failure.

@Bodigrim
Copy link
Contributor

Bodigrim commented Jan 17, 2026

@TerrorJack rebase again please.

Bodigrim
Bodigrim reviewed Jan 17, 2026
View reviewed changes
src/Data/Text/Internal/Validate/Simd.hs Outdated
import GHC.Exts (Ptr,ByteArray#)

foreign import ccall unsafe "_hs_text_is_valid_utf8" c_is_valid_utf8_ptr_unsafe
foreign import ccall unsafe "simdutf_validate_utf8" c_is_valid_utf8_ptr_unsafe
Copy link
Contributor

@Bodigrim Bodigrim Jan 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These entities are visible to users. Could we avoid changing their types (by introducing a wrapper, perhaps)?

Copy link
Contributor Author

@TerrorJack TerrorJack Jan 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the module names have Internal for a good purpose! but i don't have a strong opinion either, so i've avoided changing these in the patch.

@TerrorJack
simdutf: update to 8.0.0
3622e83 
This patch updates vendored simdutf to 8.0.0:
https://github.com/simdutf/simdutf/releases/tag/v8.0.0, and makes use
of simdutf's own C API.
@TerrorJack
ci: add simdutf-is-genuine job
8a67d66 
This commit adds a simdutf-is-genuine CI job to check if vendored
simdutf is identical to the files in the singleheader.zip upstream
release archive.
@Bodigrim Bodigrim requested a review from Lysxia January 17, 2026 17:29
@Lysxia Lysxia merged commit f6a35e5 into haskell:master Jan 17, 2026
27 checks passed
@TerrorJack TerrorJack deleted the wip/simdutf_8_0_0 branch January 18, 2026 05:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Bodigrim Bodigrim Bodigrim approved these changes

@Lysxia Lysxia Lysxia approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@TerrorJack @Bodigrim @Lysxia