Add C, Python, and Node bindings for the Rust scan ABI - #115
Conversation
Expose fuckmark-hidden-scan-v1 through a C header and host cdylib, an optional Python ctypes loader, and a Node package that wraps the committed WASM engine with the JS fallback. Co-authored-by: Byte271 <byte271@users.noreply.github.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Team Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6050f182f4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| lib = load_library() | ||
| if lib is None: | ||
| raise RuntimeError("native fuckmark-scan library is not available") | ||
| text_bytes = text.encode("utf-8") |
There was a problem hiding this comment.
When a valid Python str contains a lone UTF-16 surrogate, this strict UTF-8 encoding raises UnicodeEncodeError, so scan_text() cannot report the advertised surrogate category; clean_text() has the same failure at its corresponding encoding step. Detect these strings and delegate them to the Python reference implementation, as the Node binding already does, or otherwise provide a surrogate-preserving path.
Useful? React with 👍 / 👎.
| if (fallback && hasLoneSurrogate(source)) { | ||
| return fallback.scanText(source, categories, language); |
There was a problem hiding this comment.
For input containing a lone surrogate, this direct return exposes the fallback scanner's different result schema: findings omit index, while the result omits truncated, highest_severity, and source_length, all of which are returned for ordinary WASM-scanned input. Consumers therefore receive input-dependent object shapes and can fail specifically on the surrogate cases the fallback is intended to support; normalize the fallback result to the WASM wrapper's schema before returning it.
Useful? React with 👍 / 👎.
| cc -I crates/fuckmark-scan/include \ | ||
| crates/fuckmark-scan/examples/scan_cli.c \ | ||
| -L crates/fuckmark-scan/target/release -lfuckmark_scan \ | ||
| -Wl,-rpath,\$ORIGIN/../../crates/fuckmark-scan/target/release \ |
There was a problem hiding this comment.
With the documented output path /tmp/fm-scan, $ORIGIN is /tmp, so this rpath resolves to /crates/fuckmark-scan/target/release rather than the repository's build directory. Running the very next command consequently fails to load libfuckmark_scan.so; use an absolute build path or place the executable where this $ORIGIN-relative path is valid.
Useful? React with 👍 / 👎.
Route lone UTF-16 surrogates through the Python scanner so the native binding can report the surrogate category. Normalize the WASM JS fallback result to the same schema, and link the C example next to the built library with \$ORIGIN. Co-authored-by: Byte271 <byte271@users.noreply.github.com>
2 participants
Language bindings for the shared
fuckmark-hidden-scan-v1Rust ABI introduced in #114.Stack: merge #112 → #113 → #114 → this. Base is
cursor/wasm-scan-engine-cdf8.What landed
crates/fuckmark-scan/include/fuckmark_scan.hcdylibbuild:crates/fuckmark-scan/build-native.shcrates/fuckmark-scan/examples/scan_cli.cfuckmark.native_scan(usesFUCKMARK_SCAN_LIBortarget/release/libfuckmark_scan.*)bindings/node(@fuckmark/scan) wrapping the committed WASM module + JS fallbackbindings/README.mdWASM hidden-scanjob also builds the host library and runs binding testsCategory ABI:
*= all categories, empty string /[]= empty selection, otherwise comma-separated names.Codex P2 follow-up (
b76f013)scan_text/clean_textdetect lone UTF-16 surrogates and delegate to the Python reference scanner (UTF-8 ABI cannot carry them)index/truncated/highest_severity/source_lengthschema as a WASM scan-rpath,$ORIGINand written next tolibfuckmark_scan.soPackage version stays 0.4.1. Frozen confirmation artifacts are unchanged.