Revisit tokenizer microoptimizations #22

Open
kmcallister opened this Issue Jul 31, 2014 · 1 comment

Projects

None yet

1 participant

@kmcallister
Contributor

For example, there's an old branch to use SSE 4.2 string instructions in tokenization, which realized a speedup of about 10% and could probably be pushed much further.

@kmcallister
Contributor

https://crates.io/crates/llvmint has the necessary intrinsics. That might be a nicer route than inline assembly, even though it's still not portable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment