Revisit tokenizer microoptimizations #22

kmcallister opened this Issue Jul 31, 2014 · 1 comment


None yet

1 participant


For example, there's an old branch to use SSE 4.2 string instructions in tokenization, which realized a speedup of about 10% and could probably be pushed much further.

Contributor has the necessary intrinsics. That might be a nicer route than inline assembly, even though it's still not portable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment