Introduce SIMD optimized scanning for identifiers.

[chandlerc]

Currently, for long identifiers, a huge (>30%) fraction of time is spent
finding the end of the identifier. We can speed this up with a fun
application of SIMD and in-register lookup tables.

With this, the BM_ValidIdentifiers/12/64 benchmark goes from around 4 million
tokens/second to around 6 mt/s, so roughly 1.5x improvement. However,
there was a decent amount of noise in the measurement and I didn't study
it too closely as I was very happy with the overall result. The profile
shifted from >30% of the time in this loop to <10% of the time, so the
scan itself is 3x or more faster with this.

One concern with optimizing the lexer right now is that we don't have
full Unicode support from the design. This PR takes some steps to at least
try and avoid this pitfall -- the new routine works to classify UTF-8
code units, and has a fallback in that case that can grow the needed
logic.

[zygoloid]

This is neat!

For UTF-8 handling, we're going to need to deal with things like Unicode whitespace characters, so we can't get away with classifying everything non-ASCII as continuing an identifier, and then (presumably) doing some later checking -- we'll need to actually decode the characters during tokenization.

As a result, I wonder if maybe the best thing to do would be to fall out of the fast vectorized identifier loop whenever we see any non-ASCII character, and have the slower scalar loop also deal with UTF-8 characters (and ideally jump back into vectorized code if an identifier contains a mixture of ASCII and non-ASCII characters, but that's a rare enough case that not optimizing for it is probably fine). I think by doing that we could simplify the vectorized loop slightly, and maybe remove one of the _mm_movemask calls.

[jonmeow]

I'm happy to let zygoloid handle review here since it seems he understands the instructions better, but I did have one comment thing I'd noticed.

[jonmeow]

What do you think of switching to line comments for style? We'd been going that way in C++ because Carbon wouldn't allow /* */ comments like this.

[chandlerc]

These are already long and hard to read, and I think line comments would make that even worse.

What I'd prefer is something more like a designated initializer. I think we'll have good options for this in Carbon, but in C++ it didn't seem worth trying to find one, and instead to just use a comment.

[jonmeow]

Maybe instead put this in that kind of form, then? It looks like parameters are named b0 through b15:

_mm_setr_epi8(char __b0, char __b1, char __b2, char __b3, char __b4, char __b5,
              char __b6, char __b7, char __b8, char __b9, char __b10,
              char __b11, char __b12, char __b13, char __b14, char __b15) {

So wouldn't style be /*__b0=*/, etc?

Note, if you don't want to do this, it's probably worth thinking about what it means for Carbon if a function in a library that we don't own has parameter names you don't find useful.

[chandlerc]

I'll try this and see if I can get the parameter name approach to work well. Even though they're not the most readable, this would have actually caught a bug I wrote the first time (I used _mm_set_epi8 instead).

That said, not sure it makes sense to generalize too much to Carbon here -- I feel like there are a bunch of overlapping challenges from C++ here and we might better off just working with Carbon code that ends up with bit tables and similar constructs in its API design, and see what the best tools are or what changes would give better results there.

[chandlerc]

PTAL!

[chandlerc]

These are already long and hard to read, and I think line comments would make that even worse.

What I'd prefer is something more like a designated initializer. I think we'll have good options for this in Carbon, but in C++ it didn't seem worth trying to find one, and instead to just use a comment.

[zygoloid]

Feels like we might be able to do ... something ... better now we're only using 4 bits of mask, but I don't see it. (We could fold the two lookup tables into one, which would save a register but add a shift and mask into the loop, which I suspect isn't a win.)

[zygoloid]

Not merging: the initial comment is probably not quite the commit message we want. (At least the reference to #3121 should be removed.) But this looks good to merge to me.