Tokenizing fails when font property line height value is set from custom property

[lauriii]

• Operating System (or Browser): MacOS
• Node Version: 8.10.0
• postcss-values-parser Version: 3.0.5

How Do We Reproduce?

Example of failing CSS:

font: 1em/var(--line-height);

I created a failing test for this: lauriii@d0bbbbc

I added two test scenarios. It seems like 1em / var(--line-height); works but 1em/var(--line-height); doesn't.

This might be caused because of a bug in an upstream package. I filed an issue there but they suggested that this package shouldn't use the PostCSS core tokenizer. Upstream issue: postcss/postcss#1298.

Expected Behavior

1em should be tokenized into number and var should be tokenized into a function.

Actual Behavior

This is tokenized into a single function with name 1em/var.

[lauriii]

It seems like the maintainer of the upstream package closed the issue and recommended to copy and paste the PostCSS core tokenizer into this library: postcss/postcss#1298.

[shellscape]

It's not a bug in PostCSS. This package extends PostCSS's parser and tokenizer to support value parsing. We'll not be copying and pasting the tokenizer either, that's not a good idea. If you examine the commits, you'll find that we moved away from that intentionally.

https://github.com/shellscape/postcss-values-parser/blob/master/lib/tokenize.js will probably have to be updated to accommodate this edge case. Happy to review a PR that resolves this.