feat: import.meta.glob support ?raw

[ygj6]

Description

#5304 #3222
Add ?raw support for import.meta.globEager()
For example,

const files = import.meta.globEager('../assets/svgIcon/*.svg?raw')
console.log(files)
// Expected results
/*
{
  '../assets/svgIcon/github.svg': 
  <svg t="1634195145119" class="icon" viewBox="0 0 1024 1024" version="1.1" ...',
  '../assets/svgIcon/logo.svg': 
    '<svg t="1634195051463" class="icon" viewBox="0 0 1024 1024" version="1.1" ...'
}
*/

Additional context

• Use cleanUrl() to remove ?raw so it will not affect glob.sync(pattern).
• Whether need to add ?url support can be discussed

---

What is the purpose of this pull request?

• Bug fix
• New Feature
• Documentation update
• Other

Before submitting the PR, please make sure you do the following

• Read the Contributing Guidelines.
• Read the Pull Request Guidelines and follow the Commit Convention.
• Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
• Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
• Ideally, include relevant tests that fail without this PR but pass with it.

[ygj6]

I have implemented a version compatible with the standard. If it feels appropriate, I can continue to implement support such as url and json.

[patak-dev]

Thanks! Lets wait til next week meeting so we check with Evan that this is the way to go

[patak-dev]

We talked about it in the last team meeting, and we think it is good to align with the standard as this PR is now implemented. Thanks for the refactor @ygj6

For reference, these are notes from Evan showcasing the alignment. We could later add options like inline, exclude and others

import './foo.js' assert { type: "worker", inline: true }
import('./foo.svg', { assert: { type: "raw" }})
import.meta.glob('./*.svg', { assert: { type: "raw" }, exclude: '...' })

I'm worried about including json5 to do the parsing. But it is already in two of our dependencies (we will probably be using another version though). @bluwy @ygj6 what do you think?

Let's target the 2.8 beta to include this feature.

[bluwy]

I think json5 may be the most reliable choice for now, since the spec doesn't seem to be explicit on its syntax, so it would likely be parsing any kind of JS objects. +1 for import assertions though!

[bluwy]

Suggested change

	assert = JSON5.parse(options.substr(commaIndex + 1))
	assert = JSON5.parse(options.substring(commaIndex + 1))

There was a recent PR that converted substr to substring. I think we can do the same here too.

[ygj6]

Thanks for your replies, I agree with @bluwy 's #5545 (comment) , I can continue to implement the rest later.

We could later add options like inline, exclude and others

@patak-js Where can I find reference docs about it to help me understand it more? thx

[patak-dev]

Yes, let's merge this PR only with raw support in 2.8

@ygj6 the other types and options would resolve to the current suffixes (url, raw, worker, inline, etc). But this API isn't yet defined, maybe we could wait a bit here.

[ygj6]

@patak-js Oh, I see, looking forward to it coming. If there is anything I can help, please call me :)

In addition, I found some minor bugs in the code of lexGlobPattern

1. when lexing the pattern, it may contain quote strings, e.g. '/Tom's files/**', results in '/Tom'
2. simply find index of ) to get the endIndex is fragile, the options code may contain annotation, e.g.

import.meta.globEager('/dir/*.json', {
    // (set type)
    assert: { type: 'raw' }
  })

get the wrong pos next to type)<--

So, I plan to use acorn lexer to further refactor the code of lexGlobPattern and push it, see if you think the new approach is acceptable.

[bluwy]

@ygj6 If we go with acorn, it might be the first time Vite does parsing in dev mode, which have been avoided for performance reasons. But given it only runs for globEager and glob, maybe it's not too bad. Otherwise we can accept this bugs upfront and document them instead to preserve performance. I'm slightly leaning on accepting the bugs though, as they can be easily worked around.

[patak-dev]

Yes, as @bluwy said, we also discussed this with the team. We are accepting a few edge cases in exchange for speed with Vite. There are a lot of places where it will be better to remove regex and replace them with an AST but DX would suffer if we do that. It is a tradeoff that at this point is working well for us. So let's keep the current scheme and see if we can polish that edge cases or document the limitations if we can't.

[ygj6]

Reasonable, more understanding of vite’s pursuit of speed and certain tradeoffs, thanks for your patient explanation!

[boehs]

I think this is safe to merge?

[jd-solanki]

I am using vite 2.8.0-beta.1 but with below code I am unable to get it working 🤔

const pageLayoutsGlob = import.meta.globEager('../../../views/generator/sneat/page-layouts/*.html?raw')
console.log('pageLayouts :>> ', pageLayoutsGlob)

Does anyone know why?

[jd-solanki]

Same if I use import.meta.glob()

[patak-dev]

@jd-0001 you should use

const pageLayoutsGlob = import.meta.globEager('../../../views/generator/sneat/page-layouts/*.html',{ assert: { type: 'raw' }})
console.log('pageLayouts :>> ', pageLayoutsGlob)

[jd-solanki]

Thanks :)

Just to make vite robust I have something to share:

const pageLayoutsGlob = import.meta.globEager('../../../views/generator/sneat/page-layouts/*.html',{ assert: { type: 'raw' }})
console.log('pageLayouts :>> ', pageLayoutsGlob)

won't work because I guess it is taking path from root dir of project.

Hence, I updated it to:

const pageLayoutsGlob = import.meta.globEager('/src/views/generator/sneat/page-layouts/*.html', {
  assert: { type: 'raw' },
})
console.log('pageLayouts :>> ', pageLayoutsGlob)

Also, I am getting linting errors:

I guess function signature isn't updated yet 🤔

I am using vite vite@2.8.0-beta.1

[Shinigami92]

I assume #6433 is not released yet

[ygj6]

@LiuJiangshan Would you like to open a new issue and describe the request more clearly in it?