Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Bun.Glob #6861

Merged
merged 82 commits into from
Nov 22, 2023
Merged

Add Bun.Glob #6861

merged 82 commits into from
Nov 22, 2023

Conversation

zackradisic
Copy link
Member

@zackradisic zackradisic commented Nov 2, 2023

What does this PR do?

Still WIP

This adds a Glob class to bun which implements glob pattern matching on filesystem/strings.

Basic example:

import { Glob } from "bun";

const glob = new Glob("**/*.ts");
const entries = await glob.match({ cwd: "src" });

console.log("Entries", entries);

This adds

  • Documentation or TypeScript types (it's okay to leave the rest blank in this case)
  • Code changes

Still need to do:

  • symlink support
  • more comprehensive tests (see blow)
  • return async iterator for Glob.scan()

How did you verify your code works?

I wrote automated tests:

  • Basic tests for glob matching/walking
  • Copy tests from fast-glob
  • Weird edge case tests (invalid surrogate pairs, filenames with latin1 characters, etc.)
  • Stress tests

If Zig files changed:

  • I checked the lifetime of memory allocated to verify it's (1) freed and (2) only freed when it should be
  • I or my editor ran zig fmt on the changed files
  • I included a test for the new code, or an existing test covers it

If new methods, getters, or setters were added to a publicly exposed class:

  • I added TypeScript types for the new methods, getters, or setters

If dependencies in tests changed:

  • I made sure that specific versions of dependencies are used instead of ranged or tagged versions

If *.classes.ts files were added or changed:

  • I ran make codegen to regenerate the C++ and Zig code

@Jarred-Sumner Jarred-Sumner merged commit 6ba4e95 into oven-sh:main Nov 22, 2023
@Jarred-Sumner
Copy link
Collaborator

🎉

@zth zth mentioned this pull request Nov 24, 2023
ryoppippi pushed a commit to ryoppippi/bun that referenced this pull request Feb 1, 2024
* initial glob impl

* Add `Bun.globMatch`

* Glob boilerplate setup

* Experiment with Rust glob implementation

* Rust impl is slow revert

* Setup glob walking

* Basic glob walker working

* Fix segfault

* Recursive directory traversal

* Fix glob match non-ascii

* Make faster lil bit

* use arena

* ASCII fast path

* Experiment with packed codepoint cursor

Results in ~4% perf boost if the glob pattern needs to create/manipulate cursors (for example when the pattern uses braces)

* Try converting to u32 array

Made it pretty slow

* Lazily create codepoint buffer

* Different walk algorithm

* Fast path optimizations

* Add `dot` option to `Glob`

* .

* Fix some bugs

* Fix bug, clean up lil bit

* Windows fix

* Non absolute paths

* use specific version of fast-glob for benchmarks and tests

* .

* Fix some stuff

* Fix more stuff

* Add `hasPendingActivity()` to glob

* accident

* Symlinks

* fast-glob e2e tests

* remove

* woops

* Fix relative paths

* Fix absolute

* add test for `onlyFiles`

* Fix invalid surrogate pairs problem

* Rename: `match/matchSync` -> `scan/scanSync` and `matchString` -> `match`

* forgot to close cwd fd

* Update types

* Add stress test

* Port `micromatch` / `glob-match` / `globlin` tests

* fix stale reference arena thing

* stupid bug

* Add builtins to classes code generator and add `Glob.scanIter()`

* all iterables

* generate fixtures, remove from git

* fix test

* Fix

* woops on test

* Fix stuff

licenses

license

`has_pending_activity` to usize

cwd threadSafe fix atomic compile errors

`GlobWalker` own `cwd`

Fix windows path and absolute test

stuff

* Fixes

* Fix stuff

* Use Syscall.close

* Use private symbols for underlying scan functions to preevent misuse

* Update types

* Fix build for zig

* Fix tests

* Fix more tests

* Prevent these tests from GC'ing too much

* Make this benchmark work in Node and Bun

* Fix memory leak

* Add leak test

* Fix windows

* comment about arena allocator use for glob walker

* Make leak test run in separate process

* Iterator api for glob

* GlobWalker.Iterator

* fix leak test

* Remove old impl

* filter functions wip start

* stuff

* wip lockfile use glob

* glob working with lockfile

* revert lockfile changes

* Update bun.lockb

* Manually set to cwd to prevent test failing on linux CI

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants