Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[model libraries] simplified setup for countDownloads #485

Merged
merged 2 commits into from
Feb 16, 2024

Conversation

julien-c
Copy link
Member

@julien-c julien-c commented Feb 15, 2024

Second part of #482

note to reviewers: I only expose the filter option because:

  • it's simpler: every library except for diffusers is using it.
  • i had typings issues/conflicts with the Chai library when defining a type that contains a property named should. cc @coyotte508. Rather than fightining the tooling, i decided to work around it

// match multiple possible filenames
| { terms?: { path: string[] } }
// match a wildcard
| { wildcard?: { path: string } };
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note to reviewers: I only expose the filter option because:

  • it's simpler: every library except for diffusers is using it.
  • i had typings issues/conflicts with the Chai library when defining a type that contains a property named should. cc @coyotte508. Rather than fightining the tooling, i decided to work around it

Copy link
Member

@coyotte508 coyotte508 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

Copy link
Contributor

@Wauplin Wauplin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

// match multiple possible filenames
| { terms?: { path: string[] } }
// match a wildcard
| { wildcard?: { path: string } };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit) FWIW, wildcards are not optimized in ES. Since we only use them to filter by suffix (*.onnx, *.sentis, etc.), we could use a reverse-token-filter + a prefix filter.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm maybe cc @SBrandeis or @co42 for information

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would need to reindex everything to add a reverse token filter. If the wildcard is fast enough let's use that

Copy link
Member

@osanseviero osanseviero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice!

Copy link
Member

@pcuenca pcuenca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool 🔥

@julien-c julien-c merged commit 2b191fd into main Feb 16, 2024
2 checks passed
@julien-c julien-c deleted the simpler-libraries-part2 branch February 16, 2024 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants