-
Notifications
You must be signed in to change notification settings - Fork 161
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Split icu_datagen
into a crate for the driver and a crate for the provider
#4721
Comments
Actually, data "generation" happens in the provider crate, so maybe the driver crate should be |
The most modular setup would probably be
|
That's my proposal, just with different names. |
We could also move |
Latest proposal:
|
ICU4X-WG discussion:
Macro structure brainstorm: macro_rules! make_exportable {
([$($marker:path,)*], [$($experimental_marker,)*]) => {
#[cfg(feature = "experimental_components")]
icu_provider::make_exportable_provider!([$($marker,)* $($experimental_marker,)*]);
#[cfg(not(feature = "experimental_components"))]
icu_provider::make_exportable_provider!([$($marker,)*]);
}
}
// uses call-site Cargo features
registry!(make_exportable);
macro_rules cb {
($($marker:path),*)) => {
fn all_keys() -> ... {
HashSet::from_iter([
$($marker),* $<marker>::KEY.path()
])
}
}
}
Shane's version:
Conclusion:
LGTM: @robertbastian @sffc |
#4721 The `icu_provider_datagen` crate can be a runtime dependency for baked data, in order to expose dependencies that are required for databake operation (I'm thinking zerotrie and writeable). The former `icu_datagen::baked_exporter` module is this new crate's `exporter` module, behind an `export` feature, analogous to `icu_provider_blob` and `icu_provider_fs`. This is also a better home for the `test-baked-source` test, because in `icu_datagen`, if the baked source wouldn't build, `make-testdata` also wouldn't, which is annoying (@younies).
#4721 `provider/datagen/src/{provider.rs, provider/, transform/}` move to `provider/bikeshed/src`. Stubdata moves to `provider/data/{component}/stubdata`, as it's a quality-of-life thing and not required for testing or reviewing the `DatagenProvider` (also, `../stubdata` is a much nicer value for `ICU4X_DATA_DIR` than `../../../bikeshed/tests/data/baked`). JSON testdata moves to `provider/bikeshed/data/debug`, as it's used only for debugging/reviewing, and shouldn't be in `tests/data`, which is data that actually affects tests.
These two components are fairly independent, and there is now a use case for the datagen driver that doesn't use the CLDR/ICU backed provider (
icu_datagen_dart
). It would be nice to be able to build it without pulling in all the logic and dependencies for parsing CLDR.Idea:
icu_datagen
-DatagenDriver
and optionsicu_provider_source
-DatagenProvider
SourceDataProvider
zip
,wasm
,ureq
etc.)icu4x-datagen
- The CLI, which depends on both cratescargo install icu_datagen
installs a binary calledicu4x-datagen
is confusingbin
feature)icu_datagen_dart
depends onicu_datagen
andicu_provider_blob
, making it a lot more lightweighticu_datagen_dart
approach, i.e. have one universal blob generated from sources (long build time and long gen time, but shared), and then filter that down for use cases (short build time and short gen time).The text was updated successfully, but these errors were encountered: