Skip to content

Commit

Permalink
Databake improvements (#2906)
Browse files Browse the repository at this point in the history
  • Loading branch information
robertbastian committed Dec 21, 2022
1 parent 22866d3 commit b74aafd
Show file tree
Hide file tree
Showing 1,310 changed files with 434,760 additions and 418,398 deletions.
1 change: 1 addition & 0 deletions .gitattributes
Expand Up @@ -8,3 +8,4 @@ ffi/diplomat/cpp/include/** linguist-generated=true
ffi/diplomat/cpp/docs/** linguist-generated=true
ffi/diplomat/js/include//** linguist-generated=true
ffi/diplomat/js/docs/** linguist-generated=true
*.rs.data linguist-language=Rust
10 changes: 10 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions components/datetime/src/provider/calendar/skeletons.rs
Expand Up @@ -88,7 +88,7 @@ impl databake::Bake for DateSkeletonPatternsV1<'_> {
}
});
databake::quote! {
[#(#vals),*]
&[#(#vals),*]
}
}
}
Expand All @@ -110,7 +110,8 @@ impl databake::Bake for DateSkeletonPatternsV1Marker {
}
}

type BakedDateSkeletonPatternsV1 = [(&'static [crate::fields::Field], PatternPlurals<'static>)];
type BakedDateSkeletonPatternsV1 =
&'static [(&'static [crate::fields::Field], PatternPlurals<'static>)];

impl zerofrom::ZeroFrom<'static, BakedDateSkeletonPatternsV1> for DateSkeletonPatternsV1<'static> {
fn zero_from(other: &'static BakedDateSkeletonPatternsV1) -> Self {
Expand Down
20 changes: 10 additions & 10 deletions docs/tutorials/data_management.md
Expand Up @@ -133,9 +133,9 @@ Rebuilding the application and rerunning datagen awards us with a 3KB data blob,

So far we've used `--format blob` and `BlobDataProvider`. This is useful if we want to ship code and data separately, but there are other options.

## `mod` and `BakedDataProvider`
## `mod` and baked data

The `mod` format will generate a Rust module that defines a data provider. This format naturally has no deserialization overhead, and allows for compile-time optimizations (data slicing isn't really necessary, as the compiler will do it for us), but cannot be dynamically loaded at runtime.
The `mod` format will generate a Rust module that contains all the required data directly as Rust code. This format naturally has no deserialization overhead, and allows for compile-time optimizations (data slicing isn't really necessary, as the compiler will do it for us), but cannot be dynamically loaded at runtime.

Let's give it a try:

Expand All @@ -154,17 +154,19 @@ $ cargo add zerovec
We can then use the data by directly including the source with the `include!` macro.

```rust,compile_fail
extern crate alloc; // required as BakedDataProvider is written for #[no_std]
extern crate alloc; // required as my-data-mod is written for #[no_std]
use icu::locid::{locale, Locale};
use icu::calendar::DateTime;
use icu::datetime::{TypedDateTimeFormatter, options::length};
const LOCALE: Locale = locale!("ja");
include!("../my-data-mod/mod.rs"); // defines BakedDataProvider
struct UnstableProvider;
include!("../my-data-mod/mod.rs");
impl_data_provider!(UnstableProvider);
fn main() {
let unstable_provider = BakedDataProvider;
let unstable_provider = UnstableProvider;
let options = length::Bag::from_date_time_style(length::Date::Long, length::Time::Medium);
Expand All @@ -180,14 +182,12 @@ fn main() {
}
```

With this provider, we can use the `unstable` constructors. These are only guaranteed to work if the `BakedDataProvider` was generated with the same version of ICU4X that you are building with, but if you build the data as part of your a build pipeline, that shouldn't be a problem.
With this provider, we can use the `unstable` constructors. These are only guaranteed to work if the data was generated with the same version of ICU4X that you are building with, but if you build the data as part of your a build pipeline, that shouldn't be a problem.

You can also make the `BakedDataProvider` implement the `AnyProvider` trait, so that it can be used with `_with_any_provider` constructors. Using these constructors is slightly less performant than the `unstable` ones, but, as the name suggests, stable across (minor) releases.
You can also implement the `AnyProvider` trait, so that it can be used with `_with_any_provider` constructors. Using these constructors is slightly less performant than the `unstable` ones, but, as the name suggests, stable across (minor) releases.

```rust,compile_fail
include!("../my-data-mod/mod.rs");
include!("../my-data-mod/any.rs");
let _any_provider = BakedDataProvider;
impl_any_provider!(MyProvider);
```

## `dir` and `FsDataProvider`
Expand Down
72 changes: 57 additions & 15 deletions ffi/ecma402/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 4 additions & 3 deletions ffi/ecma402/src/lib.rs
Expand Up @@ -64,8 +64,9 @@ impl std::fmt::Display for crate::DataLocale {
}
}

mod provider {
pub(crate) struct GlobalDataProvider;

mod baked {
include!(concat!(env!("OUT_DIR"), "/baked/mod.rs"));
impl_data_provider!(super::GlobalDataProvider);
}

pub(crate) use provider::BakedDataProvider;
2 changes: 1 addition & 1 deletion ffi/ecma402/src/list.rs
Expand Up @@ -20,7 +20,7 @@ impl ecma402_traits::listformat::Format for ListFormat {
L: Locale,
Self: Sized,
{
Self::try_new_with_provider(l, opts, &crate::BakedDataProvider)
Self::try_new_with_provider(l, opts, &crate::GlobalDataProvider)
}

fn format<I, L, W>(&self, list: L, writer: &mut W) -> fmt::Result
Expand Down
2 changes: 1 addition & 1 deletion ffi/ecma402/src/pluralrules.rs
Expand Up @@ -240,7 +240,7 @@ impl ecma402_traits::pluralrules::PluralRules for PluralRules {
L: ecma402_traits::Locale,
Self: Sized,
{
Self::try_new_with_provider(l, opts, &crate::BakedDataProvider)
Self::try_new_with_provider(l, opts, &crate::GlobalDataProvider)
}

fn select<W>(&self, number: f64, writer: &mut W) -> std::fmt::Result
Expand Down
1 change: 1 addition & 0 deletions provider/datagen/Cargo.toml
Expand Up @@ -84,6 +84,7 @@ zip = "0.6"
cached-path = "0.5"
reqwest = { version = "0.11", features = ["blocking"] }
lazy_static = "1"
rust-format = { version = "0.3.4", features = ["token_stream"] }

# Dependencies for "bin" feature
clap = { version = "2.33", optional = true }
Expand Down

0 comments on commit b74aafd

Please sign in to comment.