Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wasi][icu] Add ICU sharding for wasi #82614

Open
ilonatommy opened this issue Feb 24, 2023 · 4 comments
Open

[wasi][icu] Add ICU sharding for wasi #82614

ilonatommy opened this issue Feb 24, 2023 · 4 comments
Assignees
Labels
arch-wasm WebAssembly architecture area-Infrastructure-mono os-wasi Related to WASI variant of arch-wasm
Milestone

Comments

@ilonatommy
Copy link
Member

ilonatommy commented Feb 24, 2023

Now we are loading always full icu for wasi. Add sharding like in #80421.

Connected issues: #82895
In the sharding approach we cannot achieve automatic shard selection - we would need to bundle all icu to wasi first and then make the decision which shard to choose, that would in fact increases the upload size -> no point.
Instead, we could let the user choose the shard explicitly, e.g. using custom icu loading mechanism (it should bundle one, chosen icu file from runtime pack). For that we would use the same mechanism as for TZ/full icu - embedding into dotnet.wasm file.

@ilonatommy ilonatommy added the os-wasi Related to WASI variant of arch-wasm label Feb 24, 2023
@ilonatommy ilonatommy self-assigned this Feb 24, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Feb 24, 2023
@radical radical added the arch-wasm WebAssembly architecture label Feb 24, 2023
@ghost
Copy link

ghost commented Feb 24, 2023

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

Now we are loading always full icu for wasi. Add sharding like in #80421.

Author: ilonatommy
Assignees: ilonatommy, mkhamoyan
Labels:

arch-wasm, untriaged, os-wasi

Milestone: -

@radical radical added this to the 8.0.0 milestone Feb 24, 2023
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Feb 24, 2023
@ghost
Copy link

ghost commented Feb 25, 2023

Tagging subscribers to this area: @dotnet/area-system-globalization
See info in area-owners.md if you want to be subscribed.

Issue Details

Now we are loading always full icu for wasi. Add sharding like in #80421.

Author: ilonatommy
Assignees: ilonatommy, mkhamoyan
Labels:

arch-wasm, area-System.Globalization, os-wasi

Milestone: 8.0.0

@ghost
Copy link

ghost commented Feb 25, 2023

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

Now we are loading always full icu for wasi. Add sharding like in #80421.

Author: ilonatommy
Assignees: ilonatommy, mkhamoyan
Labels:

arch-wasm, area-Infrastructure-mono, os-wasi

Milestone: 8.0.0

@pavelsavara
Copy link
Member

For single file WASI:
after #82250 merge we could start thinking on how to bundle ICU shards during native relink.
The "InvariantGlobalization" is just a type of empty shard.
We could also disable loading of any shard, when env variable it set.

For default wasi - no relink + FS access
We could have all shards on the host file system and load it from disk based on env variable.

Copied from here

@ilonatommy ilonatommy modified the milestones: 8.0.0, 9.0.0 Aug 7, 2023
@lewing lewing modified the milestones: 9.0.0, 10.0.0 Jul 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch-wasm WebAssembly architecture area-Infrastructure-mono os-wasi Related to WASI variant of arch-wasm
Projects
None yet
Development

No branches or pull requests

7 participants