Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MESDK-103 - v2 createMessageBundle() and example mocks #2795

Merged
merged 6 commits into from
May 17, 2024

Conversation

jldec
Copy link
Contributor

@jldec jldec commented May 15, 2024

This is an alternative implementation of #2773 (closed) which used MF2 string format internally.
Resolves opral/inlang-sdk#64

  • createMessageBundle()
  • createMessage() with text (no placeholder patterns)
  • v2/mocks/plural contains one example message bundle with a plural select and input declaration

NOTE: Input variable declarations and variables in patterns are now wrapped inside an Expression object which adds more nesting to the ast structure. (see test for example)

Example

createMessageBundle({
  id: "hello_world_2",
  messages: [
    createMessage({ locale: "en", text: "Hello World!" }),
    createMessage({ locale: "de", text: "Hallo Welt!" }),
  ],
})

Copy link

changeset-bot bot commented May 15, 2024

⚠️ No Changeset found

Latest commit: 0230c11

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@samuelstroschein
Copy link
Member

samuelstroschein commented May 16, 2024

general notes:

  • variableReferencePattern is redundant. you are defining a utility here that needs no roundtrip capabilities and therefore no knowledge how to serialize variables again
  • as expected, we are rolling our own serialized format now. i wonder if we need this
  • heuristic: more than two arguments -> use an object to future proof the function and increase readability

Why not pass messages to createMessageBundle?

  • avoids duplicate message creation logic

instead of

const bundle: unknown = createMessageBundle(
  "hello_name",
  {
    // pass multiple messages using locale as the key here. 
    en: "Hello, {name}!",
  },
  { variableReferencePattern: ["{", "}"] }
)

have this

const bundle: unknown = createMessageBundle({
  id: "happy_human_elephant",
  messages: [
    createMessage({ locale: "en", pattern: "Hello, {name}!" }),
    createMessage({ locale: "de", pattern: "Hallo, {name}!" })
  ]
})

@LorisSigrist
Copy link
Collaborator

LorisSigrist commented May 16, 2024

Why are we suddenly using a serialized pattern in createMessage? I thought we dismissed using ICU MF2 to avoid forcing devs to learn a serialized format.

  • Which Format are we using in createMessage?
  • Does it support selectors & expressions?
  • How is this different / better than ICU2?

@felixhaeberle
Copy link
Contributor

felixhaeberle commented May 16, 2024

@jldec Could you address the above question in a LOOM (or written maybe better)? I agree with Loris&Samuel on the observations.

@jldec
Copy link
Contributor Author

jldec commented May 16, 2024

@samuelstroschein @LorisSigrist

This is not intended to provide an alternative string format for messages.

My intent was to help us work on v2 persistence with existing project data (like the monorepo website), and to start experimenting with utility functions to manipulate v2 MessageBundles without having to operate directly on the AST.

@jldec
Copy link
Contributor Author

jldec commented May 16, 2024

@samuelstroschein

Why not pass messages to createMessageBundle?

🙏 i like that better. I have modified the signatures to use an args object and pass an array of Message objects into createMessageBundle as you suggested.

variableReferencePattern is redundant

I was thinking we might use that for migrating messages from existing i18next projects, but we can revisit that later. For now only {var} is supported.

[args] heuristic: more than two arguments

👍 changed

@jldec
Copy link
Contributor Author

jldec commented May 16, 2024

@felixhaeberle

Could you address the above question in a LOOM

I'll do a loom as soon I can actually demo something useful

The goal is supprt v2 MessageBundles with persistence in a way which is similar to what I demo'ed in https://www.loom.com/share/9f6e47997a784728a492c17e00ee030b (NOTE that video still shows persitence with the legacy internal Message ast)

@samuelstroschein
Copy link
Member

@LorisSigrist Why are we suddenly using a serialized pattern in createMessage?

I was wondering about this, too, and thought you and @jldec dismissed my arguments against a serialized format.

I am asking again if a serialized format is needed now/if the utilities are even needed. Isn't it OK if you manually type some messages for your tests for now @LorisSigrist until we get clarity on what "good" utilities will look like and if we even need them?

@LorisSigrist
Copy link
Collaborator

I was wondering about this, too, and thought you and @jldec dismissed my arguments against a serialized format.

I'm on your side here, I was also surprised to see a serialized format.

Isn't it OK if you manually type some messages for your tests for now

Yes, writing the AST fine for the moment.

In the long term it will become a maintenance burden IMO. It's a bit like writing half your tests in byte code. It's the wrong level of abstraction for a lot of the tests. Many of paraglide's tests are "take this message, run it through the complier & see if it renders correctly". Introducing an abstraction here will eventually be necessary, but can wait until we have more experience.

For now writing the AST is fine.

@jldec
Copy link
Contributor Author

jldec commented May 16, 2024

@samuelstroschein I'd prefer to use createMessageBundle() and createMessage() in tests (like load-test) because it will simplify the changes required as we transition to v2 persistence. It's always better for tests like that to use more realistic data (which is possible but harder to do with hand-rolled AST).

I agree that introducing the full MF2 string format dependency now, would mean commiting to that while our requirements are still in flux, so not doing that now.

I am also nervous about the AST api changing again, and would like to wrap that in an api layer for apps instead of just opening it up directly - but we will have other opportunities to do that before v2 goes GA.

@jldec jldec changed the title MESDK-103 - v2 createMessageBundle() and related functions MESDK-103 - v2 createMessageBundle() and example mocks May 16, 2024
@jldec
Copy link
Contributor Author

jldec commented May 16, 2024

@LorisSigrist @martin-lysk
Would you mind looking at the mock example in v2/mocks/messageBundles'ts (I just added into this PR)
Does the AST structure look correct for input declarations, expressions, and functions (like plural)?

cc: @felixhaeberle (mock copied from #2793)

@martin-lysk martin-lysk reopened this May 16, 2024
@github-actions github-actions bot locked and limited conversation to collaborators May 16, 2024
@martin-lysk
Copy link
Contributor

Wow wrong button during scrolling on mobile - sorry for the ping.

I am already afk - I can have a Quick Look at the ast tomorrow.

Reagarding the discussion:
I think I made my points for an api using parsed strings based on an existing bases (icu2) already.
I understand the upsides of having a textual format in favour of an unreadable json inside of test you @LorisSigrist and you @jldec referring.
But introducing our own parsing and format as input seems like the wrong direction.

If we want an interims string based api - we should use current state of icu2. If not it’s an object based typed api - that was at least the two path we discussed (?) @samuelstroschein ?

@samuelstroschein
Copy link
Member

samuelstroschein commented May 16, 2024

If we want an interims string based api - we should use current state of icu2. If not it’s an object based typed api - that was at least the two path we discussed (?) @samuelstroschein ?

Yes. I wouldn't cook our own serialized format and I am hesitant to introduce unstable icu2 now. Hence, delay all those utility functions until either or? :

  1. we can't live without utilities anymore
  2. icu2 is stable

@jldec
Copy link
Contributor Author

jldec commented May 17, 2024

ok, parsePattern is gone - I'll get by without the feature for now.

type: "expression",
arg: {
type: "variable",
name: "count: plural",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this expression is what you meant, this literally reads as "a variable called count: plural"

I think you wanted to apply the plural function to the variable count.

The way to express this is by adding a function annotation to the expression:

{
  type : "expression",
  arg: { type : "variable", name: "count"},
  annotation: { type: "function", name: "plural", options: [] }
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was my fault 😅 I naively created a mock message (which @jldec copied) without respecting the correct structure.

pattern: [
{
type: "text",
value: "Show {count} message.",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The value: "Show {count} message." doesn't treat "{count}" as a variable, but as plain text.

The way to mix in variables into the AST is by adding expressions to the pattern.

pattern: [
  { type: "text", value: "Show "},
  { type: "expression", arg: { type: "variable", name: "count" }},
  { type: "text", value: " message."}
]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same ⤴️

@felixhaeberle
Copy link
Contributor

I'll take myself out of the review loop here. Nothing to add besides @samuelstroschein & @LorisSigrist already valuable comments.

@felixhaeberle felixhaeberle removed their request for review May 17, 2024 09:42
@samuelstroschein
Copy link
Member

I am also taking myself out of this discussion now. @LorisSigrist @jldec @martin-lysk you'll make the right choice

@samuelstroschein samuelstroschein removed their request for review May 17, 2024 14:36
@jldec
Copy link
Contributor Author

jldec commented May 17, 2024

Thanks for the feedback and suggestions.
To move things along, I will merge this now with a simplified example bundle in sdk/src/v2/mocks/plural

@felixhaeberle, @LorisSigrist, @martin-lysk please feel free to add additional examples using a similar directory naming pattern.

@jldec jldec merged commit 6702c29 into main May 17, 2024
3 checks passed
@jldec jldec deleted the create-message-bundle branch May 17, 2024 16:47
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CreateMessageBundle and related utilities
5 participants