Migrated code generation logic to it's own crate

[ianmarmour]

Description of the change

Splits Code Generation logic out into it's own crate (from CLI).

Things I attempted to do pro-actively here,

• Remove unnecessary dependencies from both the javy-codegen crate and the cli crate.
• Added documentation and ensured all public interfaces are documented in javy-codegen.
• Re-exported commonly used structs for ease of use.

Things we might want to change/add,

• Crate specific tests ? (Maybe)
• ???

Why am I making this change?

To enable runtime generation of WASM modules without shelling out to a CLI.

Checklist

• I've updated the relevant CHANGELOG files if necessary. Changes to javy-cli and javy-plugin do not require updating CHANGELOG files.
• I've updated the relevant crate versions if necessary. Versioning policy for library crates
• I've updated documentation including crate documentation if necessary.

Related PR's

877, 879

[jeffcharles]

Can we add the crate feature we talked about before so we can reduce visibility for the methods for working with the v2 and default plugins to the CLI crate?

Otherwise, I think it's a structurally sound change besides removing some methods from the public API.

[ianmarmour]

Can we add the crate feature we talked about before so we can reduce visibility for the methods for working with the v2 and default plugins to the CLI crate?

Otherwise, I think it's a structurally sound change besides removing some methods from the public API.

Totally forgot, of course!

[jeffcharles]

I noticed a few more things while looking at this today.

While the CLI integration tests provide some level of test coverage, I feel like those tests are not testing at least a couple default behaviours of the generator. So might be good to write some tests to ensure those behaviours are what we want and aren't changed unintentionally.

I imagine at some point in the future, it might make sense to migrate a bunch of the integration tests to use this crate instead of using the CLI which should help with more direct test coverage but it doesn't make sense to do that in this PR.

[jeffcharles]

Can we also add an automated integration test for this example code in this crate?

[ianmarmour]

For the integration tests (and I suppose some of the unit tests too) how would you prefer that we deal with the plugin.wasm dependency? Would you prefer that I just commit a .wasm like we did for V2 in the other crate or do we want to use a similar build.rs setup like we do in the CLI crate to point to the built plugin?

[jeffcharles]

Could we run javy emit-plugin -o <some_known_path> before running the tests in the Makefile and in the CI workflow so the codgen tests have the plugin available? I know it breaks the typical Rust workflow of just being able to run cargo test -p javy-codegen but that's already the case for all the other crates. I generally run make tests when I need to run the test suite and that's what I advise other people to do as well.

[jeffcharles]

Almost there! Just need the integration test to verify a byte vector containing a valid Wasm module is returned and removing the setter for the plugin.

[jeffcharles]

I was hoping you would use the default plugin as emitted by javy emit-plugin instead of the test plugin. Re-reading https://github.com/bytecodealliance/javy/pull/890/files#r1977733306 I'm realizing I was not explicit about using the default plugin (just emit-plugin). Specifically if we want to run the generated Wasm, it's easier with the default plugin because it doesn't have the extra non-WASIp1 import. It's not a big deal though.

[jeffcharles]

I guess playing off the comment above, can we validate the returned vector to ensure it returned valid Wasm bytecode when there are no errors?

[ianmarmour]

Are you interested in verifying all the exports like in the UninitializedPlugin struct? (In which case we should move that over to this crate and make it public). Or will just a simple validation sufficient like below?

E.X. Simple verification.

    // Generate valid Wasm module.
    let wasm = generator.generate(&js)?;

    // Verify generated bytes are valid Wasm.
    walrus::Module::from_buffer(wasm.as_slice())?;

[jeffcharles]

Just a simple verification is fine! Just a sanity check to ensure it's not returning an empty vector or some other byte buffer.

[jeffcharles]

I'll give this another look over tomorrow morning and most likely merge it. There's a bit of follow-up I think I want to do before publishing the crate but nothing too major.

[jeffcharles]

Thank you for all your work and patience that went into this change!