About use of Scarb

[mkaput]

Hey!

@tarrencev reached to me on private channels about the idea of integrating Scarb's compiler ops in Dojo. That's a discussion worth publicizing, so I'll write down my thoughts here:

Preface

1. Dojo-cli and dojo-plugin as standalone components #40 seems to do a lot of stuff that is already worked out by Scarb's ops::compile.
2. Scarb is meant to be available as a crate. The only blocker from publishing it to crates.io as of the time of writing this, is that we still depend on unpublished Cairo revisions.
   1. One thing though that I am still thinking of is whether it wouldn't be better to extract Scarb compiler logic (i.e. the ops::compile function to a separate crate), and perhaps even make the compiler an extension provided as a separate library
   2. But this is a nuance at this moment, Dojo and others shouldn't feel this change super hard if it would happen
3. The only trick for you, I can see is that currently Scarb doesn't provide an entrypoint to boot the compiler with custom plugins. This may be something I would welcome you to contribute if you wish because definitely I and @maciektr don't have spare time at this moment to tackle this 😞

Scarb and Dojo current state context

1. The general framework of compiler extensibility is the targets mechanism.
   • This is inspired by Cargo's targets ([lib], [[bin]] etc.)
   • But in Scarb that's totally differently done, because Rust's and Cairo1's compilations models fundamentally differ:
     • Rust has C model, with compilation units built for particular crates and separate linking phase, and Cargo has a huge unit graph machinery,
     • while Cairo1 compilation unit describes the entire output artefact:
       • one contract JSON = one compilation unit,
       • one Sierra code /CASM file path = another compilation unit etc.
   • So Scarb main branch (that's going to be released in very next alpha hopefully next week) has support for two target kinds:
     • Lib, the fundamental one, which means building pure Cairo code.
     • External, the one meant to be delegated to extensions, such as starknet-contract
       • But currently there are no extensions, and contract compiler is hardcoded
     • TargetKind definition: https://github.com/software-mansion/scarb/blob/6ba612b08c2e1b7ab5f1998e3d014648164c7eb7/scarb/src/core/manifest/target.rs
   • Then, out of these two kinds, there is a trait named TargetCompiler which should be obvious from name what it does.
     • Routing TargetKind to a particular TargetCompiler implementation: https://github.com/software-mansion/scarb/blob/main/scarb/src/compiler/targets/mod.rs#L49-L61
     • lib compiler: https://github.com/software-mansion/scarb/blob/6ba612b08c2e1b7ab5f1998e3d014648164c7eb7/scarb/src/compiler/targets/lib.rs
     • starknet-contract compiler: https://github.com/software-mansion/scarb/blob/6ba612b08c2e1b7ab5f1998e3d014648164c7eb7/scarb/src/compiler/targets/starknet_contract.rs
2. https://github.com/software-mansion/scarb/blob/6ba612b08c2e1b7ab5f1998e3d014648164c7eb7/scarb/src/compiler/targets/starknet_contract.rs basically has the code that is reimplemented in Dojo-cli and dojo-plugin as standalone components #40
   • I didn't dive into Dojo-cli and dojo-plugin as standalone components #40's code, but as far as I can logically infer, the only "extension" that Dojo-cli and dojo-plugin as standalone components #40 does is adding the DojoPlugin?
   • OK, you also extend corelib but that's a hack anyway that has to be fleshed out on Cairo language level, not ours.
   • I assume you might also want to provide a dojo package, but Scarb already has a solution for this: users should simply put dojo = ... in their Scarb.tomls [dependencies].

Plan for extending Scarb to support Dojo's needs

I see two phases here: one that does the meat done, but is still kinda hacky, and second phase which cleans everything up:

Things that I am open to accept contributions

What is crucially missing for Dojo are two things:

1. There should be a possiblity to register custom targets dynamically:
   • That anyway has to be done when we'll deal with extension targets.
   • Something like type TargetRepository = HashMap<TargetName, Box<dyn Fn(...) -> Box<dyn TargetCompiler>> is missing.
   • This repository object could live within Config, i.e. add target_repository: TargetRepository field to the Config struct.
   • Dojo would then be able to initialize config, inject their target to the repository, and then call ops::compile 🎉
2. Next, it would be shame if Dojo had to copy-paste https://github.com/software-mansion/scarb/blob/6ba612b08c2e1b7ab5f1998e3d014648164c7eb7/scarb/src/compiler/targets/starknet_contract.rs, so extracting a common interface out of this would be fine.
   • No idea how this should look like, I would happily leave it to the contributor to think out and show me a suggestion.

The hacky part: all of this is still within scarb crate and is pubed from scarb::compiler module.

Things that should be done by myself, but can be deferred

Disclaimer: I am really not sure this should look like this because I didn't think into this deeply.

1. The compiler part of the scarb crate probably has to be extracted into separate crates:
   1. scarb-build which will contain ops::compileand everything downstream, like TargetCompiler trait and lib target implementation.
   2. scarb-target-starknet-contract which will contain starknet-specific compiler logic.
2. scarb-target-starknet-contract will depend on scarb-build, and both will depend on slimmed down scarb.
3. The big benefit of this is that scarb will not depend on cairo-lang-* crates, that's funny, but this will help compilation times significantly I think. (This implies also extracting scarb fmt which is is going to land today).

---

I'll be very happy to hear your thoughts on this, and also criticism of my design decisions in Scarb. Many of decisions I made in Scarb haven't been validated yet publicly by anyone.