New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tracking] Upstream Mirage/Solo5 #36

Closed
mato opened this Issue Jun 15, 2016 · 8 comments

Comments

Projects
None yet
3 participants
@mato
Member

mato commented Jun 15, 2016

This issue tracks the individual tasks required to upstream (or otherwise release) Mirage/Solo5, referring to dependent issues or PRs as appropriate:

Must have:

Nice to have:

  • TLS stack working. Requires a Solo5 "port" of mirage-entropy and adapting the nocrypto build system to use it, plus sorting out the resulting OPAM dependencies (djwillia/mirage#3, Solo5/opam-solo5#4).
  • Standalone test programs for Solo5 APIs (#37).

Out of scope / future release:

  • Support for >1 network or block device.

mato referenced this issue in mato/mirage Jun 22, 2016

Rename Solo5 'qemu' target to 'virtio'.
This is consistent with the Solo5 package names and also reflects that
'virtio' will run on hypervisors other than QEMU.

Whitespace has been cleaned up to match the conventions used in Mirage.

Part of djwillia/solo5#36.

mato referenced this issue in mirage/mirage-console-solo5 Jul 13, 2016

Refactor as independent mirage-console-solo5
- Based on consensus at hackathon, refactor as independent OPAM package
  and library with the Solo5 backend only. Depend on mirage-console for
  type signatures.
- Part of djwillia/solo5#36.

mato referenced this issue in Solo5/opam-solo5 Jul 13, 2016

mirage-console -> mirage-console-solo5
Part of djwillia/solo5#36.

mato referenced this issue in djwillia/mirage Jul 13, 2016

Use independent mirage-console-solo5
Part of djwillia/solo5#36.
@mato

This comment has been minimized.

Show comment
Hide comment
@mato

mato Jul 13, 2016

Member

Preparation for and upstreaming of Mirage/Solo5

  1. Decide on repository naming and ownership. We have a bunch of repos that are Solo5 specific. Should we move them into a solo5/ org or into mirage/. Decision: move into mirage/ since they are parallels to the existing xen and unix variant repositories that are already there.

    Overall picture of repository/opam/org relationships:

    OPAM package            Git repository (current)          Git repository (upstream)
    
    mirage                  djwillia/mirage#solo5             mirage/mirage (upstream via pr)
    mirage-solo5            djwillia/mirage-platform#solo5    
                            >> djwillia/mirage-solo5          mirage/mirage-solo5
    mirage-block-solo5      djwillia/mirage-block-solo5       mirage/mirage-block-solo5
    mirage-net-solo5        djwillia/mirage-net-solo5         mirage/mirage-net-solo5
    mirage-bootvar-solo5    djwillia/mirage-bootvar-solo5     mirage/mirage-bootvar-solo5
    mirage-console          djwillia/mirage-console#solo5
    >> mirage-console-solo5 >> djwillia/mirage-console-solo5  mirage/mirage-console-solo5
    mirage-entropy-solo5    mato/mirage-entropy#solo5         TBD, need to discuss with @hannesm
    nocrypto                mato/ocaml-nocrypto#solo5         TBD, need to discuss with @hannesm
    ocaml-freestanding      mato/ocaml-freestanding           mirage/ocaml-freestanding
    solo5-kernel-ukvm       djwillia/solo5                    solo5/solo5
    solo5-kernel-virtio     djwillia/solo5                    solo5/solo5
    gmp-freestanding        - (opam only)                     -
    zarith-freestanding     - (opam only)                     -
    -                       djwillia/opam-solo5               leave as is
    
    (>> means needs to be renamed)
    
  2. Clean up package and repository relationships based on above:

    • Extricate mirage-solo5 from mirage-platform fork.
    • Extricate mirage-console-solo5 from mirage-console fork.
    • Email Github support to convert forks to normal repositories.
    • Once Github "unforks" the requested repositories, transfer them to Mirage org on Github.
    • Decide whether solo5 should be in a solo5 org or the mirage org
  3. Publish the mirage-solo5* packages to opam.ocaml.org.

    • Clean up opam files for these packages (correct maintainer, repo URLs, ...)
    • Cut releases of each package that will be published.
    • @mato to Figure out with @hannesm what to do about mirage-entropy-solo5 and nocrypto to avoid any code duplication here.
  4. Mirage frontend changes.

    • Submit PR for merging Mirage frontend changes from djwillia/mirage#solo5
      into mirage/mirage.
Member

mato commented Jul 13, 2016

Preparation for and upstreaming of Mirage/Solo5

  1. Decide on repository naming and ownership. We have a bunch of repos that are Solo5 specific. Should we move them into a solo5/ org or into mirage/. Decision: move into mirage/ since they are parallels to the existing xen and unix variant repositories that are already there.

    Overall picture of repository/opam/org relationships:

    OPAM package            Git repository (current)          Git repository (upstream)
    
    mirage                  djwillia/mirage#solo5             mirage/mirage (upstream via pr)
    mirage-solo5            djwillia/mirage-platform#solo5    
                            >> djwillia/mirage-solo5          mirage/mirage-solo5
    mirage-block-solo5      djwillia/mirage-block-solo5       mirage/mirage-block-solo5
    mirage-net-solo5        djwillia/mirage-net-solo5         mirage/mirage-net-solo5
    mirage-bootvar-solo5    djwillia/mirage-bootvar-solo5     mirage/mirage-bootvar-solo5
    mirage-console          djwillia/mirage-console#solo5
    >> mirage-console-solo5 >> djwillia/mirage-console-solo5  mirage/mirage-console-solo5
    mirage-entropy-solo5    mato/mirage-entropy#solo5         TBD, need to discuss with @hannesm
    nocrypto                mato/ocaml-nocrypto#solo5         TBD, need to discuss with @hannesm
    ocaml-freestanding      mato/ocaml-freestanding           mirage/ocaml-freestanding
    solo5-kernel-ukvm       djwillia/solo5                    solo5/solo5
    solo5-kernel-virtio     djwillia/solo5                    solo5/solo5
    gmp-freestanding        - (opam only)                     -
    zarith-freestanding     - (opam only)                     -
    -                       djwillia/opam-solo5               leave as is
    
    (>> means needs to be renamed)
    
  2. Clean up package and repository relationships based on above:

    • Extricate mirage-solo5 from mirage-platform fork.
    • Extricate mirage-console-solo5 from mirage-console fork.
    • Email Github support to convert forks to normal repositories.
    • Once Github "unforks" the requested repositories, transfer them to Mirage org on Github.
    • Decide whether solo5 should be in a solo5 org or the mirage org
  3. Publish the mirage-solo5* packages to opam.ocaml.org.

    • Clean up opam files for these packages (correct maintainer, repo URLs, ...)
    • Cut releases of each package that will be published.
    • @mato to Figure out with @hannesm what to do about mirage-entropy-solo5 and nocrypto to avoid any code duplication here.
  4. Mirage frontend changes.

    • Submit PR for merging Mirage frontend changes from djwillia/mirage#solo5
      into mirage/mirage.

mato referenced this issue in mirage/mirage-solo5 Jul 13, 2016

mato referenced this issue in Solo5/opam-solo5 Jul 14, 2016

@avsm avsm referenced this issue Jul 14, 2016

Merged

Use topkg #41

@avsm

This comment has been minimized.

Show comment
Hide comment
@avsm

avsm Jul 14, 2016

Contributor

See mirage/mirage-console#41 for the split of the mirage-console packages into separate OPAM packages. That will stay in one repo for now but can easily be split up later once things settle -- the important thing is that there is now one OPAM package per ocamlfind package installed by the console variants.

Contributor

avsm commented Jul 14, 2016

See mirage/mirage-console#41 for the split of the mirage-console packages into separate OPAM packages. That will stay in one repo for now but can easily be split up later once things settle -- the important thing is that there is now one OPAM package per ocamlfind package installed by the console variants.

@hannesm

This comment has been minimized.

Show comment
Hide comment
@hannesm

hannesm Jul 18, 2016

Contributor

Entropy support: I started off with @mato branch for entropy and massaged it a bit to avoid code duplication. See the branch over here. The remaining piece of work is to, instead of having depopts and a single package, revert this change, and use no depopts, but two packages (mirage-entropy-xen and mirage-entropy-solo5). The C stub is a symbolic link, thus no code duplication there. I moved most of the OCaml code (apart from connect, which requires the OS module from mirage-xen/mirage-solo5) into the common Entropy module. I've no clue how to embed the call to postconf in a system using topkg (but likely the build documentation will help (the ?pre looks promising)).

nocrypto: same story, see branch here. after separating the entropy packages into two, the opam file will need some attention. There is code duplication (nocrypto_entropy_solo5 and nocrypto_entropy_xen are nearly the same, apart from the module E = Entropy_XXX line). Should investigate further how to share the OCaml code. As preparations, I added two PRs to nocrypto with cleanups.

In order to get the dependencies adjusted, I added a mirage-no-solo5 package (similar to mirage-no-xen). The purpose is to be able to express 'either there is solo5 support in nocrypto, then we also need a freestanding gmp and zarith, or there is no solo5 support in nocrypto'.

Contributor

hannesm commented Jul 18, 2016

Entropy support: I started off with @mato branch for entropy and massaged it a bit to avoid code duplication. See the branch over here. The remaining piece of work is to, instead of having depopts and a single package, revert this change, and use no depopts, but two packages (mirage-entropy-xen and mirage-entropy-solo5). The C stub is a symbolic link, thus no code duplication there. I moved most of the OCaml code (apart from connect, which requires the OS module from mirage-xen/mirage-solo5) into the common Entropy module. I've no clue how to embed the call to postconf in a system using topkg (but likely the build documentation will help (the ?pre looks promising)).

nocrypto: same story, see branch here. after separating the entropy packages into two, the opam file will need some attention. There is code duplication (nocrypto_entropy_solo5 and nocrypto_entropy_xen are nearly the same, apart from the module E = Entropy_XXX line). Should investigate further how to share the OCaml code. As preparations, I added two PRs to nocrypto with cleanups.

In order to get the dependencies adjusted, I added a mirage-no-solo5 package (similar to mirage-no-xen). The purpose is to be able to express 'either there is solo5 support in nocrypto, then we also need a freestanding gmp and zarith, or there is no solo5 support in nocrypto'.

@hannesm

This comment has been minimized.

Show comment
Hide comment
@hannesm

hannesm Jul 19, 2016

Contributor

entropy:

nocrypto:

once I get the solo5 branch in mirage-entropy-xen to work, the upcoming changes to nocrypto will only be the addition of the compilation of the C stubs (plus maybe some copy of the xen subfolder)!

Contributor

hannesm commented Jul 19, 2016

entropy:

nocrypto:

once I get the solo5 branch in mirage-entropy-xen to work, the upcoming changes to nocrypto will only be the addition of the compilation of the C stubs (plus maybe some copy of the xen subfolder)!

@hannesm

This comment has been minimized.

Show comment
Hide comment
@hannesm

hannesm Jul 20, 2016

Contributor

updates:
entropy compiles now (and works for xen)
nocrypto adjusted

I opened a PR to opam-solo5 Solo5/opam-solo5#6 (while figuring out that I need some minor adjustments to the mirage tool for renaming).

as far as I'm concerned, this is a pretty clean and good solution to our demands.

Contributor

hannesm commented Jul 20, 2016

updates:
entropy compiles now (and works for xen)
nocrypto adjusted

I opened a PR to opam-solo5 Solo5/opam-solo5#6 (while figuring out that I need some minor adjustments to the mirage tool for renaming).

as far as I'm concerned, this is a pretty clean and good solution to our demands.

@mato

This comment has been minimized.

Show comment
Hide comment
@mato

mato Jul 21, 2016

Member

PRs to opam-repository submitted:

Member

mato commented Jul 21, 2016

PRs to opam-repository submitted:

@hannesm

This comment has been minimized.

Show comment
Hide comment
@hannesm

hannesm Jul 21, 2016

Contributor

mirage/mirage#565 mirage changes (massaged into a single commit)

Contributor

hannesm commented Jul 21, 2016

mirage/mirage#565 mirage changes (massaged into a single commit)

@mato

This comment has been minimized.

Show comment
Hide comment
@mato

mato Aug 9, 2016

Member

This is now complete, closing.

Member

mato commented Aug 9, 2016

This is now complete, closing.

@mato mato closed this Aug 9, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment