Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use pixi to replace our bash scripts and many of the Dockerfile layers #3717

Closed
7 of 9 tasks
jleibs opened this issue Oct 5, 2023 · 3 comments
Closed
7 of 9 tasks
Assignees
Labels
🧑‍💻 dev experience developer experience (excluding CI) enhancement New feature or request
Milestone

Comments

@jleibs
Copy link
Member

jleibs commented Oct 5, 2023

Pixi should be able to replace all of our current just commands while bringing the added benefit of running things in an environment with managed dependencies. For example, this gives us access to our code-gen dependencies, the arrow-cpp we need for building rerun-cpp, etc.

This lets us replace all of the commands in just with versions that handle the dep installation first.
This also lets us collapse many of the layers of our Dockerfile into a single pixi install invocation.

TODO

@jleibs jleibs added enhancement New feature or request 🧑‍💻 dev experience developer experience (excluding CI) labels Oct 5, 2023
@emilk emilk self-assigned this Oct 13, 2023
emilk added a commit that referenced this issue Oct 16, 2023
### What
* Part of #3717

Add a proof-of-concept `pixi.toml` file for the project.

This is like Just but also takes care of dependency management for us.
No need to run `setup_dev.sh` or `pip install -r
scripts/requirements-dev.txt`.

This even manages the installation of `arrow-cpp` for us.

Just:
```
cargo install pixi
```

Then have fun:
```
pixi run codegen
pixi run py-test
pixi run cpp-test
```

Verified this works in a clean Ubuntu-20.04 docker container. 

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/3707) (if
applicable)

- [PR Build Summary](https://build.rerun.io/pr/3707)
- [Docs
preview](https://rerun.io/preview/071e0d4c0cff28c2262b966d985cf0f68ad9f456/docs)
<!--DOCS-PREVIEW-->
- [Examples
preview](https://rerun.io/preview/071e0d4c0cff28c2262b966d985cf0f68ad9f456/examples)
<!--EXAMPLES-PREVIEW-->
- [Recent benchmark results](https://ref.rerun.io/dev/bench/)
- [Wasm size tracking](https://ref.rerun.io/dev/sizes/)

---------

Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>
Co-authored-by: Andreas Reich <r_andreas2@web.de>
@emilk emilk removed their assignment Oct 16, 2023
@abey79
Copy link
Member

abey79 commented Oct 24, 2023

Vaguely relevant issue for me:

@emilk
Copy link
Member

emilk commented Oct 24, 2023

We should probably keep our requirements.txt files, at least most of them (e.g. for examples)

@emilk emilk changed the title Use pixi to replace just and many of the Dockerfile layers Use pixi to replace our bash scripts and many of the Dockerfile layers Nov 6, 2023
@emilk emilk added this to the Triage milestone Nov 7, 2023
Wumpf added a commit that referenced this issue Mar 18, 2024
### What

Almost everything has been taken over by `pixi` by now and these
outdated / partially broken scripts keep confusing new-comers to the
repo, so at this point they're clearly doing more bad than good.

* Fixes #5545
* Fixes #4170
* There's still some stuff that should be moved to pixi but that's
better covered by #3717


### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using newly built examples:
[app.rerun.io](https://app.rerun.io/pr/5552/index.html)
* Using examples from latest `main` build:
[app.rerun.io](https://app.rerun.io/pr/5552/index.html?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[app.rerun.io](https://app.rerun.io/pr/5552/index.html?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!

- [PR Build Summary](https://build.rerun.io/pr/5552)
- [Docs
preview](https://rerun.io/preview/8da4ef3b503b5459274c73ea40bc0926b61dbae9/docs)
<!--DOCS-PREVIEW-->
- [Examples
preview](https://rerun.io/preview/8da4ef3b503b5459274c73ea40bc0926b61dbae9/examples)
<!--EXAMPLES-PREVIEW-->
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)

---------

Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>
@Wumpf Wumpf mentioned this issue Mar 20, 2024
@Wumpf Wumpf self-assigned this Apr 10, 2024
Wumpf added a commit that referenced this issue Apr 11, 2024
### What

Part of 
* #3717

After this PR, the only .sh files in the repo are `ci_docker/publish.sh`
and `pre-push.sh`. The former is hopefully going away together with the
docker, the later can stay.

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using examples from latest `main` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/5922?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/5922?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!

- [PR Build Summary](https://build.rerun.io/pr/5922)
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)
@Wumpf
Copy link
Member

Wumpf commented Apr 12, 2024

the only thing we didn't do is to publish rerun on pixi itself, but otherwise we ticked all the boxes now!

@Wumpf Wumpf closed this as completed Apr 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🧑‍💻 dev experience developer experience (excluding CI) enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants