Skip to content

Conversation

@doorgan
Copy link
Collaborator

@doorgan doorgan commented Jun 27, 2025

Updates Expert to use Burrito for packaging

This builds both Expert and Engine with the same elixir and otp versions, the idea is that next we will build several Engine versions in CI and Expert will download the right one for the current project, or default to the one included in it.

To build a release for your machine:

make release.local

To build releases for macos, linux and windows:

make release

To only build Engine(this will generate namespaced files at apps/engine/_build/dev_ns:

make build.engine

Note that Engine will be built for dev and not prod, this is because it makes use of Mix.Project and related code, which would not be available if it were build with prod. The idea behind this command is that we will have a CI step on release that will run make build.engine against a matrix of supported elixir+otp versions and publish those artifacts, which Expert will then download so it matches the exact project versions.


There are some other relatively minor differences or things to note:

  • Due to the way Burrito generates the executable, we lose the ability to include a debug_shell.sh script in it, as far as I can see. We need to rethink how to do this, we can still connect an observer to the server node with :observer.start(:"manager-project-entropy@127.0.0.1") but I'm not sure how to open a remote session like one would with a regular release script.
  • A few tests in Engine were calling code that in production would run in the Expert node(like Engine.Api). Because that code was moved to Expert and is no longer accessible by Engine, those tests were moved to Expert to avoid circular dependencies on tests.

@doorgan doorgan changed the base branch from main to genlsp-server June 27, 2025 11:40
@doorgan doorgan force-pushed the doorgan/burrito_packaging branch 3 times, most recently from 327323d to 2c94e28 Compare June 28, 2025 00:21
@doorgan doorgan marked this pull request as ready for review June 29, 2025 00:07
@doorgan doorgan force-pushed the doorgan/burrito_packaging branch from 4d8aed0 to 32644c2 Compare June 29, 2025 21:20
@doorgan doorgan changed the base branch from genlsp-server to doorgan/isolate_engine June 29, 2025 21:20
@doorgan doorgan force-pushed the doorgan/burrito_packaging branch 2 times, most recently from 1864da9 to 0047f4d Compare June 29, 2025 23:37
@doorgan doorgan changed the title refactor: Use Burrito for packaging refactor: use Burrito for packaging Jul 3, 2025
Base automatically changed from doorgan/isolate_engine to main July 3, 2025 04:53
@doorgan doorgan force-pushed the doorgan/burrito_packaging branch 6 times, most recently from a1101ae to 2ff1264 Compare July 3, 2025 18:36
Copy link
Member

@mhanberg mhanberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets merge #35 into this PR and then we can merge this.

i'll work on the release workflow, and you can get the precompiled engine matrix going

@doorgan doorgan force-pushed the doorgan/burrito_packaging branch from a7a7498 to 15ce974 Compare July 8, 2025 04:32
@doorgan doorgan force-pushed the doorgan/burrito_packaging branch from 15ce974 to 3043a23 Compare July 8, 2025 04:38
* refactor: integrate GenLSP

* fixup!: fix tests

* fixup!: use persistent_term instead of adding .lsp to Project

* fixup!: fix dialyzer issues

* fixup!: Don't use parenthesis on 0-arity function

* fixup!: cleanup credo changes

* fixup!: Remove outdated comments

* fixup!: use info severity for normal shutdown log

* fixup!: cleanup credo changes

* refactor: isolate Engine from Expert

* refactor: rename project node modules and simplify EngineApi/EngineNode code

* refactor: use Burrito for packaging

* chore: update github release workflow

* feat: add plain(non burrito) releases

* chore: Remove release-please-action

* chore: Remove build/release tasks

* chore: fix unordered aliases

* chore: update lockfile

* fix: rollback change to elixir version

* chore: remove ci integration tests

Using burrito we're no longer trying to activate the user's elixir executable on boot, making those tests stale

* Revert changes to the release workflow

It will be done in a follow-up PR

* chore: add ci test for release builds

* wip

---------

Co-authored-by: doorgan <dorgandash@gmail.com>
@mhanberg
Copy link
Member

To recap the trouble I have been having testing this locally on x86_64 linux:

  • Zig was failing to compile the burrito wrapper, I thought this was due to my Linux kernel being too knew. I had recently updated my desktop to Ubuntu 25.04.
  • I re-install my operating system with NixOS, which uses a LTS version of the linux kernel. Error still occurs. I think maybe its Nix being weird (I was using Nix on my Ubuntu install)
  • I try a plain jane x86_64 Ubuntu 24.04 virtual machine, still the same error.

I am left with confusion why it passed in CI, but can't seem to pass for me locally. Digit is going to investigate, and I am going to invite him to the repository so it's easier for him.

@mhanberg
Copy link
Member

To recap the trouble I have been having testing this locally on x86_64 linux:

* Zig was failing to compile the burrito wrapper, I thought this was due to my Linux kernel being too knew. I had recently updated my desktop to Ubuntu 25.04.

* I re-install my operating system with NixOS, which uses a LTS version of the linux kernel. Error still occurs. I think maybe its Nix being weird (I was using Nix on my Ubuntu install)

* I try a plain jane x86_64 Ubuntu 24.04 virtual machine, still the same error.

I am left with confusion why it passed in CI, but can't seem to pass for me locally. Digit is going to investigate, and I am going to invite him to the repository so it's easier for him.

Digit was able to get it to wrap in a VM, so I'm just going to classify this as I'm cursed and I'll figure it out later

Copy link
Member

@mhanberg mhanberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine burrito wise, please merge at your discretion.

@doorgan doorgan merged commit e655dfe into main Jul 22, 2025
0 of 16 checks passed
@doorgan doorgan deleted the doorgan/burrito_packaging branch July 22, 2025 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants