Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cli): make tauri/cli fully support projects with non-standard structure #11258

Merged
merged 4 commits into from
Oct 17, 2024

Conversation

regexident
Copy link
Contributor

@regexident regexident commented Oct 7, 2024

Usually I'd first open a feature suggestion issue before embarking on the pull request itself for something like this, but I already have the implementation (as it was necessary to unblock myself), so I'm gonna skip the issue for this one.

The tl;dr is that we would like to propose adding explicit support for more flexible project structures (i.e. moving src-tauri out of the frontend app), which surprisingly took only minimal and non-invasive changes for the tauri cli tool.

(FYI: we'll be using web-app and tauri-app as dummy-names for the top-most app and app/src-tauri directories here throughout, they don't matter!)

With the patched cargo-tauri binary one is able to run pnpm tauri …/cargo tauri … from both, the app, as well as src-tauri directory (and even —very convenient!— from the top-most directory!), regardless of how the project structured, like so:

Given a project structured like this:

my-app
├── ...
├── tauri-app
│   └── ...
└── web-app
    └── ...

… one would be able to run this:

cd my-app

# probably something you'd put in an `.env` file:
export TAURI_APP_PATH="./web-app"
export TAURI_FRONTEND_PATH="./tauri-app"

pnpm tauri dev
# or
cargo tauri dev
Currently the CLI tool crashes when running from projects with non-standard structure

Currently running pnpm tauri …, such as pnpm tauri migrate from the web app directory of a project with non-standard structure leads to the following crash:

$ cd ./web-app
$ pnpm tauri migrate

thread '<unnamed>' panicked at crates/tauri-cli/src/helpers/app_paths.rs:107:5:
Couldn't recognize the current folder as a Tauri project. It must contain a `tauri.conf.json`, `tauri.conf.json5` or `Tauri.toml` file in any subfolder.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
 ERR_PNPM_RECURSIVE_EXEC_FIRST_FAIL  Command was killed with SIGABRT (Aborted): tauri migrate

Running the equivalent cargo command from the tauri src directory (in again, a non-standard structure) works, for what it's worth:

$ cd ./tauri-app
$ cargo tauri migrate

Info Installing Cargo dependency "tauri-plugin-dialog"...
Updating crates.io index
  Adding tauri-plugin-dialog v2 to dependencies
...
Added permission `dialog:default` to `migrated` at <SNIP>/tauri-app/capabilities/migrated.json
Info Adding plugin to <SNIP>/tauri-app/src/main.rs
Info Running `cargo fmt`...

But since the pnpm route seems to be the far more common (and is usually the one that's dominantly recommended in the official guides) one I'd expect very few people to realize that while one crashes, the other works, even in non-standardly structured projects.

Motivation

The standard structure of a tauri project looks something like this:

my-app
├── package.json
├── src
├── src-tauri
│   ├── Cargo.toml
│   ├── src
│   ├── tauri.config.json
│   └── ...
└── ...

The tauri project gets added as a sub-directory of the existing web project. This is certainly convenient for some things (like being able to have a single place for scripts: package.json, etc.) but also rather inconvenient for others.

Flexible project structure

This rather rigid structure seems both, unnecessarily limiting and inconvenient, compared to say a structure like this:

my-app
├── tauri-app
│   ├── Cargo.toml
│   ├── src
│   ├── tauri.config.json
│   └── ...
└── web-app
    ├── package.json
    ├── src
    └── ...

The main source of inconvenience of the standard structure comes from the fact that web dev projects tend to be littered with top-level configuration junk.

Junk, junk everywhere

What we mean by "top-level configuration junk"

As such the directory diagrams above where lying. Rather than being neat and clean:

web-app
├── package.json
├── src
└── ...

… the web-app directory's actual contents look like this garbage fire:

web-app
├── .gitignore
├── .npmrc
├── .prettierignore
├── .prettierrc
├── .svelte-kit
│   └── ...
├── build
│   └── ...
├── bundlemeta.json
├── components.json
├── eslint.config.mjs
├── node_modules
│   └── ...
├── package.json
├── packages
│   └── ...
├── playwright.config.ts
├── pnpm-lock.yaml
├── pnpm-workspace.yaml
├── postcss.config.ts
├── setupTest.ts
├── src
│   └── ...
├── static
│   └── ...
├── svelte.config.js
├── tailwind.config.ts
├── tests
│   └── ...
├── tsconfig.json
├── vite.config.ts
└── ...

Poor developer experience & productivity

Other than src/tests/package.json hardly any of the many top-level files within a run-of-the-mill web-app project tend to be of actual regular(!) importance in one's day-to-day work, yet there is no way to hide them, making it at times hard to see the forest for the trees.

Unfortunately there really isn't much one can do about that (afaik), it's just the mess we find ourselves in when doing web dev work.
But forcing the same nonsense upon everybody working on the Rust bits of a Tauri project is really just adding insult to injury.

Poor CI experience & performance, and significantly increased costs

Speaking of which, another major issue with the classic structure is that there simply is no straight-forward and reliable way to be smart about which jobs to run for a PR and on which CI runners, when everything is stuffed in the same directory.

This is particularly problematic in the context of a multi-platform Tauri project (which probably 90% of Tauri projects are) and the reality of macOS-based CI runners commonly being 10x as expensive as their Linux-based counterparts.

From a cost-perspective one would thus probably at least want to run the platform-agnostic frontend tests on the cheap CI runner and possibly even try to carve out any macOS-specific backend tests so only those get frequently run on the expensive macOSs (10x) runner, and everything else on cheap Linux (1x) or Windows (2x) runners.

Frontend and backend tests competing for CI time

One might decide to split frontend and backend test jobs into separate workflows and have them run in parallel, in order to reduce the time spent by devs waiting for their pull requests to finish running their actions.

And while this might help reduce the average time to wait it ends up increasing the average money spent as unlike a single serial workflow that can abort on the first error with parallel workflows your "other" workflow will keep running and thus keep blocking the queue when your workflow encounters an error. (There probably are arcane Github action spells to solve this, but shouldn't this stuff just do the right thing by default?)

One could certainly split up the workflows and add extensive on.pull_request.paths/on.pull_request.exclude_paths lists to each of them and achieve a more fine-grained CI workflow selection, but such lists tend to be frickle at best and at worst could lead to CI not running a workflow where it absolutely should have.

By turning web-app and tauri-app into sibling directories selecting the right CI jobs and the right CI runner for a given PR becomes almost trivial:

How to effortlessly run separate workflows for the app and tauri
# .github/workflows/tauri-app.yml

on:
    pull_request:
        branches: [main, "**"]
        paths:
            - "tauri-app/**"
            - ".github/workflows/tauri-app.yml"
# ...
# .github/workflows/web-app.yml

on:
    pull_request:
        branches: [main, "**"]
        paths:
            - "web-app/**"
            - ".github/workflows/web-app.yml"
# ...

As a result we found our CI times to greatly improve as said sibling structure allowed us to limit the expensive macOS tests to just the tauri-app directory. (We're still running full tests on a fixed schedule as a safety net to catch any regressions introduced at the integration layer of web-app and tauri-app, but so far those have't caught anything slipping through.)

Benefits of a non-standard project structure

Anecdotally we found the developer experience and productivity to greatly improve upon switching the app/src-tauri directories from the standard parent-child to the alternative sibling structure.

While focussing on one side of a tauri app other's directory can be folded away in the IDE, allowing for working uninterruptedly and with focus.

For larger teams where the web-frontend and tauri-backend are developed by different people having a clear separation has clear benefits that probably don't need to be illustrated here on their own.

Additionally having separate directories helps with VCS features like Github's code owners, which afaik doesn't even support exclude paths, so rather than directory globs for ./web-app/** and ./tauri-app/** every top-level file would need to be listed separately, leading to additional maintenance overhead.

When it comes to CI ergonomics and economics we —again anecdotally—found that having straight-forward control (yet fine-grained where necessary) greatly helped with streamlining the project's CI, significantly reducing the mean waiting time per CI action, as well as greatly reduced cost overall.

(From a pure architectural and usability point of view one could go as far as argue that the sibling structure should probably be the default, not the other way round. But that's not what this PR is about!)

That said we would really like to see Tauri support a more flexible project structure. Especially since it doesn't seem to take that much to make its tools work with them.

We are fully aware that a change like this (i.e. introducing env vars) has to be taken carefully and will thus probably take more than just "it works, let's ship it!". As such this PR is sort of a proof-of-concept for how things could be done.

(Disclaimer: we have so far only run the patched CLI with a few commands on our project with a sibling structure, but all of them successfully so, which is promising! That said every non-trivial tool inevitably will have accrued implicit assumptions about the currently rather rigid project structure and thus it is expected that more changes may be necessary elsewhere.)

@regexident regexident requested a review from a team as a code owner October 7, 2024 19:24
Copy link
Contributor

github-actions bot commented Oct 7, 2024

Package Changes Through 4a3eb71

There are 4 changes which include @tauri-apps/api with patch, tauri with patch, @tauri-apps/cli with patch, tauri-cli with patch

Planned Package Versions

The following package releases are the planned based on the context of changes in this pull request.

package current next
@tauri-apps/api 2.0.2 2.0.3
tauri 2.0.4 2.0.5
@tauri-apps/cli 2.0.3 2.0.4
tauri-cli 2.0.3 2.0.4

Add another change file through the GitHub UI by following this link.


Read about change files or the docs at github.com/jbolda/covector

@regexident
Copy link
Contributor Author

The tests were already failing for the upstream commit prior to this PR's changes. Not sure what to do about them. 🤔

@lucasfernog lucasfernog self-assigned this Oct 14, 2024
@lucasfernog
Copy link
Member

I've renamed TAURI_SRC_DIR to TAURI_FRONTEND_PATH and TAURI_APP_DIR to TAURI_APP_PATH to be more clear on what path does the env var refer to, what do you think? app and src are too obscure IMO, as both the frontend and the tauri app directories contain the "app" and the "src" 😂

@lucasfernog
Copy link
Member

tests were failing because you changed the tauri.conf lookup function to start from the <cwd>/src-tauri folder, where we actually lookup from <cwd> instead to partially support your use case - having the Tauri app in a subfolder that is not named src-tauri

lucasfernog
lucasfernog previously approved these changes Oct 14, 2024
@regexident
Copy link
Contributor Author

I've renamed TAURI_SRC_DIR to TAURI_FRONTEND_PATH and TAURI_APP_DIR to TAURI_APP_PATH to be more clear on what path does the env var refer to, what do you think? app and src are too obscure IMO, as both the frontend and the tauri app directories contain the "app" and the "src" 😂

Works for me! 👍 I wasn't exactly sure about the somewhat overloaded names to begin with.

tests were failing because you changed the tauri.conf lookup function to start from the <cwd>/src-tauri folder, where we actually lookup from <cwd> instead to partially support your use case - having the Tauri app in a subfolder that is not named src-tauri

Oh I see. Interesting. 🤔 When I noticed the tests failing I checked out the PR's parent commit to check if I broke something and it failed the same way. I guess I somehow didn't clear the caches properly.

@regexident regexident force-pushed the custom-tauri-app-and-src-dirs branch from 366a728 to 8af2d51 Compare October 15, 2024 09:22
@lucasfernog lucasfernog merged commit e4c9268 into tauri-apps:dev Oct 17, 2024
12 of 14 checks passed
@regexident regexident deleted the custom-tauri-app-and-src-dirs branch October 18, 2024 05:59
@StreetStrider
Copy link

Hi. Great work. What changes must be done to existing project to migrate? I feel like I've been using this approach already (even without the patch): My build.frontendDist = '../web'. If someone's wanting to use this, what actions must be done? Is it enougt to just move directories, update build.frontendDist and go on? Or is something else required?

@regexident
Copy link
Contributor Author

@lucasfernog just updated the tauri CLI and got this:

$ cd ./web-app
$ pnpm tauri dev

thread '<unnamed>' panicked at crates/tauri-cli/src/helpers/app_paths.rs:117:5:
Couldn't recognize the `TAURI_FRONTEND_PATH` folder as a Tauri project. It must contain a `tauri.conf.json`, `tauri.conf.json5` or `Tauri.toml` file in any subfolder.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
 ERR_PNPM_RECURSIVE_EXEC_FIRST_FAIL  Command was killed with SIGABRT (Aborted): tauri dev

It looks like the env vars got mixed up in 8af2d51. 😦

@regexident
Copy link
Contributor Author

regexident commented Oct 21, 2024

@StreetStrider what made the above structure work for us was adding "cwd": "../web-app" to all the npm commands in tauri.config.json:

{
  "build": {
    "beforeBuildCommand": {
      "cwd": "../web-app",
      "script": "pnpm build"
    },
    "beforeDevCommand": {
      "cwd": "../web-app",
      "script": "pnpm dev"
    },
  }
}

Update: no longer necessary, see comment below!

@regexident
Copy link
Contributor Author

@StreetStrider I just checked and the above is actually no longer necessary (and might even break things) with the env vars introduced with this PR (and fixed in #11492).

See #11492 (comment) for a complete step-by-step of the minimal steps necessary to switch from tauri's standard structure to a custom one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants