Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate signalling server away from Heroku free tier #2

Open
5 of 8 tasks
ErnWong opened this issue Aug 28, 2022 · 30 comments
Open
5 of 8 tasks

Migrate signalling server away from Heroku free tier #2

ErnWong opened this issue Aug 28, 2022 · 30 comments
Assignees

Comments

@ErnWong
Copy link
Owner

ErnWong commented Aug 28, 2022

Consider either upgrade or move to another PaaS.
Going to move to railway.

Plan:

  • Deploy signalling server instance Railway
  • Update code to point to new signalling server instance
  • Shutdown Heroku signalling server instance.

Places to update in the code:

  • The RustConfig file (used by Heroku to know which crate we want to build)
    RUST_CARGO_BUILD_FLAGS="--release --package signalling-server"
  • Heroku endpoint URL hardcoded to the client now needs to be updated to the Railway endpoint
    let endpoint_url = format!("http://dango-daikazoku.herokuapp.com/join/{}", host_id);
  • Heroku endpoint URL hardcoded to the server also needs to be updated to the Railway endpoint
    let endpoint_url = "ws://dango-daikazoku.herokuapp.com/host".to_string();

To check:

  • Cross origin policies
  • SSL
@ErnWong ErnWong self-assigned this Aug 28, 2022
@ErnWong
Copy link
Owner Author

ErnWong commented Nov 13, 2022

[Edited: moved comment contents into issue description]

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 13, 2022

Railway's Nixpacks automatically-deduced build plan did not work:

==============
Using Nixpacks
==============
Nixpacks build failed
 
Error: Error reading crates/bevy_prototype_frameshader/src/Cargo.toml
 
Caused by:
No such file or directory (os error 2)

Going to look at setting up the build plan manually or help prod the Nixpack in the right direction

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 13, 2022

Why is Nixpacks trying to read a Cargo.toml in a src directory? Is it because Nixpacks is interpreting the wildcard differently for the Cargo workspace?

members = ["crates/*", "client", "server", "shared", "signalling-server"]

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 13, 2022

Looks like the corresponding Nixpacks configuration for RustConfig's --package setting is the NIXPACKS_CARGO_WORKSPACE environment variable: https://nixpacks.com/docs/providers/rust#workspaces

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 13, 2022

Environment variable NIXPACKS_CARGO_WORKSPACE=signalling-server has been manually set in Railway.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 13, 2022

Now we're further ahead in the deployment. The next error is:

 error: opening file '/app/.nixpacks/rust-toolchain': No such file or directory

Full logs:


==============
Using Nixpacks
==============
 
╔══════════════════════════════ Nixpacks v0.14.0 ══════════════════════════════╗
║ setup      │ binutils, gcc, (rust-bin.fromRustupToolchainFile ./rust-        ║
║            │ toolchain)                                                      ║
║──────────────────────────────────────────────────────────────────────────────║
║ build      │ mkdir -p bin                                                    ║
║            │ cargo build --release --package signalling-server               ║
║            │ cp target/release/signalling-server bin                         ║
║──────────────────────────────────────────────────────────────────────────────║
║ start      │ ./target/release/signalling-server                              ║
╚══════════════════════════════════════════════════════════════════════════════╝
 
 
#1 [internal] load build definition from Dockerfile
#1 sha256:e52de91af0a36b6086dbfb6b10dcd89c9962c2b33c6ba6508a13ba705d2dae49
#1 transferring dockerfile: 2.34kB done
 
#1 DONE 0.1s
 
#2 [internal] load .dockerignore
#2 sha256:9c5447916487526753159a1c919227b4851a6fbd092dbed641d905eb2bd766d0
#2 transferring context: 2B done
 
#2 DONE 0.1s
 
 
#3 [internal] load metadata for ghcr.io/railwayapp/nixpacks:debian-1667866002
#3 sha256:ff4546c29b1eb585dd7df95164933728a568cb8dc35398f9f51f11afd8b9a1cb
 
#3 DONE 0.2s
 
#4 [stage-0  1/10] FROM ghcr.io/railwayapp/nixpacks:debian-1667866002@sha256:96bc6d9c576900c9a99db8f7c8d6ef86c9e8dd9e54f46a119ff8499b9f5fe3b0
#4 sha256:a6aa9f4410ef5777e49afa17be8a1e260d09acb8a74de2dab233288bf3c56052
#4 DONE 0.0s
 
#5 [stage-0  2/10] WORKDIR /app/
#5 sha256:f1ff5906bd390875dc0cc6c2f06540658835071f9bb0611cd6ce76c903f39537
#5 CACHED
 
#6 [internal] load build context
#6 sha256:a73ec9408899d8af217c0d8d4817336bfad68eb49806caaa281d3068fe2bcd04
 
#6 transferring context: 19.60MB 0.2s done
 
#6 DONE 0.2s
 
#7 [stage-0 3/9] COPY .nixpacks/nixpkgs-a0b7e70db7a55088d3de0cc370a59f9fbcc906c3.nix .nixpacks/nixpkgs-a0b7e70db7a55088d3de0cc370a59f9fbcc906c3.nix
#7 sha256:40ef9b0a8bf94dcb42856d09167eabb4e7f550255081d5b473fae7c8257f55cb
 
#7 DONE 0.3s
 
#8 [stage-0 4/9] RUN nix-env -if .nixpacks/nixpkgs-a0b7e70db7a55088d3de0cc370a59f9fbcc906c3.nix && nix-collect-garbage -d
#8 sha256:47c6e2f8c88efbc361d03db9b11d2fe3a0279f3fed0751d9b239c2fac7f6029e
 
#8 27.72 installing 'a0b7e70db7a55088d3de0cc370a59f9fbcc906c3-env'
 
#8 28.13 error: opening file '/app/.nixpacks/rust-toolchain': No such file or directory
#8 28.13 (use '--show-trace' to show detailed location information)
 
#8 ERROR: executor failed running [/bin/bash -ol pipefail -c nix-env -if .nixpacks/nixpkgs-a0b7e70db7a55088d3de0cc370a59f9fbcc906c3.nix && nix-collect-garbage -d]: exit code: 1
-----
> [stage-0 4/9] RUN nix-env -if .nixpacks/nixpkgs-a0b7e70db7a55088d3de0cc370a59f9fbcc906c3.nix && nix-collect-garbage -d:
-----
executor failed running [/bin/bash -ol pipefail -c nix-env -if .nixpacks/nixpkgs-a0b7e70db7a55088d3de0cc370a59f9fbcc906c3.nix && nix-collect-garbage -d]: exit code: 1
 
Error: Docker build failed

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 19, 2022

I speculate the error is coming from the rust nix overlay due to the fromRustupToolchainFile option, but either it's looking in the wrong place or the toolchain file didn't get copied across to the right place. Could be wrong. What is the .nixpacks directory?

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 19, 2022

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 19, 2022

Tried installing railway cli to my NixOS but my nixpkgs might be too old. Trying to install it with the -I argument specifying the latest nixpkgs didn't work either.

Tried installing nixpacks through nix flakes, but failing with:

error: builder for '/nix/store/1zfqc00vcbw5v7xnf2vqyngmqyxnafm2-nixpacks-v0.3.0.drv' failed with exit code 1;
       last 10 log lines:
       > error[E0670]: `async fn` is not permitted in Rust 2015
       >  --> /build/dp51xbncj6pxg9fnbqgjhjrzp5f6hghs-source/src/nixpacks/builder/mod.rs:9:5
       >   |
       > 9 |     async fn create_image(
       >   |     ^^^^^ to use `async fn`, switch to Rust 2018 or later
       >   |
       >   = help: set `edition = "2021"` in `Cargo.toml`
       >   = note: for more on editions, read https://doc.rust-lang.org/edition-guide
       >
       > Error writing files: failed to resolve mod `builder`: cannot parse /build/dp51xbncj6pxg9fnbqgjhjrzp5f6hghs-source/src/nixpacks/builder/mod.rs
       For full logs, run 'nix log /nix/store/1zfqc00vcbw5v7xnf2vqyngmqyxnafm2-nixpacks-v0.3.0.drv'.
error: 1 dependencies of derivation '/nix/store/bx4rxg3j9rh0vvc4d03xxvm7dyp2hkwx-nix-shell-env.drv' failed to build

I suspect it's because in the check phase, it's using rustfmt outside of cargo, and therefore isn't aware of the edition set by Cargo.toml. I worked around it by disabling checks:

{
  description = "fix-my-dango";
  inputs = {
    flake-utils.url = "github:numtide/flake-utils";
    nixpacks.url = "github:railwayapp/nixpacks?rev=a8bf45a551e6be451633da04ba1d763c997ccc32";
  };
  outputs = { self, nixpkgs, flake-utils, nixpacks }:
    flake-utils.lib.eachDefaultSystem (system:  {
      devShell = pkgs.mkShell {
        nativeBuildInputs = with pkgs; [
          (nixpacks.packages.${system}.nixpacks.overrideAttrs (finalAttrs: previousAttrs: { doCheck = false; }))
        ];
      };
    });
}

Strangely, running nixpacks locally gives me error: opening file '/app/.nixpacks/rust-toolchain': No such file or directory even when NIXPACKS_CARGO_WORKSPACE=signalling-server is specified. My workaround is to remove ./crates/* from the cargo worksace in Cargo.toml.

Next up, I need to install docker onto my system.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 19, 2022

Running NIXPACKS_CARGO_WORKSPACE=signalling-server nixpacks build ../dango-tribute/ --verbose

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 20, 2022

Fails locally

[nixos@NixOS:~/repos/fix-dango]$ NIXPACKS_CARGO_WORKSPACE=signalling-server nixpacks build ../dango-tribute/ --verbose

╔══════════════════════════════ Nixpacks v0.14.0 ══════════════════════════════╗
║ setup      │ binutils, gcc, (rust-bin.fromRustupToolchainFile ./rust-        ║
║            │ toolchain)                                                      ║
║──────────────────────────────────────────────────────────────────────────────║
║ build      │ mkdir -p bin                                                    ║
║            │ cargo build --release --package client                          ║
║            │ cp target/release/client bin                                    ║
║──────────────────────────────────────────────────────────────────────────────║
║ start      │ ./target/release/signalling-server                              ║
╚══════════════════════════════════════════════════════════════════════════════╝

#1 [internal] load build definition from Dockerfile
#1 sha256:ae5e3284bbc2039d6bb646ea2399e953c644ca00b76cd66ce98f68293102222b
#1 transferring dockerfile: 1.18kB done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:05e9b9f25881d05b34aaa4255e49918643784478b8b7509970938d742487ed6c
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for ghcr.io/railwayapp/nixpacks:debian-1667866002
#3 sha256:ff4546c29b1eb585dd7df95164933728a568cb8dc35398f9f51f11afd8b9a1cb
#3 DONE 2.9s

#4 [stage-0 1/9] FROM ghcr.io/railwayapp/nixpacks:debian-1667866002@sha256:96bc6d9c576900c9a99db8f7c8d6ef86c9e8dd9e54f46a119ff8499b9f5fe3b0
#4 sha256:a6aa9f4410ef5777e49afa17be8a1e260d09acb8a74de2dab233288bf3c56052
#4 resolve ghcr.io/railwayapp/nixpacks:debian-1667866002@sha256:96bc6d9c576900c9a99db8f7c8d6ef86c9e8dd9e54f46a119ff8499b9f5fe3b0 0.0s done
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 0B / 31.37MB 0.1s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 0B / 43.97MB 0.1s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 0B / 31.32MB 0.1s
#4 sha256:96bc6d9c576900c9a99db8f7c8d6ef86c9e8dd9e54f46a119ff8499b9f5fe3b0 1.05kB / 1.05kB done
#4 sha256:6f23d6fa754b831d051be24378aa643f53b753f57269fb9c5678a19ab673cf06 953B / 953B done
#4 sha256:af4b12c8eaa23e13d394a5847e0771fd9cbcecc43fbb534ffb360a1e8f319702 3.83kB / 3.83kB done
#4 ...

#6 [internal] load build context
#6 sha256:e0a7a3f33e7efa1312dc8cd253e37a35efd8782581beb6da5699f0c0a7520d83
#6 transferring context: 77.86MB 1.4s done
#6 DONE 1.5s

#4 [stage-0 1/9] FROM ghcr.io/railwayapp/nixpacks:debian-1667866002@sha256:96bc6d9c576900c9a99db8f7c8d6ef86c9e8dd9e54f46a119ff8499b9f5fe3b0
#4 sha256:a6aa9f4410ef5777e49afa17be8a1e260d09acb8a74de2dab233288bf3c56052
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 2.10MB / 31.32MB 1.7s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 2.10MB / 31.37MB 2.1s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 4.19MB / 31.32MB 2.4s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 6.29MB / 31.32MB 2.9s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 4.19MB / 31.37MB 3.3s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 3.15MB / 43.97MB 3.3s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 8.39MB / 31.32MB 3.5s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 10.49MB / 31.32MB 5.3s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 6.29MB / 31.37MB 5.5s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 12.58MB / 31.32MB 6.8s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 5.24MB / 43.97MB 8.4s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 14.68MB / 31.32MB 9.7s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 7.34MB / 31.37MB 10.6s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 6.29MB / 43.97MB 13.5s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 16.78MB / 31.32MB 14.4s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 9.44MB / 31.37MB 14.5s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 18.87MB / 31.32MB 15.0s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 11.53MB / 31.37MB 15.7s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 9.44MB / 43.97MB 15.7s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 20.97MB / 31.32MB 15.7s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 23.07MB / 31.32MB 16.3s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 13.63MB / 31.37MB 16.8s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 25.17MB / 31.32MB 17.0s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 12.58MB / 43.97MB 17.3s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 15.73MB / 31.37MB 17.8s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 27.26MB / 31.32MB 17.8s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 29.36MB / 31.32MB 18.4s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 17.83MB / 31.37MB 18.9s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 15.73MB / 43.97MB 19.1s
#4 sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 31.32MB / 31.32MB 19.2s done
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 19.92MB / 31.37MB 19.7s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 18.87MB / 43.97MB 20.0s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 22.02MB / 31.37MB 20.3s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 24.12MB / 31.37MB 21.0s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 22.02MB / 43.97MB 21.0s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 26.21MB / 31.37MB 21.6s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 25.17MB / 43.97MB 21.9s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 28.31MB / 31.37MB 22.4s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 30.41MB / 31.37MB 22.9s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 28.31MB / 43.97MB 22.9s
#4 sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 31.37MB / 31.37MB 23.1s done
#4 extracting sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 31.46MB / 43.97MB 23.4s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 34.60MB / 43.97MB 23.9s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 37.75MB / 43.97MB 24.3s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 40.89MB / 43.97MB 24.8s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 43.97MB / 43.97MB 25.3s
#4 sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 43.97MB / 43.97MB 25.3s done
#4 extracting sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7 2.8s done
#4 extracting sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 0.1s
#4 extracting sha256:178aa227edbde8b2fdd551b4ae9f6926fbe04923f910779888cf58588fefd3a9 4.0s done
#4 extracting sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930
#4 extracting sha256:2f73ae600eeca70e016decc7577a9be5f4daff0d315a19173dbd97ce21e14930 2.3s done
#4 DONE 32.9s

#5 [stage-0 2/9] WORKDIR /app/
#5 sha256:f1ff5906bd390875dc0cc6c2f06540658835071f9bb0611cd6ce76c903f39537
#5 DONE 0.7s

#7 [stage-0 3/9] COPY .nixpacks/nixpkgs-a0b7e70db7a55088d3de0cc370a59f9fbcc906c3.nix .nixpacks/nixpkgs-a0b7e70db7a55088d3de0cc370a59f9fbcc906c3.nix
#7 sha256:34aa8b39f06b0d6eba231927cc021fc32d75113f7981774572c2f7229ad57c61
#7 DONE 0.0s

#8 [stage-0 4/9] RUN nix-env -if .nixpacks/nixpkgs-a0b7e70db7a55088d3de0cc370a59f9fbcc906c3.nix && nix-collect-garbage -d
#8 sha256:6eca83260e93852bf497edcb86ca656f85a0474fd6801cf8e69a123007b924c3
#8 0.318 error: Operation not permitted
#8 ERROR: executor failed running [/bin/bash -ol pipefail -c nix-env -if .nixpacks/nixpkgs-a0b7e70db7a55088d3de0cc370a59f9fbcc906c3.nix && nix-collect-garbage -d]: exit code: 1
------
 > [stage-0 4/9] RUN nix-env -if .nixpacks/nixpkgs-a0b7e70db7a55088d3de0cc370a59f9fbcc906c3.nix && nix-collect-garbage -d:
------
executor failed running [/bin/bash -ol pipefail -c nix-env -if .nixpacks/nixpkgs-a0b7e70db7a55088d3de0cc370a59f9fbcc906c3.nix && nix-collect-garbage -d]: exit code: 1
Error: Docker build failed

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 20, 2022

Setting security.unprivilegedUsernsClone in my nixos configuration to true didn't help. It would be weird if it's a misconfiguration of the debian image used as the base of the docker image, since it works fine on the railway website. Looks very similar to the error here: https://github.com/railwayapp/nixpacks/issues/441. I might try updating my nixos.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 20, 2022

After updating nixos, I now get pass the Operation not permitted error, and instead get the same error as in Railway about rust-toolchain not found.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 20, 2022

Indeed if I follow https://github.com/railwayapp/nixpacks/blob/1a37a464e187781d5670d8b0abf9d3a823574663/CONTRIBUTING.md#debugging and run NIXPACKS_CARGO_WORKSPACE=signalling-server nixpacks build ../dango-tribute/ --out out, and inspect the ./out directory, I don't see any out/.nixpacks/rust-toolchain file in there: just the Dockerfile and the nix file.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 20, 2022

If that rust-toolchain is added via Phase::add_file_dependency, I assume it should've been picked up by dockerfile_generation.rs to generate the corresponding COPY commands.

When I log the Phase::only_include_files field in the rust setup struct in the rust provider, I indeed see the rust-toolchain file added in.

However, when I log the Phase::only_include_files field in generate_dockerfile, I only get None? Is it the same Phase object as before?

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 20, 2022

Ah, looks like I was only looking at the build phase in the generate_dockerfile. Where is the setup phase then?

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 20, 2022

Instead, when generating the dockerfile for the build plan, it generates a noop dockerfile for the setup phase.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 20, 2022

This is because of Phase::run_docker_commands() returns false. Is this meant to return true?

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 20, 2022

I suspect it's a regression introduced in railwayapp/nixpacks@be4c282

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 20, 2022

Temporary workaround: I'll see if I can specify an older NixPacks before this commit

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 20, 2022

Even with that part commented out, I still get an error as it seems it's looking for the rust-toolchain file in the wrong directory.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 20, 2022

Looks like we need the rust-toolchain before the setup phase, as that's when the nix file (with the rust overlay) is being run.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 21, 2022

Change of plans do to the lack of time. I'll try other builders available on railway, and if that fails, will look at creating a custom nix flake file.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 21, 2022

Using dream2nix to nixify this project. Hitting a problem where it thinks my patches are part of the workspace and gets confused.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 21, 2022

Wait, I see nixpacks has an example of using rust with custom toolchain. I'm going to try run that example.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 21, 2022

Still fails for me. Looks like examples/rust-custom-toolchain is not covered by the tests.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 21, 2022

dream2nix didn't seem to work - it got confused when resolving the impurities for a patch, saying that it thinks it's in a workspace when it's not. Algoods, since dream2nix is still in its very early stages as well.

Going to try out cargo2nix.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 21, 2022

Fumbled around for a while with cargo2nix due to a typo I had when specifying a custom rustToolchain from the rust-toolchain file - I accidentally set the wrong attribute (rustVersion instead of rustToolchain). With that fixed, cargo2nix worked fine!

Then I created a simple Dockerfile to run nix build. Started off with https://github.com/LnL7/nix-docker but it didn't have nix flakes configured out of the box, so I went with https://github.com/nix-community/docker-nixpkgs (https://hub.docker.com/r/nixpkgs/nix-flakes).

Got an error when using nixos-21.11

warning: unknown setting 'accept-flake-config'
warning: Git tree '/app' is dirty
error: flake 'git+file:///app' does not provide attribute 'packages.x86_64-linux.defaultPackage.x86_64-linux', 'legacyPackages.x86_64-linux.defaultPackage.x86_64-linux' or 'defaultPackage.x86_64-linux'

but this error looks familiar - it's probably because I ran cargo2nix that is only compatible with newer versions of nix. Indeed, running nix flake show shows slightly different attribute names.

Updating the dockerfile to use a more up-to-date base image with a newer version of nix (nixos-22.05) solved it.

It's now building on railway now.

Next step would be to maybe use multi-stage builds to only copy over the actual runtime dependencies and not the build time dependencies.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 23, 2022

Multi-stage build done. Thank you https://marcopolo.io/code/nix-and-small-containers/ for the guide. Next up, I'll point the code to the new signalling server.

@ErnWong
Copy link
Owner Author

ErnWong commented Nov 23, 2022

Hmmm, server websocket seems to connect fine, but then the client http request doesn't go through... perhaps it's something to do with the SSL redirection?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant