[Deployment Issue] 'Captain-definition file does not exist' after updating repo structure

[TylerWanta]

IMPORTANT: Please avoid posting issues that are not specific to CapRover. Issues and questions related to Docker will get closed.

If you are trying to see how to deploy a particular app using a customized dockerfile - please change your questions to Dockerfile and post it where appropriate such as StackOverflow. This is a Docker question, not a CapRover specific issue.

---

What is the problem?
Previously my repo only had a single .net core project in it and structure looked like this

- VaulticServer\
----- captain-definition
----- other files

The deploy for it worked and all was ok. But, In order to improve the app structure, I split it out into multiple projects like so:

- VualticServer\
----- API\
---------- captain-definition
---------- other files
----- STS\
----------- captain-definition
----------- other files
----- Business\
----------- other files

The plan being that each project with a captain-definition will get its own caprover app. I use github actions and thought I would be able to just update the path to point to the new apps directory and life would be good but its not working.

Previous action (works)

name: Build & Deploy

on:
  workflow_dispatch:

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [18.x]
        
    if: github.ref == 'refs/heads/master'

    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - uses: a7ul/tar-action@v1.1.0
        with:
          command: c
          cwd: "../"
          files: |
            ./VaulticServer
          outPath: deploy.tar

      - name: Deploy App to CapRover
        uses: caprover/deploy-from-github@v1.0.1
        with:
          server: '${{ secrets.CAPROVER_SERVER }}'
          app: '${{ secrets.APP_NAME }}'
          token: '${{ secrets.APP_TOKEN }}'

New action (works but causes Captain Definition file does not exist in caprover)

 name: Build & Deploy API

on:
  workflow_dispatch:

jobs:
  build-and-deploy-api:
    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [18.x]
        
    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - uses: a7ul/tar-action@v1.1.0
        with:
          command: c
          cwd: "../"
          files: |
            ./VaulticServer/API
          outPath: deploy.tar

      - name: Deploy App to CapRover
        uses: caprover/deploy-from-github@v1.0.1
        with:
          server: '${{ secrets.CAPROVER_SERVER }}'
          app: '${{ secrets.APP_NAME }}'
          token: '${{ secrets.APP_TOKEN }}'

Note: I even tried specifying the captain-definition file like so:

  - uses: a7ul/tar-action@v1.1.0
    with:
      command: c
      cwd: "../"
      files: |
        ./VaulticServer/API
        ./VaulticServer/API/captain-definition
      outPath: deploy.tar

the github action still succeeds but same error in caprover. I tried the advice from this question and ran

git archive --format tar --output ./test.tar refactorRepo. This gave me a tar folder with the correct file structure, i.e.

- test.tar
----- captain-definition
----- other files

I've also double checked to make sure that the captain-definition is in fact pushed to my branch.

If applicable, content of captain-definition file:

 {
  "schemaVersion" :2 ,
  "dockerfilePath" :"./Dockerfile"
 }

my docker file is pretty simple as well
(old)

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build-env
WORKDIR /app

# copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore

# copy everything else and build
COPY . ./
RUN dotnet publish -c Release -o out

# build runtime image
FROM mcr.microsoft.com/dotnet/sdk:8.0
WORKDIR /app
COPY --from=build-env /app/out .
ENTRYPOINT ["dotnet", "VaulticServer.dll"]

updated (only changed the .dll name)

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build-env
WORKDIR /app

# copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore

# copy everything else and build
COPY . ./
RUN dotnet publish -c Release -o out

# build runtime image
FROM mcr.microsoft.com/dotnet/sdk:8.0
WORKDIR /app
COPY --from=build-env /app/out .
ENTRYPOINT ["dotnet", "API.dll"]

Steps to reproduce the problem:
Not entirely sure. Maybe its just an issue with a nested folder structure?

Output of the following command on your server:

uname -a && lsb_release -a && free -h

Linux caprover1111onubuntu2004-s-2vcpu-2gb-90gb-intel-nyc3-01 5.4.0-162-generic #179-Ubuntu SMP Mon Aug 14 08:51:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.6 LTS
Release:        20.04
Codename:       focal
              total        used        free      shared  buff/cache   available
Mem:          1.9Gi       913Mi       127Mi        60Mi       930Mi       808Mi
Swap:            0B          0B          0B

[githubsaturn]

Try this one, does it work?

      - uses: a7ul/tar-action@v1.1.0
        with:
          command: c
          cwd: "./"
          files: |
            API/
          outPath: deploy.tar

[TylerWanta]

Hmmm, that seemd to solved it. Why does it matter if I use ../ and then reference the path vis VaulticServer/API instead of ./ and just API? Isn't it the same folder?

I'm also getting a second error, dockerfilePath should not refer to parent directory!. This is in result of trying to include the Business project since the API project references it.

Following some stackoverflow threads, it seemed like the easiest solution was to have my dockerfile at the solution level. I moved it there so my file structure looks like:

-API/
----- captain-definition
-Business/
-APIDockerFile

I then just updated my captain definition to:

{
    "schemaVersion": 2,
    "dockerfilePath": "../APIDockerfile"
}

which causes the error.

new docker file just for reference:

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
WORKDIR /app

# copy csprojs and restore as distinct layers
WORKDIR /src
COPY ["./API/API.csproj", "API/"]
COPY ["./Business/Business.csproj", "Business/"]
RUN dotnet restore "API/API.csproj"

# copy everything else and build
COPY . ./

WORKDIR "/src/API"
RUN dotnet build "API.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "API.csproj" -c Release -o /app/out

# build runtime image
FROM mcr.microsoft.com/dotnet/sdk:8.0
WORKDIR /app
COPY --from=publish /app/out .
ENTRYPOINT ["dotnet", "API.dll"]

Is there anyway to have a docker file in the parent directory or do I instead need to change my docker file to work from within the API directory (hopefully possible)?

EDIT:
I am finding out docker does not allow copying files from outside where the docker file lies.

EDIT 2:
Ok I think I got it working. I had to move both the captain-definition and docker file to the solution level. Final structure:

- API/
- Business/
- api-captain-definition
- APIDockerFile

Include everything in github workflow:

  command: c
  cwd: "./"
  files: |
    ./API
    ./Business
    ./api-captain-definition
    ./APIDockerFile
  outPath: deploy.tar

I obviously also had to update my default captain-definition path for my caprover app.

Final question though is, is this the best way to do this? I'm eventually going to have 3 captain definitions and docker files in my solution directory, which, for lack of a better phrase, is a bit ugly IMO. Is there a better way to set this up while still having all projects in the solution and repo?

EDIT 3:
One more small question. After setting up both my API and STS projects, the only one I was able to hit both and not get a 502 was to set the container http port for both to be 8080. Is this correct? Why / how can both apps be on port 8080?

[githubsaturn]

Isn't it the same folder?

Yes, but the content of the tar file is a bit different. You want your captain definition to be at the root level of your tar file, not deep in a directory.

Final question though is, is this the best way to do this? I'm eventually going to have 3 captain definitions and docker files in my solution directory, which, for lack of a better phrase, is a bit ugly IMO. Is there a better way to set this up while still having all projects in the solution and repo?

If you want to share dockerfiles / captain-definition across multiple apps (API and Business), that's the best way to do it.

One more small question. After setting up both my API and STS projects, the only one I was able to hit both and not get a 502 was to set the container http port for both to be 8080. Is this correct? Why / how can both apps be on port 8080?

This is the container port, not the host port.

                                                                         
                                         ┌────────► App1:8080            
                                         │                               
                                         │                               
      User────► Port80───►CapRover(nginx)│                               
                                         │                               
                                         │                               
                                         │                               
                                         └────────► App2:8080            
                                                                         
                                                                         

[TylerWanta]

Ahh, that all makes sense. I appreciate you taking the time to explain it all!