Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

csm-backup in docker with dotnet #49

ruXlab opened this issue Apr 26, 2020 · 5 comments

csm-backup in docker with dotnet #49

ruXlab opened this issue Apr 26, 2020 · 5 comments


Copy link

ruXlab commented Apr 26, 2020

Hello here

I'm trying to get up and running backups on the linux via docker but stumbled upon this issue:

SCM Backup 1.4.0.ae7edc3
Backing up config files
Getting repositories for source 'xxx-bitbucket' from hoster 'bitbucket'
Trying to find SCMs on this machine...
Found Git 2.20.1 
Backing up source: xxx
fatal: Unable to create temporary file '/scm-backup-out/xxx-bitbucket/xxxx#proj/repo/objects/pack/tmp_pack_XXXXXX': Permission denied
fatal: index-pack failed

Backup failed!
The application will close in 0 seconds!

I don't see what is wrong with permissions because if I do docker exec in this container I can easily create file:

# touch /scm-backup-out/xxx-bitbucket/xxxx#proj/repo/objects/pack//tmp_pack_XXXXXX
# ls -l /scm-backup-out/xxx-bitbucket/xxxx#proj/repo/objects/pack//tmp_pack_XXXXXX
-rw-r--r-- 1 1000 1000 0 Apr 26 13:11 /scm-backup-out/xxx-bitbucket/xxxx#proj/repo/objects/pack/tmp_pack_XXXXXX

Also, git clone inside container works fine as expected

My Dockerfile:


WORKDIR /scm-backup

# Install .NET core and SCM Backup
RUN apt-get update && \
    apt-get -y install curl sudo wget unzip apt-transport-https git && \
    wget -O && \

CMD dotnet ScmBackup.dll


➜  docker build . -t scm-backup
➜  docker run -it -v `pwd`/bitbucket-backups:/scm-backup-out:rw -v `pwd`/settings.yml:/scm-backup/settings.yml scm-backup 
Copy link

Sorry, but I don't have any Linux and Docker experience.
I'm developing this on Windows and my only "contact" with Docker so far was setting up some CI on Bitbucket Pipelines, which happens to use Docker containers under the hood.

So I don't have the knowledge to directly help you with any permission issues.
And SCM Backup just calls the Git executable, so there's nothing special happening in SCM Backup that I could "fix".
That error message sounds like something deep from the bowels of Git.

Also, git clone inside container works fine as expected

SCM Backup uses git fetch, not git clone. Here are the exact parameters.

So for your example, it would execute this:

git -C "/scm-backup-out/xxx-bitbucket/xxxx#proj/repo/" fetch --force --prune refs/heads/*:refs/heads/* refs/tags/*:refs/tags/*

Does this work in your Docker container?

Copy link

ruXlab commented Apr 26, 2020

@christianspecht thanks for prompt response

I tried to reproduce what your code does but in Docker:

  1. Create folder mkdir "/scm-backup-out/xxx-bitbucket/xxxx#proj/repo/" -p
  2. I noted git init is used, so I did git init --bare in that folder
  3. Run command above and it failed!
git -C "/scm-backup-out/xxx-bitbucket/xxxx#proj/repo/" fetch --force --prune refs/heads/*:refs/heads/* refs/tags/*:refs/tags/*
remote: Enumerating objects: 324, done.
remote: Counting objects: 100% (324/324), done.
remote: Compressing objects: 100% (194/194), done.
fatal: Unable to create temporary file '/scm-backup-out/xxx-bitbucket/xxxx#proj/repo/objects/pack/tmp_pack_XXXXXX': Permission denied
fatal: index-pack failed

So, you were right about issue - it was caused by git and/or docker themself. Will look into this later

Thank you!

It's very interesting - git clone works as expected

git clone 
Cloning into 'scm-backup'...
remote: Enumerating objects: 324, done.
remote: Counting objects: 100% (324/324), done.
remote: Compressing objects: 100% (194/194), done.
remote: Total 3884 (delta 205), reused 212 (delta 128), pack-reused 3560
Receiving objects: 100% (3884/3884), 486.04 KiB | 1.36 MiB/s, done.
Resolving deltas: 100% (2756/2756), done.

So I'd assume it has to do with git init and the way it creates git index files

git init --bare produces valid permissions (uid 1000)

root@fa3ca74f5265:/scm-backup-out/xxx-bitbucket# tree -u    
`-- [1000    ]  xxxx#proj
    `-- [1000    ]  repo
        |-- [1000    ]  FETCH_HEAD
        |-- [1000    ]  HEAD
        |-- [1000    ]  branches
        |-- [1000    ]  config
        |-- [1000    ]  description
        |-- [1000    ]  hooks
        |   |-- [1000    ]  applypatch-msg.sample
        |   |-- [1000    ]  commit-msg.sample
        |   |-- [1000    ]  fsmonitor-watchman.sample
        |   |-- [1000    ]  post-update.sample
        |   |-- [1000    ]  pre-applypatch.sample
        |   |-- [1000    ]  pre-commit.sample
        |   |-- [1000    ]  pre-push.sample
        |   |-- [1000    ]  pre-rebase.sample
        |   |-- [1000    ]  pre-receive.sample
        |   |-- [1000    ]  prepare-commit-msg.sample
        |   `-- [1000    ]  update.sample
        |-- [1000    ]  info
        |   `-- [1000    ]  exclude
        |-- [1000    ]  objects
        |   |-- [1000    ]  info
        |   `-- [1000    ]  pack
        |       `-- [1000    ]  eeeeeeee
        `-- [1000    ]  refs
            |-- [1000    ]  heads
            `-- [1000    ]  tags

11 directories, 17 files

Copy link

FYI: someone else got it to work in Docker, see #51

@ruXlab ruXlab mentioned this issue Apr 28, 2020
Copy link

juris commented Apr 30, 2021

Hi there, here is a sample of working Dockerfile:

FROM alpine:3.13

ARG DOTNET_FILE=dotnet-runtime-3.1.14-linux-musl-x64.tar.gz
ARG DOTNET_FILE_SHA512=77856f6183ef7ee561fe8510e89148566972ae21e3915242ae4ac7ef987b1aa78cda09bb06fdae96cd03758975dc5eb0e8652dea79b96db327ac5de2a4d41961

ENV SCM_ROOT=/opt/scm-backup
ENV DOTNET_ROOT=/opt/dotnet


RUN apk add --no-cache \
 # Install deps
        bash icu-libs krb5-libs \
        libgcc libintl libssl1.1 \
        libstdc++ zlib curl git \
 # Install dotnet runtime
 && curl -fsL -o ${DOTNET_FILE}${DOTNET_FILE} \
 && echo "${DOTNET_FILE_SHA512}  ${DOTNET_FILE}" > ${DOTNET_FILE}.sha512 \
 && sha512sum -c ${DOTNET_FILE}.sha512 \
 && mkdir -p ${DOTNET_ROOT} \
 && tar zxf ${DOTNET_FILE} -C ${DOTNET_ROOT} \
 # Install scm-backup
 && curl -fsL -o ${SCM_FILE}${SCM_FILE} \
 && unzip ${SCM_FILE} \
 && rm -f ${SCM_FILE} ${DOTNET_FILE} ${DOTNET_FILE}.sha512 \
 # Install yq (config templating)
 && curl -fSL${YQ_VERSION}/yq_linux_amd64 -o /usr/local/bin/yq \
 && chmod +x /usr/local/bin/yq

COPY /usr/local/bin
COPY settings.yml .
RUN chmod +x /usr/local/bin/


Very raw for Dockerfile with variable replacement:

#!/usr/bin/env bash

yq e -i '(.sources[] | select(.title == "bitbucket_backup").name) = env(BITBUCKET_TEAM)' settings.yml
yq e -i '(.sources[] | select(.title == "bitbucket_backup").authName) = env(BITBUCKET_USER)' settings.yml
yq e -i '(.sources[] | select(.title == "bitbucket_backup").password) = env(BITBUCKET_PASS)' settings.yml

exec dotnet ScmBackup.dll

Sample settings.yml template with bitbucket config:

localFolder: 'backup'

  - title: bitbucket_backup
    hoster: bitbucket
    type: org
    name: team
    authName: username
    password: app_password

Just create all 3 files in a single directory and run docker build . -t scm-backup
Then you can use a container and mount your target backup repo to /opt/scm-backup/backup

docker build . -t scm-backup
docker run -v ${PWD}:/opt/scm-backup/backup -e BITBUCKET_USER=mybbusername -e BITBUCKET_PASS=mybbpassword -e BITBUCKET_TEAM=mybbteam scm-backup

christianspecht added a commit to christianspecht/scm-backup-docs that referenced this issue Aug 25, 2021
Copy link

Closing this for now.

I just added a new section to the documentation which lists all Dockerfiles / images that I know of:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet

No branches or pull requests

3 participants