How to Prevent .env and upload from Being Deleted During git-sync (Similar to .gitignore)

[1ocate]

The issue is that after git-sync successfully completes, all untracked files are deleted, including the external file links and .env file connections. However, recreating the links after syncing feels a bit odd, as the file connections are lost during the sync process.

How can this be solved? Should I recreate the symbolic links immediately after the sync finishes?

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-deployment
spec:
  replicas: 1
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: example
  template:
    metadata:
      labels:
        app: example
    spec:
      containers:
        - name: example
          image: docker.redgistry.kr/example:latest
          imagePullPolicy: Always
          volumeMounts:
          - name: web-source
            mountPath: /var/www/html
          - name: storage
            mountPath: /var/www/html/upload
          - name: htaccess
            mountPath: /var/www/html/.htaccess
            subPath: .htaccess
          - name: config
            mountPath: /var/www/html/.env
            subPath: .env
          command: ["/bin/sh", "-c"]
          args:
            - |
              chown -R root:www-data /var/www/html/upload && \
              chmod -R 775 /var/www/html/upload && \
              ln -s /var/www/html/.env /var/www/html/web/.env && \
              ln -s /var/www/html/.htaccess /var/www/html/web/.htaccess && \
              ln -s /var/www/html/upload /var/www/html/web/upload && \
              apache2-foreground
          ports:
            - containerPort: 80
          env:
          - name: TZ
            value: Asia/Seoul
        - name: git-sync
          image: registry.k8s.io/git-sync/git-sync:v4.0.0
          securityContext:
            runAsUser: 1000
            runAsGroup: 1000
          args:
            - "--repo=https://$(GIT_USERNAME):$(GIT_TOKEN)@github.com/redgistry-kr/example.git"
            - "--branch=$(BRANCH)"
            - "--root=/git"
            - "--dest=web"
            - "--one-time=false"
            - "--period=60s"
            - "--wait=60"
          env:
            - name: BRANCH
              value: branch
            - name: GIT_USERNAME
              valueFrom:
                secretKeyRef:
                  name: git-credentials
                  key: GIT_USERNAME
            - name: GIT_TOKEN
              valueFrom:
                secretKeyRef:
                  name: git-credentials
                  key: GIT_TOKEN
          volumeMounts:
            - name: web-source
              mountPath: /git
      imagePullSecrets:
        - name: redgistry-registry-secret
      volumes:
        - name: storage
          persistentVolumeClaim:
            claimName: web-pvc
        - name: htaccess
          configMap:
            name: htaccess
        - name: config
          configMap:
            name: web-env
        - name: web-source
          emptyDir: {}

[thockin]

I'm not sure I understand. Are you creating extra files in the get repo dir that are not managed by git? How are those being created? Or am I misunderstanding?

git-sync has hooks, but they only run after the symlink is published.

Have you looked at the hook s that git itself offers?

[thockin]

Revisiting this before it ages out.

You are mounting the git-sync volume into the webserver as /var/www/html. Inside that directory is a symlink web which points to the current git hash. Presumably apache's serving root is /var/www/html/web (which is not the default) ? Or are you redirecting everything URL to /web/$original_path ?

And you want to add files into that dir (which is a git repo) which are NOT managed by git.

We can't "ignore" the files because git-sync makes a new directory to sync each hash.

The existing web and exec hooks run AFTER the symlink (web, here) is flipped, which means even if you recreate the files in the hook, there is a moment when they don't exist.

I could imagine something like --exechook-when=BEFORE_PUBLISH which would change the hook behavior to run before the symlink flips, so you could create the extra files the way you want.

[tylenwells]

We're having a similar issue using git-sync with airflow. We're using git-sync to pull a repo of dags from git. We're using a exec hook to pull manifests into the synced dir. These files are then used by dag runs. The manifests are hosted in an object storage bucket not git, otherwise we could just pull them recursively as a submodule.

This --exechook-when flag would work well for us, as we just need to make sure the symlink isn't swapped over until we are done pulling the manifests.

Alternatively, a feature to delay the symlink swap until after the exechook finishes would work as well.

Actually, a --delay-symlink=post-exechook feature would probably work better for us, as exechook and git-sync were async right? In that case even if we started the exechook first, gitsync could finish first and still be an issue.

[thockin]

PRs are welcome - I am a little underwater at the moment so it will be tough to make time in the near future.

[tylenwells]

No problem at all. I've forked the repo and will take a look at it. Thanks for the quick response!