Add http endpoint for triggering a forced (out of schedule) sync

[sed-i]

Running git-sync --one-time with the same params as an existing git-sync process may interfere each other if both syncs happen to occur at the same time.
An http endpoint for triggering a sync on demand (temporarily skipping throught the remaining wait-time) could be handy.

---

Perhaps .git/index.lock guarantees no interference, in which case running a second instance of git-sync with --one-time could be the go-to solution.

[thockin]

Allowing a manual trigger means we have to talk about authorization - who is allowed to trigger that? git-sync is often (usually!) run in a container, so putting such a mechanism on the network without authz is a bad idea.

I don't think git-sync is responsible for synchronizing between instances of itself - why are you doing that?

See #149 and #226

Again, I am not exactly saying no, but I'd need someone to show a clear example of how auth works.

[sed-i]

I don't think git-sync is responsible for synchronizing between instances of itself - why are you doing that?

I'm using git-sync for prometheus alert rules: when those are updated, I would like to be able to trigger an update right away (if indeed urgent) instead of waiting for wait-time to elapse.

The auth argument is solid.

If starting a second git-sync process with git-sync --one-time is safe (relying on index.lock to fail either process), then that would probably work for me.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[the-xentropy]

This was actually fairly easy. Only took a few hours - and I have it dynamically installing my python requirements too. No more rebuilding images for this guy. (:

0. Configure the deployment to use a podSecurityContext setting the fsGroup to e.g. 1000. Your cluster might need runAsUser and runAsGroup too, but just set them all to the same value.
1. Add git-sync as a sidecarContainer, pulling in code to a known location. If you don't do the above you'll get file issues since git-sync will write with unusable permissions.
2. Add a startupProbe to the git-sync sidecarContainer which checks for the existence of the definitions file (I do stat <package root - I.E. below the first absolute import>/somepackagename/__init__.py)
3. Set the dagster user-code-example gRPC command line options to:

          dagsterApiGrpcArgs:
            - "--working-directory"
            - "<package root - I.E. below the first absolute import>"
            - "--python-file"
            - "<package root - I.E. below the first absolute import>/somepackagename/__init__.py"

init.py imports the rest using e.g. from somepackagename import <blah>.

For the python requirements, I just modified my python code to run pip install if it detects it's being run under dagster lazily before it needs specific imports and bumped up the gRPC readinessProbe failureThreshold so it has time to install everything. You probably can't use an init container for this unless you want to manually restart your deployments all the time, because git-sync would pull in code which has requirements that aren't installed yet - so I just went the "easy" route by lazily pulling down requirements before I import them.

I'll handle optimization timing on the tail-end by just pruning instances where I can detect I don't have to run pip. I dunno if dagster itself caches imports even across file changes, and if so I may have to add some code to manually delete cached imports from globals in the future, but this works.

For those with more static dependencies or that want reliability instead of dynamic import insanity, it's probably easier to just build a derivative image of the user-code-example and pip install your requirements then. Way less prone to breakage.

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten