Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schedule doesn`t start from command line #1812

Closed
yuturiy opened this issue Oct 5, 2019 · 5 comments
Closed

Schedule doesn`t start from command line #1812

yuturiy opened this issue Oct 5, 2019 · 5 comments
Assignees

Comments

@yuturiy
Copy link
Contributor

yuturiy commented Oct 5, 2019

I use
dagster schedule start --start-all
inside docker container for script in docs https://dagster.readthedocs.io/en/0.6.0.post0/sections/learn/tutorial/scheduler.html

Dockerfile:

FROM python:3.7.4-slim-stretch

RUN apt-get update
RUN apt-get -y install cron 

RUN pip install dagster
RUN pip install dagit
RUN pip install dagster-cron

WORKDIR /dagster

EXPOSE 3000

ENTRYPOINT [ "dagit", "-h", "0.0.0.0", "-p", "3000" ]

docker-compose.yml

version: '3'
services:
  dagster:
    build: .
    environment:
      - LC_ALL=C.UTF-8
      - LANG=C.UTF-8
    ports:
      - 3000:3000
    volumes:
      - ./dagster:/dagster

And get
image

@yuturiy yuturiy changed the title tab Schedule doesn`t start from command line Oct 5, 2019
@natekupp
Copy link
Contributor

natekupp commented Oct 6, 2019

I believe this will be fixed by https://dagster.phacility.com/D1226

@natekupp
Copy link
Contributor

natekupp commented Oct 6, 2019

one workaround you can use in the interim:

for name in `dagster schedule list --name`; do
    dagster schedule start $name;
done

@natekupp
Copy link
Contributor

natekupp commented Oct 6, 2019

also @helloworld I think we're going to have to update the docs to describe how to get cron working in a container, the above Dockerfile won't launch cron. @newturok FYI you'll need something like the following in an entrypoint.sh:

#!/bin/sh

# see: https://unix.stackexchange.com/a/453053 - fix link-count
touch /etc/crontab /etc/cron.*/*

service cron start
export DAGSTER_HOME=/opt/dagster/dagster_home

for name in `/usr/local/bin/dagster schedule list --name --running`; do
    echo $name;
    /usr/local/bin/dagster schedule stop $name;
    /usr/local/bin/dagster schedule start $name;
done

/usr/local/bin/dagit -h 0.0.0.0

then in your Dockerfile, add the lines:

COPY entrypoint.sh /
ENTRYPOINT ["/entrypoint.sh"]

@yuturiy
Copy link
Contributor Author

yuturiy commented Oct 6, 2019

Maybe I don't understand something, but it looks like cron is not installed in Docker image, at least dagster/dagster-py37
image

Only after

RUN apt-get update
RUN apt-get -y install cron 

in Dockerfile i get

image

@yuturiy
Copy link
Contributor Author

yuturiy commented Oct 6, 2019

@natekupp
dagster schedule up is needed to start from scratch

Full working demo for scheduler:
https://github.com/newturok/dagster_scheduler_demo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants