New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Background (non-parallel) task/code execution #194
Comments
Actually-parallel tasks fall under #63, FWIW - this is a little different from that even though similar solutions might apply. tl;dr, probably a good use for Off the top of my head the most straightforward (maybe only) way to do this is to update So maybe something like this? @task
def mytask():
event = threading.Event()
# Giving non-None to 'event' could implicitly trigger backgrounding.
# Could also make it explicit though this needs 2 args every time which is bluh
run('start pelican', event=event)
# ... do random other foreground things here
event.set() # the backgrounded run() notices this & halts itself We're considering requiring contextualization for all tasks, in which case the event could be an explicit attribute of the shared context, perhaps (again - top of my head here): @task
def mytask(c):
c.run('start pelican', background=True)
# ... other things here ...
c.event.set() # or a 'nicer' API call like c.stop() or c.finish(); or even bake it into what @task does. Suspect we'd need both versions of this - folks doing a "I just want to kick this off and have it clean up when I exit" use case like yours might prefer the Task/Context based implicit setup where you just say |
Thanks for looking into it. I'm not sure I understand all the options, but yes, I'm looking for a "I just want to kick this off and let it do it's thing". I like the second code example you wrote: @task
def develop():
run('pelican -r -s pelicanconf.py', background=True)
run('cd ' + env_deploy_path + ' && python -m http.server', background=True) that seems to make intuitive sense to me. |
EDIT: Have run into some subsequent issues where EDIT 2: I have updated the code below with a fix. @MinchinWeb I think we're trying to solve the same problem using pelican and invoke on Python 3.x I was able to get this to work by doing the following (Disclaimer: I don't have lots of knowledge on threading, but this works for me) from invoke import task
import os
import sys
import socketserver
import threading
from pelican.server import ComplexHTTPRequestHandler
ENV_DEPLOY_PATH = 'output'
PORT = 8000
@task
def regenerate(ctx,start_dir=''):
"""Automatically regenerate site upon file modification"""
ctx.run('pelican -r -v -s {}pelicanconf.py'.format(start_dir))
@task
def serve(ctx):
"""Serve site at http://localhost:8000/"""
os.chdir(ENV_DEPLOY_PATH)
class AddressReuseTCPServer(socketserver.TCPServer):
allow_reuse_address = True
server = AddressReuseTCPServer(('', PORT), ComplexHTTPRequestHandler)
sys.stderr.write('Serving on port {0} ...\n'.format(PORT))
server.serve_forever()
@task
def develop(ctx):
"""`regenerate` and `serve`"""
start_dir = os.getcwd() + '/'
t_regenerate = threading.Thread(target=regenerate,args=[ctx,start_dir])
t_serve = threading.Thread(target=serve,args=[ctx])
t_regenerate.start()
t_serve.start()
t_regenerate.join()
t_serve.join() Hopefully this is useful to you while the invoke team work on #63 |
Actually, somewhere along the line, it just started working for me. Originally, I would call invoke using start, and that worked. (i.e. Somewhere along the line, invoke started working to invoke 'start' commands. My current from pathlib import Path
from invoke import run, task
p = Path.cwd()
deploy_path = p.parents[0] / 'blog.minchin.ca-temp' # used for local testing
@task
def serve(ctx):
"""Serve the local blog output on port 8000."""
ctx.run('cd {} && start python -m http.server'.format(deploy_path))
# ... I'm running invoke v0.14.0 on Python 3.6.0 x64 on Windows 10. |
I'm guessing there has been minimal forward progress on this? I'm using v0.22.1 and |
@aikchar That's gonna happen either way, because That's why this sort of thing needs the backgrounding to be done on the master/control side, thus the notes about threading and similar above. This comes up periodically and while users can (as noted) do their own threading, I definitely want some easy common simple API, kinda similar to |
Today I discovered Start-Process which allows you to do this from PowerShell directly, and can be included in a
|
Didn't find this when I searched the other day, but #682 subsumes this. Of note, my assertion about shell backgrounding and blocking actually only applies to some situations like remote sshd-driven shell sessions; local ones, at least on some operating systems, seem to actually work as users expect (but not on Invoke right now, for reasons - see #682). The notes about Windows' |
@bitprophet Still around! Let me know what you want me to test. |
I'm using Windows 7, Python 3.4.2, and Invoke 0.9.0 (installed via
pip
).I want to run 'backgound' tasks. In Windows, if you place the
start
command at the beginning of the command, it will start your task in a new cmd window and then return focus to your original cmd window.Invoke
will start the task, but waits for the new cmd window to close before continuing on. I would likeInvoke
to start the new task, and then continue with its next command, and eventually exit on 'completion.'My
tasks.py
file looks like this:The text was updated successfully, but these errors were encountered: