For consideration: an API to start / teardown a request context #596

Closed
wichert opened this Issue May 4, 2012 · 9 comments

Projects

None yet

2 participants

@wichert
Member
wichert commented May 4, 2012

When writing console scripts you often need a configured pyramid, which is handled by pyramid.paster.bootstrap. Sometimes you need more than that though: you also need a fully configured request context. For example to get transaction handling or other things provided by tweens, or to have extra request properties setup. A possible API for that would be something like:

environment = bootstrap(config_file)
with environment['request-context']:
    # Do my thing

where the request context would setup the tweens, issue INewRequest, etc. on __enter__ and do all the necessary teardown on __exit__.

Owner

What events would you expect to be emitted? What response would be propagated back into the tweens? These are serious questions because I've toyed with implementing this feature but didn't have a specific use-case in mind.

There is already an outstanding pull request (#520 and #538) for adding the request properties to the request that were set via config.set_request_property.

Member
wichert commented May 4, 2012

INewRequest mostly, I don't think there is anything else that makes sense at first sight.

I don't think there is a useful response to be returned since you are not rendering a view - None would work for me. I don't know if all current tweens can deal with that though.

The use case where I need this is for integration with things like celery or rq where you have a separate process that executes tasks. In that case I have tweens that do things that are important for a task (configuring RQ itself, managing stransaction), and for tasks that render a template I need the extra request properties as well.

Owner

Well at one point I had the notion of effectively treating the bootstrap script as a continuation. Thus while you're in the with you are "in a view" and when the with exits request.response is propagated back out. I ran into some issues implementing that, but a context manager may make it possible.

On the other hand, at that point it's not much different from running prequest where you place your script in some view. Biggest difference there is that your script isn't publicly available, but that view could be disabled unless run in a certain context.

Member
wichert commented May 4, 2012

The prequest model does not work for queueing systems (celery/rq/retools/etc.) though.

Owner

You'll have to elaborate. My conceptual idea is that the view is something mounted at '/__pviews__/myscript' and you run prequest dev.ini /__pviews__/myscript, your code is executed in the full pipeline including tweens/middleware.

Member
wichert commented May 4, 2012

This fails because you assume that I have a single request during the process lifetime. For queueing systems you have a long-running process that runs many different asks, and each task must get its own request context.

Owner

Okay, I see what you're getting at. So conceptually you are running a script that may execute prequest multiple times.

Owner

The original idea for bootstrap was really just to be able to send emails that were rendered using templates. For this all you needed was 1) the mailer from the registry, 2) the correct host name (set via Request._blank(..)) to generate URLs and 3) the ability to call render() and route_url() which require a request.

The case that's made it interesting is the one with pyramid_tm where you have to call transaction.commit() manually in a script but it might be cool if it was committed automatically when you call env['closer']().

In general, you don't want to run your entire WSGI pipeline in a script. There's potentially a lot of unwanted side-effects that make me feel like a script that simulates requests to do its job is a bad fit. It only really makes sense in very simple applications that don't have a ton of side-effects.

@mcdonc mcdonc added a commit that referenced this issue Sep 11, 2012
@mcdonc mcdonc - Request properties and methods added via ``config.set_request_prope…
…rty`` or

  ``config.add_request_method`` are now available to tweens.

- Request properties and methods added via ``config.set_request_property`` or
  ``config.add_request_method`` are now available in the request object
  returned from ``pyramid.paster.bootstrap``.

Related partially to issue #520 and issue #538 and issue #596, although it doesn't actually allow us to close any of them, because we still don't issue a newrequest event when bootstrap is used.
7359873
Owner
mmerickel commented Apr 28, 2016 edited

I just ran across this issue and it should be mostly solved at this point. The main feature is that request methods and properties are now attached properly which replaces most uses of the NewRequest event.

For a long-running script it's possible now to build a registry using the building blocks of bootstrap (bootstrap itself is coupled to a single request, but the building blocks are not).

app = pyramid.paster.get_app(inipath)
registry = app.registry

for job in jobs:
    req = Request.blank(...)
    env = pyramid.scripting.prepare(registry=registry, request=request)
    env['closer']()

Using this method you can load the app once and re-use it across several dummy requests. You are still responsible for defining your own pipeline however, including invoking any events you care about and managing transactions.

If you really need to simulate a request with a full router lifecycle (tweens and events) then prequest is a good option or you can define your logic in a pyramid view, sending your request through the router.

def your_worker_view(request):
    # do some work

config = Configurator()
config.include(your_app)
config.add_view(your_worker_view, name='worker1234')
app = config.make_wsgi_app()

req = Request.blank('worker1234', ...)
response = req.get_response(app)

Anyway I'm closing this because I don't see where else this discussion can go. If I'm wrong let me know and we can reopen the issue!

@mmerickel mmerickel closed this Apr 28, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment