Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Support long-running tasks and scheduled operations #1211

Open
FooBarWidget opened this Issue Jun 13, 2014 · 12 comments

Comments

Projects
None yet
9 participants
Owner

FooBarWidget commented Jun 13, 2014

From http://www.reddit.com/r/ruby/comments/281bha/phusion_passenger_design_architecture_and_code/ci6g2av:

"The thing that I find lacking in passenger is support for long running task and scheduled operations.
I have tried various combinations of threads, forks, detached processes etc and it seems like passenger will shut down the parent process and child processes no matter what.
So you always have to launch a separate rails environment and redis just to manage background tasks and scheduled tasks. If you have lots of apps this ends up being a needlessly memory intensive ordeal."

rensm commented Aug 20, 2014

+1

wrkrb33 commented Oct 8, 2014

+1 - this also makes it impossible to create a persistent dblink to a remote database

Owner

FooBarWidget commented Oct 8, 2014

@wrkrb33 Why does it make persistent db links impossible? What does that have to do with supporting long-running tasks?

wrkrb33 commented Oct 8, 2014

@foorbarwidget I have a postgresql dblink that is instantiated (with username and password) at app startup through an initializer and is expected to remain accessible by name as long as the app is running. It works just fine under 'rails console' and 'rails server', but when running the app under passenger it throws an error:

PG::SREProhibitedSqlStatementAttempted: ERROR: password is required DETAIL: Non-superusers must provide a password in the connection string.

which I am interpreting to mean that the originally instantiated dblink has gone missing. So I think it's a differing symptom of the same root problem, that is, passenger shutting down processes that hang around for what it considers to be too long.

Owner

FooBarWidget commented Oct 8, 2014

@wrkrb33 I don't think that's related to this issue at all. Issue 1211 is about the lack of the ability to keep a process alive so that background threads can keep running. Dblinks are just socket connections that are only used during the life time of a single process, and there's no need to keep them open when request handling has ended, so they should be completely unrelated to issue 1211. I think you just have a wrong configuration file.

wrkrb33 commented Oct 8, 2014

You may be right, I am a complete n00b when it comes to dblinks. But I do know the configuration is correct, because as I said it works everywhere except under passenger. I've found a workaround, which is to not use a named link, but simply use the database connection string directly in dblink() calls. Which might still prove a problem if I needed more than one, since postgresql only permits one unnamed dblink at a time.

In any case, I'd be happy to remove my comments if you think they are irrelevant to your issue here.

khoan commented Feb 1, 2016

+1

@OnixGH OnixGH removed the SupportCentral label Feb 1, 2016

Is this issue still valid for people? I, having not much previous experience with Passenger, had some trouble getting an infinite running SuckerPunch job staying alive with Rails, as all the info I found said it will not work, but after some studying I have it working. I can share the details if someone wants them, but for all I know this is something that was addressed in an update like a year ago. Anyways it was all about having the runner in the preloader process and configuring that one to never time out.

@OnixGH OnixGH removed the SupportCentral label Jun 16, 2016

amukherj commented Sep 5, 2016

@skycaptainskycaptain, could you share what worked for you.

skycaptainskycaptain commented Sep 6, 2016 edited

@amukherj , sure. :)

What I have is a Rails engine gem, which runs the SuckerPunch job. So in the lib folder, I have code like this:

module Webasto
  class CacheRefresh
    include SuckerPunch::Job

    def perform()
      #my implementation code...

      CacheRefresh.perform_in(10.seconds)
    end
  end
end

So I want to keep that one running. I init it like this, from the railtie.rb ie. the "engine" file:

module Webasto
  class Railtie < ::Rails::Railtie
    config.after_initialize do
      Rails.logger.debug "Webasto enabled: " + Webasto.configuration.enabled.to_s
      if Webasto.configuration.enabled then
        CacheRefresh.perform_async
      end
    end
  end
end

The reason I'm posting the above details is that in my case, the job is started in the so called preloader process. What Passenger does, is that it starts the preloader process, which has the whole framework, it will run initializers etc. After that it clones only the http request handler part and uses that as the "instances" most of the Passenger configuration deals with. So when it's creating more instances to handle increased load, it's only dealing with the request handling part of the application, the controllers and such.

In my case, the job is running as part of the preloader process, and the normal max- and min-instances config don't apply to it. By default, the preloader process does its initial thing, and then passenger kills it off after 5min.

So what causes most of the confusion, is that people (like me) will try to adjust this setting:

https://www.phusionpassenger.com/library/config/standalone/reference/#--pool-idle-time-pool_idle_time

But it only affects the request handling instances, not the preloader process.

The correct setting is this one, so all you need to do in order to keep the preloader process alive, is to set this setting to 0. :D

https://www.phusionpassenger.com/library/config/standalone/reference/#--max-preloader-idle-time-max_preloader_idle_time

My links point to the standalone version's config, but the equivalent settings should exist for using with nginx etc, just double check the syntax, as it differs if you use it from the command line or in a config file etc.

I recommend reading this explanation about the spawn methods, as it really opened my eyes to what is going on in there. :)

https://www.phusionpassenger.com/library/indepth/ruby/spawn_methods/

amukherj commented Sep 6, 2016

@skycaptainskycaptain Thanks. That cleared quite a bit of confusion. Am I right to assume that this scheme will work when the background task is known in advance and can be started at bootstrap. On the other hand, if I want to start such a job in response to a user request and let it run in the background for some time, I can't rely on the preloader anymore - unless I use some sort of an inter-process communication mechanism. In this case the SuckerPunch task in the preloader process could just be the listener for incoming requests on a queue say (AMQP?). And because the preloader process isn't going anywhere, it could actually process this request in a thread.

@CamJN CamJN removed the SupportCentral label Jan 23, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment