-
-
Notifications
You must be signed in to change notification settings - Fork 188
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Polling logging can be noisy. Ways to redirect or silence? #1221
Comments
Thanks for opening this issue. My initial thought on it is that it's tricky: GoodJob's polling query is the same query used by all other mechanisms to "fetch and lock the next job" i.e. it's the same query that happens after on LISTEN/NOTIFY trigger too. The query for polling could be silenced, but that would make jobs that are executed by polling have an inconsistent query log. My thought would be to add an option to silence (or change the reporting level to Error) of all of GoodJob's Active Record queries, which would just leave the logs that happen as part of Active Job.
I tried doing something really naive like this and it didn't have the affect I wanted, so it probably would need to be dug into. # config/initializers/good_job.rb
ActiveSupport.on_load(:good_job_base_record) do
GoodJob::BaseRecord.logger = Logger.new(IO::NULL)
end |
@bensheldon, I've explored this too and ran into the same issue with Some kind of @andynu, we ended up conditionally disabling the ActiveRecord logs. That might be viable in development if you run GJ in a separate process (eg: via foreman). If you run GJ inside the main Rails process or log ActiveRecord queries in production, then this won't be helpful. if GoodJob::CLI.within_exe && Rails.env.development? && !ENV['ALL_LOGS']
ActiveRecord::LogSubscriber.detach_from :active_record
end |
FWIW, just happened to see that SolidQueue is using |
Ben, I see what you mean. I tried assorted on_load hooks, and was unable to effectively silence the logger that way. I do think that approach is a lovely idea, if something like that worked then the library user would have lots of control.
None of these were effective. Trying to debug a little: # app/models/good_job/execution.rb:268
query = unfinished.dequeueing_ordered(parsed_queues).only_scheduled.limit(1)
warn "query: #{query.logger.inspect}"
query.with_advisory_lock(select_limit: queue_select_limit) do |executions| What I found especially odd was that I could see the logger configuration change as I introduced those different on_load logger overrides. They would appear on the active relation above, but never affected the actual logging when that query was executed. I don't really understand why yet (timing? threads?). As @zarqman suggests a silence block can effectively silence a particular query. # app/models/good_job/execution.rb:268
ActiveRecord::Base.silence{
unfinished.dequeueing_ordered(parsed_queues).only_scheduled.limit(1)
}.with_advisory_lock(select_limit: queue_select_limit) do |executions| However, that is ultra specific. And having a configuration option for one specific query may be too niche a need. Maybe there are a set of queries that together it might make sense to have a collective silence feature for, but it is not quite as nice as being able to swap out the logger. I do think that if there were a way to just reset the logger used on GoodJob::Exectution and GoodJob::Process models, that would be ideal. |
The logging that I find particularly noisy is related to polling.
In production, when execution_mode is :async, the following query happens every 10 seconds by default.
In a quiet application, this query piles up in the logs.
I'm curious if we could change something to quiet this down.
Some options that come to mind:
I think it is very important to retain logs of the jobs being executed, and any ActiveRecord logging related to the jobs itself.
Just a thought, happy to work with you on it. Thank you so much for this wonderful library.
The text was updated successfully, but these errors were encountered: