Skip to content

Support :source_location tag option for query log tags - #50969

Merged
byroot merged 1 commit into
rails:mainfrom
fatkodima:query_logs-line
Feb 6, 2024
Merged

Support :source_location tag option for query log tags#50969
byroot merged 1 commit into
rails:mainfrom
fatkodima:query_logs-line

Conversation

@fatkodima

@fatkodima fatkodima commented Feb 5, 2024

Copy link
Copy Markdown
Member

Follow up to #42240 (as discussed in #42240 (comment)).

We lost the ability to use the :line option with QueryLogs (#42240 (comment)), while there is one in the marginalia. It was costly to use it before in the gem, but we made it fast (basecamp/marginalia#138), so it is not that costly and people deliberately decide if they want to use it.

I would like to have the :line option (upd: decided to name it as :source_location), it is very much useful. Without it, if we have a slow db query log, for example, it is very hard to know from which part of the codebase the query was generated. Even having an :action component, you need to investigate the unfamiliar codebase to find out where the query was triggered.

cc @byroot

end
else
def query_source_location # :nodoc:
LogSubscriber.backtrace_cleaner.clean(caller(1).lazy).first

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this lazy make much sense. Might as well explicitly do caller_location(1).each.

if Thread.respond_to?(:each_caller_location)
def query_source_location # :nodoc:
Thread.each_caller_location do |location|
frame = LogSubscriber.backtrace_cleaner.clean_frame(location)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
frame = LogSubscriber.backtrace_cleaner.clean_frame(location)
frame = LogSubscriber.backtrace_cleaner.clean_frame(location.path)

Might as well pass only the path, no?

module QueryLogs
mattr_accessor :taggings, instance_accessor: false, default: {}
mattr_accessor :taggings, instance_accessor: false, default: {
line: -> { query_source_location }

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be set in activerecord/railtie.rb like the others, otherwise it risks being overidden by users.

@fatkodima

Copy link
Copy Markdown
Member Author

Updated, please take a look.

@byroot byroot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks OK now, but there is a surprising amount of tests failures. I suspect they're not related but might be worth trying a rebase?

# * +socket+
# * +db_host+
# * +database+
# * +line+

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I wonder if line really makes sense as a name, as it's not just the line, but the whole location. Maybe location or source_location?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I named it after the marginalia's :line, but yeah, changed it to :source_location.
CI is still red, but looks like is not related to this PR.

@fatkodima fatkodima changed the title Support :line tag option for query log tags Support :source_location tag option for query log tags Feb 5, 2024
@byroot

byroot commented Feb 6, 2024

Copy link
Copy Markdown
Member

It's weird because main is green, and you got the same failures after your rebase. I'll try to check if main is hosed.

@byroot

byroot commented Feb 6, 2024

Copy link
Copy Markdown
Member

Yeah, similar failures on other PRs...

@byroot

byroot commented Feb 6, 2024

Copy link
Copy Markdown
Member

Nevermind, it's a minitest regression, @yahonda pined it in #50978 ❤️

@byroot
byroot merged commit 0602e1e into rails:main Feb 6, 2024
@fatkodima
fatkodima deleted the query_logs-line branch February 6, 2024 10:05
@andyatkinson

Copy link
Copy Markdown
Member

🎉

@andyatkinson

Copy link
Copy Markdown
Member

Having line number logging is hugely helpful with Marginalia, so it's great to have this within Rails!

I added this to my Rideshare app I used for testing and for my book, Rails 8, source_location enabled, works great.

  1. Using Postgres
  2. Cleared out pg_stat_statements using the pg_stat_statements_reset() function
  3. Ran a "simulate activity" rake task in the app that sends requests to API endpoints to generate requests and DB queries.
  4. Looked at the new results captured in pg_stat_statements

Here's an example annotation from the end of one of the result rows:

/*action='create',application='Rideshare',controller='trip_requests',source_location='app%2Fcontrollers%2Fapi%2Ftrip_requests_controller.rb%3A4%3Ain%20%60create%27'*/

Here it's unescaped:

 CGI.unescape('app%2Fcontrollers%2Fapi%2Ftrip_requests_controller.rb%3A4%3Ain%20%60create%27')
=> "app/controllers/api/trip_requests_controller.rb:4:in `create'"

Works great! Thanks for your work on this @fatkodima @byroot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants