Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add INFO log for each table being fingerprinted #41365

Merged
merged 1 commit into from Apr 12, 2024

Conversation

calherries
Copy link
Contributor

@calherries calherries commented Apr 12, 2024

This PR adds INFO-level logs per table being fingerprinted. These logs will help debug performance issues with fingerprinting, because you can correlate issues with specific tables being fingerprinted.

If you add <Logger name="metabase.sync.analyze.fingerprint" level="INFO"/> to the log4j2.xml file, you'll get these additional lines in your logs when syncing a table for the first time:

[clojure-agent-send-off-pool-7] INFO  metabase.sync.analyze.fingerprint - Fingerprinting 3 fields in table Table 17 ''PUBLIC.USERS''
[clojure-agent-send-off-pool-7] INFO  metabase.sync.analyze.fingerprint - Fingerprinting 12 fields in table Table 18 ''PUBLIC.PEOPLE''
[clojure-agent-send-off-pool-7] INFO  metabase.sync.analyze.fingerprint - Fingerprinting 5 fields in table Table 19 ''PUBLIC.REVIEWS''
[clojure-agent-send-off-pool-7] INFO  metabase.sync.analyze.fingerprint - Fingerprinting 8 fields in table Table 20 ''PUBLIC.ORDERS''
[clojure-agent-send-off-pool-7] INFO  metabase.sync.analyze.fingerprint - Fingerprinting 5 fields in table Table 21 ''PUBLIC.VENUES''
[clojure-agent-send-off-pool-7] INFO  metabase.sync.analyze.fingerprint - Fingerprinting 1 fields in table Table 22 ''PUBLIC.CATEGORIES''
[clojure-agent-send-off-pool-7] INFO  metabase.sync.analyze.fingerprint - Fingerprinting 7 fields in table Table 23 ''PUBLIC.PRODUCTS''
[clojure-agent-send-off-pool-7] INFO  metabase.sync.analyze.fingerprint - Fingerprinting 3 fields in table Table 24 ''PUBLIC.CHECKINS''

Copy link

replay-io bot commented Apr 12, 2024

Status Complete ↗︎
Commit 6fa8f14
Results
⚠️ 12 Flaky
2395 Passed

@calherries calherries added the backport Automatically create PR on current release branch on merge label Apr 12, 2024
@calherries calherries requested a review from a team April 12, 2024 12:47
Comment on lines +198 to +199
(do
(log/infof "Fingerprinting %s fields in table %s" (count fields) (sync-util/name-for-logging table))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stylistically I'd prefer putting this as a _ binding at the start of the let block to avoid so many levels of nesting.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also a smaller diff 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mmm yeah that could look nicer. i'll try it next time

@calherries calherries merged commit 92181ab into master Apr 12, 2024
130 of 150 checks passed
@calherries calherries deleted the log-fingerprinting-per-table branch April 12, 2024 14:50
Copy link

@calherries Did you forget to add a milestone to the issue for this PR? When and where should I add a milestone?

@calherries calherries added this to the 0.49.6 milestone Apr 12, 2024
metabase-bot bot added a commit that referenced this pull request Apr 12, 2024
Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport Automatically create PR on current release branch on merge .Team/BackendComponents also known as BEC
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants