Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add profile events for fsync #39179

Merged
merged 1 commit into from Aug 2, 2022
Merged

Add profile events for fsync #39179

merged 1 commit into from Aug 2, 2022

Conversation

azat
Copy link
Collaborator

@azat azat commented Jul 13, 2022

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Add profile events for fsync

The following new provile events had been added:

  • FileSync - Number of times the F_FULLFSYNC/fsync/fdatasync function was called for files.
  • DirectorySync - Number of times the F_FULLFSYNC/fsync/fdatasync function was called for directories.
  • FileSyncElapsedMicroseconds - Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for files.
  • DirectorySyncElapsedMicroseconds - Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for directories.

@robot-clickhouse robot-clickhouse added the pr-improvement Pull request with some product improvements label Jul 13, 2022
@azat azat force-pushed the fsync-profile-events branch 2 times, most recently from 219e7a1 to d621843 Compare July 14, 2022 04:00
@azat
Copy link
Collaborator Author

azat commented Jul 14, 2022

Stateless tests (release, s3 storage, actions) — fail: 1, passed: 4208, skipped: 58

  • 02361_fsync_profile_events - s3 does not have fsync, added no-s3-storage tag

@azat
Copy link
Collaborator Author

azat commented Jul 21, 2022

Can someone please take a look?

@azat azat force-pushed the fsync-profile-events branch 2 times, most recently from 715f001 to 93d7bf0 Compare July 30, 2022 09:26
@alexey-milovidov alexey-milovidov self-assigned this Jul 30, 2022
alexey-milovidov
alexey-milovidov approved these changes Jul 30, 2022
View reviewed changes
Copy link
Member

@alexey-milovidov alexey-milovidov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything else LGTM.

@alexey-milovidov
Copy link
Member

#39164

@alexey-milovidov
Copy link
Member

@azat the test should be disabled for s3 storage.

@alexey-milovidov
Copy link
Member

@KochetovNicolai
this reproducing a logical error very easily, PTAL:
#39164 (comment)

Comment on #39164 Invalid number of columns in chunk pushed to OutputPort. Expected 3, found 4
SELECT NULL FROM
(SELECT [1048575, NULL] AS ax, 2147483648 AS c) t1 ARRAY JOIN ax
INNER JOIN (SELECT NULL AS c) t2 USING (c)

@devcrafter
please check your test, it is flaky due to non-deterministic order: https://s3.amazonaws.com/clickhouse-test-reports/39179/5bd90b71600931dec874cd547190cd8025eb0f10/stateless_tests__release_.html

@azat
Add profile events for fsync
3e627e2 
The following new provile events had been added:

- FileSync - Number of times the F_FULLFSYNC/fsync/fdatasync function was called for files.
- DirectorySync - Number of times the F_FULLFSYNC/fsync/fdatasync function was called for directories.
- FileSyncElapsedMicroseconds - Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for files.
- DirectorySyncElapsedMicroseconds - Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for directories.

v2: rewrite test to sh with retries
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
@azat
Copy link
Collaborator Author

azat commented Jul 31, 2022

@azat the test should be disabled for s3 storage.

Yep, incorrect comment after rewriting to .sh - fixed now.

@alexey-milovidov
Copy link
Member

2022-08-01 02:43:12 Thread 831 "clickhouse-serv" received signal SIGSEGV, Segmentation fault.
2022-08-01 02:43:12 [Switching to Thread 0x7f69d7f2d700 (LWP 54462)]
2022-08-01 02:43:12 0x000000000b3a8007 in __tsan_func_entry ()
2022-08-01 02:44:49 [Inferior 1 (process 660) detached]

Bug in TSan.

@alexey-milovidov alexey-milovidov merged commit 3165288 into ClickHouse:master Aug 2, 2022
@azat azat deleted the fsync-profile-events branch August 4, 2022 07:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alexey-milovidov alexey-milovidov alexey-milovidov approved these changes

Assignees

@alexey-milovidov alexey-milovidov

Labels
pr-improvement Pull request with some product improvements
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@azat @alexey-milovidov @robot-clickhouse