Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add listener API that notifies on IOError #9177

Closed

Conversation

akankshamahajan15
Copy link
Contributor

Summary: Add a new API in listener.h that notifies about IOErrors on
Read/Write/Append/Flush etc. The API reports about IOStatus, filename, Operation
name, offset and length.

Test Plan: In progress

Reviewers:

Subscribers:

Tasks:

Tags:

@facebook-github-bot
Copy link
Contributor

@akankshamahajan15 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@akankshamahajan15 has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@akankshamahajan15 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

include/rocksdb/listener.h Outdated Show resolved Hide resolved
file/writable_file_writer.h Show resolved Hide resolved
file/writable_file_writer.h Show resolved Hide resolved
file/writable_file_writer.h Outdated Show resolved Hide resolved
Comment on lines 222 to +223
NotifyOnFileTruncateFinish(start_ts, finish_ts, s);
if (!interim.ok()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure this is related to this specific change but this code looks generally broken to me. The NotifyOnFiileTruncateFinish passes in the status "s", but that status is not the status of Flush() at line 205. The same is true later for NotifyOnFileSyncFinish and NotifyOnFileCloseFinish.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. It doesn't seems consistent. Here we are passing the status of Flush() but at other places we are passing the status returned by FileSystem Operation.

Comment on lines 243 to 246
if (!interim.ok()) {
NotifyOnIOError(interim, "Fsync", file_name());
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would the code end up being simpler if in all of the NotifyOnFileXXX methods, if the input status was an error, you called NotifyOnIOError? There seems like places now that are missing the NotifyOnIOError (cases in Flush() and Append()) ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added the error notification when writable_file_-> calls the underlying File system APIs. I will update the PR to specify this. So in Flush(), it will report error when it calls writable_file_->Flush.

Copy link
Contributor

@anand1976 anand1976 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a follow-up PR, we should also notify on file open failure.

Summary: Add a new API in listener.h that notifies about IOErrors on
Read/Write/Append/Flush etc. The API reports about IOStatus, filename, Operation
name, offset and length.

Test Plan: In progress

Reviewers:

Subscribers:

Tasks:

Tags:
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
@akankshamahajan15
Copy link
Contributor Author

akankshamahajan15 commented Nov 18, 2021

In a follow-up PR, we should also notify on file open failure.

@anand1976 you mean whenever we call fs->NewRandomAccessFile or fs->NewWritableFile in rocksdb (in different code paths) we want to report the error? I don't think we have access to options.listeners everywhere. But wherever we have, I can add that for now.

@facebook-github-bot
Copy link
Contributor

@akankshamahajan15 has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@akankshamahajan15 has updated the pull request. You must reimport the pull request before landing.

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
@facebook-github-bot
Copy link
Contributor

@akankshamahajan15 has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@akankshamahajan15 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Copy link
Contributor

@anand1976 anand1976 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
@facebook-github-bot
Copy link
Contributor

@akankshamahajan15 has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@akankshamahajan15 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

1 similar comment
@facebook-github-bot
Copy link
Contributor

@akankshamahajan15 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@akankshamahajan15 merged this pull request in 4a7c1dc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants