Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test refactoring for Backups+Temperatures #9655

Closed
wants to merge 5 commits into from

Conversation

pdillinger
Copy link
Contributor

@pdillinger pdillinger commented Mar 3, 2022

Summary:
In preparation for more support for file Temperatures in BackupEngine,
this change does some test refactoring:

  • Move DBTest2::BackupFileTemperature test to
    BackupEngineTest::FileTemperatures, with some updates to make it work
    in the new home. This test will soon be expanded for deeper backup work.
  • Move FileTemperatureTestFS from db_test2.cc to db_test_util.h, to
    support sharing because of above moved test, but split off the "no link"
    part to the test needing it.
  • Use custom FileSystems in backupable_db_test rather than custom Envs,
    because going through Env file interfaces doesn't support temperatures.
  • Fix RemapFileSystem to map DirFsyncOptions::renamed_new_name
    parameter to FsyncWithDirOptions, which was required because this
    limitation caused a crash only after moving to higher fidelity of
    FileSystem interface (vs. LegacyDirectoryWrapper throwing away some
    parameter details)
  • backupable_options_ -> engine_options_ as part of the ongoing
    work to get rid of the obsolete "backupable" naming.

Test Plan: test code updates only

Summary:
In preparation for more support for file Temperatures in BackupEngine,
this change does some test refactoring:
* Move DBTest2::BackupFileTemperature test to
BackupEngineTest::FileTemperatures, which some updates to make it work
in the new home. This test will soon be expanded for deeper backup work.
* Move FileTemperatureTestFS from db_test2.cc to db_test_util.h, to
support sharing because of above moved test.
* Use custom FileSystems in backupable_db_test rather than custom Envs,
because going through Env file interfaces doesn't support temperatures.
* Fix RemapFileSystem to map DirFsyncOptions::renamed_new_name
parameter to FsyncWithDirOptions, which was required because this
limitation caused a crash only after moving to higher fidelity of
FileSystem interface (vs. LegacyDirectoryWrapper throwing away some
parameter details)

Test Plan: test code updates only
@facebook-github-bot
Copy link
Contributor

@pdillinger has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@pdillinger has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@pdillinger has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

}
};

TEST_F(DBTest2, BackupFileTemperature) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the refactor. I think we can also remove the include:

#include "rocksdb/utilities/backup_engine.h"

logger_options.env = db_chroot_env_.get();
// TODO: This should really be an EXPECT_OK, but this CreateLogger fails
// regularly in some environments with "no such directory"
CreateLoggerFromOptions(dbname_, logger_options, &logger_)
Copy link
Contributor

@jay-zhuang jay-zhuang Mar 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm getting the following exception on macos occasionally, the logger_ is not a valid Logger, could it because it's not created here?:
image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure. I fixed the non-OK issue on Linux. See if that works.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix a UAF based on ASAN trace so should be fixed now.

@facebook-github-bot
Copy link
Contributor

@pdillinger has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@pdillinger has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@pdillinger has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@pdillinger has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Copy link
Contributor

@jay-zhuang jay-zhuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants