Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unify yb::Env and rocksdb::Env #1661

Open
ttyusupov opened this issue Jun 30, 2019 · 0 comments
Open

Unify yb::Env and rocksdb::Env #1661

ttyusupov opened this issue Jun 30, 2019 · 0 comments
Assignees
@ttyusupov
Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue

Comments

@ttyusupov
Copy link
Contributor

ttyusupov commented Jun 30, 2019 •
edited by yugabyte-ci
Loading

Jira Link: DB-2523
Currently, we have both yb::Env and rocksdb::Env which are originally inherited from LevelDB, but diverged other time. Although most of the code is the same and when we need to add new env or file-related functionality to use on multiple layers - we have to do it both in yb::Env and rocksdb::Env. So, the goal is to unify this code and keep it in one place.

Also Env functionality scope is too broad - it contains code for working with filesystem, with threads and something else. It would be cleaner and more effective for future development to extract filesystem-related code into a separate class.
@ttyusupov ttyusupov self-assigned this Jun 30, 2019
yugabyte-ci pushed a commit that referenced this issue Jun 30, 2019
@ttyusupov
#1661: Replaced gscoped_ptr with std::unique_ptr for yb::Env.
5adb348 
Summary: Replaced `gscoped_ptr` with `std::unique_ptr` for `yb::Env`. This is needed as a prerequisite step for unifying `yb::Env` and `rocksdb::Env`.

Test Plan: Jenkins

Reviewers: sergei, mikhail

Reviewed By: mikhail

Subscribers: ybase, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D6835
@rkarthik007 rkarthik007 added the area/docdb YugabyteDB core features label Jul 1, 2019
yugabyte-ci pushed a commit that referenced this issue Jul 5, 2019
@ttyusupov
#1661: Unified yb::SequentialFile and rocksdb::SequentialFile
ad0d41c 
Summary: Unified `yb::SequentialFile` and `rocksdb::SequentialFile` as a part of `Env` unification.

Test Plan: Jenkins

Reviewers: mikhail, sergei

Reviewed By: sergei

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D6838
yugabyte-ci pushed a commit that referenced this issue Jul 5, 2019
@ttyusupov
#1661: Fixed OS_LINUX ifdef by replacing to __linux__
3b23e5b 
Summary: Due to incorrect rebase of ad0d41c, obsolete  `OS_LINUX` ifdef was introduced again. Replaced with `__linux__` and included missing headers.

Test Plan: Jenkins

Reviewers: mikhail, sergei, alex

Reviewed By: alex

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D6862
mbautin pushed a commit that referenced this issue Jul 11, 2019
@ttyusupov @mbautin
This commit provides additional functionality that is logically relat…
124b6c2 
…ed to the

earlier commit 5adb348

Original commit message:

#1661: Replaced `gscoped_ptr` with `std::unique_ptr` for `yb::Env`.

Summary: Replaced `gscoped_ptr` with `std::unique_ptr` for `yb::Env`. This is needed as a prerequisite step for unifying `yb::Env` and `rocksdb::Env`.

Test Plan: Jenkins

Reviewers: sergei, mikhail

Reviewed By: mikhail

Subscribers: ybase, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D6835
mbautin pushed a commit that referenced this issue Jul 11, 2019
@ttyusupov @mbautin
This commit provides additional functionality that is logically relat…
787c4a2 
…ed to the

earlier commit ad0d41c

Original commit message:

#1661: Unified yb::SequentialFile and rocksdb::SequentialFile

Summary: Unified `yb::SequentialFile` and `rocksdb::SequentialFile` as a part of `Env` unification.

Test Plan: Jenkins

Reviewers: mikhail, sergei

Reviewed By: sergei

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D6838
mbautin pushed a commit to mbautin/yugabyte-db that referenced this issue Jul 16, 2019
@ttyusupov
yugabyte#1661: Replaced gscoped_ptr with std::unique_ptr for `yb:…
93fe083 
…:Env`.

Summary: Replaced `gscoped_ptr` with `std::unique_ptr` for `yb::Env`. This is needed as a prerequisite step for unifying `yb::Env` and `rocksdb::Env`.

Test Plan: Jenkins

Reviewers: sergei, mikhail

Reviewed By: mikhail

Subscribers: ybase, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D6835

Note:
This commit provides additional functionality that is logically related to
the earlier commit yugabyte@5adb348
and supersedes the commit yugabyte@124b6c2
mbautin pushed a commit to mbautin/yugabyte-db that referenced this issue Jul 16, 2019
@ttyusupov
yugabyte#1661: Unified yb::SequentialFile and rocksdb::SequentialFile
5963934 
Summary: Unified `yb::SequentialFile` and `rocksdb::SequentialFile` as a part of `Env` unification.

Test Plan: Jenkins

Reviewers: mikhail, sergei

Reviewed By: sergei

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D6838

Note:
This commit provides additional functionality that is logically related to
the earlier commit yugabyte@ad0d41c
and supersedes the commit yugabyte@787c4a2
yugabyte-ci pushed a commit that referenced this issue Jul 17, 2019
@ttyusupov
#1661: Unified yb::RandomAccessFile and rocksdb::RandomAccessFile
2c46126 
Summary: Unified `yb::RandomAccessFile` and `rocksdb::RandomAccessFile` as a part of `Env` unification.

Test Plan: Jenkins

Reviewers: mikhail, sergei

Reviewed By: sergei

Subscribers: ybase, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D6863
ttyusupov added a commit that referenced this issue Aug 1, 2019
@ttyusupov
#1661: Move iostats and perf_level from rocksdb to util/stats
03c7ae0 
Summary: Moved iostats and related utility classes from rocksdb to util/stats, so we can unify `yb::WritableFile` and `rocksdb::WritableFile` in next diff as a part of `Env` unification.

Test Plan: Jenkins

Reviewers: mikhail, sergei

Reviewed By: sergei

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D6861
@yugabyte-ci yugabyte-ci added kind/bug This issue is a bug priority/medium Medium priority issue labels Jun 9, 2022
@ttyusupov ttyusupov added kind/enhancement This is an enhancement of an existing feature and removed kind/bug This issue is a bug labels Jul 20, 2022
@yugabyte-ci yugabyte-ci added kind/bug This issue is a bug and removed kind/enhancement This is an enhancement of an existing feature labels Jul 20, 2022
ttyusupov added a commit that referenced this issue Jul 26, 2022
@ttyusupov
[#1661] docdb: Extracted Env::IOPriority from rocksdb::Env into separ…
21b333a 
…ate enum inside yb/util

Summary: Extracted Env::IOPriority from rocksdb::Env into separate enum inside yb/util as a part of effort to separate filesystem-related logic from Env and unify rocksdb::Env and yb::Env.

Test Plan: Jenkins

Reviewers: mbautin

Reviewed By: mbautin

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D18459
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature and removed kind/bug This issue is a bug labels Jul 29, 2022
ttyusupov added a commit that referenced this issue Aug 17, 2022
@ttyusupov
[#1661] docdb: Moved rocksdb::WritableFile to yb/util
9c5ce39 
Summary:
Made the following changes as a part of effort to separate filesystem-related logic from Env and unify `rocksdb::Env` and `yb::Env`:

1) Moved `use_mmap_writes`, `allow_fallocate`, `fallocate_with_keep_size` options from `rocksdb::EnvOptions` to `yb::FileSystemOptions`.
2) Moved `TEST_KILL_RANDOM` test macros and related code from `rocksdb/util/sync_point` to `yb/util/test_kill`.
3) Moved `rocksdb::WritableFile` and subclasses from yb/rocksdb/util to yb/util/file_system*.

No logic has been changed as a part of this refactoring revision, unification of `rocksdb::WritableFile` and `yb::WritableFile` is to be done in a separate diff for easier and faster review cycles.

Test Plan: Jenkins

Reviewers: mbautin

Reviewed By: mbautin

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D18560
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ttyusupov ttyusupov

Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ttyusupov @rkarthik007 @yugabyte-ci