Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reconcile possible duplicated columns in object and versioned object tables #480

Closed
0xavi0 opened this issue Apr 26, 2023 · 1 comment · Fixed by aquarist-labs/ceph#150
Closed

Reconcile possible duplicated columns in object and versioned object tables #480

0xavi0 opened this issue Apr 26, 2023 · 1 comment · Fixed by aquarist-labs/ceph#150
Assignees
@0xavi0
Labels
area/rgw-sfs RGW & SFS related kind/quality Quality improvements, Refactoring, Automation via CI, E2E, Integration, CLI or REST API
Milestone
v0.17.0

Comments

@0xavi0
Copy link
Contributor

0xavi0 commented Apr 26, 2023

Right now there are a few columns in objects and versioned objects tables in sqlite that are repeated or describe the same thing.

We should decide which table holds those columns.

Columns we can discuss about:

  • size
  • etag
  • creation_time
  • mtime
  • set_mtime
  • detele_at_time
@0xavi0 0xavi0 added the kind/quality Quality improvements, Refactoring, Automation via CI, E2E, Integration, CLI or REST API label Apr 26, 2023
@irq0
Copy link
Contributor

irq0 commented May 8, 2023

https://github.com/aquarist-labs/s3gw/pull/497, design review 2023-05-05 concluded that we use the versioned object for metadata. Changes:

           sqlite_orm::make_column("bucket_id", &DBObject::bucket_id),
           sqlite_orm::make_column("name", &DBObject::name),
-          sqlite_orm::make_column("size", &DBObject::size),
-          sqlite_orm::make_column("etag", &DBObject::etag),
-          sqlite_orm::make_column("mtime", &DBObject::mtime),
-          sqlite_orm::make_column("set_mtime", &DBObject::set_mtime),
-          sqlite_orm::make_column("delete_at_time", &DBObject::delete_at_time),

@jhmarina jhmarina changed the title [FR] Reconcile possible duplicated columns in object and versioned object tables Reconcile possible duplicated columns in object and versioned object tables May 9, 2023
@0xavi0 0xavi0 self-assigned this May 15, 2023
@jhmarina jhmarina added this to the v0.17.0 milestone May 15, 2023
This was referenced May 15, 2023
Closed
Closed
@votdev votdev added the area/rgw-sfs RGW & SFS related label May 16, 2023
0xavi0 referenced this issue in 0xavi0/ceph May 16, 2023
@0xavi0
rgw/sfs: change db schema as described in ADR.
8e6cf59 
Changes the db schema following [this
ADR](https://github.com/aquarist-labs/s3gw/pull/497).

It also deletes repeated fields that we had in objects vs
versioned_objects tables.

Logic of `sfs` was intentionally not updated in this PR.
(For example new field `version_type` is added to the schema, but still
not used).
Updates in logic, specially in versioning logic, will come in a future
PR.

New Features:

**Sqliteorm type bindings for the following types.**
* `uuid_d`
* `ceph::real_time`
* `rgw::sal::sfs::ObjetState`
* `rgw::sal::sfs::VersionType`

In general, any enum type that has a `LAST_VALUE` item assigned to the
last valid value and that also starts with 0 is eligible to be binded to
sqliteorm, as the type binding for enums is generic.

For example: you can create an enum like:

```c++
enum class TestEnum  {
  WHATEVER = 0,
  SOMETHING_ELSE,
  AND_THIS,
  LAST_VALUE = AND_THIS
};
```
and use it right away in sqliteorm.

**Object has no explicit conversion code**
As all the types in the `object` are compatible with `sqliteorm` we
don't need extra conversion layer.
This is a preview of what we can do with the rest of database objects
once `BLOBS` are also type binded in the future.
That was not done in this PR to avoid too many changes.

Note: The new type `VersionType` was added as a new file `version_type.h`
because we are in a re-design process and I wasn't sure if it should be
located in any other existing file that might be eligible to be deleted
in the near future.

Fixes: https://github.com/aquarist-labs/s3gw/issues/480

Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
@0xavi0 0xavi0 mentioned this issue May 16, 2023
Merged
11 tasks
0xavi0 referenced this issue in 0xavi0/ceph May 16, 2023
@0xavi0
rgw/sfs: change db schema as described in ADR.
b726c3f 
Changes the db schema following [this
ADR](https://github.com/aquarist-labs/s3gw/pull/497).

It also deletes repeated fields that we had in objects vs
versioned_objects tables.

Logic of `sfs` was intentionally not updated in this PR.
(For example new field `version_type` is added to the schema, but still
not used).
Updates in logic, specially in versioning logic, will come in a future
PR.

New Features:

**Sqliteorm type bindings for the following types.**
* `uuid_d`
* `ceph::real_time`
* `rgw::sal::sfs::ObjetState`
* `rgw::sal::sfs::VersionType`

In general, any enum type that has a `LAST_VALUE` item assigned to the
last valid value and that also starts with 0 is eligible to be binded to
sqliteorm, as the type binding for enums is generic.

For example: you can create an enum like:

```c++
enum class TestEnum  {
  WHATEVER = 0,
  SOMETHING_ELSE,
  AND_THIS,
  LAST_VALUE = AND_THIS
};
```
and use it right away in sqliteorm.

**Object has no explicit conversion code**
As all the types in the `object` are compatible with `sqliteorm` we
don't need extra conversion layer.
This is a preview of what we can do with the rest of database objects
once `BLOBS` are also type binded in the future.
That was not done in this PR to avoid too many changes.

Note: The new type `VersionType` was added as a new file `version_type.h`
because we are in a re-design process and I wasn't sure if it should be
located in any other existing file that might be eligible to be deleted
in the near future.

Fixes: https://github.com/aquarist-labs/s3gw/issues/480

Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
0xavi0 referenced this issue in 0xavi0/ceph May 16, 2023
@0xavi0
rgw/sfs: change db schema as described in ADR.
e805143 
Changes the db schema following [this
ADR](https://github.com/aquarist-labs/s3gw/pull/497).

It also deletes repeated fields that we had in objects vs
versioned_objects tables.

Logic of `sfs` was intentionally not updated in this PR.
(For example new field `version_type` is added to the schema, but still
not used).
Updates in logic, specially in versioning logic, will come in a future
PR.

New Features:

**Sqliteorm type bindings for the following types.**
* `uuid_d`
* `ceph::real_time`
* `rgw::sal::sfs::ObjetState`
* `rgw::sal::sfs::VersionType`

In general, any enum type that has a `LAST_VALUE` item assigned to the
last valid value and that also starts with 0 is eligible to be binded to
sqliteorm, as the type binding for enums is generic.

For example: you can create an enum like:

```c++
enum class TestEnum  {
  WHATEVER = 0,
  SOMETHING_ELSE,
  AND_THIS,
  LAST_VALUE = AND_THIS
};
```
and use it right away in sqliteorm.

**Object has no explicit conversion code**
As all the types in the `object` are compatible with `sqliteorm` we
don't need extra conversion layer.
This is a preview of what we can do with the rest of database objects
once `BLOBS` are also type binded in the future.
That was not done in this PR to avoid too many changes.

Note: The new type `VersionType` was added as a new file `version_type.h`
because we are in a re-design process and I wasn't sure if it should be
located in any other existing file that might be eligible to be deleted
in the near future.

Fixes: https://github.com/aquarist-labs/s3gw/issues/480

Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
0xavi0 referenced this issue in aquarist-labs/ceph Oct 5, 2023
@0xavi0
rgw/sfs: change db schema as described in ADR.
b641e93 
Changes the db schema following [this
ADR](https://github.com/aquarist-labs/s3gw/pull/497).

It also deletes repeated fields that we had in objects vs
versioned_objects tables.

Logic of `sfs` was intentionally not updated in this PR.
(For example new field `version_type` is added to the schema, but still
not used).
Updates in logic, specially in versioning logic, will come in a future
PR.

New Features:

**Sqliteorm type bindings for the following types.**
* `uuid_d`
* `ceph::real_time`
* `rgw::sal::sfs::ObjetState`
* `rgw::sal::sfs::VersionType`

In general, any enum type that has a `LAST_VALUE` item assigned to the
last valid value and that also starts with 0 is eligible to be binded to
sqliteorm, as the type binding for enums is generic.

For example: you can create an enum like:

```c++
enum class TestEnum  {
  WHATEVER = 0,
  SOMETHING_ELSE,
  AND_THIS,
  LAST_VALUE = AND_THIS
};
```
and use it right away in sqliteorm.

**Object has no explicit conversion code**
As all the types in the `object` are compatible with `sqliteorm` we
don't need extra conversion layer.
This is a preview of what we can do with the rest of database objects
once `BLOBS` are also type binded in the future.
That was not done in this PR to avoid too many changes.

Note: The new type `VersionType` was added as a new file `version_type.h`
because we are in a re-design process and I wasn't sure if it should be
located in any other existing file that might be eligible to be deleted
in the near future.

Fixes: https://github.com/aquarist-labs/s3gw/issues/480

Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
0xavi0 referenced this issue in 0xavi0/ceph Oct 5, 2023
@0xavi0
rgw/sfs: change db schema as described in ADR.
26e826e 
Changes the db schema following [this
ADR](https://github.com/aquarist-labs/s3gw/pull/497).

It also deletes repeated fields that we had in objects vs
versioned_objects tables.

Logic of `sfs` was intentionally not updated in this PR.
(For example new field `version_type` is added to the schema, but still
not used).
Updates in logic, specially in versioning logic, will come in a future
PR.

New Features:

**Sqliteorm type bindings for the following types.**
* `uuid_d`
* `ceph::real_time`
* `rgw::sal::sfs::ObjetState`
* `rgw::sal::sfs::VersionType`

In general, any enum type that has a `LAST_VALUE` item assigned to the
last valid value and that also starts with 0 is eligible to be binded to
sqliteorm, as the type binding for enums is generic.

For example: you can create an enum like:

```c++
enum class TestEnum  {
  WHATEVER = 0,
  SOMETHING_ELSE,
  AND_THIS,
  LAST_VALUE = AND_THIS
};
```
and use it right away in sqliteorm.

**Object has no explicit conversion code**
As all the types in the `object` are compatible with `sqliteorm` we
don't need extra conversion layer.
This is a preview of what we can do with the rest of database objects
once `BLOBS` are also type binded in the future.
That was not done in this PR to avoid too many changes.

Note: The new type `VersionType` was added as a new file `version_type.h`
because we are in a re-design process and I wasn't sure if it should be
located in any other existing file that might be eligible to be deleted
in the near future.

Fixes: https://github.com/aquarist-labs/s3gw/issues/480

Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
0xavi0 referenced this issue in aquarist-labs/ceph Oct 18, 2023
@0xavi0
rgw/sfs: change db schema as described in ADR.
4130d0a 
Changes the db schema following [this
ADR](https://github.com/aquarist-labs/s3gw/pull/497).

It also deletes repeated fields that we had in objects vs
versioned_objects tables.

Logic of `sfs` was intentionally not updated in this PR.
(For example new field `version_type` is added to the schema, but still
not used).
Updates in logic, specially in versioning logic, will come in a future
PR.

New Features:

**Sqliteorm type bindings for the following types.**
* `uuid_d`
* `ceph::real_time`
* `rgw::sal::sfs::ObjetState`
* `rgw::sal::sfs::VersionType`

In general, any enum type that has a `LAST_VALUE` item assigned to the
last valid value and that also starts with 0 is eligible to be binded to
sqliteorm, as the type binding for enums is generic.

For example: you can create an enum like:

```c++
enum class TestEnum  {
  WHATEVER = 0,
  SOMETHING_ELSE,
  AND_THIS,
  LAST_VALUE = AND_THIS
};
```
and use it right away in sqliteorm.

**Object has no explicit conversion code**
As all the types in the `object` are compatible with `sqliteorm` we
don't need extra conversion layer.
This is a preview of what we can do with the rest of database objects
once `BLOBS` are also type binded in the future.
That was not done in this PR to avoid too many changes.

Note: The new type `VersionType` was added as a new file `version_type.h`
because we are in a re-design process and I wasn't sure if it should be
located in any other existing file that might be eligible to be deleted
in the near future.

Fixes: https://github.com/aquarist-labs/s3gw/issues/480

Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@0xavi0 0xavi0

Labels
area/rgw-sfs RGW & SFS related kind/quality Quality improvements, Refactoring, Automation via CI, E2E, Integration, CLI or REST API
Projects
None yet
Milestone
v0.17.0
Development

Successfully merging a pull request may close this issue.

rgw/sfs: change db schema as described in ADR. 0xavi0/ceph
4 participants
@irq0 @votdev @jhmarina @0xavi0