Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[YSQL] Catalog manager namespace and table name maps to contain YSQL tables #1476

Closed
ndeodhar opened this issue Jun 3, 2019 · 4 comments
Closed
Assignees
Milestone

Comments

@ndeodhar
Copy link
Contributor

@ndeodhar ndeodhar commented Jun 3, 2019

Currently, YSQL table information is maintained in namespace and table ID maps instead of namespace and table name maps and this causes problems while trying to look up tables by their names. YSQL table maps also need to store schema since the same database can have same table name in different schemas.

This is needed by CDC as well as some yb-admin commands.

@ndeodhar ndeodhar self-assigned this Jun 3, 2019
@ndeodhar ndeodhar added this to To do in YSQL via automation Jun 3, 2019
@ndeodhar ndeodhar added this to To Do in YBase features via automation Jun 3, 2019
@ndeodhar ndeodhar added this to the v2.0 milestone Jul 11, 2019
@ndeodhar ndeodhar moved this from To do to In progress in YSQL Jul 17, 2019
@m-iancu
Copy link
Contributor

@m-iancu m-iancu commented Jul 17, 2019

Related issue is that those maps need to be updated when the relevant metadata is altered.
For instance, currently, after running:

postgres=# create table foo(a int);
CREATE TABLE
postgres=# alter table foo rename to bar;
ALTER TABLE
postgres=# select * from bar;
 a
---
(0 rows)

postgres=# \d
        List of relations
 Schema | Name | Type  |  Owner
--------+------+-------+----------
 public | bar  | table | postgres
(1 row)

The UI will still show the old name:

image

Loading

d-uspenskiy added a commit that referenced this issue Sep 24, 2019
…m yb-admin to catalog_manager

Summary:
`yb-admin` reads `keyspace` and `table` names as command argument. And use it as a table identifier
when sends requests to `catalog_manager`.
For now there are three kind of databases: `cql`, `sql` and `redis`. And `keyspace` name can be non-unique
across different db types.
Instead of `keyspace` name its id should be used in requests to `catalog_manager`.
`yb-admin` resolves keyspace id by using its name and db type. Db type is added by `yb-admin` as a prefix to
`keyspace` name.

YCQL keyspace `test` -> ycql.test keyspace name
YSQL database `test` -> ysql.test keyspace name
If database type prefix is omitted `cql` database type is used as default value (or `redis` db type
in case keyspace name is equal to `system_redis`)

```
14:38 $ ./build/latest/bin/yb-admin list_tables_with_db_types
ycql.system.local
ycql.system.partitions
ycql.system.peers
ycql.system.size_estimates
ycql.system_auth.resource_role_permissions_index
ycql.system_auth.role_permissions
ycql.system_auth.roles
ycql.system_schema.aggregates
ycql.system_schema.columns
ycql.system_schema.functions
ycql.system_schema.indexes
ycql.system_schema.keyspaces
ycql.system_schema.sys.catalog
ycql.system_schema.tables
ycql.system_schema.triggers
ycql.system_schema.types
ycql.system_schema.views
ysql.postgres.sql_features
ysql.postgres.sql_implementation_info
ysql.postgres.sql_languages
ysql.postgres.sql_packages
...

14:44 $ ./build/latest/bin/yb-admin list_tablets cql.system local
Tablet UUID                           Range                                                   Leader
b414ed947d6f4720915a3620a361692a                                                              127.0.0.1:7100
```

**Note:** Now `yb-admin` is able to operate on `ysql` tables

Test Plan:
Run existed unit test
```
yb_build.sh --cxx-test yb-admin-test
```

Reviewers: neha

Reviewed By: neha

Subscribers: ybase, yql, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D7101
@d-uspenskiy
Copy link
Contributor

@d-uspenskiy d-uspenskiy commented Sep 24, 2019

Phase 1 is completed. yb-admin is able to operate on YSQL tables.
Phase 2 is in progress. YSQL has no table schema yet as a result multiple tables can has same name and yb-admin can't distinguish them

Loading

@kmuthukk
Copy link
Collaborator

@kmuthukk kmuthukk commented Sep 27, 2019

@d-uspenskiy -- increasing priority on this to help address issues like #1991.

Loading

d-uspenskiy added a commit that referenced this issue Nov 18, 2019
…ce name uniqueness

Summary:
`catalog_manager` uses single map to store all keypaces by name. In spite of the fact keyspaces can belong to different db types (SQL, CQL, Redis). As a result keyspace name is not not unique identifier. And SQL keyspaces (database name) are not added to this map to avoid conflicts with CQL and Redis keyspaces.
To make keyspace name unique identifier single map is divided into 3 individual maps (CQL, Redis, SQL).
Note: 4th map is used for `YQL_DATABASE_UNKNOWN`. It  is illegal case and it is guarded with `CHECK`
Also `namespace_type` is added into `YBTableName` structure as it stores keyspace name only

Test Plan: Run existed unit tests

Reviewers: hector, mihnea, neha

Reviewed By: neha

Subscribers: kannan, yql, ybase, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D7106
@d-uspenskiy
Copy link
Contributor

@d-uspenskiy d-uspenskiy commented Apr 15, 2020

yb-admin command for YSQL tables are now supported by e6318c0

Loading

YBase features automation moved this from To Do to Done Apr 15, 2020
YSQL automation moved this from In progress to Done Apr 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
YSQL
  
Done
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants