Are the federated metastores read-only?

[KochankovID]

I'm a little confused about the documentation. There is a READ_AND_WRITE_ON_DATABASE_WHITELIST setting that, in theory, should allow write operations with meta in federated metastores.

But also, there is a sentence in the Note section which says:

Metadata read operations are routed only. Write and destructive operations can be performed on the local metastore only.

[nskylark]

Moreover, there is a paragraph in the "Access control" section that states:
"There are a number of write operations in the metastore whose requests do not contain database/table name context, and so cannot be routed to federated metastore instances configured with a writeable access control level."
So it means that write to a federated metastore is possible but limited to a certain operations.
Please clarify.

[patduin]

Metadata read operations are routed only. Write and destructive

operations can be performed on the local metastore only. I'll update the sentence to mention access controls, We probably wrote this before we added the access control feature. Thanks for bringing that up. But you are correct with that setting you can do writes (alters) on tables on the list of databases configured. Creates are not allowed.

So it means that write to a federated metastore is possible but limited to

a certain operations.

Please clarify.

This holds for both read and write operations. Any thrift api endpoint request we get can only be routed if it contains a database name, so any endpoint that does not contain a database name will be routed to the primary by default. Example: get_open_txns ( https://github.com/ExpediaGroup/waggle-dance/blob/main/waggle-dance-core/src/main/java/com/hotels/bdp/waggledance/server/FederatedHMSHandler.java#L1202 ) This opens a transaction but we can't know on which metastore. These are limitations of using WaggleDance. The API hive provides is big so best to test if your use case fits most often used write operations like alters should all work. Hope this helps, Patrick Op wo 17 jan 2024 om 10:47 schreef nskylark ***@***.***>:

…

Moreover, there is a paragraph in the "Access control" section that states: "There are a number of write operations in the metastore whose requests do not contain database/table name context, and so cannot be routed to federated metastore instances configured with a writeable access control level." So it means that write to a federated metastore is possible but limited to a certain operations. Please clarify. — Reply to this email directly, view it on GitHub <#306 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAP6JGAZ2NNPJAJOZFT3RXLYO6M3JAVCNFSM6AAAAABB6GFV4CVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DCNJUGQ2TC> . You are receiving this because you are subscribed to this thread.Message ID: <ExpediaGroup/waggle-dance/repo-discussions/306/comments/8154451@ github.com>