Adding support for more than 1 bucket/catalog

[lefebsy]

Hello,

Some other Iceberg catalog implementations are allowing to define more than one bucket by catalog.

First reaction : it is not a key feature

Then I tried to register in my new Lakekeeper catalog some tables previously associated with an other catalog I want to leave. I figure that I will need to move data to Lakekeeper defined bucket, instead of simply add the old storage in the list of my lakekeeper catalog.
Because when I run the register procedure https://iceberg.apache.org/docs/latest/spark-procedures/#register_table I encounter the mismatch bucket conflict...

So multi-bucket support could avoid to move data in this scenario. There is probably other interesting use cases :-)

[v-kessler]

@lefebsy interesting. What was the reason to use multiple buckets per a single warehouse?

[Aloha-dancer]

I guess this feature would be really nice.
Firstly, it is really a user friendly approach, when you can migrate all the LakeHouse meta and data from something like HMS to LakeKeeper simply by switching already existing object to the last one.
Secondly, I guess it is more native approach to store different kind of data into separated buckets for better data management across s3. Also in my case(I have LakeHouse storage architecture on self hosted MINIO S3 with Iceberg) and i usually run into the case, when big amount of snapshots and orphan files greatly slows down queries across one bucket, while others were functioning as usual (or a bit slower but only a bit)
And finally i guess that the concept of the REST catalog architecture is not about to store all data into single bucket, but about managing metadata in the stateless way, whenever data can be distributed in any way
So, would you please answer, if it is possible to provide such feature?(╥﹏╥)