Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Bucket listing API should include non-default bucket types #463

Open
michellep-basho opened this Issue Jan 3, 2014 · 13 comments

Comments

Projects
None yet
5 participants

DESCRIPTION
Per Randy:
As of 2.0 build pre7 the list buckets operation only includes buckets of the 'default' bucket type. If bucket types are added (say through usage of Yokozuna), those buckets are no longer listed. There is a riak-admin command which uses internal Erlang APIs to list bucket types; but this is only useable to operational staff using Riak. This change also breaks developer / web admin tools (such as rekon), used for data modeling.

IMPACT:
We have 4 known customers potentially impacted by this. Additionally, it impact the ProServ developers. Please contact me or Randy S for further more details.

Contributor

seancribbs commented Jan 3, 2014

We have tests that demonstrate that listing buckets within known bucket-types works. Whether bucket-types themselves should be able to be listed from a client API is a separate issue, I believe (caveats around security/disclosure, etc).

Contributor

Vagabond commented Jan 3, 2014

We can certainly add a permission to allow you to list bucket types, along the lines of the permission that allows bucket listing.

Contributor

jrwest commented Jan 3, 2014

We have discussed a list bucket types API previously as part of the larger bucket type API discussion. We are aware that several features are missing from that API and do not plan to expose any of it this release (the only way to work w/ bucket types should be the command line w/ Riak 2.0). I believe we plan to revist and fully implement the API in the future (caveats re: security).

Additionally, we need to make a distinction between "listing bucket types" and "listing buckets under a type". The current list bucket operation maps to "listing buckets under the default type". I believe we have extended that to including "listing buckets under any specific type" however, it sounds like there is a feature request for "listing buckets across multiple types at once"?

Sent from my iPhone

On Jan 3, 2014, at 11:40 AM, Andrew Thompson notifications@github.com wrote:

We can certainly add a permission to allow you to list bucket types, along the lines of the permission that allows bucket listing.


Reply to this email directly or view it on GitHub.

...however, it sounds like there is a feature request for "listing buckets across multiple types at once"?
yes

Contributor

jrwest commented Jan 3, 2014

ok. just wanted to clarify. considering that would be an entirely new feature I was confused by the report that customers are currently affected...Even with the current state of non-existence of the API one could continue to operate web admin tools that use the "listing buckets under/within a type" as @seancribbs mentioned. Riak 2.0 operates just as it used to under the default bucket type and web-admin tools and other applications should not be affected just because they exist.

In addition, bucket types, from a data modeling perspective, at least at this time, should not be created dynamically (something that would obviously be difficult anyways since there is no write operations via API currently). This should make it easy to enumerate them all even w/o an API to do so. Tools may continue to operate but a w/ degraded user experience but they are not broken. Once the API is available they can of course be improved, and that is why we would like to provide the API in a future release.

I would be curious to know more about the customers that may potentially have issues w/ this to see if we can find a workaround, in the meantime, as well as to ensure we build the API they want, in the future. Would be happy to discuss that elsewhere.

Regardless, I think it would be should to use this issue to discuss the future work on the bucket types API, so we have a central place. It has been scattered across several issues which makes the discussion harder to follow.

For completeness, here is the current state of bucket types as has been shown to users (the only mention of an API is there isn't one): https://gist.github.com/jrwest/3652ecd1db609e25cdaa

[1] https://github.com/basho/riak_test/blob/master/tests/bucket_types.erl#L125

I will ask Randy to weigh in more on the existing customer use cases and security considerations, so we have more context. I understand that there are other things we need to consider like security, but I can also see how from a customer perspective it might seem like basic functionality is missing in this initial implementation. thanks

Contributor

jrwest commented Jan 3, 2014

Sounds good. I agree there is functionality missing. Unfortunately, it was a casualty of the release cycle and the lack of time necessary to really map out how things played w/ security on/off, etc. Customers also probably won't love that we are using JSON on the command line either, but we were unable to fix that in time as well...

Randy is available for next Tue's core/kv mumble. We can discuss more.

Contributor

jrwest commented Jan 4, 2014

Unfortunately I probably won't be able to make that mumble -- not that I am required for the conversation but fyi. I will be out of town. I may be able to set something up for the evening (pacific) while I'm away otherwise I will be back the following Tuesday.

Let's plan on discussing this the following week when Jordan is back. thanks

Contributor

jaredmorrow commented Mar 24, 2014

Any status on this since the last planned meeting? (3 months ago)

@jaredmorrow jaredmorrow added this to the 2.0-RC milestone Mar 24, 2014

Contributor

jrwest commented Mar 24, 2014

The tl;dr is there is no user-facing API for bucket types (code exists for some of it, but it is not documented or supported in Riak 2.0). We should use this issue to track work for 2.1 to add it.

@jrwest jrwest modified the milestones: 2.1, 2.0-RC Mar 24, 2014

Contributor

jrwest commented Mar 24, 2014

moved to 2.1 release. @michellep @gburd happy to discuss again if necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment