List objects FSM v2 bugfix and tests #788
Conversation
- Two EQC properties for listing objects (GET Bucket),
one without delimiter and one with delimiter "/".
Those properties assert following two are equal:
1. keys from listing objects by repetitive usage of
riak_cs_list_objects_fsm_v2
2. active keys from generated [{State, Key},...]
- When counterexample is found, states and keys are
written to file $PWD/.riak_cs_list_objects_fsm_v2_eqc.txt.
Each line consists of line number, status and key.
Example output:
1,active,0123456/10
2,scheduled_delete,0123456/1000
- To generate manifests, generator of whole manifests is avoided
and generate only states and keys of manifests.
The property list-all-objects generates several thousands of nested tuples,
#lfs_manifest_v3 and #acl_v2 in this case.
But generators of nested tuples seem not so fast.
TODOs remaining
- prefix of request parameter
- non-default max-keys
Clean up the list objects EQC test and start threading the 2i request page size through spawn_link of the FSM. This allows us to test with constants far below 1002. 1002 is a nice size for production 2i queries, but is terrible for getting good shrinks out of QuickCheck.
Don't slice the result-set until `enough_results` has been determined.
Update the list objects FSM and EQC test to support the 'NextMarker' XML node. This is used when a `delimiter` is included in a request. The client uses this node to page through the next request, as opposed to simply using the last `Key` in the contents.
Only use the `skip_past_prefix_and_delimiter` trick if the common delimiter is already in the result-set. It will _not_ be in the result-set if the sole object which 'created' it is not in the `active` state.
|
Might be worth getting a third-person (@kellymclaughlin or @andrewjstone) to review, since both @shino and me have put in quite a bit of code. |
|
I can start reviewing this a bit later today. Looks like a good one :) On Mon, Feb 3, 2014 at 2:50 PM, Reid Draper notifications@github.comwrote:
|
|
|
@shino you mind giving this a once over too to make sure I didn't miss anything? |
| @@ -271,31 +278,52 @@ enough_results(#state{req=?LOREQ{max_keys=UserMaxKeys}, | |||
| objects=Objects, | |||
| common_prefixes=CommonPrefixes}) -> | |||
| riak_cs_list_objects_utils:manifests_and_prefix_length({Objects, CommonPrefixes}) | |||
| >= UserMaxKeys | |||
| > UserMaxKeys | |||
There was a problem hiding this comment.
Would you mind add some comments here?
I wonder why there is no equal sign when the name of function says "enough results".
There was a problem hiding this comment.
Yes, good catch. I'll update in the morning.
|
@reiddraper Everything looks nice 💃 . Just a misc comment above. |
|
👍 epic testing Great work @shino and @reiddraper |
…squashed-reid-squashed List objects FSM v2 bugfix and tests
reiddraper
deleted the
feature/list-objects-eqc-addition-squashed-reid-squashed
branch
Fixes #781.
This is a combination of work from @shino and @reiddraper, intended to fix #781, and add further testing. I also intend for it to replace PR #784, as I have re-ordered some of the commits to ensure Riak CS and the tests compile at each commit. Please compare with the other WIP branches to make sure no changes have been missed.