Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop 2.9 #1675

Merged
merged 299 commits into from
Jan 28, 2019
Merged

Develop 2.9 #1675

merged 299 commits into from
Jan 28, 2019

Conversation

martinsumner
Copy link
Contributor

Merge develop-2.9 changes into main basho repo for riak_kv.

The changes include:

  • Changes to GET and PUT FSM (and riak_kv_vnode) to support use of HEAD capability within backends;

  • Integration of leveled backend;

  • Addition of support for kv_index_tictactree based anti-entropy;

  • Support for aae_folds to provide inter-cluster anti-entropy building blocks based on kv_index_tictactree;

  • Integration with core node worker pool and ability to queue async work based on previously snapped snapshots;

  • Vnode soft limit monitoring when selecting PUT co-ordinators.

fold_heads receives a binary proxy object not a riak_object.
As previously this disn't work
Start to plumb in query options, rather than generating filters - as the filter can always be generated as part of generating the object folder
Make it a bit easier to write other fold functions by generalising the validation of input options in riak_kv_mapfold_fsm
Try and make abstraction between mapfold_fsm modules clearer and add in support for segment_folder in new code style
Detcted by dialyzer
Removed the need to fully convert the object to a riak-object to extract key details (e.g. sibling count, object size and vector clock)
This supports passing check_presence from the request through to the backend
Unit test JSON decoding ... kept making mistakes
It wouldn't work - and there's no code behind it yet
This isn't yet configurable.

Also tidy up logging
Helps with verifying results in test.
FoldObjects fun gets transformed based on some conditions in leveled from a 4 arity to a 3 arity function.  This uses a tag to reverse that
Switch to use mas-aae-segmentfoldplus.  This should use the new hashing algorithms for tictac trees, and is a pre-cursor to using accelerated segment folds
martinsumner and others added 22 commits December 10, 2018 08:56
Mistake - leveled capability was removed, but was required by tictac_aae
Use version brought in from riak_core to avoid confusion
Mainly outcome from testing, some changes where I had wrong
assumptions about how wrq functions worked with the path. Some
docs/specs changes in the cluster aae fsm that testing showed to be needed.
Set Request Type in prepare.
Some changes, mainly making the ordering requirement on get core
results explicit in documentation. I renamed override_nodes, as I
found it confusing. I renamed head_merge/2 to merge_heads/2 since it
had nothing to do with head_merge/1 and made code navigation harder. I
hope that none of these are too contentious
@martinsumner
Copy link
Contributor Author

@martincox - this gets all the work on 2.9.0 into the develop-2.9 branch on riak_kv. Can you +1?

@martincox
Copy link
Contributor

This is a behemoth of a PR!

Am I right in seeing that the additional features introduced here, such as HEAD requests are exposed exclusively through the HTTP API? Therefore, we should maybe raise an issue for these to be exposed through PB (if needed / required)?

@martinsumner
Copy link
Contributor Author

The only API features are the aae_fold API and the new riak_repl feature which allows an object to be touched (prompting re-replication without modification). These two features are only available via HTTP - so I'll create an issue for the PB follow-up (although I'm not sure with aae_fold if in the long run the right answer is exposure via API or access direct using riak_client).

The use of HEAD request is internal only at the moment (and only changes behaviour when the backend supports the head capability, and when the user has made a GET request). There is no change to the API involved in this. I will add an issue with changing the behaviour if the user submits a HEAD request (currently this is still a use of GET under the hood with the body stripped at the end, and this could be improved)

Re-point at basho - requires other PRs to be accepted
@martinsumner martinsumner merged commit a3b8f7d into basho:develop-2.9 Jan 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants