Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor trickle DAG builder #4730

Merged
merged 1 commit into from
Mar 23, 2018
Merged

Refactor trickle DAG builder #4730

merged 1 commit into from
Mar 23, 2018

Conversation

schomatis
Copy link
Contributor

@schomatis schomatis commented Feb 23, 2018

Refactoring of the logic of the trickle DAG builder to simplify the future patch that will add support for Filestore (#4052).

The Layout function is mirroring the logic in fillTrickleRec duplicating almost entirely its code. The refactored fillTrickleRec now absorbs both scenarios of building a trickle tree with or without a maximum depth (Layout didn't have a maximum depth, only fillTrickleRec did). This is accomplished abusing the depth parameter (now renamed as maxDepth) to make it optional (a negative value is interpreted as no maximum). This hack is due to the fact that Go doesn't support optional parameters, suggestions are welcome for a more elegant solution.

I'm submitting this WIP PR to have feedback on this refactoring, as it will be useful to extend it to other functions (Append, appendFillLastChild, appendRec) which also replicate similar loop patterns of the sort:

https://github.com/ipfs/go-ipfs/blob/cc01b7f188622e7148ce041b9d09252c85041d9f/importer/trickle/trickledag.go#L212-L224

@schomatis schomatis changed the title [WIP] dag: refactor trickle importer [WIP] Refactor trickle DAG builder Feb 23, 2018
License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>
@schomatis
Copy link
Contributor Author

As commented in #4052, the support for Filestore will be added only to the Layout function for now, so I'm restricting this refactoring to just that function and leaving the Append function (and related) as is (unless I'm suggested otherwise).

This PR is then ready for review.

@schomatis schomatis changed the title [WIP] Refactor trickle DAG builder Refactor trickle DAG builder Feb 23, 2018
@schomatis
Copy link
Contributor Author

@whyrusleeping Could you review this please?

Copy link
Member

@Stebalien Stebalien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks right to me but I'm not familiar with this code.

for j := 0; j < layerRepeat && !db.Done(); j++ {
next := db.NewUnixfsNode()
if err := fillTrickleRec(db, next, i); err != nil {
for depth := 1; ; depth++ {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code nit: You can write this as for depth := 1; depth < maxDepth || maxDepth < 0; depth++ {.

Copy link
Contributor Author

@schomatis schomatis Mar 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I put it separately to highlight the fact that maxDepth is an optional parameter (that will most commonly be unset, which is the base case) but I can move it all together to make it more compact if you think it's more in line with Go coding principles.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds reasonable. I don't feel strongly either way.

for layer := 0; layer < layerRepeat; layer++ {
if db.Done() {
return nil
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why move do this separately?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid repeating the same condition in both fors, and make it more clear that if db.Done is true nothing more needs to be done in this function and it should just exit there (removing the final return from the code).

@schomatis
Copy link
Contributor Author

@Stebalien Thanks for the review!

Copy link
Member

@Kubuxu Kubuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM, hash stability in case of big files was unaffected. I wasn't sure if it was covered by sharness so I checked it manually.

@Kubuxu Kubuxu added the RFM label Mar 9, 2018
@Kubuxu Kubuxu added this to the go-ipfs 0.4.15 milestone Mar 9, 2018
@whyrusleeping whyrusleeping merged commit b8e7b63 into ipfs:master Mar 23, 2018
@whyrusleeping
Copy link
Member

Thanks @schomatis, and congrats on your first PR landing :)

@schomatis schomatis deleted the fix/dag/trickle branch March 23, 2018 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants