Size tracking component

[jfantinhardesty]

What type of Pull Request is this? (check all applicable)

• Refactor
• Feature
• Bug Fix
• Optimization
• Documentation Update

Describe your changes in brief

This adds a new size tracker component which tracks the size of the mounted directory to prevent calls to the cloud to get the size of the mount. This in particular is useful when mounting subdirectories as there is no call in S3 to retrieve the size of the storage. This component goes in between the file_cache and attr_cache to track write information.

Checklist

• Tested locally
• Added new dependencies
• Updated documentation
• Added tests

Related Issues

• Related Issue #
• Closes #

[foodprocessor]

That's quite the testing infrastructure! Good work!

[Dabnsky]

overall, pretty cool stuff. one thing I ran into when I ran it was this error:

Error: failed to initialize new pipeline [config error in Pipeline [component size_tracker is out of order]]

so a naïve user who just tacks that on their config file won't know that there is a certain order that it needs to be listed in the components section. we can consider adding sanitation support for the config file when it is being processed.

Great job!

[jfantinhardesty]

overall, pretty cool stuff. one thing I ran into when I ran it was this error:

Error: failed to initialize new pipeline [config error in Pipeline [component size_tracker is out of order]]

so a naïve user who just tacks that on their config file won't know that there is a certain order that it needs to be listed in the components section. we can consider adding sanitation support for the config file when it is being processed.

Great job!

Yes, good point. Yeah there is an implicit order in the component section, and this does require being after the file_cache section and before the attr_cache section. Once merged, I would update the current documentation on this that exists on our wiki at https://github.com/Seagate/cloudfuse/wiki/Config-File where we note the order of the configuration file and the priority that should be set if users want to use this component. Though, this component should really only be used by the plugin most likely.