s3, allow to ignore get/put errors #54
Conversation
There was a problem hiding this comment.
i noticed you haven't added this option for multipart uploads. but that's ok. if you fix the minor issues that i mentioned in the comments, then i can merge this and will work on top of your patch to add this to other IO operations and to add the log file option, as discussed in #52. (or you can of course also try adding the log file option if you prefer.)
source/ProgArgs.h
Outdated
| @@ -146,6 +146,7 @@ namespace bpt = boost::property_tree; | |||
| #define ARG_S3BUCKETACLPUT_LONG "s3baclput" | |||
| #define ARG_S3ENDPOINTS_LONG "s3endpoints" | |||
| #define ARG_S3FASTGET_LONG "s3fastget" | |||
| #define ARG_S3IGNOREERRORS_LONG "s3ignoreerrors" | |||
There was a problem hiding this comment.
alignment of the word "s3ignoreerrors" not correct.
tab width is 4. i actually switched from tabs to 4 spaces recently for new patches. but not a showstopper if you keep tabs for this patch (because i know i should really write a code style guide for these things. some day... :-) )
There was a problem hiding this comment.
yeah, sorry. some of the parts of the file was still tabs and some other spaces. would love the mega patch from you to standardize at some point <3
There was a problem hiding this comment.
i'm still thinking about that mega patch, but probably won't do it, because this change to all code lines would make "git blame" harder and thus i wouldn't even know who to yell at if there ever is a bug in the code ;-)
source/workers/LocalWorker.cpp
Outdated
| "Exception: " + s3Error.GetExceptionName() + "; " + | ||
| "Message: " + s3Error.GetMessage() + "; " + | ||
| "HTTP Error Code: " + std::to_string( (int)s3Error.GetResponseCode() ) ); | ||
| if (!ignoreS3Errors) { |
There was a problem hiding this comment.
opening curly bracket ({) to new line
source/workers/LocalWorker.cpp
Outdated
| "Exception: " + s3Error.GetExceptionName() + "; " + | ||
| "Message: " + s3Error.GetMessage() + "; " + | ||
| "HTTP Error Code: " + std::to_string( (int)s3Error.GetResponseCode() ) ); | ||
| if (!ignoreS3Errors) { |
There was a problem hiding this comment.
opening curly bracket ({) to new line
source/ProgArgs.cpp
Outdated
| @@ -496,6 +496,8 @@ void ProgArgs::defineAllowedArgs() | |||
| "Send downloaded objects directly to /dev/null instead of a memory buffer. This option " | |||
| "is incompatible with any buffer post-processing options like data verification or " | |||
| "GPU data transfer.") | |||
| /*s3f*/ (ARG_S3IGNOREERRORS_LONG, bpo::bool_switch(&this->ignoreS3Errors), | |||
There was a problem hiding this comment.
/*s3f*/ needs to be changed to /s3i*/. as you were probably aware already, these comments are just for us humans to make it easier to find the right place so that the --help-all output is sorted alphabetically by option strings.
|
@breuner added ignores for multipart uploads as well, plz let me know if there were any other s3 put/get operations i missed yeah, maybe we can merge this first and then do the logging part |
|
thanks for your contribution! if you want to tell me your real name for this then i'll add that to the thanks in the changelog. otherwise i'll just use your github username there. |
|
thanx @breuner! github username is just fine any chance this can be pulled into 3.0.8 and released soon? 🙏 |
|
indeed i'm planning to build a new release in the couple of days. (it's still a bit of manual work for me to make a release, because i enjoy writing C++ code more than writing automation scripts, and thus i'm making releases a bit less frequently than i should, but i'm working on improving this :-) ). since you mentioned |
|
hi @russki, i just wanted to let you know that this is now included in the latest elbencho v3.0.9. |
@breuner 🙏
partially resolves #52