Create a tool for verifying WAL assumptions #123
Conversation
* Set exit code on failure for scripting * remove cruft Co-authored-by: billierinaldi <billie.rinaldi@gmail.com>
There was a problem hiding this comment.
FWIW, the build failure in Travis is because this testing framework is for testing non-public API, whereas the expectation is that the accumulo-testing repo be restricted to testing public API. However, I can see the value in providing a testing framework for WAL behavior for different file system types.... so I'm not sure if we should:
- relax the restriction,
- find a new home for the proposed test,
- promote some portion of WAL code to public API (or SPI), or
- just work around it (perhaps temporarily) by using fully-qualified class names in the code rather than use import statements.
I haven't reviewed the specific test in detail, but I did look it over and, in general, I don't care if we add new tests to the accumulo-testing package, since nothing in there is public API, we don't do releases of it, and the addition of a new test is not an imposition on anybody using the existing tests. So, the test looks fine to me (other than the build failure).
|
Looking at the test its use of internal accumulo config and volume manager are likely susceptible to breakage between accumulo versions as those areas of the code are changing as we pay down tech debt. Ideally the test could only ref the pluggable log closer which is more stable. However not sure if that is possible, will look into it more next week. |
|
Thanks for taking a look, @ctubbsii and @keith-turner. That's a good point. One option would be for this to be a standalone tool in Accumulo itself, like the troubleshooting tools. Regarding technical debt, working on this has made me think it would be a lot easier if we had an interface that had all the WAL file system operations in the same place. That is, if the LogCloser also opened the files in addition to closing them (closing meaning lease recovery in this case). That might allow us to wrap a file system that doesn't have leases with our own custom lease handling, if we wanted. |
|
This is quite old. Has anybody looked at this recently and interested in picking it back up? |
Suggestions for additional verifications that could be performed or other improvements are welcome!