v2: Rework hooks system

[Acconut]

This branch is used to develop the new hook system. The main concepts will stay the same (i.e. file, HTTP, gRPC hooks) but the exact interface has to change.

Goals:

• Experiment with new plugins using https://github.com/hashicorp/go-plugin
• Allow hooks to control status code, headers and body of response (only for pre-* hooks and post-receive hook)
• Make error responses the exception and not the norm (errors should not be used to reject uploads anymore)
• Allow additional response values from hooks (e.g. whether upload should be rejected or stopped)

TODO (will be addressed after merge):

• docs: Update documentation about hooks
• docs: Add plugin hooks to documentation
• cli: Consider moving hooks in pkg/
• cli: Add tests for hook implementations and system
• handler: Allow modifying the HTTP response when stopping an upload
• grpc: Add headers to hook request
• grpc: Add documentation to proto files
• grpc: Move proto file into better, public location
• plugin: Stop zombie processes when tusd terminates
• plugin: Move into public package. The plugin should not import for the cmd directory
• plugin: Clean up hooks/plugin.go

[sagikazarmark]

Have you considered only supporting go-plugins for hooks and implementing http, grpc and file hooks as plugin implementations? It would simplify the code you have to maintain in tusd. Granted, it would also introduce a slight performance hit.

[Acconut]

Have you considered only supporting go-plugins for hooks and implementing http, grpc and file hooks as plugin implementations?

Yes, but it's a tradeoff. I doubt it would reduce code complexity as every hook implementation would need the code for the go-plugin setup. Furthermore, it would make tusd not a single, all-containing binary anymore as users need to ensure that the plugin files are present during runtime.