METRON-2347: [BRO-PLUGIN-KAFKA] Use docker compose in end to end tests

[JonZeolla]

Contributor Comments

This is a first step towards docker-compose in our end to end tests.

Testing

Run ./run_end_to_end.sh and ./finish_end_to_end.sh

Pull Request Checklist

Thank you for submitting a contribution to Apache Metron's Bro kafka writer plugin.

In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:

For all changes:

• Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
• Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
• Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

• Have you included steps to reproduce the behavior or problem that is being changed or addressed?
• Have you included steps or a guide to how the change may be verified and tested manually?
• Have you ensured that the full suite of tests and checks have been executed via:

  bro-pkg test $GITHUB_USERNAME/metron-bro-plugin-kafka --version $BRANCH
  

• Have you written or updated unit tests and or integration tests to verify your changes?
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• Have you verified the basic functionality of the build by building and running locally with Apache Metron's Vagrant full-dev environment or the equivalent?

[JonZeolla]

@ottobackwards Open to suggestions - would prefer not need to go back to using wait-for-it.sh but that seems to be a rather standard approach to solving this problem.

[ottobackwards]

https://docs.docker.com/compose/compose-file/compose-file-v2/#depends_on

[JonZeolla]

It doesn't work that way. Same with healthcheck. If you look in the docker compose file I'm already using depends_on

[ottobackwards]

then, why don't you stick with wait_for_it?

[JonZeolla]

I may, let me think on it. Was hoping to remove the MIT license and all the overhead but compose isn't as capable as k8s readiness

[JonZeolla]

I have a solution for this which is docker-compose native, just need to find some time to wrap it up.

[ottobackwards]

I am +1 on this. Review and run, seems good to me. The wait for it can be a follow on. Please create a jira.

[JonZeolla]

Thanks - give me a bit to wrap up responses to your feedback, hopefully I'll get time in the next couple of days.

[JonZeolla]

Ok, back to you @ottobackwards should have addressed all of your feedback now

[ottobackwards]

+1 nice work