Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
HDDS-2588. Consolidate compose environments #238
What changes were proposed in this pull request?
There are a few slightly different sample docker compose environments:
Plus: also run
I also propose to rename
Consolidating these 4 environments would slightly reduce both code duplication and the time needed for acceptance tests.
How was this patch tested?
Ran acceptance test in
Clean CI in private branch.
Thanks to work on this @adoroszlai . I always felt the same, that we have too many environments, but I didn't know the right answer. therefore I just share my thoughts this is not a
It's -- at least partially -- a philosophical question, what is Ozone. (And as it's philosophy, I am interested about the opinion of our philosopher of Ozone cc @anuengineer)
By default I would say:
But we can also say what what this patch says:
(at lest for Apache Ozone, this is not true for proprietary distributions).
The other question is usability: Do we need Prometheus and Jaeger all the time. Do we need to start them when we would like to test any of the features of ozone?
My third (actual) problem is the guarantee of a cluster. To avoid same flakiness I would like to declare that we need at least 3 datanodes for ozone compose clusters (see. HDDS-2606). But I am not happy with the usability that Ozone can't work if you don't do a
We may need a smaller subset of ozone (
Thanks @elek for your thoughts. I think (1) and (2) are addressed by the followup commit, which extracts monitoring and profiling into separate configs. (Let me know if any of the configs are miscategorized.) They can be mixed in as desired by one of:
I need to think (or talk) about your third point (3) more.
Thanks the update @adoroszlai This approach is very smart, but I have some fear how easy is to understand it. (One additional function of the compose folders to provide simple examples to use ozone.)
But let's try out this approach. I am fine with it.
Can you please update the README.txt inside
Changes in last two commits besides README update:
I'm fine with reworking this one to accommodate whatever changes are done in #282. I wanted to avoid blocking the acceptance test fix for a usability improvement.
I am okay with what this patch says -- in reality, irrespective of what we say, there will be monitoring , tracing and logging collectors in place for most data centers. So irrespective of what we do (Prometheus, Jaeger, Grafana, Fluentd) the system admins will do the right thing for them.
We are just show casing that fact that, it is trivial to do this with Ozone. So when someone is evaluating Ozone, the question of how can I really run this service in production is answered via the presence of these tools. I would go a step ahead add these as recipes in the Ozone documentation too.