JPERF-1106: Gather Apache2 logs

[pczuj]

No description provided.

[pczuj]

This requires at least 4 changes:

• Get rid of problematic API changes, e.g. Jira 2 new constructors are added. We want to wait for JPERF-1079 Make it possible to stop Jira nodes with a dedicated script. #162, as it introduces Builder for Jira
• Need a test
• Need to find better names for new classes
• Need a CHANGELOG entry

[pczuj]

First time used this as? syntax. Casts are usually bad, however given the circumstances of our current model I think this is a better solution than putting a new field into ProvisionedLoadBalancer and I'm also not certain if adding methods to LoadBalancer would be a good idea. Extending the interface and detecting it allows for wrapping of DiagnosableLoadBalancer implementations without losing the feature.

[dagguh]

Indeed casts are bad. See https://github.com/atlassian/aws-infrastructure/blob/release-2.29.0/src/main/kotlin/com/atlassian/performance/tools/awsinfrastructure/api/jira/DataCenterFormula.kt#L360:

if (loadBalancer is ApacheProxyLoadBalancer) listOf(loadBalancer::updateJiraConfiguration) else emptyList()

Fortunately, ApacheProxyLoadBalancer will match both is/as? casts, so it will function.
We need hooks API to avoid further tight coupling. If the apache logs are necessary, then we can merge the tech debt. But if not, then let's skip that part.

[pczuj]

Name suggestions welcome

[pczuj]

Name suggestions welcome

[pczuj]

We could make that LoadBalancer-agnostic, however that would make the model more abstract and that possibly could make it harder to understand. Looking for thoughts on that.

[pczuj]

I migrated the new tests to use AssertJ

[pczuj]

@dagguh do you see value in this change? If not we may want to decline it.

We used the logs in our internal experiment to diagnose problem and we already know that the problem is with Jira memory usage and not load balancer. Load balancer proxy error was only a symptom of product having CPU/IO problems.

[dagguh]

Overall reuse of MeasurementSource is good. Now the branch needs more work due to breaking changes on master. I trust your call to continue or cancel this.

[pczuj]

This is ready for review again. I'm not planning to add integration test for the logs.

[dagguh]

Let's split those changes. One sounds good, while the other needs some further considerations.

[dagguh]

Indeed casts are bad. See https://github.com/atlassian/aws-infrastructure/blob/release-2.29.0/src/main/kotlin/com/atlassian/performance/tools/awsinfrastructure/api/jira/DataCenterFormula.kt#L360:

if (loadBalancer is ApacheProxyLoadBalancer) listOf(loadBalancer::updateJiraConfiguration) else emptyList()

Fortunately, ApacheProxyLoadBalancer will match both is/as? casts, so it will function.
We need hooks API to avoid further tight coupling. If the apache logs are necessary, then we can merge the tech debt. But if not, then let's skip that part.

[pczuj]

If the apache logs are necessary, then we can merge the tech debt. But if not, then let's skip that part.

Not sure how to judge that.

We wanted to take a look at the appache logs when diagnosing a problem recently. We used JAR built from this code to do so, however didn't really find anything useful other than confirmation that the load balancer just times out on communication with the product.

We no longer have an immediate problem to diagnose, so that would mean that the logs are not necessary. Question is if we wouldn't waste the effort if the diagnostic need will repeat itself, yet the code for it won't be merged and e.g. will have conflicts with the current master.