Add Elasticsearch 5.x storage support

[oopschen]

Add Elasticsearch 5.x storage support.
Enhance date format for index name resolvition.
Enhance client options.

[k8s-ci-robot]

Hi @oopschen. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with @k8s-bot ok to test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[dashpole]

@k8s-bot ok to test

[dashpole]

@timstclair this looks like an entirely new storage driver. Is it not possible just to update the old ES driver? If it is a new one, this is probably blocked on #1458...

[dashpole]

@k8s-bot test this

[dashpole]

@oopschen apologies for the delay. Is it possible to update the existing ES storage driver rather than adding a new one while maintaining compatibility with older versions? I don't think we should have a driver for each version of each driver if we can avoid it.

[oopschen]

@dashpole There is two major release of elasticsearch: 2.x and 5.x, and there are not compatible. I think it is better to support two of them to offer a better user experience for developers.

It definitely possible to merge two major release ES storage, if it is required, should i open a new pull request or merge them in this pull request?

[dashpole]

@timstclair should this wait on #1458?

[timstclair]

It depends on whether it would be an update to the existing elasticsearch plugin, or a new one. I was blocking new plugins, but not changes to existing ones.

…

On Thu, Apr 20, 2017 at 9:19 AM, David Ashpole ***@***.***> wrote: @timstclair <https://github.com/timstclair> should this wait on #1458 <#1458>? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1597 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADHtaA4RwCZW9zPK1N6nfMuTUuj0-Rcdks5rx4WKgaJpZM4MGxZn> .

[dashpole]

It is a new plugin, since elasticsearch 2.x and 5.x are not compatible

[rikatz]

Since the drivers from 2 and 5 aren't compatible, we need to have 2 drivers :(

I've faced this very same issue in Heapster.

There are some prettier solutions here (maybe using an interface, configuring the version in storage URI when starting cAdvisor), but the point here is that we still need 2 drivers :)

Haven't tested this implementation by itself. I've missed an _test file.

About the refactor of cAdvisor storage (as in #1458), can't say something about this (as I'm no expert in developing) but sounds better to me anyway to have plugable storage instead of maintaining each solution market. By the way, would prometheus be a 'storage' implementation, or should we keep this separate as a '/metric' implementation? :)

[jjduhamel]

Any update on this? Would be good to add support for this asap.

[jindov]

Waiting for this support, I use ELK 5.5 stack on system

[mostolog]

Already added +1.

Having this blocked by #1458 for so long doesn't make any sense to me.

Adding this driver as "elasticsearch5" while the other is still mantained as "elasticsearch" would do the trick.

[liangyuanpeng]

any update?

maybe current the es version supported is too old, we should update version of es for stoarge 😢

[bobbypage]

I agree with the direction in #1458 (comment) we want to avoid adding new storage drivers in cAdvisor and instead rely on prometheus metrics + external backends that can ingest those metrics. As a result, closing this PR for now.