[New Integration] GitLab

[tehbooom]

• Enhancement

Proposed commit message

New integration for GitLab

Some examples logs were pulled from the documentation

The other logs were pulled from a test GitLab instance using the free tier.

Not all logs from GitLab are collected since we need a dev license and more example logs.

The following logs will need to be ingested from another integration:

• nginx
• postgresql
• redis
• prometheus
• mattermost

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

• [ ]

How to test this PR locally

Related issues

• Relates Feature request: Gitlab integration #9122
• Relates Gitlab #1741

[elasticmachine]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[Alphayeeeet]

@tehbooom Thanks for looking into this topic. Is it possible to use use the mentioned other integrations inside the Gitlab integration as a one for all solution?

I hope it's understandable what I mean. So that I don't need an integration policy for each of the components as the GitLab integration should manage all that (preferably configurable in the policy settings).

[tehbooom]

The following logs will need to be ingested from another integration:

• nginx
• postgresql
• redis
• prometheus
• mattermost

Hey @Alphayeeeet , there have been discussions around using one ingest pipeline for two integrations however at this time it is not supported. Ive decided to not recreate the pipelines in this integration incase we do in the future move towards utilizing more than one integration pipeline for a single integration. As of now, you will need to add more integrations to the policy to fully monitor all of GitLab.

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 04ea8c6

History

• 💚 Build #11648 succeeded 316f7a4
• 💚 Build #11590 succeeded 6ad87a4
• 💔 Build #11587 failed 6634256

[elastic-sonarqube]

Quality Gate failed

Failed conditions
73.5% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

[efd6]

Two comments on this.

1. There is a GitLab CE PR open at the moment. It may be good to think about whether it's possible to simplify the surface for integration users in some way.
2. The size of this PR is too large to reasonably review with any real quality control. It would be good to reduce the chunk size of review objects. I suggest prioritising the data streams and making a series of PR that add them in priority order.

[tehbooom]

Closing since it is a duplicate of #9747. After it is merge will start to add more datastreams for this PR.