[filebeat][log] Enable status reporter for log input

[VihasMakwana]

Proposed commit message

Enable StatusReporter for log input which was introduced in #39209

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

• Closes #39733

Use cases

This PR allows the log input to report its status to elastic-agent. This would keep the status of the unit up-to-date and would help with diagnostics.

Screenshots

Logs

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @VihasMakwana? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[pierrehilbert]

@belimawr is now in PTO.
@rdner could you please have a look here?

Tiago is out.

[rdner]

The log input is deprecated since 7.16.0.

I don't think we should spend any time on extending its functionality and testing. Instead I'd expect status reporting to be introduced in the filestream input.

[pierrehilbert]

The log input is deprecated since 7.16.0.

@nimarezainia what is your opinion, should we do it for Log Input or consider closing this PR as this is deprecated?

I don't think we should spend any time on extending its functionality and testing. Instead I'd expect status reporting to be introduced in the filestream input.

FYI we already have it merged for Filestream input: #40121

[jlind23]

I do agree with @rdner statement above, if it is deprecated this would be another reason why users need to switch to filestream.
Also another reason why we need to migrate the container input.

[VihasMakwana]

I do agree with @rdner statement above, if it is deprecated this would be another reason why users need to switch to filestream. Also another reason why we need to migrate the container input.

@jlind23 Agree. But container input uses log input under the hood.
If we add the status reporting to log input, it would be added for container input as well

[jlind23]

@jlind23 Agree. But container input uses log input under the hood.
If we add the status reporting to log input, it would be added for container input as well

That is why my suggestion was rather to close this issue and spend time on migrating the container input to rely on filestream.

[VihasMakwana]

That is why my suggestion was rather to close this issue and spend time on migrating the container input to rely on filestream.

@jlind23 Oh, sorry. I misunderstood.
I thought you were referring to add StatusReporter to container.

[jlind23]

@VihasMakwana @rdner as spoke over slack, this has to be delivered even if the log input is marked as deprecated.
So @VihasMakwana please reply back to @rdner's comment and let's continue the review process here.

[nimarezainia]

if the PR is ready and tested, why not go through with it? doesn't seem like a risky endeavor.

We should have made this decision before pursuing the PR IMO. but now it's here what doe we lose in merging? many are still using 7.16 and on logs input.

[pierrehilbert]

@nimarezainia this is already merged. However, we are waiting for 8.15.1 before merging it into 8.15 branch as the time frame is now too short to merge it for 8.15.0.

[cmacknz]

doesn't seem like a risky endeavor.

The risk is we suddenly have the majority of agents report themselves as degraded and get a flood of support cases for things we could have identified and fixed ourselves if we let this change soak longer.