One more fields is added, and the data cannot be queried by this fields

[ly0511]

Describe the bug

My log aggregation has a path dimension, as shown in the figure below, I can’t query the data when I use the path, only occasionally, but in most cases, I can’t query it, why is this?

To Reproduce

Nothing more, see bug description

Version

version is docker.io/victoriametrics/victoria-logs:v0.2.0-victorialogs

Logs

No response

Screenshots

No response

Used command-line flags

No response

Additional information

No response

[f41gh7]

@zekker6 PTAL

[ly0511]

The problem can be reproduced stably. The data cannot be queried after adding the path, but can be found after removing the path. Why?

[zekker6]

@ly0511 I've tried to reproduce this issue on our environments, but it doesn't seem to be the case there.

Is it possible to share the data with us to have a reproducible way to check the issue?
If it is possible, you can send it to z.bessarab@victoriametrics.com

If this is not possible, could you share your write configuration(configuration of stream fields, msg and time fields) and some artificial examples. This might help to reproduce an issue with data format similar to yours.

[ly0511]

@zekker6
I use vector to collect logs, and add path as the log label to query. The configuration is as follows:

Vector is programmed into my docker test environment. After the environment runs, it will be cleaned up and the docker container will be stopped. Only when the vector is running will the log be uploaded to Victorialogs, which is mainly used for log collection during CICD

My Victorialogs query statement is to splice the _stream_fields I added into url for query. Except for the path field, other fields are valid

The path query field does not take effect. It is a few hours after the environment has finished running. I don’t know why the query is fine when the environment is first running. After a while, it cannot be queried.

All query fields are as follows：

[zekker6]

@ly0511 Could you try to use the following queries and check if issue is still present?

# Regex match as a part of _stream_fields
_stream:{...,path=~".*/result/tmp/nohup_madx.log.*",...}

# Searching as a separate filter
_stream:{..} and path:re(".*/result/tmp/nohup_madx.log.*")

Also, does search using filters only for path on those logs work as expected?

[XLONG96]

Hi, I submitted the PR #5295 for this issue. You can fetch the patch and try if it resolves the problem.

[valyala]

The bug should be resolved in the commit cef7a39 . This commit will be included in the next release. In the mean time it is possible to check the bugfix by running VictoriaLogs built from this commit according to these instructions.

[valyala]

The bugfix from this commit has been included in v0.4.2-victorialogs release. Closing the issue as fixed then.

@ly0511 , feel free re-opening the issue if it still present at v0.4.2-victorialogs.