Passing JSON object as search term for /v2/broadcasts/list/{offset}/{size} in cluster mode fails

[timantmedia]

Short description

A Customer is using a JSON object as the name for a live stream on creation.

When making an API request to /v2/broadcasts/list/{offset}/{size} and passing a JSON object as the search term, it works on a standalone instance but fails if in cluster mode.

Error log shows:

java.lang.IllegalArgumentException: Invalid character found in the request target [/LiveApp/rest/v2/broadcasts/list/0/100?search={%22tenantId%22:%22fd216128-e4da-11ed-9c89-bfe723cdc436%22,%22erCollectionId%22:%225fb69a0a-26d7-4ce7-81c5-ac7ab4949a27%22,%22ConnectorComponentId%22:%22[6b8d7491-a86a-4c64-a982-0f8a2d3d393b.9c9df16c-eac1-4593-8010-d28e92f8a694]%22} ]. The valid characters are defined in RFC 7230 and RFC 3986

The customer reported the following information from the log file:

2023-06-27 18:25:43,372 [http-nio2-0.0.0.0-5080-exec-9] INFO  io.antmedia.datastore.db.MongoStore - Server side search in broadcast for the text -> {"tenantId":"fd216128-e4da-11ed-9c89-bfe723cdc436","erCollectionId":"5fb69a0a-26d7-4ce7-81c5-ac7ab4949a27","ConnectorComponentId":"[6b8d7491-a86a-4c64-a982-0f8a2d3d393b.9c9df16c-eac1-4593-8010-d28e92f8a694]"}
2023-06-27 18:25:43,376 [http-nio2-0.0.0.0-5080-exec-9] ERROR io.antmedia.datastore.db.MongoStore - com.mongodb.MongoQueryException: Query failed with error code 51091 with name 'Location51091' and error message 'Regular expression is invalid: range out of order in character class' on server mongo:27017

Environment

• Operating system and version:
• Java version:
• Ant Media Server version: 2.5.3
• Browser name and version:

Steps to reproduce

1. Create a live stream that is named {"tenantId":"fd216128-e4da-11ed-9c89-bfe723cdc436","erCollectionId":"5fb69a0a-26d7-4ce7-81c5-ac7ab4949a27","ConnectorComponentId":"[6b8d7491-a86a-4c64-a982-0f8a2d3d393b.aeda6dc0-8445-4938-9fe5-397384261da1]"}
2. Make an API request to http://localhost:5080/LiveApp/rest/v2/broadcasts/list/0/100/?search=%7B%22tenantId%22%3A%22fd216128-e4da-11ed-9c89-bfe723cdc436%22%2C%22erCollectionId%22%3A%225fb69a0a-26d7-4ce7-81c5-ac7ab4949a27%22%2C%22ConnectorComponentId%22%3A%22%5B6b8d7491-a86a-4c64-a982-0f8a2d3d393b.9c9df16c-eac1-4593-8010-d28e92f8a694%5D%22%7D in standalone mode and cluster mode
3. Observe different response: Standalone mode sends expected result, cluster mode sends blank response.

Expected behavior

The API request should return a valid result in both cluster mode and standalone mode.

Actual behavior

Only standalone mode returns the result.

Logs

Place logs on pastebin or elsewhere and put links here

Ask your questions on Ant Media Github Discussions

[mustafaboleken]

Hi @timantmedia ,

It's not related to Ant Media Server directly. The problem is there is a character limit in mongo db. Please talk with the customer about the ConnectorComponentId value. It should be shorter. Whenever I shortened it, the error is solved.
Because of that, we use MapDB in standalone mode, it's worked without any problem.

[mekya]

Hi @mustafaboleken,
Could you give some info about how to change the character limit in mongodb?
In this way, we save good time for support process, emails communication etc.

[mustafaboleken]

I investigate more about this issue and found that the list item and string with minus character interpreted as range character like /^[Z-a]$/ and in search term we shouldn't put minus character inside square brackets to prevent this error.

[mekya]

Hi @mustafaboleken,

Don't we have an option to let the mongodb search as a free text search or escape this characters in searching?

[mustafaboleken]

Hi @mekya
I try to put escape character in front of square brackets but it throws illegal escape character error.
Instead of filter.regex, we can use search function. Are we sure that 100% of the users use for text search only. It can be caused to broke some cases.

[mekya]

Let's use the search function, we're not sure that 100% of the users use for text search only. We don't need to have %100 coverage for this issue. This function is for text search and please don't be perfectionist to solve this issue. It's blocking you.

Just fix it and have the test code for this bug.

If one day some user asks for regular expression support, we can think of adding regular expression support.

[mustafaboleken]

@timantmedia it's merged, fyi