filter string and updated tests

[vietnguyengit]

No description provided.

[utas-raymondng]

Instead of change in code, is it easier to do it in elastic ?
``
"index": {
"analysis": {
"filter": {
"remove_uuids": {
"type": "pattern_replace",
"pattern": "[a-fA-F0-9]{32}", // Regular expression to match UUIDs
"replacement": ""
},
"my_shingle_filter": {
"type": "shingle",
"min_shingle_size": 2,
"max_shingle_size": 3,
"output_unigrams": true
}
},
"analyzer": {
"my_analyzer": {
"type": "custom",
"tokenizer": "standard",
"filter": ["lowercase", "remove_uuids", "my_shingle_filter"]
}
}
}

[utas-raymondng]

Also pure number can be remove ?

        "remove_numbers": {
          "type": "pattern_replace",
          "pattern": "\\b\\d+\\b",
          "replacement": ""
        },

[utas-raymondng]

And single char like "m"

[vietnguyengit]

Instead of change in code, is it easier to do it in elastic ? `` "index": { "analysis": { "filter": { "remove_uuids": { "type": "pattern_replace", "pattern": "[a-fA-F0-9]{32}", // Regular expression to match UUIDs "replacement": "" }, "my_shingle_filter": { "type": "shingle", "min_shingle_size": 2, "max_shingle_size": 3, "output_unigrams": true } }, "analyzer": { "my_analyzer": { "type": "custom", "tokenizer": "standard", "filter": ["lowercase", "remove_uuids", "my_shingle_filter"] } } }

didn't think about that path, I'll try

[utas-raymondng]

LGTM, please reload after deployed to edge thanks