TokenClassificationRecord metadata menu seems to limit number of entries to 100 even if there are more in the dataset.

[coreyelowsky]

It seems as though the metadata drop down menu limits the amount of entries to only 100. Is it possible to expand this amount?

[frascuchon]

Hi @coreyelowsky and thanks a lot for all of your feedback.

The number of allowed metadata fields is configurable by the METADATA_FIELDS_LIMIT environment variable. As the default is defined as 50 and should not be greater than 100. In that sense, it should not be possible to have a dataset with a greater number of 100 different metadata fields.

I would like to know if you are able to manage a larger number of these data. We can also consider expanding the maximum number of metadata fields allowed, although it could affect the performance of the search engine.

[coreyelowsky]

@frascuchon

What I meant was not for more than 50 different metadata fields. But for just one specific metadata field, we have more than 100 unique values. As in the image above, for the specific metadata field called 'note_id', only 100 unique values show up in that dropdown shown, when we really have many more unique values for that specific field. Unless we mean the same thing?

[frascuchon]

Ok, got it. Yes, we've set a limit to a max number of results for metadata filters, since they can directly impact performance.

We will increase the default value. And we can allow configuration by env var in future releases.