[BUG] Mapping size is larger than 10MB - Why is it hardcoded!?

[asfoorial]

Good day all,

I checked mappings.js in the latest OpenSearch Dashboards source code and I found the following (line 323). Apparently, the maximum number of fields fetched is hardcoded to 10MB. Is there a reason for this? And will this be replaced with a configruation instead?

if (mappings) {
const maxMappingSize = mappings[0].length > 10 * 1024 * 1024;
if (maxMappingSize) {
console.warn(
Mapping size is larger than 10MB (${mappings[0].length / 1024 / 1024} MB). Ignoring...
);
mappingsResponse = '[{}]';
} else {
mappingsResponse = mappings[0];
}
loadMappings(JSON.parse(mappingsResponse));
}

[joshuarrrr]

I imagine there are a number of performance and UI concerns we'd need to address if we decided to increase the mapping limit. But first we'd need to better understand the use case you have and figure out whether that's really the best approach (for example getting flattened results may be preferable.) Can you explain more about your index patterns, document structure, and how you ran into this limitation in the UI?

[asfoorial]

In fact, I am using the Dashboards query console (Dev Tools). The auto-completion fails giving that error because I have many fields in my indices. So probably the way field auto-completion works in Dashboards needs to be fixed.

[asfoorial]

I also think that all fields are loaded in the client automatically even if I don't request them!

[asfoorial]

Note also that the size that makes fail is 19MB. I don't think that this is too much for the browser to handle.