Importing data from a file is considerably slower on map view than table view

[sxsws]

Describe the Bug

Currently importing large amount of data (2 million rows - postcode codes with lat/long) and it's a lot slower when displaying the collection in map view compared to when importing in table view. I guess this is from the map view wanting to update the view regularly (auto refresh is not on)?

So, I'm thinking disable updating the index view whilst importing?

To Reproduce

import a large amount of data via a file when the index view is on map view

[rijkvanzanten]

Moving this to a discussion as I believe it's more a conversation on what's happening / how to improve rather than a single bug we can fix on the map

I think it's simply because the map layout takes way more resources from the browser to render than the table does! It could also be the difference in number of records fetched; the table layout defaults to a pretty small page size, whereas the map layout renders way more points at the same time 🤔

[sxsws]

Right, plotting is way more intense than just spitting out text but I guess my point was not to update the index view whilst importing because even in a table view, if a field is sorted, and even if it does have an index, it's still going to slow down the import than if no update happened. And the larger the data, the slower it's going to get!

There's no real benefit to updating the index view whilst importing. It may encourage the user to click away from the index page... which will kill the import.

Maybe the import should be a modal to prevent this clicking away, or at least some kind of message underneath whilst importing stating not to be silly and click on something!

[rijkvanzanten]

(Also, as per our other discussion in #13299, the current CSV import endpoint isn't really suited for big data imports, but we'll keep that discussion on the other discussion 😄 )

[sxsws]

lol, I don't think batching imports will solve the core issue of this problem!

[rijkvanzanten]

I don't know if I agree! For the batched import, I'd imagine the user is requested to upload the full file first in a more dedicated dialog, and the import would happen asynchronously. That small change in UI also affects how the layout itself behaves during the import 🙂

[sxsws]

Ahh, yeah I understand. Yes, I think if there's a modal, there's no real need for an update of the view at all?

[rijkvanzanten]

Yup exactly!

[sxsws]

I guess I'll mark this question as answered as I don't envision it popping up once #13299 and related is resolved.