Large description causes market to hang

[corrie-sloot]

A large description (or possibly any large metadata) results in the market UI hanging (with the pink spinning wheel) and the browser eventually asking to kill the tab.

To Reproduce
Steps to reproduce the behaviour:

1. Download and configure ocean.js-cli
2. Create a metadata json file with a large amount of text in the additionalInformation.description (at least 58 kB)
3. Publish the dataset to Rinkeby npm run cli publishDataset metadata/testDataset.json
4. Using a browser, navigate to the market
5. Under networks, select "ETH Rinkeby"
6. Page should take too long to render, and browser will ask to kill

testData.json:

{
	"main": {
		"type": "dataset",
		"name": "Test",
		"dateCreated": "2021-08-05T12:00:00Z",
		"author": "Test",
		"license": "Commercial",
		"files": [{
			"contentLength": "71872",
			"url": "https://example.com/test.csv",
			"contentType": "text/csv",
			"encoding": "UTF-8",
			"index": 0
		}],
		"datePublished": "2021-08-05T12:00:00Z"
	},
	"additionalInformation": {
		"description": "Lorem ipsum dolor sit amet, consectetur adipiscing elit...",
		"tags": [],
		"copyrightHolder": "Test",
		"inLanguage": "en",
		"updateFrequency": null
	}
}

where the description has ~58kB of text.

[mihaisc]

Thanks for the feedback. Is the home page that is hanging for you or the asset details page? Can you provide a did for ease of testing?

[corrie-sloot]

It was on the homepage. The asset details page seemed to work find, so I changed the description to something smaller.

I have created a new asset on Rinkeby. In the description I used 100 paragraphs of Lorem Ipsum which was 59,501 bytes.

https://market.oceanprotocol.com/asset/did:op:9c63B34bDe74E3200c2ea745BdeDBDE0917b4988

When I try to load the homepage right now (at least while the DID shows up under recently published), the pink wheel spins and after a little while with my laptop fan spinning faster and faster, my Chome tells me the page is unresponsive and asks if I want to exit the page.

[mihaisc]

Ok, thanks, I replicated the issue. It is not stuck, it just takes a while. Probably because the laptop cpu is slower the chrome thread hangs. Anyway this is not a good/desired behavior, we will come up with a solution.

[KatunaNorbert]

Any suggestion on how we can fix this on the frontend side?
I was thinking to limit the description length, doesn't make sense to have such a long description and while we can't upload other media files just add text, the text length seems to be the only thing affecting the file size.

[mihaisc]

What do you mean by limiting the description length? If you mean in the publish/edit form that is just one part of the problem. The issue is what happens when it is edited outside. The main problem is that it breaks the search/home page so we need to see why it takes so long to load, is it the http call, is it slow on loading in memory or what is going on.

[KatunaNorbert]

Looks like the http call takes a long time to receive a response

[mihaisc]

Not sure what you are testing there. Search for Large Dataset . Total duration of query is around 0.6 seconds, and then you wait for 20 seconds or more and it's highly cpu dependent

[kremalicious]

so we have lots going on with asset teasers descriptions, so my first gut feeling would be checking those 2 libraries we use to process them: https://github.com/oceanprotocol/market/blob/main/src/components/molecules/AssetTeaser.tsx#L47-L51

Take them out and measure again, pretty sure one of those is what's causing large CPU usage. Simple fix then would be to limit the description length before passing it in asset teasers, as there we know we never want any long description anyway

[mihaisc]

Dotdotdot is killing it. Displaying the whole description is instant. So there is some useless processing in that component

[kremalicious]

yeah, that most likely does a lot of live calculation to figure out when it should cut a line. It's only use case is for nicer typography (to prevent widows, orphans, runts) but performance is way more important here.

We can remove it in asset teasers and kinda replicate it at least for webkit/blink browsers without the performance overhead:

• limit description based on character count to slightly more characters than the lines we want to cut
• use -webkit-line-clamp to get same functionality Dotdotdot provides

Still would have drawback that browsers which do not support -webkit-line-clamp will get slightly longer description. And maybe would be not that bad if we simply rely on cutting it based on character count only

[mihaisc]

well, we can do something way simpler, slice the description before passing it to dotdotdot, to an arbitrary size that it is for sure bigger than what will fit in the teaser.

edit : pr was already done 😄