Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exponential sets can be computationally expensive to humanize #69

Closed
GreenReaper opened this issue Apr 27, 2021 · 2 comments
Closed

Exponential sets can be computationally expensive to humanize #69

GreenReaper opened this issue Apr 27, 2021 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@GreenReaper
Copy link

The set [Y3378E4..Y3389E4] results in a gzipped API response of 223.90 KB (1.05 MB decompressed) and takes seconds to generate (on WBStack). Starting the range with Y3278E4 resulted in an internal server error after five seconds. (The 'Y's are optional.)

I'm guessing a large part of the issue is how it humanizes the resulting set, which seems liable to exhaust memory:

(One of these: 33780000, 33780001, 33780002, 33780003, 33780004, 33780005, {... ~9995 more years}, 33890000)

rather than something like:

(One of these: 33780000 through 33890000)

More generally it may be worth limiting the length of the output; ProfessionalWiki/WikibaseEdtf#11 seems to touch on this but perhaps didn't consider this case. Beyond a certain point the humanization becomes non-useful, and if anything it might be best to skip to the last item. Or maybe each range in the set could only contribute a certain number of entries to the humanization, including its first and last items?

(One of these: 1990, 1991, 1992, 1993, [...], 2000, 33780000, 33780001, 33780002, 33780003, [...], 33890000) {with [...] as shown}

@JeroenDeDauw
Copy link
Member

Thanks for reporting this issue!

I'm afraid this might go deeper than just the humanization but I need to double check.

@JeroenDeDauw JeroenDeDauw transferred this issue from ProfessionalWiki/WikibaseEdtf Apr 27, 2021
@JeroenDeDauw
Copy link
Member

Partially a duplicate of #58

@JeroenDeDauw JeroenDeDauw added the bug Something isn't working label Apr 27, 2021
@JeroenDeDauw JeroenDeDauw self-assigned this Apr 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants