[SVCS-675] Add file size limit to tabular renderer. #325
Conversation
* Add more documentation for the `sheets` data structure. Rename or create some derived variables to make code clearer. * Log the number of columns and rows when one or the other exceeds the maximum allowed number.
* Tabular data has to be slurped into memory to be displayed, meaning large files can cause resource shortages or oom errors. Throw an informative error if the user tries to render a file larger than `MAX_FILE_SIZE` bytes. * Fix order of imports, removed old commented-out code.
| import os | ||
| import json | ||
| import logging | ||
| import os |
There was a problem hiding this comment.
Fix import order which should be length based.
There was a problem hiding this comment.
I thought for MFR we were doing alphabetical order, or am I misremembering that?
|
|
||
| from mfr.core import extension | ||
| from mfr.extensions.tabular import settings | ||
| from mfr.extensions.tabular import exceptions |
There was a problem hiding this comment.
How about from mfr.extensions.tabular import settings, exceptions?
| from mfr.extensions.tabular import settings | ||
| from mfr.extensions.tabular import exceptions | ||
|
|
||
| logger = logging.getLogger(__name__) |
| import logging | ||
| import os | ||
|
|
||
| from humanfriendly import format_size |
There was a problem hiding this comment.
👍
A nice package that I was trying to find a while ago.
| self.attr_stack.append([self.__TYPE, {'formatting_function': self.formatting_function}]) | ||
|
|
||
|
|
||
| class FileTooLargeError(TabularRendererError): |
There was a problem hiding this comment.
I remember you mentioned in WB but I am not sure about MFR: when an exception is added, should we update the tests somewhere to make sure that serialization works?
There was a problem hiding this comment.
Not needed for MFR. That test is to make sure the error survives the pickling/unpickling done by WB's celery worker.
felliott
changed the title
3 participants
Ticket
Not created yet
Purpose
Refuse to render tabular files if they exceed a certain size.
Changes
Add a configurable setting for tabular file size limits. If files is above this size, refuse to render it.
Side effects
QA Notes
Deployment Notes