fix: PDF reports #456
Merged
fix: PDF reports #456
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Allows easier calculation of pages prior to layout of images
Avoid pass-through of props to components nested in components
Support starting the report preview at a page > 1, which requires previous pages to be rendered first in order to establish which observation should appear on the current page. Also adds caching of observation PDFs (each observation is rendered to its own PDF during preview) and eagerly rendering the next observation, to reduce delay.
Previous code was causing React warnings of re-using the same context in multiple renderers, because we created a new renderer instance for each PDF preview doc (one for each observation). This changes the code to re- use the same renderer, and update it with the doc for each page. It should also be faster, since it's updating existing nodes rather than re-creating the react tree.
This can happen if the user changes the filter whilst the preview is still loading. If the filter has changed, then we want to cancel async functions that started with the previous filter
Known issues
Current behaviour: If the user's screen is small such that the width of the report is larger than their screen, then a horizontal scroll bar will show.
Current behaviour: During export a dialog will appear showing a spinning progress animation, but there is no indication about how long it will take. The progress animation can get stuck.
This is a hard fix: it's hard to calculate how long the report will take to render, it's hard to get updates during rendering (it's a single task that we start and wait for it to finish) and because it takes a lot of processing power, it freezes the loading animation. There are work-arounds for all of this, but it is quite a bit of work to fix.
It seems like if the user clicks around lots, back and forth to reports, then there is a very rare case where the report will just never appear, and Mapeo needs to be restarted. I have done a few hours of bug-hunting and am pretty sure this is a bug in the underlying PDF library, and it is very hard to reproduce. @jencastrodoesstuff do you think we can work around this for now? Still TODO
The page navigation buttons (for next page and previous page) are disabled if the user is on the first page or the last page (e.g. if the user clicks then it does nothing). However, we do not have a design for what a disabled button should look like. Normally a disabled button is grayed out (see https://material.io/design/interaction/states.html#disabled) but I was not sure how to design these buttons. @macandrow can you suggest a design for disabled navigation buttons?
In testing it seems like report pages seem to show all fields for a preset (or maybe all fields that appear across all data in Mapeo?) and for each shows "No Answer". I think we should maybe not display fields that are not filled in, only showing completed fields? What do you think @jencastrodoesstuff ?
Currently if you navigate away (e.g. to map or media view) and back, then the report preview will return to page 1. I know @jencastrodoesstuff identified this as an issue before, but does it remain an issue now that you can edit the observation from the report view? |
|
the animation bug is fine for now. |
|
Agree about hiding empty fields. |
|
switching views resetting to page 1 is ok for now. Yes with edit available from report pages this is less of a workflow issue. |
Using the same renderer instance and updating the container was causing race conditions, and does not seem to change performance much.
Changes babel preset-env to reduce polyfills and transpilation
ErikSin
pushed a commit
that referenced
this pull request
Aug 3, 2021
* feat: auto-update Mapeo when internet is available (#346) * chore(build): remove parallel requirement * chore(release): 5.4.0-beta.0 * chore(build): Typo for uploading excecutables to AWS * chore(release): 5.4.0-beta.1 * chore(release): Workaround attempt to fix an electron-builder bug * chore(build): Add providers for github and s3 for each target * chore(build): Manually add app-update.yml * chore(release): 5.4.0-beta.2 * chore(build): try new app-update.yml through extraResources * chore(release): 5.4.0-beta.3 * fix: Multiselect field should not error if has select_one data (#390) * fix: Only allow updater when on compatible platforms * fix: update to electron 9.1.2 (#389) * chore(release): 5.4.0-beta.4 * fix: Use https://downloads.mapeo.app for updater endpoint * fix: Error where download speed would always show 0 seconds * fix: Only fetch download speed when opening the Updater tab. * Revert "fix: Only fetch download speed when opening the Updater tab." This reverts commit 6e5645f. * fix: Properly return error when updater is inactive * chore(release): 5.4.0-beta.5 * chore(release): 5.4.0-beta.5 * fix: Make left-hand side copy more simple * fix: Ensure all dialogs will work with Electron 9 (#391) * fix(typo): add missing 'notes' field in fallbackFields * chore: Remove unnecessary matrix config from github action CD (#397) * chore: Remove un-used architecture option for actions/setup-node (#394) There is a PR to add an architecture option but it is not yet merged: actions/setup-node#42 * fix: Notarize Mapeo for MacOS — fix warning / unable to open app (#396) * fix: Notarize Mapeo for MacOS — fix warning / unable to open app * fix: Add secrets * fix: remove .map that should not have been there * fix: Fix icon layout in dmg for MacOS (#399) * chore: Put 'Check for Updates' in the help menu * fix: Only save the language to config when changed manually. Fixes #400 Mapeo was saving the system locale to the config always on startup, so if you change system languages and reopened Mapeo it wouldn't also change system languages. This now only persists the change in language if the user explicitly requests to change the language through the User Interface. * fix: Mapeo config directory should be removable (#401) * chore: Merge mapFilter@3.2.5 & add bundling & code-splitting (#393) Co-authored-by: okdistribute <633012+okdistribute@users.noreply.github.com> * New Report Generator (#303) Co-authored-by: Gregor MacLennan <gmaclennan@digital-democracy.org> * chore: package-lock * chore(release): 5.4.0-beta.6 * fix: Ensure global OsmServerHost is always set * fix: Mapeo should always open. Removes png editing for now. * chore(lint): remove unused dependencies * Update PULL_REQUEST_TEMPLATE.md * chore: update translations * chore(release): 5.4.0-beta.7 * fix: Cleanup report code - Show toolbar during load - Create ReportContentComponent -> allows memoizing PDFReport - More robust loading code (patch for @react-pdf see diegomura/react-pdf#995) * chore: gitignore generated worker files * fix: Fix print button (still downloads, does not print) PrintButton was using `this.url` instead of `this.props.url`. However also removed some unnecessary code in this component * fix: Fix map size (make square) * fix: Render portrait images correctly and fix image width * fix: Pull MAPBOX_ACCESS_TOKEN from a static location for both front & backend * feat: Add marker to map (first rough attempt) * fix: make some progress on flow and types, not fixing all things but many * feat(staticMaps): Add concurrency and caching to speed up static maps NB: Caching will not be noticable in dev, because currently npm start and npm run dev will turn off http caching, but have tested this works when http caching is enabled. * feat(staticMaps): Support style and accessToken as URL query params * feat(Reports): Use user defined map style for report maps * fix(ReportMap): Add border around photos and simplify CSS * fix(ReportDetails): Only show "Details" header when there are fields * chore(lint) * fix: translate report headers * chore: translations * fix: only display the first 5 character of the project secret key (#419) * fix: only display the first 5 character of the project secret key The rest is masked as "*" * trigger GitHub actions * fixed length mask to avoid potential UI scaling issues * feat: display additional metadata in SyncFooter * feat: show 'MAPEO' when there's no encryption key * Add desktop architecture diagram * chore(test): add proper mocks to storybook * fix: Cleanup report code - Show toolbar during load - Create ReportContentComponent -> allows memoizing PDFReport - More robust loading code (patch for @react-pdf see diegomura/react-pdf#995) * chore: gitignore generated worker files * fix: Fix print button (still downloads, does not print) PrintButton was using `this.url` instead of `this.props.url`. However also removed some unnecessary code in this component * fix: Fix map size (make square) * fix: Render portrait images correctly and fix image width * fix: Pull MAPBOX_ACCESS_TOKEN from a static location for both front & backend * feat: Add marker to map (first rough attempt) * fix: make some progress on flow and types, not fixing all things but many * feat(staticMaps): Add concurrency and caching to speed up static maps NB: Caching will not be noticable in dev, because currently npm start and npm run dev will turn off http caching, but have tested this works when http caching is enabled. * feat(staticMaps): Support style and accessToken as URL query params * feat(Reports): Use user defined map style for report maps * fix(ReportMap): Add border around photos and simplify CSS * fix(ReportDetails): Only show "Details" header when there are fields * chore(lint) * fix: translate report headers * chore: translations * Update README.md * fix: Ensure any stale processes are cleaned up before starting ipc workers * fix: Save PDF now prints all observations * fix: Refactor Save PDF to include typed options & fix flow * chore(flow): Ensure datefilter value is defined when accessed * fix: Styling of loading dialog & Add date/time report name * fix: Map printer should work on the current offline style * chore: Update desktop architecture documentation * fix: write correct pid file * chore: lint * fix: Ensure mapeo always closes and show closing window while processes are being destroyed. Fixes #398 * fix: wrap win/splash.close in try/catch as electron can throw * fix: Show Single & Multi-select fields in Report View * chore: Cleanup unused files * chore(test): add proper mocks to storybook (#422) * chore: move flow types into mapeo-schema (#420) * fix: Bugs in menu * Update QA.md * fix: config now included in bundle * fix: If subprocess failed with fatal error, it should not prevent Mapeo from closing. * fix: Ensure process is always closed properly and mapeo can reopen easily in dev mode * fix: Updater should not throw visible error when offline * fix: standard * fix: Sometimes Mapeo would continue running if Splash was closed before window finished loading * fix: Leave port & map printer port control to Electron main process * fix: if Mapeo fails to close gracefully, kill the process * fix: Map printer will close when electron window closes it, no need to wait for graceful close * fix: cleanup debug messaging on killing node proc * fix: ensure background code gets included in bundle * fix: refactor node-ipc main process listeners * fix: remember debugging settings on rotate and mapeo start * fix: Splash screen would pause for a second with black background before starting animation * chore(flow): update to latest mapeo-schema * chore(deps): remove react-mapfilter, update package-lock.json * fix: Use mapeo.ipc directly * chore: Add debugger statements for errors when using menu * chore: lint * chore(flow): fix flow types * fix: dont start heavier processes until splash screen is loaded * fix: Multiselect fields with commas were not displaying correctly in Territory view * chore(release): 5.4.0-beta.8 * fix: Prevent a bug that caused MapEditor to crash if you have no nodes after a sync. * chore(release): 5.4.0-beta.9 * chore: Update CHANGELOG.md * fix: Fix pdf report image wrapping (#436) * fix: Extract English language messages for CrowdIn * fix: Add separators in Help menu * fix: Fallback to exporting preview-sized images if originals are missing (#437) Co-authored-by: Karissa McKelvey <633012+okdistribute@users.noreply.github.com> * Fix: Display error to user when it is a fatal mapeo core error (#439) * New Crowdin updates (#388) Co-authored-by: Karissa McKelvey <633012+okdistribute@users.noreply.github.com> * chore: Add error log messages * chore: Improve onboarding documentation for new devs. * fix: Add Vietnamese, Thai, and Khmer to available languages. * fix: Remove duplicate styling for updater loading progress. * chore(release): 5.4.0-beta.10 * chore: lint * chore: Add Thai translations for menus and main process strings (#445) * New translations en.json (French) * New translations en.json (Spanish) * New translations en.json (Vietnamese) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Khmer) * New translations en.json (Thai) * New translations en.json (Burmese) * New translations en.json (Lao) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Vietnamese) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Spanish) * chore: Remove untranslated strings (#450) * New translations en.json (French) * New translations en.json (Spanish) * New translations en.json (Vietnamese) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Khmer) * New translations en.json (Thai) * New translations en.json (Burmese) * New translations en.json (Lao) * New translations en.json (Thai) * New translations en.json (French) * New translations en.json (Spanish) * New translations en.json (Vietnamese) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Khmer) * New translations en.json (Burmese) * New translations en.json (Lao) * feat: Add Thai translations (#451) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Thai) * fix: Ensure that fallback presets do not break iD Editor iD Editor requires that presets `point`, `area`, `line`, `relation` are always defined. The user can override these via mapeo config to give them custom names and fields, however the user could also accidentally set the `tags` and `geometry` properties on these presets to something that breaks iD. This fix ensures that these "special" presets always have the correct tags so that iD does not break. * fix(MapExportDialog): fix React controlled/uncontrolled state warning (#459) * fix: PDF reports (#456) * chore(PDFReport): Code formatting * feat(Reports): Fit images within a square Allows easier calculation of pages prior to layout of images * chore(Reports): Refactor PDF Report component Avoid pass-through of props to components nested in components * Allow navigation through entire report * tweak CSS to show report paper outline clearly * Smooth loading state transitions * Cache and eagerly render PDF pages, and support initial page > 1 Support starting the report preview at a page > 1, which requires previous pages to be rendered first in order to establish which observation should appear on the current page. Also adds caching of observation PDFs (each observation is rendered to its own PDF during preview) and eagerly rendering the next observation, to reduce delay. * chore: tidy up * Add initialPageNumber option and add stories * Add current page number * chore: Small type fixes * chore: Rename and add code comments * fix: Queue PDF renders to use the same react-pdf instance Previous code was causing React warnings of re-using the same context in multiple renderers, because we created a new renderer instance for each PDF preview doc (one for each observation). This changes the code to re- use the same renderer, and update it with the doc for each page. It should also be faster, since it's updating existing nodes rather than re-creating the react tree. * fix bug where loop could continue to infinity * add missing dep to useEffect * Update intl messages * commit case change to git * Add save button * Add support for intl characters in PDF report * fix webpack config to load fonts * Add new design page navigator * Format report toolbar and add x out of y observations count * chore: fix types * fix page navigator so that buttons are equally sized * fix toolbar so that text is centered to page * remove bottom border radius on page navigator * 🎉 New report design & layout * remove eager rendering (premature optimization, and caused problems) * 🚀 page numbers in footer * Fix re-render of PDF after cancelling edit of observation * chore: Catch errors in PDF render and report them * fix: Reset to page 1 when the user is on an invalid page * chore: Change startPage to actually mean statePage (not zero-based idx) * Turn eager rendering back on * chore: Unused var * Add storybook for observation with no location * chore: Add types * 🥳 Add icons to observations in report * fix: patch hidden-mapbox so it does not hang * fix: timeout PDF render * fix: Cache category icon * fix: Don't cache failed preview renders * fix: pageIndex can end up with `null` values sometimes (not sure how!) * fix: If preview is cancelled then bail This can happen if the user changes the filter whilst the preview is still loading. If the filter has changed, then we want to cancel async functions that started with the previous filter * fix: Catch infinite loop in preview * feat: Use same map marker for observation as mapeo mobile * fix: Fix patch to static map renderer * fix: Only show fields with answers in report * chore: Add Story for testing long preset names * Add disabled state styling to page navigation buttons * fix: Use new instance of ract-pdf for each render Using the same renderer instance and updating the container was causing race conditions, and does not seem to change performance much. * chore: Use forked react-pdf/renderer Changes babel preset-env to reduce polyfills and transpilation * chore: Remove hidden-mapbox patch * chore(release): 5.4.0-beta.11 * fix: Fix errors from Wifi Status display and disable on MacOS (#470) Fixes #469 * fix: Fix "zoom to data": include deletions & correct density calc (#475) * fix: Fix "zoom to data": include deletions and correct density calc * fix: In territory view, zoom to data also considers observations * chore: Update node version in Github Action * feat: Update iD Editor background imagery (fix mapbox satellite) (#477) Fixes #476 * fix: Fix crash when trying to resize sidebar (#478) * fix: Fix crash when trying to resize sidebar Fixes #473 * fix: Add button to un-collapse sidebar if collapsed * fix: Fix Mapeo window randomly not opening at startup (#465) * chore: Add incremental Typescript checking Add // @ts-check to the top of files in src/background and src/main to check JS files with JSDoc hints * Refactor app startup code & background processes * Manage background processes together, and time async startup actions * Read backend state in main window * Startup sequence in parallel * remove console.log * Fix startup sequence * Fix background state updates (emit after state is updated) * Update to latest electron@9 * Fix ipc message validation * Fix settings extraction * Wait for first render before showing main window * Improve logging * fixes for breaking mkdirp change * add note about React devtools * fix lint error * chore(tsconfig): fix typo * fix packaging config * fix: Fix errors from Wifi Status display and disable on MacOS Fixes #469 * fix: Package default config with app, rather than extracting at runtime * fix: Fix IPC messaging with main process - call port.start() * fix: Fix "zoom to data": include deletions and correct density calc * fix: Fix crash when syncing by removing non-transferable objects from IPC * Add message error handlers to IPC * fix: Fix reload of Mapeo when config changes * chore: update ecstatic & dedupe * Fix package-lock * Catch errors on startup * Extend background process start timeout * throw error if starting background process fails * Pass through error events from background processes * Cleanup subscriptions * Clear closing timeout Co-authored-by: Kira Oakley <kira@eight45.net> * chore: Fix storybook * fix: Don't allow observation location to be changed in territory view Fixes #479 * chore: Remove unused files * feat: Add Khmer, Dutch & French translations, fix th,pt,vi (#460) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Spanish) * New translations en.json (Vietnamese) * New translations en.json (Thai) * New translations en.json (Dutch) * New translations en.json (Dutch) * New translations en.json (Sranan Tongo) * New translations en.json (Sranan Tongo) * New translations en.json (French) * New translations en.json (French) * New translations en.json (Khmer) * New translations en.json (Khmer) * New translations en.json (Khmer) * New translations en.json (Khmer) * New translations en.json (Khmer) * New translations en.json (Khmer) * New translations en.json (Khmer) * New translations en.json (Khmer) * New translations en.json (Thai) * New translations en.json (Khmer) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Khmer) * New translations en.json (Khmer) * New translations en.json (Khmer) * New translations en.json (Khmer) * New translations en.json (Khmer) * New translations en.json (Khmer) * New translations en.json (Khmer) * New translations en.json (Vietnamese) * New translations en.json (Vietnamese) * New translations en.json (Vietnamese) * New translations en.json (Vietnamese) * New translations en.json (Vietnamese) * New translations en.json (Vietnamese) * New translations en.json (Vietnamese) * New translations en.json (Vietnamese) * New translations en.json (Vietnamese) * New translations en.json (Vietnamese) * New translations en.json (Vietnamese) * New translations en.json (Vietnamese) * New translations en.json (Khmer) * New translations en.json (Khmer) * New translations en.json (Dutch) * New translations en.json (Dutch) * New translations en.json (Dutch) Co-authored-by: Gregor MacLennan <gmaclennan@digital-democracy.org> * fix: Fix npm ci failure in Github actions by retrying until it works * chore: update to react-pdf latest (#484) * chore: Update to @react-pdf/renderer@2.0.15 * Fix layout changes in react-pdf@2 * Fix: Move PDF reports to a web worker (#482) * WIP: Move PDF reports to a web worker * uri-templates * Testing pdf in worker * Fix inset maps and preset icons * Fix: Show observation ID in mono-spaced font * fix: Update import/export labels for territory data (#486) * Update import data menu item label * update territory export button label * New translations en.json (Khmer) * New translations en.json (Thai) * New translations en.json (Thai) * New translations en.json (Khmer) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Vietnamese) * New translations en.json (Vietnamese) * New translations en.json (Dutch) * New translations en.json (Dutch) * New translations en.json (Spanish) * New translations en.json (Spanish) * New translations en.json (French) * New translations en.json (Spanish) * New translations en.json (Dutch) * New translations en.json (Vietnamese) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Khmer) * New translations en.json (Thai) * New translations en.json (Portuguese, Brazilian) Co-authored-by: Karissa McKelvey <633012+okdistribute@users.noreply.github.com> Co-authored-by: okdistribute <karissa@noreply.github.com> Co-authored-by: Gregor MacLennan <gmaclennan@digital-democracy.org> Co-authored-by: Poga Po <poga.po@gmail.com> Co-authored-by: Lou Huang <lou@louhuang.com> Co-authored-by: Kira Oakley <kira@eight45.net>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Work in Progress to fix the new PDF reporting feature to meet user requirements for release:
Tasks:
Cache and use page renders for final render of page to speed up export (stretch goal)Show progress during rendering of whole report on save (stretch goal)Reduce report preview width to screen width on smaller displaysStorybook preview of report view (Requires Google Chrome)