v0.2.0

[github-actions[bot]]

Security

• A run that can't reach Paperless, because Paperless is down or its name or certificate fails, no longer writes the Paperless API token into the Nextcloud log. Before, the log received the token in plain text with the trace of the error, with every failed run by hand or by the schedule, and so did everyone who reads the log or receives a copy of it. Every earlier version is affected: create a new API token in Paperless, enter it in the settings, and delete the old Nextcloud logs and their copies, such as in backups or log collectors.
• The API token goes only to the Paperless URL that it was saved with, so another URL needs the token again. Before, saving without a token sent the stored token to the new URL to test the connection, so whoever could save the settings, with the session of an administrator for example, could receive it at a server of their own. The settings page asks for the token as soon as the URL changes. A URL changed past the settings page, with occ or Nextcloud's provisioning API, doesn't get the token either: synchronization pauses until the token is entered again. The update keeps the stored token for the URL that is configured.
• Saving the settings and disconnecting Paperless ask for the password of the administrator, as Nextcloud does for its own sensitive settings, unless it was confirmed in the last 30 minutes.
• A Paperless at an address of your network, such as http://paperless:8000 in Docker or http://192.168.1.20:8000, works with the new setting Paperless runs in my local network, which allows such an address only for the requests of Paperless Sync to the saved Paperless URL. Before, saving failed with Host "paperless" violates local access rules, and the only way out was the switch allow_local_remote_servers, which turns off Nextcloud's protection against server-side request forgery for every app and every user. Saving and a run by hand now say what to do instead, also for a name that only the hosts file of the server knows, and the settings page moves the focus to the setting. While the setting is off, a download that Paperless redirects to an address of your network is refused, also when Nextcloud's dns_pinning is turned off. If you turned that switch on for this app, turn the setting on and save, then turn the switch off again: occ config:system:delete allow_local_remote_servers.

Features

• The settings page shows how the last run ended at every visit, also a scheduled run that nobody watched: its state, in red when it failed or had errors, when it finished, the last successful run, the next scheduled run, the errors of the run in a box of their own, and its report. While a run is active, the page follows it until it ends.
• The Nextcloud log has a warning, with the ID and the path of the document, when an error of a document first occurs or changes. An error that comes back in every run is logged once.

Bug fixes

• On Paperless-ngx 3.x, the .error.txt of a failed import of the inbox has the reason of Paperless again, such as a file type that it doesn't take, and the ID of the Paperless task. Before, it said only Paperless import failed: Paperless-ngx 3.x answers in version 10 of its API by default, whose shape the app didn't read.
• A file of the inbox whose task Paperless cancelled, or has not finished a day after the upload, no longer waits in the inbox for good. A cancelled task, REVOKED in the task list of Paperless, moves the file to the error folder with that reason. A day after the upload, a file whose content Paperless has, found by its checksum, counts as imported. A task that Paperless still lists as pending or started moves the file to the error folder, with a note that the document may still appear, because Paperless-ngx 3.x would import a second upload as a duplicate. A task that Paperless no longer lists, as after a restart of its queue, sends the file up again after 15 minutes, with an error in the report and a warning in the Nextcloud log.
• Every request asks for version 9 of the Paperless API, so that a new default of Paperless can't change the answers that the app reads. The settings page names the Paperless that the last run reached, such as The last run reached Paperless-ngx 3.3.0 through version 9 of its API., and the report names the document that a successful import created, such as IMPORT SUCCESS: Dokumente/Paperless/Eingang/police.pdf -> P456. Paperless Sync now needs Paperless-ngx 2.16 or newer, the first version with version 9 of the API; with an older Paperless, saving and every run say so.
• A new content of a document, such as a new archive version after Paperless processed it again, updates its file in Nextcloud in place, so the file keeps its shares and public links, comments, tags and favorites, and in a scheduled run its previous content becomes a version of the file. Before, the app deleted the file and wrote a new one with a new file ID: shares and public links of the file broke, its comments, tags and favorites were lost, the old copy filled the trash bin of the target user, and the desktop and mobile clients downloaded it as a new file. A document whose content and path change moves first and is updated at its new path.
• A document that Paperless restores from its trash, or lists again after it was missing, keeps the copy that the trash behavior Keep archive file in place left in place, without a download. Before, the run downloaded it and wrote it again.
• With the conflict policy Skip and report conflict, the app updates its own copies: a document whose content changed while its path stayed the same, or which came back after it was missing, is no longer reported as a conflict in every run while its copy stays outdated. A file that someone else put at the path, and a copy that someone changed in Nextcloud, also one whose document gets a new path, are still left alone and reported as a conflict.
• You can pause scheduled synchronization while Paperless is down, its token revoked or the target user disabled: uncheck Enable scheduled synchronization and save. Saving tests the connection only when the schedule is on or a new token is entered, and says so when it didn't. Before, every save had to pass the test, so the only ways to stop a schedule that failed every few minutes were Disconnect, which deleted every setting, and occ.
• Disconnect deletes the API token and turns off scheduled synchronization and Paperless runs in my local network, and keeps the other settings, also your deletion policy, as its question says. Before, it deleted every setting while the page went on showing the old values.
• The settings page always shows what Nextcloud stored, also after a save or a disconnect, such as the URL without a slash at its end, and says when the form has changes that aren't saved, also changes that you make while a save is in progress. Run dry-run and Synchronize now use the saved settings, so Synchronize now waits until you save, and the browser asks before you leave the page with unsaved changes. Before, a run could apply settings that the page didn't show.
• Results and errors appear next to the buttons, which stay at the bottom of the window, and an error takes the focus. Before, they appeared below the long form, out of view. The page no longer shows the HTML of a proxy, such as its 504 Gateway Time-out during a long run: while the status shows the run, it says that the run continues and follows it there. A lost connection and an expired session are explained in your language, a run with errors says how many, and a failed refresh of the status no longer replaces the result of the run.
• The save button says Save when saving doesn't test the connection, with the schedule off and no new token, and Test connection and save when it does. An error that names a setting, such as Enable scheduled synchronization after a failed test, scrolls that setting into view, where the buttons at the bottom of the window no longer hide it. The hint to turn off the schedule is in German for German administrators. When neither a run nor the status answers, the status no longer shows the run as running for good, and the message asks you to refresh it.
• The questions before a run and before Disconnect, and the other messages of the settings page itself, are in German for German administrators.
• A dry-run no longer pushes back the next scheduled run: the interval counts from the last real run, also one started with Synchronize now. Before, a dry-run at 10:00 with an interval of an hour moved the next run to 11:00. Only a dry-run that is still running holds the next run back, until it ends.
• The interval is at least 5 minutes, as a hint below it explains: Nextcloud starts background jobs every 5 minutes at most. Before, the settings accepted 1 to 4 minutes, which acted as 5. An interval below 5 minutes that you saved before acts as 5 minutes, as it did, and the settings show it as such.
• Disconnect sits at the right end of the bar of buttons, away from Synchronize now, and every checkbox sits next to the first line of its label. Before, the styles of Nextcloud put Disconnect right after Synchronize now and pushed every checkbox below its label. The titles of the sections no longer wrap next to their description, and each section links to its part of the guide.
• The settings of deletion say in plain words what they do: turn on permanent deletion only after a few runs with deletion off looked right, and Also delete copies of documents that disappeared without passing the Paperless trash, which replaces Also allow deletion when the app never observed the Paperless trash state.
• On Paperless-ngx 2.x, a change of the title, tags, correspondent or type of a document moves its file to the new path, as on Paperless-ngx 3, instead of downloading the document again and replacing the file. Before, the app only recognized the SHA-256 checksums of Paperless-ngx 3, so on 2.x every such change gave the file a new ID in Nextcloud, which lost its shares, comments and favorites, left the old copy in the trash bin and made the desktop and mobile clients download it again. The app now keeps the SHA-256 and the MD5 of every download. For a file that an earlier version exported, the first such change reads the file in Nextcloud once to add its MD5, instead of downloading the document again, also when its move waits for a later run because the batch is full or the quota is reached.
• A large document no longer ends the run with a fatal error of PHP's memory_limit. Before, the app read every download into memory to compute its checksum, so a document larger than the free memory stopped the run before it saved its state, the status stayed running, and every following run died on the same document. Downloads now go from a temporary file to Nextcloud in chunks.
• The report of a run with more than 100 lines lists its errors, deletions, removals of excluded copies and moves to the deleted folder first and says how many lines it leaves out. Before, it dropped every line after the hundredth without a word, so that a dry-run with many files in the inbox could hide the deletions that the run would make.
• A run that fails no longer counts as the last completed run; the status shows its end as Last finished.
• A run no longer counts a document as missing when Paperless answered a page of its list incompletely, such as an empty page from a proxy, or when documents were added or deleted while the run read the pages. The report says SCAN INCOMPLETE, the Nextcloud log has a warning, and the next complete scan carries on. Before, a document that still existed in Paperless could move to the deleted folder, or be deleted when direct deletion is on.
• A document that Paperless lists again starts its count of missing scans anew, also when its export fails or waits for the next run, so that only scans in a row count.
• Every export that a run tries counts against the batch size, also one that fails, and a document whose export failed waits behind the others. Before, a run that couldn't write, against a full quota or a read-only storage for example, downloaded every pending document again in every run and failed on each one. Failed exports don't take the places of moves of trashed or missing documents, and a move or deletion that fails takes its place in the batch too.
• Exports respect the quota of the target user. Nextcloud applies no quota to the temporary file of a write, so before, an export could fill the quota beyond its limit. When the quota is full, or the archive is read-only or its storage unavailable, a run stops exporting and moving at the first failure and reports EXPORT STOPPED; the other documents follow in the next run. A file name that Nextcloud refuses concerns only its document.
• A run exports the most recently added documents first, also a newly scanned letter with an old date, so that it doesn't wait behind a backlog such as the first export of a large archive.
• A run fails, before it changes anything in the archive, when Paperless answers its list of tags, correspondents, document types or storage paths incompletely. Before, a document with an excluded tag could be exported when the tag was missing from a short page.
• The report keeps the lines SCAN INCOMPLETE and EXPORT STOPPED also after a hundred other lines.
• With the conflict policy Skip and report conflict, a file in the way is reported before the document is downloaded, and a dry-run reports the conflict too. Before, the run downloaded the document in vain.
• A $ in a title or a name stays in the path of the file: "Refund $100 from Shop" no longer becomes "Refund 0 from Shop", and "Tarif $0 Basis" no longer takes in a piece of the template. A title that contains a variable, such as {{ correspondent }}, stays as it is too.
• Documents with a long title or name in Cyrillic, Chinese, emoji or another script of several bytes per character reach Nextcloud again. Names are shortened to the bytes that Nextcloud accepts, with room for the temporary name of a write. Before, such a document failed in every run.
• A long title, correspondent or original file name no longer cuts the marker [P123] and the extension off a file name. Before, two such documents could get the same path, and with Replace synchronized file the second one replaced the file of the first. Now only the free text is shortened, the longest first, and a template that leaves out the marker or the extension gets them added at the end of the file name.
• The path template accepts variables without spaces, such as {{id}} for {{ id }}, and the message about an unknown variable names it as {{ owner }}.
• Control characters beyond ASCII and invalid bytes become _ in names too, and names of Windows devices such as COM0, LPT¹ or CONIN$ get a leading _ as well.
• The next run moves the files whose names these fixes change once to their corrected path. Two documents that shared one file are downloaded again instead, so that each gets its own content back.

Every pull request of this release

⚠ BREAKING CHANGES

• inbox: read Paperless 3.x task results, give up on lost tasks and pin the API version (#144)

Features

• status: keep and show the errors and the report of the last run (#106) (68f172f)

Bug fixes

• archive: render path templates safely and always keep the [P id] marker and the extension (#94) (2642dec)
• inbox: read Paperless 3.x task results, give up on lost tasks and pin the API version (#144) (817cfd5)
• security: keep the Paperless API token out of logs and ask for it again for a new address (#105) (e0898f9)
• security: reach a Paperless in the local network without turning off Nextcloud's SSRF protection (#111) (31c5bc3)
• settings: an honest save button, visible focus, no endless running state and a translated schedule hint (#146) (628c886)
• settings: honest interval, dry-runs that don't postpone the schedule, aligned checkboxes and Disconnect (#137) (cf70a84)
• settings: keep the form in step with the server, allow pausing while Paperless is down, and make feedback reliable (#122) (a5a18df)
• sync: keep every run within the batch size and never treat an incomplete scan as deletions (#92) (f9030c0)
• sync: read a mirrored file once also when its move waits for a later run (#133) (1cd1a77)
• sync: recognize Paperless 2.x checksums and hash downloads without loading them into memory (#120) (a25b4e3)
• sync: update mirrored files in place so shares, versions and the file ID survive (#125) (dd59ca4)

Compare with the previous release

---

This discussion was created from the release v0.2.0.