JSON.parse error if multiple pages are scanned/processed

[atkalcec]

First of all, thanks a lot for that amazing piece of software. It enabled me to still use my old hp3900 scanner without worrying about drivers and vendor specific UIs. Furthermore, scans just being accessible via a awesome web interface makes it even more brilliant!

Describe the bug
I'm on version v2.11.0 and running on a RaspberryPi (RPi 3 Model B). I've installed scanservjs manually. All worked fine. However, I just experienced that if scanning multiple pages (>20 and more), clicking afterwards on the Finish button, I get an error after a while telling "SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data" - the webpage switches then still to the 'Files' page, but I can't see the converted scan (PDF) within the files. However, it looks like that the Pi is still busy processing, so if I just wait a while, clicking back to the 'Scan' page and back to the 'Files' page, I'm able to see the converted scan (PDF). For me it looks like a 'race condition' between the Pi still processing something and the scanservjs already expecting it to be finished (just a guess).

Settings
{ version: '2.11.0', params: { deviceId: 'hp3900:libusb:001:004', top: 0, left: 28, width: 192, height: 289, resolution: 150, mode: 'Color', source: 'Flatbed' }, filters: [], pipeline: 'PDF (JPG | Medium quality)', batch: 'manual', index: -1 } }

Client:

• Browser Firefox
• Version 88.0

Server:
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)" NAME="Raspbian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=raspbian ID_LIKE=debian HOME_URL="http://www.raspbian.org/" SUPPORT_URL="http://www.raspbian.org/RaspbianForums" BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

Logs
scanservjs.log.installed.txt

[sbs20]

First of all, thanks a lot for that amazing piece of software. It enabled me to still use my old hp3900 scanner without worrying about drivers and vendor specific UIs. Furthermore, scans just being accessible via a awesome web interface makes it even more brilliant!

No, thank you! Comments like this make it worthwhile!

Thanks also for raising the issue - I can from the logs that something odd is happening. I'll need time to look into it.

[sbs20]

This is odd.

The final request to finish up the scan happens:

May 01 15:22:59 ty-beta server.js[521]: [2021-05-01T13:22:59.327Z] DEBUG (Http): request:  { method: 'POST',

That process finishes up, as expected here:

May 01 15:24:41 ty-beta server.js[521]: [2021-05-01T13:24:41.151Z] DEBUG (Scan): Written data to: ./data/output/scan_2021-05-01 15.24.40.pdf

Which is about 1min 42 seconds later.

But 42 seconds earlier the browser requested the files:

May 01 15:23:59 ty-beta server.js[521]: [2021-05-01T13:23:59.740Z] DEBUG (Http): request:  { method: 'GET', path: '/files' }

The file request in the browser should wait until the previous process has finished before requesting files. But it isn't.

One thing that makes me suspicious is that the file request is exactly 60 seconds after the scan finish request. Not only that but you see SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data which also indicates that the browser hasn't seen a legitimate JSON response.

My hypothesis is that the server is terminating its response after a timeout - with no JSON. My guess is that this isn't normally visible because:

• Scans take less than a minute
• Post processing (where it collates all the imates into a PDF) takes less than a minute

This is likely exacerbated because this is running on a slower CPU.

Next steps:

• I will attempt to reduce the timeout to reproduce the issue (then increase it as a fix, assuming I can repro).
• If you have time, could you try doing it again with, say, 19 pages? 34 pages is taking 102 seconds - so 19 should be 57. 20 or 21 pages should tip it over the edge.

[sbs20]

I have not been able to reproduce this issue. Although I have been able to get some odd behaviour by changing timeouts.

Out of interest, would you be prepared to change ./server/configure.js (it's in /var/www/scanservjs by default) and update the following code (around line 159)

  app.post('/scanner/scan', async (req, res) => {
    // Add this line for a 3 minute timeout
    req.setTimeout(180000);
    logRequest(req);
    try {

It's not a solution - but it will help debug to see if I'm on the right lines.

Update: the only other thought I had was... You're not running this behind a proxy are you? Like nginx? That might cause issues

[atkalcec]

Hi Sam,
Sorry for the late replay. Typically I do my scans during weekends. Hopefully I haven't wasted your time, but just saw your last comment: "You're not running this behind a proxy are you? Like nginx?". Actually yes, I do run scanservjs via nginx. Shall I still try your suggestions in previous comments (i.e. continue debugging)? If it boils down to a issue most likely cause by nginx, feel free to close this issue. Appreciate all your help :)

BTW, one thing, is there a way to also modify 'Batch mode' e.g. from 'None' to 'Manual (with prompt)' via config.local.js?

[sbs20]

Hopefully I haven't wasted your time, but just saw your last comment

Not at all - I'm pleased! Don't bother changing any of the code though. Instead, just increase the nginx timeout. See something like: https://ubiq.co/tech-blog/increase-request-timeout-nginx/

[sbs20]

is there a way to also modify 'Batch mode' e.g. from 'None' to 'Manual (with prompt)' via config.local.js?

Do you mean, change the default? If so, not yet. I'll have a think. Is it not enough that the last used setting is saved?

See #265

[sbs20]

See #266