Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Network failure can cause the program to be stuck in httpStream #80

Closed
binsee opened this issue Feb 2, 2023 · 2 comments · Fixed by juzibot/file-box#31
Closed

Network failure can cause the program to be stuck in httpStream #80

binsee opened this issue Feb 2, 2023 · 2 comments · Fixed by juzibot/file-box#31

Comments

@binsee
Copy link

binsee commented Feb 2, 2023 •
edited
Loading

When filebox saves files from url, if the network fails, the program will be stuck and not return.

Test code

// src/test.ts

import { randomUUID } from 'crypto'
import { rm } from 'fs'
import { setTimeout } from 'node:timers/promises'
import { promisify } from 'node:util'
import { tmpdir } from 'os'
import { join } from 'path'
import { FileBox } from './mod.js'

async function testFileBox() {
  const localPath = join(tmpdir(), randomUUID())
  // from www
  const url = 'https://nodejs.org/dist/v18.13.0/node-v18.13.0.pkg'

  for (let i = 0; i < 1000; i++) {
    try {
      await promisify(rm)(localPath)
    } catch (error) {}
    console.log(`${new Date().toLocaleTimeString()} [${i}] start`)

    await FileBox.fromUrl(url).toFile(localPath, true).catch((e: Error) => {
      console.error(`${new Date().toLocaleTimeString()} [${i}] toFile failed!`, e.stack)
      throw e
    })

    console.error(`${new Date().toLocaleTimeString()} [${i}] save done`)
    await setTimeout(2000)
  }
}

void testFileBox()

Test step

Run: NODE_DEBUG=net NODE_OPTIONS="--no-warnings --loader=ts-node/esm" node src/test.ts

While outputting NET 33028: _read, disconnect from the network and reconnect after a few seconds. It will be found that the program is still stuck there, and has not been executed downward, and no error has been reported.

NET 33028: pipe false null
NET 33028: connect: find host nodejs.org
NET 33028: connect: dns options { family: undefined, hints: 1024 }
NET 33028: _read
NET 33028: _read wait for connection
NET 33028: afterConnect
NET 33028: _read
NET 33028: Socket._handle.readStart
NET 33028: _read
NET 33028: _read
NET 33028: _read
NET 33028: _read

After fixed

Fixed by #81

  • log 1
01:49:42 [0] start
NET 63846: pipe false null
NET 63846: connect: find host nodejs.org
NET 63846: connect: dns options { family: undefined, hints: 1024 }
NET 63846: _read
NET 63846: _read wait for connection
NET 63846: afterConnect
NET 63846: _read
NET 63846: Socket._handle.readStart
NET 63846: _onTimeout
NET 63846: destroy
NET 63846: close
NET 63846: close handle
NET 63846: emit close
01:49:53 [0] toFile failed! Error: Http request timeout!
    at ClientRequest.<anonymous> (file:///Users/binsee/CodeWork/file-box/src/misc.ts:141:19)
    at Object.onceWrapper (node:events:627:28)
    at ClientRequest.emit (node:events:513:28)
    at ClientRequest.emit (node:domain:489:12)
    at TLSSocket.emitRequestTimeout (node:_http_client:839:9)
    at Object.onceWrapper (node:events:627:28)
    at TLSSocket.emit (node:events:525:35)
    at TLSSocket.emit (node:domain:489:12)
    at TLSSocket.Socket._onTimeout (node:net:550:8)
    at listOnTimeout (node:internal/timers:559:17)
testFileBox error: Error: Http request timeout!
    at ClientRequest.<anonymous> (file:///Users/binsee/CodeWork/file-box/src/misc.ts:141:19)
    at Object.onceWrapper (node:events:627:28)
    at ClientRequest.emit (node:events:513:28)
    at ClientRequest.emit (node:domain:489:12)
    at TLSSocket.emitRequestTimeout (node:_http_client:839:9)
    at Object.onceWrapper (node:events:627:28)
    at TLSSocket.emit (node:events:525:35)
    at TLSSocket.emit (node:domain:489:12)
    at TLSSocket.Socket._onTimeout (node:net:550:8)
    at listOnTimeout (node:internal/timers:559:17)
  • log 2
01:50:23 [0] start
NET 63953: pipe false null
NET 63953: connect: find host nodejs.org
NET 63953: connect: dns options { family: undefined, hints: 1024 }
NET 63953: _read
NET 63953: _read wait for connection
NET 63953: afterConnect
NET 63953: _read
NET 63953: Socket._handle.readStart
NET 63953: _read
NET 63953: _read
NET 63953: _read
NET 63953: _read
NET 63953: _read
NET 63953: _read
NET 63953: _read
NET 63953: _read
NET 63953: _onTimeout
NET 63953: destroy
NET 63953: close
NET 63953: close handle
NET 63953: emit close
01:50:30 [0] toFile failed! Error: Http request timeout!
    at ClientRequest.<anonymous> (file:///Users/binsee/CodeWork/file-box/src/misc.ts:141:19)
    at Object.onceWrapper (node:events:627:28)
    at ClientRequest.emit (node:events:513:28)
    at ClientRequest.emit (node:domain:489:12)
    at TLSSocket.emitRequestTimeout (node:_http_client:839:9)
    at Object.onceWrapper (node:events:627:28)
    at TLSSocket.emit (node:events:525:35)
    at TLSSocket.emit (node:domain:489:12)
    at TLSSocket.Socket._onTimeout (node:net:550:8)
    at listOnTimeout (node:internal/timers:559:17)
testFileBox error: Error: Http request timeout!
    at ClientRequest.<anonymous> (file:///Users/binsee/CodeWork/file-box/src/misc.ts:141:19)
    at Object.onceWrapper (node:events:627:28)
    at ClientRequest.emit (node:events:513:28)
    at ClientRequest.emit (node:domain:489:12)
    at TLSSocket.emitRequestTimeout (node:_http_client:839:9)
    at Object.onceWrapper (node:events:627:28)
    at TLSSocket.emit (node:events:525:35)
    at TLSSocket.emit (node:domain:489:12)
    at TLSSocket.Socket._onTimeout (node:net:550:8)
    at listOnTimeout (node:internal/timers:559:17)
@binsee binsee mentioned this issue Feb 2, 2023
Closed
@huan
Copy link
Owner

huan commented Feb 6, 2023

We need a unit test to reproduce it automatically instead of manually.

BTW: It seems the issue is triggered by the Big CDN, which I believe the Big CDN itself is not reliable enough. My suggestion would be also considering to change your CDN service as well.

@binsee
Copy link
Author

binsee commented Feb 8, 2023

The above test shows that it has nothing to do with BIGCDN, and it can be reproduced only by calling FileBox.

@binsee binsee mentioned this issue Feb 16, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@huan @binsee