Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fsPromise - read files from positions >4GB doesn't work #21121

Closed
freiit opened this issue Jun 4, 2018 · 1 comment · Fixed by #21148
Closed

fsPromise - read files from positions >4GB doesn't work #21121

freiit opened this issue Jun 4, 2018 · 1 comment · Fixed by #21148

Comments

@freiit
Copy link

freiit commented Jun 4, 2018

  • node10.3.0:
  • tested on x86_64 GNU/Linux and armv6l GNU/Linux
  • fs / fsPromise

reading files > 4GB doesn't read the correct bytes.
(Maybe related to fix here: #21003)

Code example shows identical code, one with fsPromise, one the old style cb. Example uses sparse file for fast reproduction, but works also with non-sparse files.

let {promises: fs} = require('fs')
let fsOld = require('fs')

let {strict: assert} = require('assert')

const FILE = '/tmp/demo'
const GB = 4
const BUFFER = 2

async function p() {
	let input = Buffer.alloc(BUFFER, '-')

	let fh = await fs.open(FILE, 'w')
	let bytes = await fh.write(input, 0, BUFFER, GB * 1024 * 1024 * 1024)
	console.log('bytes', bytes)
	await fh.close()

	fh = await fs.open(FILE, 'r')
	let buffer = Buffer.allocUnsafe(BUFFER)
	let result = await fh.read(buffer, 0, BUFFER, GB * 1024 * 1024 * 1024)
	await fh.close()

	assert.deepEqual(input, result.buffer) // <-- fails
}

async function cb() {
	let input = Buffer.alloc(BUFFER, '-')

	let fh = await new Promise((res, rej) => fsOld.open(FILE, 'w', (err, data) => err ? rej(err) : res(data)))
	let bytes = await new Promise((res, rej) => fsOld.write(fh, input, 0, BUFFER, GB * 1024 * 1024 * 1024, (err, data) => err ? rej(err) : res(data)))
	console.log('bytes', bytes)
	await new Promise((res, rej) => fsOld.close(fh, e => e ? rej(e) : res()))

	fh = await new Promise((res, rej) => fsOld.open(FILE, 'r', (err, data) => err ? rej(err) : res(data)))
	let buffer = Buffer.allocUnsafe(BUFFER)
	let result = await new Promise((res, rej) => fsOld.read(fh, buffer, 0, BUFFER, GB * 1024 * 1024 * 1024, (err, bytes, buffer) => err ? rej(err) : res({ bytes, buffer })))
	console.log('result', result)
	await new Promise((res, rej) => fsOld.close(fh, e => e ? rej(e) : res()))

	assert.deepEqual(input, result.buffer) // <-- works
}

(async function main() { await cb(); await p() })().catch(e => console.error(e))

@ryzokuken
Copy link
Contributor

I believe that this should've been fixed in 10.3.0, after #21003.

/cc @mafintosh

cjihrig added a commit to cjihrig/node that referenced this issue Jun 7, 2018
PR-URL: nodejs#21148
Fixes: nodejs#21121
Refs: nodejs#21003
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Ujjwal Sharma <usharma1998@gmail.com>
targos pushed a commit that referenced this issue Jun 7, 2018
PR-URL: #21148
Fixes: #21121
Refs: #21003
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Ujjwal Sharma <usharma1998@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants