Fix support for blobs larger than 64 KB on Android

[tomekzaw]

Summary

Fixes #31774.

This pull request resolves a problem related to accessing blobs greater than 64 KB on Android. When an object URL for such blob is passed as source of <Image /> component, the image does not load.

This issue was related to the fact that pipe buffer has a limited capacity of 65536 bytes (https://man7.org/linux/man-pages/man7/pipe.7.html, section "Pipe capacity"). If there is more bytes to be written than free space in the buffer left, the write operation blocks and waits until the content is read from the pipe.

The current implementation of BlobProvider.openFile first creates a pipe, then writes the blob data to the pipe and finally returns the read side descriptor of the pipe. For blobs larger than 64 KB, the write operation will block forever, because there are no readers to empty the buffer.

react-native/ReactAndroid/src/main/java/com/facebook/react/modules/blob/BlobProvider.java

Lines 86 to 95 in 41eccce

	ParcelFileDescriptor readSide = pipe[0];
	ParcelFileDescriptor writeSide = pipe[1];
	try (OutputStream outputStream = new ParcelFileDescriptor.AutoCloseOutputStream(writeSide)) {
	outputStream.write(data);
	} catch (IOException exception) {
	return null;
	}
	return readSide;

This pull request moves the write operation to a separate thread. The read side descriptor is returned immediately so that both writer and reader can work simultaneously. Reading from the pipe empties the buffer and allows the next chunks to be written.

Changelog

[Android] [Fixed] - Fix support for blobs larger than 64 KB

Test Plan

A new example has been added to RN Tester app to verify if the new implementation properly loads the image of size 455 KB from a blob via object URL passed as image source.

[analysis-bot]

Code analysis results:

• google-java-format found some issues. See https://github.com/google/google-java-format

[analysis-bot]

Platform	Engine	Arch	Size (bytes)	Diff
android	hermes	arm64-v8a	9,108,821	-11,768
android	hermes	armeabi-v7a	8,637,234	-9,392
android	hermes	x86	9,547,891	-12,043
android	hermes	x86_64	9,514,289	-11,820
android	jsc	arm64-v8a	10,754,449	-8,713
android	jsc	armeabi-v7a	9,674,123	-6,342
android	jsc	x86	10,788,908	-9,007
android	jsc	x86_64	11,396,831	-8,764

Base commit: fa0518d

[analysis-bot]

Platform	Engine	Arch	Size (bytes)	Diff
ios	-	universal	n/a	--

Base commit: fa0518d

[tomekzaw]

We can also optimize the new implementation to start a new thread only if the blob is larger than 64 KB, otherwise write to pipe synchronously.

[yungsters]

What happens in this situation? Should we somehow close or notify with failure?

Previously, the parent method would return null instead of readSide.

[tomekzaw]

Since the new implementation returns a ParcelFileDescriptor immediately, i.e. before calling write (which possibly could fail later on), we cannot return null as previously.

AutoCloseOutputStream ensures that the write side descriptor always gets closed, no matter if the write was successful or not. In case of an I/O error, the write side descriptor gets closed, and reading from a broken pipe will also result in an IOException on the reader side.

According to Android documentation, the reader is responsible for closing the read side descriptor:

The returned ParcelFileDescriptor is owned by the caller, so it is their responsibility to close it when done.

So it should be safe to return ParcelFileDescriptor instead of null, since the reader is responsible for closing it.

The opposite case is when the reader closes the read side descriptor on purpose, for example when unmounting an image component before it was fully loaded (i.e. partial read). In such case, the write call fails and the write side descriptor gets closed as well, so there is no resource leak.

[yungsters]

Amazing! Thank you for the detailed explanation.

[facebook-github-bot]

@yungsters has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[yungsters]

One pretty valid concern raised by @ShikaSD is that creating a new thread for each URI is a good idea can consume quite a bit of memory by accident.

What do you think about going ahead with your suggestion about conditionally creating he thread when the blob is bigger than the pipe?

Additionally, @ShikaSD suggested using an executor here to schedule things on a single thread.

[tomekzaw]

What do you think about going ahead with your suggestion about conditionally creating he thread when the blob is bigger than the pipe?

I tried to determine the pipe capacity using Os.fcntlInt, but this method is only available from API 30 (see the documentation).

int F_GETPIPE_SZ = 1032; // Linux 2.6.35+
FileDescriptor fd = writeSide.getFileDescriptor();
int pipeCapacity = Os.fcntlInt(fd, F_GETPIPE_SZ, 0);

Probably we could call fcntl(2) from native code instead. Another way to determine the pipe capacity would be to open and read /proc/sys/fs/pipe-max-size. Hopefully it's enough to assign 65536 directly for now.

I've also updated the example in RNTester so now it includes two images – one smaller and one larger than 64 KB.

Just to be safe, I've also checked if it works properly for an image exactly of size equal to pipe capacity. In order to generate images of arbitrary size, I've implemented a HTTP server in Flask which appends null bytes to an existing JPG image and returns the modified image in the response:

import urllib

from flask import Flask, request, make_response

app = Flask(__name__)

url = 'https://upload.wikimedia.org/wikipedia/commons/thumb/9/97/The_Earth_seen_from_Apollo_17.jpg/240px-The_Earth_seen_from_Apollo_17.jpg'
original = urllib.request.urlopen(url).read()

@app.route("/image.jpg")
def image():
    size = int(request.args['size'])
    diff = size - len(original)
    assert diff >= 0

    output = original + b'\0' * diff
    assert len(output) == size

    response = make_response(output)
    response.headers.set('Content-Type', 'image/jpeg')
    return response

In RNTester app I've replaced the URLs:

<BlobImageExample
  urls={[
    'http://10.0.2.2:5000/image.jpg?size=65534',
    'http://10.0.2.2:5000/image.jpg?size=65535',
    'http://10.0.2.2:5000/image.jpg?size=65536', // max pipe capacity
    'http://10.0.2.2:5000/image.jpg?size=65537',
    'http://10.0.2.2:5000/image.jpg?size=65538',
  ]}
/>

Before (always synchronous write)	After (conditional write)

Using the new implementation with conditional write, all images are loaded properly, so it should be safe to use the condition data.length <= PIPE_CAPACITY.

Additionally, @ShikaSD suggested using an executor here to schedule things on a single thread.

Good idea! Done.

[analysis-bot]

Code analysis results:

• google-java-format found some issues. See https://github.com/google/google-java-format

[facebook-github-bot]

@yungsters has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[facebook-github-bot]

@yungsters merged this pull request in f00e348.