Skip to content

Commit

Permalink
ARROW-12578: [JS] Remove Buffer in favor of TextEncoder API to suppor…
Browse files Browse the repository at this point in the history
…t bundlers such as Rollup

Bundlers such as Rollup do not recognize the `_Buffer` import, which breaks their builds. This change resolves this issue by removing Buffer in favor of `TextEncoder`. Note that change incurs a performance penalty on Node as `Buffer` is often faster.

Co-authored-by: Adam Lippai <adam@rigo.sk>
Co-authored-by: Paul Taylor <paul.e.taylor@me.com>

Closes #10332 from domoritz/remove-buffer-js

Authored-by: Dominik Moritz <domoritz@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
  • Loading branch information
domoritz authored and kszucs committed May 17, 2021
1 parent efdfb6d commit eaac875
Showing 1 changed file with 4 additions and 23 deletions.
27 changes: 4 additions & 23 deletions js/src/util/utf8.ts
Expand Up @@ -15,34 +15,15 @@
// specific language governing permissions and limitations
// under the License.

import { toUint8Array } from './buffer';
import {
TextDecoder as TextDecoderPolyfill,
TextEncoder as TextEncoderPolyfill,
} from 'text-encoding-utf-8';

/** @ignore @suppress {missingRequire} */
const _Buffer = eval("typeof Buffer === 'function' ? Buffer : null");
const decoder = new (typeof TextDecoder !== 'undefined' ? TextDecoder : TextDecoderPolyfill)('utf-8');
/** @ignore */
const useNativeEncoders = typeof TextDecoder === 'function' && typeof TextEncoder === 'function';
export const decodeUtf8 = (buffer?: ArrayBuffer | ArrayBufferView) => decoder.decode(buffer);

const encoder = new (typeof TextEncoder !== 'undefined' ? TextEncoder : TextEncoderPolyfill)();
/** @ignore */
export const decodeUtf8 = ((TextDecoder) => {
if (useNativeEncoders || !_Buffer) {
const decoder = new TextDecoder('utf-8');
return (buffer?: ArrayBuffer | ArrayBufferView) => decoder.decode(buffer);
}
return (input: ArrayBufferLike | ArrayBufferView) => {
const { buffer, byteOffset, length } = toUint8Array(input);
return _Buffer.from(buffer, byteOffset, length).toString();
};
})(typeof TextDecoder !== 'undefined' ? TextDecoder : TextDecoderPolyfill);

/** @ignore */
export const encodeUtf8 = ((TextEncoder) => {
if (useNativeEncoders || !_Buffer) {
const encoder = new TextEncoder();
return (value?: string) => encoder.encode(value);
}
return (input = '') => toUint8Array(_Buffer.from(input, 'utf8'));
})(typeof TextEncoder !== 'undefined' ? TextEncoder : TextEncoderPolyfill);
export const encodeUtf8 = (value?: string) => encoder.encode(value);

0 comments on commit eaac875

Please sign in to comment.