Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider alternate (browser friendly) buffer #578

Open
rolanday opened this issue Feb 6, 2023 · 14 comments
Open

Consider alternate (browser friendly) buffer #578

rolanday opened this issue Feb 6, 2023 · 14 comments

Comments

@rolanday
Copy link

rolanday commented Feb 6, 2023

Hello! :-) I'd love to be able to use this module in my SvelteKit app, but node:buffer dep is not something I've been able to workaround via Vite polyfill (same issue for other frameworks like Nextjs as well).

Have or can you consider alt buffer impl that is browser friendly? Or, any tips on configuring for use in SvelteKit, Vue, Nextjs, etc?

Thanks!

@sindresorhus
Copy link
Owner

Ideally, all code should be using Uint8Array instead of Buffer, but there's just too much Node.js code that depends on Buffer.

If anyone wants to work on this, the first task would be to convert the dependencies of file-type to use Uint8Array instead of Buffer.

@Borewit
Copy link
Collaborator

Borewit commented Feb 13, 2023

That work has started to some extend Borewit/token-types#246, but yet there is much more work to do.

@volarname
Copy link

+1
this library really needs a browser version without node and webpack deps
something like fromFile(file: File)
because file.mimeType doesnt seems to work properly all times

@Borewit
Copy link
Collaborator

Borewit commented Mar 6, 2023

+1 this library really needs a browser version without node and webpack deps something like fromFile(file: File) because file.mimeType doesnt seems to work properly all times

That function already exist @volarname : fileTypeFromBlob(blob). Where you can pass you file as blob, since File inherits from Blob.

@akkie
Copy link

akkie commented Mar 9, 2023

+1 this library really needs a browser version without node and webpack deps something like fromFile(file: File) because file.mimeType doesnt seems to work properly all times

That function already exist @volarname : fileTypeFromBlob(blob). Where you can pass you file as blob, since File inherits from Blob.

Yes, but it doesn't work in modern frameworks like SvelteKit as example. It uses vite under the hood and shows the warnings:

Module "node:buffer" has been externalized for browser compatibility. Cannot access "node:buffer.Buffer" in client code.

@Borewit
Copy link
Collaborator

Borewit commented Mar 13, 2023

This module has been primary written for Node.js. It does depend on Buffer. There is browser friendly buffer module available, but you depend on your framework how easy that is to fit it in.
There is not much more we can do, as these core dependencies are not shared between browser and Node.js.

@sindresorhus
Copy link
Owner

@Borewit
Copy link
Collaborator

Borewit commented Oct 25, 2023

https://sindresorhus.com/blog/goodbye-nodejs-buffer

Interesting!, very nice guidance. I will give it try (starting with the dependent modules first).

@Borewit
Copy link
Collaborator

Borewit commented Nov 1, 2023

It is not only node:buffer file-type depends on, also safe-buffer

graph TD;
    FT-->S
    FT(file-type)-->RWNS(readable-web-to-node-stream)
    S(strtok3)-->P(peek-readable)
    S(strtok3)-->TO("@tokenizer/token")
    TY(token-types)-->TO
    TY-->IE("ieee754")
    FT-->TY
    TY-->NB(node:buffer)
    RWNS-->RS(readable-stream)
    RS-->SD(string_decoder)
    SD-->SB(safe-buffer)
    RS-->UD(util-deprecate)
    RS-->I(inherits)
    style NB fill:#F88,stroke:#A44
    style SB fill:#F88,stroke:#A44
    style SD fill:#CCC,stroke:#888
    style IE fill:#CCC,stroke:#888
    style UD fill:#CCC,stroke:#888
    style I fill:#CCC,stroke:#888

@Borewit
Copy link
Collaborator

Borewit commented Nov 1, 2023

I still don't see a good alternative for the string encoding node:buffer supports: ascii, utf8, utf-8, utf16le, ucs2, ucs-2, base64, base64url, latin1, binary, hex;

e.g.:

Buffer.from(uint8Array).toString(encoding);

@Songkeys
Copy link

Songkeys commented Nov 4, 2023

I still don't see a good alternative for the string encoding node:buffer supports: ascii, utf8, utf-8, utf16le, ucs2, ucs-2, base64, base64url, latin1, binary, hex;

I think this library can help? https://github.com/sindresorhus/uint8array-extras

I can see there are methods to hex, string, base64. We have to construct every encoding method from sketch though.

@SgtPooki
Copy link

SgtPooki commented Feb 5, 2024

i've got a working browser-consumable fork at https://github.com/sgtpooki/file-type, but stream and file support has been disabled for now (since I don't need it for my use-case). All tests for fileTypeFromBuffer are passing.

@andrewbrennanfr
Copy link

Hi 👋 Facing similar issues after upgrading from a very old fork of this lib.
I feel like I've read a 101 different issues/comments/responses & ultimately keeping ending up in the same situation.

So just to be crystal clear - this lib can't be used without polyfiling node:buffer right? (in my case with something like this)
There's no "out-of-the-box" way to use this on the client-side?

@mgcrea
Copy link

mgcrea commented Apr 22, 2024

Encountering this while playing with Cloudflare workers, would be nice for fileTypeFromStream to also handle ReadableStream, when I try to pass one (from File.stream()) I get Error: Expected an instance of stream.Readable.

EYHN pushed a commit to toeverything/blocksuite that referenced this issue Apr 22, 2024
Fix toeverything/AFFiNE#6603

This pull request switches the [file-type](https://github.com/sindresorhus/file-type) to [@sgtpooki/file-type](https://github.com/sgtpooki/file-type).

It also includes other minor changes and updates.

~~The `file-type` is no longer useful since the toeverything/AFFiNE#3245 resolved in toeverything/AFFiNE#5565

---

More context see toeverything/AFFiNE#3245 #3555 #4818 #5372 sindresorhus/file-type#578

---

## Caveat

The [@sgtpooki/file-type](https://github.com/sgtpooki/file-type) is an unofficial fork package from the community. It may need to be source code audited before updating.

Also, thanks to sgtpooki for their strong contribution to @sgtpooki/file-type
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

9 participants