Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API for Base64.encode / Base64.decode with an IO as the source #14603

Open
jgaskins opened this issue May 21, 2024 · 3 comments
Open

Add API for Base64.encode / Base64.decode with an IO as the source #14603

jgaskins opened this issue May 21, 2024 · 3 comments

Comments

@jgaskins
Copy link
Contributor

Related to #2964
Originally mentioned on the forum

In order to base64-encode data with Base64.encode, that data must exist in RAM. Technically, it just needs to be something that responds to to_slice, but at least in the stdlib all of those things are actually stored in RAM:

➜  crystal git:(b64-encode-io-to-io) ✗ ag 'def to_slice( :|$)' src
src/io/memory.cr
418:  def to_slice : Bytes

src/llvm/memory_buffer.cr
14:  def to_slice

src/slice.cr
848:  def to_slice : self

src/string.cr
5449:  def to_slice : Bytes

src/bit_array.cr
626:  def to_slice : Bytes

src/static_array.cr
369:  def to_slice : Slice(T)

My use case is to base64-encode a file of arbitrary size. Some APIs accept JSON but allow you to send/receive file data inside that payload. I've had to do this a few times — for example, some email service providers allow you to send attachments embedded in JSON to their HTTP API, one integration I had to build for mortgage lenders to integrate with their loan origination system base64-encoded documents inside XML payloads, things like that.

In addition to certain email providers' specific APIs, you have to do the same in SMTP itself. SMTP is a text-based protocol, so attachments are typically encoded into a text-friendly format. The shard I typically use to send emails has to load the entire file into RAM.

So since base64-encoding nontrivial amounts of data is not an uncommon requirement, I figured this might be a good addition to the Base64 stdlib.

@BlobCodes
Copy link
Contributor

BlobCodes commented May 21, 2024

Implemented in #13574 (which I will work on soon.. probably)

@jgaskins
Copy link
Contributor Author

Implemented in #13574 (which I will work on soon.. probably)

That PR looks like it only implements this overload for decode. Am I overlooking encode?

@BlobCodes
Copy link
Contributor

Ah, seems like I misremembered what I did in that PR..
Yes, it only implements IO-to-IO decode

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants