Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

base64 decoding function #47

Closed
Downchuck opened this issue Nov 17, 2012 · 17 comments

Comments

@Downchuck
Copy link

commented Nov 17, 2012

Add a function to base64 encode and decode a string result. Some serialization packages such as Jackson, a popular Java package, include base64 encoding for some data fields.

stedolan added a commit that referenced this issue Dec 28, 2012

@nicowilliams

This comment has been minimized.

Copy link
Collaborator

commented Dec 11, 2013

@stdeolan FYI there's several variants of base64 encoding :( Also, a decoder would be equally nice now that we have fromjson.

@schroederc

This comment has been minimized.

Copy link

commented Jun 3, 2015

I find myself also wanting a base64 decoder. Are there any updates on this feature?

@nicowilliams

This comment has been minimized.

Copy link
Collaborator

commented Jun 3, 2015

l8nite added a commit to l8nite/jq that referenced this issue Jan 24, 2016

l8nite added a commit to l8nite/jq that referenced this issue Jan 24, 2016

l8nite added a commit to l8nite/jq that referenced this issue Jan 24, 2016

l8nite added a commit to l8nite/jq that referenced this issue Jan 26, 2016

l8nite added a commit to l8nite/jq that referenced this issue Jan 26, 2016

@antonosmond

This comment has been minimized.

Copy link

commented Mar 24, 2016

+1 for base64 decoding

@Overbryd

This comment has been minimized.

Copy link

commented Mar 19, 2018

As this is showing up on Google a lot, and good documentation on jq is sparse, here is to everybody who lands here:

Example:

echo '{"foo": "Ym9iIGxpa2VzIGFsaWNlCg=="}' | jq '.foo | @base64d'

Or even use it when building new objects:

echo '{"foo": "Ym9iIGxpa2VzIGFsaWNlCg=="}' | jq '{encoded: .foo, decoded: .foo | @base64d}'
@rmetzler

This comment has been minimized.

Copy link

commented Mar 19, 2018

Thank you @Overbryd! After building jq in a Docker container (fixing the Dockerfile first), this works for me.

@ameissnersofi

This comment has been minimized.

Copy link

commented Apr 12, 2018

@Overbryd , when I run your example of decoding, I get an error.

$ jq --version
jq-1.5
$ echo '{"foo": "Ym9iIGxpa2VzIGFsaWNlCg=="}' | jq '.foo | @base64d'
jq: error (at <stdin>:1): base64d is not a valid format

However, if I run @base64 to encode things, that works just fine. Am I doing something wrong?

@Overbryd

This comment has been minimized.

Copy link

commented Apr 12, 2018

@ameissnersofi My best guess is that you have to have a recent version of jq. @base64d was only added recently.

@ameissnersofi

This comment has been minimized.

Copy link

commented Apr 12, 2018

@Overbryd oh I'm sorry. I thought you were actively developing jq. My bad. :/

@l8nite - can you verify my issue (assuming you are developing - the commit history seems to indicate so lol) @stedolan ?

Thanks!

@l8nite

This comment has been minimized.

Copy link
Contributor

commented Apr 12, 2018

@ameissnersofi I believe @base64d isn't in 1.5 release, it might be in 1.6 though.

Edit: I built latest from source and confirmed it is there and working:

$ ./jq --version
jq-1.6rc1-10-g7fd9e86-dirty
$ echo '{"foo": "Ym9iIGxpa2VzIGFsaWNlCg=="}' | ./jq '.foo | @base64d'
"bob likes alice\n"
@ameissnersofi

This comment has been minimized.

Copy link

commented Apr 12, 2018

@l8nite awesome! I see that 1.6 is in rc1 right now, which is probably why brew didn't pick it up for me. Thanks so much!

@pkoppstein

This comment has been minimized.

Copy link
Contributor

commented Apr 12, 2018

@ameissnersofi - Have you tried brew install --HEAD jq ?

@Downchuck

This comment has been minimized.

Copy link
Author

commented Apr 19, 2018

Great job all, on the feature and documenting its access.

@Downchuck Downchuck closed this Apr 19, 2018

@mterron

This comment has been minimized.

Copy link

commented Jun 21, 2019

@base64d seems to have issues with binary values, or at least does not match base64 -d:

$ echo "V0MsL2hwbyCcGs2AMrFAKaSjPl8OuX4OWAEip+idGVU=" | jq -Rr '@base64d' | xxd -p -c64
57432c2f68706f20efbfbd1acd8032efbfbd4029efbfbdefbfbd3e5f0eefbfbd7e0e580122efbfbdefbfbd19550a
$ echo "V0MsL2hwbyCcGs2AMrFAKaSjPl8OuX4OWAEip+idGVU=" | base64 -d | xxd -p -c64
57432c2f68706f209c1acd8032b14029a4a33e5f0eb97e0e580122a7e89d1955
$ echo "V0MsL2hwbyCcGs2AMrFAKaSjPl8OuX4OWAEip+idGVU=" | base64 -d | hexdump -C
00000000  57 43 2c 2f 68 70 6f 20  9c 1a cd 80 32 b1 40 29  |WC,/hpo ....2.@)|
00000010  a4 a3 3e 5f 0e b9 7e 0e  58 01 22 a7 e8 9d 19 55  |..>_..~.X."....U|
$ echo "V0MsL2hwbyCcGs2AMrFAKaSjPl8OuX4OWAEip+idGVU=" | jq -Rr '@base64d' | hexdump -C
00000000  57 43 2c 2f 68 70 6f 20  ef bf bd 1a cd 80 32 ef  |WC,/hpo ......2.|
00000010  bf bd 40 29 ef bf bd ef  bf bd 3e 5f 0e ef bf bd  |..@)......>_....|
00000020  7e 0e 58 01 22 ef bf bd  ef bf bd 19 55 0a        |~.X.".......U.|
@l8nite

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2019

Thanks for the report @mterron - I'd suggest starting a new issue so we can track a fix for it there. I'll try to take a look at this tomorrow.

Edit: Actually, I remember now that if the encoded value is not a UTF-8 string than the results of the decode are undefined.

@mterron

This comment has been minimized.

Copy link

commented Jun 21, 2019

Thanks for the report @mterron - I'd suggest starting a new issue so we can track a fix for it there. I'll try to take a look at this tomorrow.

Edit: Actually, I remember now that if the encoded value is not a UTF-8 string than the results of the decode are undefined.

Then it's not actually base64 and should be renamed to something else maybe @utfb64 or something along those lines?

I opened #1931 to track this.

@pkoppstein

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2019

@mterron -- jq is JSON-oriented with further limitations based on UTF-8 requirements. In particular, @base64 should only be expected to operate on UTF-8 strings. Similarly, @base64d has the necessary limitation that it has, as described in the documentation. Thus it's fair to say that both jq filters are appropriately named, despite these inherent limitations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
10 participants
You can’t perform that action at this time.