Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jsonlz4: lz4: invalid source or destination buffer too short #198

Closed
sck opened this issue Sep 28, 2022 · 1 comment
Closed

jsonlz4: lz4: invalid source or destination buffer too short #198

sck opened this issue Sep 28, 2022 · 1 comment

Comments

@sck
Copy link

sck commented Sep 28, 2022

I tried using pierrec/lz4 to access mozilla's jsonlz4 files using the way outlined here: #28 , but it always shows this error:

lz4: invalid source or destination buffer too short

mozlz4a.py can decompress them without problems

Demo: https://go.dev/play/p/A32NWBYkchg

package main

import (
	"fmt"
	"crypto/md5"
	"github.com/pierrec/lz4"
)

// Payload created with this Python script: https://gist.github.com/Tblue/62ff47bef7f894e92ed5
//
// $ printf 'mozLz40\x00!\x00\x00\x00\xF0\x12{\"version\":[\"sessionrestore\",1]}\n'  | md5sum
// 12c5a86eaafe57bbb0345f52505610bf  -
// printf 'mozLz40\x00!\x00\x00\x00\xF0\x12{\"version\":[\"sessionrestore\",1]}\n'  | python3.7 mozlz4a.py  -d -
// {"version":["sessionrestore",1]}

func md5sum(s string) (r string) {
	digest := md5.New()
	digest.Write([]byte(s))
	return fmt.Sprintf("%x", digest.Sum(nil))
}

var payload string = "mozLz40\x00!\x00\x00\x00\xF0\x12{\"version\":[\"sessionrestore\",1]}\n"

func main() {
	fmt.Println(md5sum(payload))
	out := make([]byte, len(payload)*1000)
	_, e := lz4.UncompressBlock([]byte(payload), out)
	if e != nil {
		panic(e)
	}
	fmt.Print(string(out))
}
@sck
Copy link
Author

sck commented Sep 30, 2022

I got it working now; I forgot to skip the header.

Playground: https://go.dev/play/p/Oj2-WFYLkYl

package main

import (
	"fmt"

	"crypto/md5"

	"github.com/pierrec/lz4"
)

// Payload created with this Python script: https://gist.github.com/Tblue/62ff47bef7f894e92ed5
//
// $ printf 'mozLz40\x00!\x00\x00\x00\xF0\x12{\"version\":[\"sessionrestore\",1]}\n'  | md5sum
// 12c5a86eaafe57bbb0345f52505610bf  -
// printf 'mozLz40\x00!\x00\x00\x00\xF0\x12{\"version\":[\"sessionrestore\",1]}\n'  | python3.7 mozlz4a.py  -d -
// {"version":["sessionrestore",1]}

func md5sum(s string) (r string) {
	digest := md5.New()
	digest.Write([]byte(s))
	return fmt.Sprintf("%x", digest.Sum(nil))
}

var payload string = "mozLz40\x00!\x00\x00\x00\xF0\x12{\"version\":[\"sessionrestore\",1]}\n"

func main() {
	fmt.Println(md5sum(payload))
	p := payload[12:len(payload)]
	out := make([]byte, len(payload)*1000)
	_, e := lz4.UncompressBlock([]byte(p), out)
	if e != nil {
		panic(e)
	}
	fmt.Print(string(out))
}

@pierrec pierrec closed this as completed Jan 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants