go-xdelta extremely slow sometimes

[frontierpsycho]

I found a case where using go-xdelta is significantly slower than using the xdelta3 executable, to the point of making it unusable. I'd like to share it with you in case you find it useful. I tried to see if I can figure out why myself, but unfortunately I'm not familiar enough with xdelta to do so.

Just because it fit my use case, I downloaded electron 2.0.17 and electron 5.0.12 and tried to patch the electron executable alone, from one version to the other. I generated the vcdiff using the xdelta3 executable, and then tried to apply it both with that and using a simple go program using go-xdelta:

package main

import (
	"context"
	"fmt"
	xd "github.com/konsorten/go-xdelta"
	"os"
	"path/filepath"
)

func main() {
	sourceFile := os.Args[1]
	filename := filepath.Base(sourceFile)
	diffFileArg := os.Args[2]
	patchOutputFileArg := os.Args[3]

	sourceFileReaderSeeker, err := os.OpenFile(sourceFile, os.O_RDWR|os.O_CREATE, 0755)
	if err != nil {
		fmt.Printf("Error: %v\n", err)
		return
	}

	patchOutputFileWriter, err := os.OpenFile(patchOutputFileArg, os.O_RDWR|os.O_CREATE, 0755)
	if err != nil {
		fmt.Printf("Error: %v\n", err)
		return
	}

	diffFileReader, err := os.Open(diffFileArg)
	if err != nil {
		fmt.Printf("Error: %v\n", err)
		return
	}

	var decoderOptions = xd.DecoderOptions{
		FileID:    filename,
		FromFile:  sourceFileReaderSeeker,
		ToFile:    patchOutputFileWriter,
		PatchFile: diffFileReader,
	}

	decoder, err := xd.NewDecoder(decoderOptions)
	if err != nil {
		fmt.Printf("Error: %v\n", err)
		return
	}
	defer decoder.Close()

	fmt.Printf("Processing %v\n", filename)
	err = decoder.Process(context.TODO())
	if err != nil {
		fmt.Printf("Error: %v\n", err)
		return
	}
	fmt.Printf("Processing of %v ended\n", filename)
}

I timed both with the linux command line utility time, and xdelta3 took 0,7 seconds, while my program with go-xdelta took 13 minutes and 5 seconds.

At first I thought this was because the files were large (from ~80MB to ~110MB), but then I tried the same with the debian amd64 netinst and i386 netinst CD images, which clock at around 300-400MB, and the results were very different: xdelta3 again took around 0,5 seconds to complete the patch, while my program above took 11 seconds. Still a lot slower, but not as prohibitively so.

I wonder if it relates to the fact that the electron executables were executables, although since xdelta3 performs very well, I expect it's not an issue with the library, but probably with this wrapper.

Anyway, this makes the wrapper not usable for me at the moment, but thank you very much for producing it anyway, hope this gets figured out and fixed at some point! Let me know if you need help reproducing it.

[fkollmann]

Hi, thanks for investing the time to build this test case! I honestly never did test the performance but I will look into this. Have a nice holiday and I will provide feedback within the next two weeks. Best Regards, Felix

[frontierpsycho]

Thank you! Happy holidays to you too!

[fkollmann]

Hi,

I looked into the issue and in the end it was a simple misconfiguration (inherited from another project I used Xdelta in). Please feel free to test with the new v0.3.1 release.

You can compare the test setup with mine, if you want to: https://github.com/konsorten/go-xdelta-perftest-electron .

Best Regards, Felix

[fkollmann]

Hi, I will close this ticket. Please feel free to reopen it should the issue persist from your POV, Best Regards, Felix