Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initialization fails when running go test -race on a module which includes gozstd. #40

Open
Villenny opened this issue Nov 23, 2021 · 1 comment

Comments

@Villenny
Copy link

Villenny commented Nov 23, 2021

Running any test or benchmark from this file with -race will cause the below error

==17204==ERROR: ThreadSanitizer failed to allocate 0x000000a61000 (10883072) bytes at 0x200dc6f9fc000 (error code: 87)
exit status 66
package crashOnLoad


import (
	"bytes"
	"compress/gzip"
	"encoding/base64"
	"io/ioutil"
	"sync"
	"testing"

	"github.com/stretchr/testify/assert"
	"github.com/valyala/gozstd"
)


func TestThisCantRunEither(t *testing.T) {
}

func BenchmarkCompressValyalaZstd(b *testing.B) {
	b.ResetTimer()
	for i := 0; i < b.N; i += 1 {
		func() {
			compressedBytes := gozstd.Compress(make([]byte, 0, len(SampleVast)), SampleVast)
			compressionRatio := float64(len(SampleVast)) / float64(len(compressedBytes))
			if compressionRatio < 3.8 {
				panic("WTF")
			}
		}()
	}
}

func BenchmarkDecompressValyalaZstd(b *testing.B) {
	compressedBytes := gozstd.Compress(make([]byte, 0, len(SampleVast)), SampleVast)
	compressionRatio := float64(len(SampleVast)) / float64(len(compressedBytes))
	if compressionRatio < 3.8 {
		panic("WTF")
	}

	b.ResetTimer()
	for i := 0; i < b.N; i += 1 {
		func() {
			decompressedBytes, err := gozstd.Decompress(make([]byte, 0, 8192), compressedBytes)
			if err != nil {
				panic("wtf")
			}
			if len(decompressedBytes) <= 0 {
				panic("wtf")
			}
		}()
	}
}
go test -benchmem -run=^$ -coverprofile=go-code-cover -bench ^TestThisCantRunEither$ crashOnLoad -race -v -memprofile memprofile.out -cpuprofile profile.out -benchtime 3s
@valyala
Copy link
Owner

valyala commented Jan 31, 2022

The error failed to allocate 0x000000a61000 (10883072) bytes at 0x200dc6f9fc000 (error code: 87) suggests that the host system doesn't have enough RAM for completing the test with race detector enabled. Try running the test on a machine with more RAM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants