Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of coopering file data #83

Closed
kozmod opened this issue Jun 3, 2023 · 0 comments
Closed

Improve performance of coopering file data #83

kozmod opened this issue Jun 3, 2023 · 0 comments
Labels
bug Something isn't working enhancement New feature or request

Comments

@kozmod
Copy link
Owner

kozmod commented Jun 3, 2023

The current realization of the File (config package) use string type for keeping raw file data:

type File struct {
	Path  string  `yaml:"path"`
	Data  *string `yaml:"data"` 👈🏻👈🏻👈🏻
	Get   *Get    `yaml:"get"`
	Local *string `yaml:"local"
}

Data casts to []byte to creating domain entity (entity.DataFile):

...
		var producer entity.FileProducer
		switch {
		case f.Data != nil:
			file := entity.DataFile{
				FileInfo: tmpl,
				Data:     []byte(*f.Data),, 
			}
...

These conversions create a new slice or string, and therefore have time complexity proportional to the number of bytes that are processed.

import (
	"github.com/kozmod/progen/internal/entity"
	"testing"
)

const x = `matrix:
 version: 1.19
steps:
 name: Setup Go 1.19`

var fd = func(d FileData) *FileData { return &d }(FileData(`matrix:
 version: 1.19
steps:
 name: Setup Go 1.19`))

func Benchmark_copy_string(b *testing.B) {
	b.ReportAllocs()
	res := entity.DataFile{}
	for i := 0; i < b.N; i++ {
		res = entity.DataFile{Data: *fd}
	}
	_ = res
}

func Benchmark_copy_pointer_of_bytes(b *testing.B) {
	b.ReportAllocs()
	res := entity.DataFile{}
	for i := 0; i < b.N; i++ {
		res = entity.DataFile{Data: []byte(x)}
	}
	_ = res
}
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
Benchmark_copy_string
Benchmark_copy_string-12              	1000000000	         0.5203 ns/op	       0 B/op	       0 allocs/op
Benchmark_copy_pointer_of_bytes
Benchmark_copy_pointer_of_bytes-12    	38351860	        27.81 ns/op	      64 B/op	       1 allocs/op

Changing string to []byte will be more efficient

type File struct {
	Path  string    `yaml:"path"`
	Data  *[]byte `yaml:"data"`
	Get   *Get      `yaml:"get"`
	Local *string   `yaml:"local"`
}
@kozmod kozmod added bug Something isn't working enhancement New feature or request labels Jun 3, 2023
kozmod added a commit that referenced this issue Jun 3, 2023
kozmod added a commit that referenced this issue Jun 3, 2023
kozmod added a commit that referenced this issue Jun 3, 2023
[#83] change `File.Data` field: `string` -> `[]byte`
@kozmod kozmod closed this as completed Jun 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant