Getting rid of reflect2 dependency/package

[advancedwebdeveloper]

Hi.
People are experiencing compilation issues, with gollvm, when compiling Kubernetes.

The reason is in reflect2 - it a dependency of this project, in turn being a dependency of Kubernetes.

It was pretty shocking to admit that three different implementations of reflection related package exist:

https://golang.org/pkg/reflect/
Previously mentioned reflect2
https://github.com/goccy/go-reflect
.
Reflection is indeed a core feature for many programming languages.
As for the fact that json-iterator is depending on no-more-maintained sub-package - that smells like the current system design/architecture of Kubernetes, which doesn't change frequently (and farther re-design is pretty predictable) , depends on json-iterator itself only at the current stage.
I have nothing against this project - but it makes sense to split API implementations into (at least) two different Golang packages, each of which wouldn't have no common sub-dependencies/packages.

I have pinged key committers of k8s - and they pointed me into those links:

Reduce sizes of binaries #76506
Use json-iterator instead of ugorji for JSON. #48287

As for the practical side of the story - here is a list on already opened bugs:

undefined reference to 'reflect.typelinks' for gccgo and gollvm
Build kubernetes with gollvm
bugs with gollvm

Thanks to @mrVanboy - he pointed me into https://github.com/json-iterator/go/blob/master/reflect.go#L40-L41 .

[mrVanboy]

There are much more implementations of reflect libraries: https://libs.garden/go/search?q=reflect

I think, that the non-stdlib reflect library has been chosen primarily because of better performance.

One of the solutions which cross my mind is introducing an interface for the reflection dependency and allowing a user to decide (e.g. via options or build tags) which implementation them prefers.

This approach also allows us to write benchmarks and compile different reflect packages and how they are affecting performance. I don't think that replacing reflect2 package blindly without benchmarks is great.

[goccy]

Hi, I'm the author of https://github.com/goccy/go-reflect .
Also, I'm the author of https://github.com/goccy/go-json that I develop using the goccy/go-reflect technique.
goccy/go-json is much faster than the json-iterator .

github.com/goccy/go-reflect call the standard reflect library without allocation.
By using this library for the type determination part of the json-iterator, you can encode the value passed at Marshal without escaping it.
This eliminates unnecessary allocations and is faster.

---

I don't think it's a good idea to keep using an unmaintained library .
But if you're going to replace the library, I also think it's a good idea to get a benchmark along with it.
I'm sure you'll get good results.

Also, although this isn't really what I should be writing about here,
the github.com/goccy/go-json has the same concept as json-iterator/go , but it's so faster.
I would be happy to refer to it in conjunction with goccy/go-reflect .

[advancedwebdeveloper]

It was pretty shocking to admit that three different implementations of reflection related package exist:

https://golang.org/pkg/reflect/
CC @paleozogt @heylinn

[advancedwebdeveloper]

CC @fedebongio @deads2k

[advancedwebdeveloper]

Hi, I'm the author of https://github.com/goccy/go-reflect .
Also, I'm the author of https://github.com/goccy/go-json that I develop using the goccy/go-reflect technique.
goccy/go-json is much faster than the json-iterator .

github.com/goccy/go-reflect call the standard reflect library without allocation.
By using this library for the type determination part of the json-iterator, you can encode the value passed at Marshal without escaping it.
This eliminates unnecessary allocations and is faster.

I don't think it's a good idea to keep using an unmaintained library .
But if you're going to replace the library, I also think it's a good idea to get a benchmark along with it.
I'm sure you'll get good results.

Also, although this isn't really what I should be writing about here,
the github.com/goccy/go-json has the same concept as json-iterator/go , but it's so faster.
I would be happy to refer to it in conjunction with goccy/go-reflect .

@Fenny , would you suggest specific benchmarking scenarios?

[liggitt]

I don't think compatibility with a non-standard go compiler is a good reason to make a significant change here. If there are other reasons, like better performance or closer adherence to the stdlib reflect package that fix functional bugs, that could be a reasonable motivation.

Has the incompatibility with the standard go compiler been reported to gollvm?

[advancedwebdeveloper]

I don't think compatibility with a non-standard go compiler is a good reason to make a significant change here. If there are other reasons, like better performance
This is where gollvm could show some advantage, by using the flexibility of LLVM's back-ends, for specific arch.

or closer adherence to the stdlib reflect package

Please provide some examples - I can't catch/follow your thought precisely.

that fix functional bugs, that could be a reasonable motivation.
OK

Has the incompatibility with the standard go compiler been reported to gollvm?

If you would be attentive to the reports - you would already mention that is was.
@ianlancetaylor , please provide your vision for the common Go language front-end.

[ianlancetaylor]

@ianlancetaylor , please provide your vision for the common Go language front-end.

I can't force all package maintainers to support gccgo or GoLLVM. I think it's unfortunate that people choose to run unsafe code, but ultimately that is their prerogative.

[advancedwebdeveloper]

@ianlancetaylor , I am curious about your opinion on mentioned reflection libraries

[ianlancetaylor]

I think it's risky to use them, as they are prone to break with each new Go release. I think it would be better to file bugs against the standard reflect package to look for standard ways to address whatever problems they are seeing. I don't know enough about them to know why they exist.

[advancedwebdeveloper]

@goccy, please explain your motivation (why did you decided to create an alternative reflection package; which issues where you facing, for the standard reflection package)

[advancedwebdeveloper]

#462 - another cause for refactoring