Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

byteorder: Simplify byteorder package #16201

Merged
merged 3 commits into from May 31, 2021
Merged

byteorder: Simplify byteorder package #16201

merged 3 commits into from May 31, 2021

Conversation

twpayne
Copy link
Contributor

@twpayne twpayne commented May 18, 2021

pkg/byteorder introduced in #730 uses a mix of unsafe, reflection, unchecked type assertions, and panics to handle byte order conversions, and the tests would only pass on little endian machines.

This PR replaces the package with much simpler functions and fixes the tests.

As this PR touches several parts of the code it should probably only be merged once 1.10 is released.

@twpayne twpayne added area/misc Impacts miscellaneous areas of the code not otherwise owned by another area. dont-merge/wait-until-release Freeze window for current release is blocking non-bugfix PRs release-note/misc This PR makes changes that have no direct user impact. labels May 18, 2021
@twpayne twpayne requested review from joestringer and aanm May 18, 2021 10:47
@twpayne twpayne requested a review from a team as a code owner May 18, 2021 10:47
@twpayne twpayne requested a review from a team May 18, 2021 10:47
@twpayne twpayne requested a review from a team as a code owner May 18, 2021 10:47
@twpayne twpayne requested a review from a team May 18, 2021 10:47
@twpayne twpayne requested review from a team as code owners May 18, 2021 10:47
@twpayne
Copy link
Contributor Author

twpayne commented May 18, 2021

test-me-please

@twpayne
Copy link
Contributor Author

twpayne commented May 18, 2021

CI is failing due to the current GitHub Actions outage.

Copy link
Member

@aanm aanm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, only one small question

return fmt.Sprintf("unsupported type(%v): %v", reflect.TypeOf(field).Kind(), field)
// HostSliceToNetwork32 converts b to the networking byte order.
func HostSliceToNetwork32(b []byte) uint32 {
return Native.Uint32(b[len(b)-4:])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not _ = b[3] to assert the len of b?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. And it turns out that this (and the original HostSliceToNetwork function) is completely wrong. Updated PR coming up...

@twpayne
Copy link
Contributor Author

twpayne commented May 19, 2021

test-me-please

@christarazi christarazi added the kind/enhancement This would improve or streamline existing functionality. label May 20, 2021
Copy link
Member

@christarazi christarazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@twpayne twpayne removed the dont-merge/wait-until-release Freeze window for current release is blocking non-bugfix PRs label May 21, 2021
@twpayne
Copy link
Contributor Author

twpayne commented May 21, 2021

test-me-please

@twpayne
Copy link
Contributor Author

twpayne commented May 21, 2021

ci-aks

@twpayne
Copy link
Contributor Author

twpayne commented May 21, 2021

ci-gke

@twpayne
Copy link
Contributor Author

twpayne commented May 25, 2021

test-me-please

2 similar comments
@twpayne
Copy link
Contributor Author

twpayne commented May 26, 2021

test-me-please

@twpayne
Copy link
Contributor Author

twpayne commented May 26, 2021

test-me-please

Copy link
Contributor

@errordeveloper errordeveloper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvement, thanks Tom!

The previous HostToNetwork and NetworkToHost functions used a
combination of the unsafe package, reflection, unchecked type
assertions, and panics.

Replace them with simple functions controlled by build tags.

Signed-off-by: Tom Payne <tom@isovalent.com>
Signed-off-by: Tom Payne <tom@isovalent.com>
The previous HostSliceToNetwork function used a combination of
reflection, unchecked type assertions, and panics to claim to convert a
slice of bytes in host order to a uint32 in network byte order on
machines of any endianness.

What actually happened was quite different.

HostToSliceNetwork was called with net.IP types which are []bytes of
length 4 or 16 and are already in network order with the IPv4 part of
the address in the last four bytes.

On little-endian machines the following happened:

The call to reverse in the little endian path had the side effect of not
only reversing the order of bytes in the slice, but also moving the IPv4
address in the last four bytes of the slice to first four bytes of the
slice. The call to binary.BigEndian.Uint32 (note the hardcoded
BigEndian) does not swap the bytes. So, the net effect was that it was
actually used to convert the last four bytes of a slice in network byte
order to a uint32 in host order.

This was even visible in the tests where the IP address 10.11.129.91 was
converted to 0x5b810b0a - note the position of the 10.

On big-endian machines the code would have sporadically failed. Length 4
net.IPs would have been correctly left unchanged, but length 16 net.IPs
would have their first four bytes used, all of which would be zero,
causing HostToSliceNetwork to always return zero. In any case, the tests
were hard-coded with values that would only work on little-endian
machines.

This commit replaces HostToNetworkSlice with NetIPv4ToHost32 to more
accurately reflect both what the function does and how it is used.

Signed-off-by: Tom Payne <tom@isovalent.com>
@twpayne
Copy link
Contributor Author

twpayne commented May 27, 2021

test-me-please

@twpayne
Copy link
Contributor Author

twpayne commented May 27, 2021

ci-aks

Failure is #16292.

@twpayne
Copy link
Contributor Author

twpayne commented May 27, 2021

test-gke

New flake I think: #16337

@twpayne
Copy link
Contributor Author

twpayne commented May 27, 2021

test-1.21-4.9

Could be #14959, but not sure.

@twpayne
Copy link
Contributor Author

twpayne commented May 28, 2021

test-gke

@twpayne
Copy link
Contributor Author

twpayne commented May 28, 2021

test-1.21-4.9

@twpayne
Copy link
Contributor Author

twpayne commented May 28, 2021

Hmm, the two previous comments don't seem to have re-triggered the relevant tests. Specifically, clicking on "Details" for the failing tests links to yesterday's runs. Will try again.

@twpayne
Copy link
Contributor Author

twpayne commented May 28, 2021

test-gke

@twpayne
Copy link
Contributor Author

twpayne commented May 28, 2021

test-1.21-4.9

1 similar comment
@pchaigno
Copy link
Member

test-1.21-4.9

@twpayne
Copy link
Contributor Author

twpayne commented May 31, 2021

test-gke

1 similar comment
@nebril
Copy link
Member

nebril commented May 31, 2021

test-gke

@twpayne
Copy link
Contributor Author

twpayne commented May 31, 2021

ci-aks

@twpayne
Copy link
Contributor Author

twpayne commented May 31, 2021

test-1.21-4.9 failure is flake #16237.

Almost all checks have passed, and the change is small, so marking this as ready-to-merge.

@twpayne twpayne added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label May 31, 2021
@kaworu kaworu merged commit c042c05 into cilium:master May 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/misc Impacts miscellaneous areas of the code not otherwise owned by another area. kind/enhancement This would improve or streamline existing functionality. ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/misc This PR makes changes that have no direct user impact.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet