Add golden tests. #362
Add golden tests. #362
Conversation
| // To refresh the fixtures, run: | ||
| // | ||
| // go run ./oxide/testdata/record/main.go | ||
| func TestGoldenRoundTrip(t *testing.T) { |
There was a problem hiding this comment.
"The Golden Round Trip" sounds like a cool movie 😄
oxide/golden_test.go
Outdated
| // timestampComparer returns a cmp.Option that compares timestamp strings | ||
| // by their actual time value, handling precision differences. Rust and go format timestamps slightly differently, so we need to normalize to avoid spurious differences in marshalled values. |
There was a problem hiding this comment.
Minor nit, but the different line length in the comments looks a bit weird.
There was a problem hiding this comment.
Yes, this was annoying me, so I'm proposing a linter/formatter change in #363. I'll fix here as well.
| } | ||
|
|
||
| // doRequest makes a request to the configured nexus instance. We use the standard library here and not our own sdk because we're generating test files to verify the generated code. | ||
| func doRequest(method, url, token, body string) ([]byte, error) { |
There was a problem hiding this comment.
I wonder if we should send the API version in the request to make sure we're testing the same structure? For example, if I update the golden files pointing to server that has an older released, the tests will probably fail in a (possibly) confusing way.
There was a problem hiding this comment.
I'm going to make this optional for now by adding a flag to the record command. Maybe I'm thinking about this wrong, but I think if VERSION_OMICRON is ahead of our test rack (in this case I used dogfood), we can't record responses. I think we can change this later if it causes problems.
There was a problem hiding this comment.
Another minor nit, but it may be better to keep these files in a separate directory. I think we'll start collecting a bunch of them
There was a problem hiding this comment.
Agreed, I moved these around into their own directory.
jmcarp
force-pushed
the
jmcarp/golden-tests
branch
from
2dc317c to
0b40072
Compare
It's easy to make changes to the code generation logic that generate sdk code that appears to be correct and passes unit tests, but doesn't interact correctly with nexus. This patch introduces the concept of golden tests for nexus endpoints: we fetch real responses from a few endpoints of interest, check in the resulting json, and assert that we can unmarshal it into our generated types and marshal it back to equivalent json correctly. We also include a script and make target for refreshing golden files.
jmcarp
force-pushed
the
jmcarp/golden-tests
branch
from
0b40072 to
8151648
Compare
It's easy to make changes to the code generation logic that generate sdk code that appears to be correct and passes unit tests, but doesn't interact correctly with nexus. This patch introduces the concept of golden tests for nexus endpoints: we fetch real responses from a few endpoints of interest, check in the resulting json, and assert that we can unmarshal it into our generated types and marshal it back to equivalent json correctly. We also include a script and make target for refreshing golden files.