Go: Models as Data Documentation

[egregius313]

This introduces documentation for the Models-as-Data library for Go.

[owen-mc]

Looks good. We should also have a section on version-matching. (Note that if the package does not contain a major version suffix (like "/v2") then we will match all major versions. This can be disabled by putting fixed-version: at the start of the package path.)

[owen-mc]

I'm a bit confused because this isn't a model that we currently have. (In particular it has "(string, string)" as the signature.) Is this in a PR you have open? Or are you in the middle of adapting an example from another language?

[egregius313]

I am in the middle of adapting the C# docs. There are a few things which still need re-writing.

[egregius313]

The example that had been used in the C# example was the API for opening a socket. As I understand it, Listen is roughly the equivalent in Go, and would make sense as a remote flow source.

The wording from the C# documentation mentioned it being from a existing model, because sockets were already modeled in C#.

I am fine opening a small PR to add Listen as a remote flow source if we think that makes sense. Alternatively, I can reword this documentation to (1) remove the wording about being an existing model or (2) change it to a different API if you have a suggestion for a better function to highlight.

[owen-mc]

Ah, that makes sense. I recommend you talk about this model instead, as Go is used a lot for dealing with http requests.

- ["net/http", "Request", True, "FormValue", "", "", "ReturnValue", "remote", "manual"]

Unless you think that is too close to the later example for using the field Request.Body as a source in the same package. In that case I guess you could use one of the file sources that you merged yesterday, like os.ReadFile.

[owen-mc]

Before go modules, this was much more true. Now the go.mod file specifies the URL that this module expects to be found at, which creates a canonical import path for a module (and hence the packages it contains). The reasons why we want to group models for several different import paths are:

• forks (e.g. klog was forked from glog by kubernetes);
• when a package moves, we still want to analyse code written for the old import path.

We don't have to explain this all, but I mention it so you have it in mind when writing the sentence above.

[owen-mc]

I'll give this a proper review tomorrow.

[egregius313]

@owen-mc I have changed the one example to use Request::FormValue instead of Listen.

Please let me know if you think this needs any further changes or if you think we're ready for docs review.

[owen-mc]

I wrote up the last section I think is missing.

[subatoi]

Words look good—I don't think there'll be any issues with rendering but feel free to open a follow-up and tag us again if you need any help. Thank you!

[owen-mc]

This shouldn't be merged until we fix some problems that were recently found (not in this PR). Giving it a Request Changes review to avoid us accidentally merging it.

[owen-mc]

The pre-requisites have now been merged.

[smowton]

Would be good to give some example with a field or element access, to make clear that it can be more than just an arg spec or similar.

[smowton]

Mention that this could be value and the difference between them?

[owen-mc]

The equivalent page for java has a value flow example (through a lambda). Maybe we should come up with one that includes value flow and a different access path. Maybe something that stores a value in a field, or reads one out? Or something that stores a value in an array, or reads one out?

[egregius313]

Would examples relating to the slices package make sense? Something like slices.MaxFunc is comparable to the Java example (Stream#map) in terms of:

1. Value flow element -> func
2. Value flow element -> return value

Though that brings up some other questions:

• How does MaD-for-Go handle generics?
• Are elements of slices handled by .ArrayElement? (Based on how append is modeled I think so, but want to check)

Another option is slices.Concat, but that has some annoyances because of varargs.

[egregius313]

I'm thinking something like:

for func MaxFunc[S ~[]E, E any](x S, cmp func (a, b E) int) E

- ["slices", "", True, "MaxFunc", "", "", "Argument[0].ArrayElement", "Argument[1].Parameter[0]", "value", "manual"] 
- ["slices", "", True, "MaxFunc", "", "", "Argument[0].ArrayElement", "Argument[1].Parameter[1]", "value", "manual"] 
- ["slices", "", True, "MaxFunc", "", "", "Argument[0].ArrayElement", "ReturnValue", "value", "manual"] 

[owen-mc]

• Go currently ignores specializations and just treats them as if they're the generic type. I've never come across a situation where this has been an issue.
• Yes, we treat slices just like arrays. They're quite interchangeable in Go. We don't currently use .Element anywhere in Go models (except some tests to make sure it would work).
• The go dataflow library doesn't currently deal with lambdas, so I don't think the first two of those models would work. slices.Concat might be good - yes, the variadic argument makes it more complicated, but it also gives you an opportunity to explain how to model variadic arguments, which is currently missing. (Not that these docs have to be exhaustive.)

[owen-mc]

I've looked through it more carefully now. I think the description of the subtypes column needs to be updated. And the value of the subtypes column in one case. And I think we should mention the ReturnValue[i] syntax.

[owen-mc]

I found it a bit odd that the docs would mention models that we don't have yet, so I made a PR to model slices: #18108.