Dependency Submission API - Opam Protocol

[smorimoto]

We have just done some work to submit the (still shallow) dependency graph in opam to the DS API. Next we want to make sure the GitHub team is aware of how to get stronger support on the GitHub side.

Currently, we are submitting metadata with the following content.

const purl = new PackageURL(
  "opam",
  null,
  dependency.name,
  dependency.version,
  null,
  null
);

And it's not difficult at all to link this to the official OCaml site, as you can see from its URL schema: https://ocaml.org/p/dune/3.3.1

[smorimoto]

I would also like to touch on a different point here. It seems that if there are multiple projects in the repository and multiple opam package files (*.opam), then even if you submit all the metadata, only the contents of one opam package file are rendered. Is this known behaviour?

[smorimoto]

@brphelps @juxtin I'm glad to finally get a reply from the team. As a fact, I'm still not sure there's anything wrong with my implementation, which would be great if you could see it.
Implementation: https://github.com/ocaml/setup-ocaml/tree/analysis/src/analysis
Demo: https://github.com/smorimoto/opam-dependency-submission

[brphelps]

@smorimoto : From looking at the build output, the issue seems to be the correlator issue I mentioned above:

Notice: {
      "detector": {
          "name": "ocaml/setup-ocaml/analysis",
          "url": "https://github.com/ocaml/setup-ocaml/tree/master/analysis",
          "version": "2.0.6"
      },
      "version": 0,
      "job": {
          "correlator": "build",
          "id": "3111343067"
      },
      "sha": "17adcb6b4a1ea8d69d32518e3b7b5a264a036889",
      "ref": "refs/heads/main",

The correlators for both submissions are being sent with the same value, so they are overwriting each other. If they have different correlator values for each run, they will be aggregated for a given SHA. If looks like you have a single chunk of code that submits a submission per discovered file. You could either choose to combine each manifest into a single submission (resolved is a map to support this) or just ensure that each submission is created with its own unique correlator. In general, managing different correlators for a single logical submission can be hard (we usually only do that if someone is running multiple different actions).

[smorimoto]

That approach seems to have worked: https://github.com/sneeuwballen/benchpress/network/dependencies

[smorimoto]

@brphelps The next point is how to display this pretty, but is there any kind of support from GitHub related to this right now?

[smorimoto]

Any idea?

[smorimoto]

ping @courtneycl

[brphelps]

@courtneycl if you want to reply to the question about package mapping that would be great. AFAIK we don't have plans around mapping ecosystem types that don't already exist in dependency graph (like what you can see mapped today) for dependency submission. That could change, but I don't expect it to happen that soon.

[smorimoto]

@brphelps For example, the Java ecosystem and others do community submissions, and I think GitHub has official support for them, but how do we get proper support for OCaml on GitHub? Does it simply depend on popularity from enterprise users?