What's the reasoning behind filtering packages that do not have a dev-repo field?

[kit-ty-kate]

opam-monorepo/lib/duniverse.mli

Line 66 in eec0b83

It filters out virtual packages and packages with unknown dev-repo. *)

This took 2 hours out of my life to figure out why it wasn't adding a custom package to my mirage project.
I'm not sure I understand the reasoning behind this. This seems to have been added in #131 but I'm not sure to understand why opam-monorepo silently ignores packages that do not have a dev-repo field.

[Leonidas-from-XIV]

I am not sure myself but I believe I have a decent hypothesis why this happens based on this test:

opam-monorepo/test/test_duniverse.ml

Lines 153 to 169 in 5e22093

	make_test ~name:"Aggregates repos"
	~summaries:
	[
	summary_factory ~name:"y" ~version:"v" ~url_src:(Other "u") ~dev_repo:"d" ();
	summary_factory ~name:"y-lwt" ~version:"v" ~url_src:(Other "u") ~dev_repo:"d" ();
	]
	~expected:
	(Ok
	[
	{
	dir = "d";
	url = Other "u";
	provided_packages =
	[ { name = "y-lwt"; version = "v" }; { name = "y"; version = "v" } ];
	};
	])
	();

opam-monorepo only supports one tarball per dev-repo, because if there are multiple tarballs, their contents might have overlapping source trees (and in fact, all opam packages from the same dev-repo released with dune-release share the same tarball which contains the entire source tree), so it needs to pick one or decide how to merge multiple source trees (which I would say is quite difficult, especially if the tarballs correspond to different versions of the dev-repo). The way it does so is by determining the dev-repo, but if they are missing the field, then it can't figure out which tarballs to aggregate.

It might be useful to display a warning if a package is missing the dev-repo, however this might be printing a lot of warnings for conf-packages or similar, so it would need to make sure to only warn on "normal" packages.

[kit-ty-kate]

It might be useful to display a warning if a package is missing the dev-repo, however this might be printing a lot of warnings for conf-packages or similar, so it would need to make sure to only warn on "normal" packages.

conf packages usually have the conf flag so they could be easily filtered off this way.

[Leonidas-from-XIV]

That's true; certainly worth a try implementing a warning to see whether it will output some useful messages.