Potential features for lookout!

[vsoch]

I want to open an issue to list some potential issues that we discussed today, which I'd love to give a first stab at (and likely I'll be able to make some time next week) - I love GoLang too but don't get the opportunity to work with it super often.

Lookout search

The use case I have in mind is when the user doesn't have a package installed, and isn't privy to what the latest version is. We might support either of the following:

• some kind of search, e.g., added with lookout search that would look for the package in some registry (file) of known software packages, and then return to the user the latest version (or some subset of versions in a small table depending on what they ask for).
• but arguably this same search could be integrated with the lookout add to allow for a more seamless interaction that doesn't require more than one command. For example:

$ lookout add singularity
...<does search, shows latest version, asks user "would you like to add version X from <resource> Y/n>?

I think we would want a user to be able to do lookout add <tool> without needing to know the registry or version in advance. If lookout is setup to basically try to help get to the latest version, then the default should probably be the latest found.

Lookout import

It would be super cool to have some existing lmod / spack / other install, and quickly "import" current versions into lookout. I think what I want to think about more is how these two things can be coupled easily so even a user could write some custom script to, e.g., find packages that need update, update or build a container, or some custom thing.

Feedback on organization and color of lookout

We chat about this in the call -I think we can we better organize the current and latest by either putting them each just on one line, or grouping them together more obviously for the user.

A way to import an entire already existing package base (spack,lmod)

[alecbcs]

Thanks for the feedback! I'll get started on some output UI redesigns this weekend and start experimenting with some methods for parsing & importing version information from something like Spack.

[vsoch]

I can help too! I actually already updated the UI for add, but I wanted to test by adding a repository and then hit #2. So I can hopefully get those both in as PRs sometime this weekend!

[alecbcs]

Oh awesome! Very nice! Feel free to experiment with different colors as well, I'm not attached to any yet.

[alecbcs]

Quick New Years Update!

We should be able to begin importing package metadata from Spack shorty. Before finals I finished up work on my first write of go-parspack a library to import Spack spec files into usable Go structs. Now all that's left is to write the integration into Lookout to use the library.

Second, @vsoch do you believe that we should look to implement a secondary polling method for GitHub based apps separate from Cuppa to support GitHub sources without requiring a Personal Access Token? I've recently been experimenting with go-latest and it seems to do the trick.

[vsoch]

Happy New Year @alecbcs! It indeed would be very nice to support GitHub sources without a personal access token - go-latest looks good, as long as you are OK with adding another dependency. Would you like me to test and PR when I get the chance?

[alecbcs]

Sure! That'd be great! I'd be curious if comparing the two (go-latest or cuppa), which gives more accurate results? Cuppa is still occasionally tripped up by prerelease tags with weird names and so maybe go-latest is a better option in general. Luckily Lookout is still pretty small so I'm alright adding another dependency if it means greater usability.

I'll begin checking how we might search the spack repository as a source to import app metadata.

[vsoch]

hey @alecbcs ! I started looking at this today, and one issue is that go-latest seems to require semantic versioning, meaning that any custom release names (e.g. v1.0.0 or 1.0.0a or 1.0.0rca) requires us to define a custom function to parse the core version. When I looked closer at cuppa vs go-latest, go-latest seems to find the latest by way of semantic versions, but I think cuppa just grabs the last in the result. So the benefit of cuppa (even though it request a token for the graph query) is that we don't have to worry about parsing semantic versions. But maybe this is something you want to try to do? I would err on the side of wanting to tell the user about a new release candidate over truncating that extra bit of the version string and telling them a release exists that does not (e.g., 1.0.0rc1 would be truncated to 1.0.0). We could also just try skipping any of these special patterns, but then we would need to know all the patterns. Let me know what you think! Here is the part of the README where it mentions the special function.

https://github.com/tcnksm/go-latest#github-tag

[vsoch]

Another idea (if we are considering adding another library) is to just look for a standard GitHub API (v3, Restful) client, e.g., https://github.com/google/go-github. For the restful API, each entry has a published at date, so we would only need that to get the latest (as opposed to using semantic versioning). For example:

$ curl   -H "Accept: application/vnd.github.v3+json"   https://api.github.com/repos/singularityhub/sregistry/releases | jq .[].published_at
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 16318  100 16318    0     0  60661      0 --:--:-- --:--:-- --:--:-- 60661
"2020-12-22T18:52:38Z"
"2020-12-21T06:39:20Z"
"2020-06-04T06:00:21Z"
"2020-06-04T05:59:34Z"
"2019-06-27T21:02:27Z"
"2019-06-26T19:10:15Z"
"2017-10-15T11:45:24Z"
"2017-09-28T03:02:05Z"