Method-based return types for libindex & libvuln

[hdonnay]

I had an thought about how claircore's data moves around in a program: the results of functions like (*Libindex).IndexReport and (*Libvuln).Scan return types that could be seen variously as large documents or small databases.
So, what if we treated them as databases?

There's already some of this style of handling being done with the current "document" approach, but what if we pushed this further and returned unserializable types that delayed reading from the database as long as possible?
I think some notable benefits emerge:

• Memory reduction (in theory)
  Currently the core types are pointer-heavy and "complete."
  For example, there's no way to ask a question like, "is package X present?" without loading all packages, repositories, environments, etc. into memory first.
  The current types also make the memory allocated harder for the GC to handle.
• Externalization of serialization
  By no longer returning a record type with members, we have greater leeway to make changes to claircore internals and the claircore interfaces.
  Our internal domain types and return types wouldn't have to be 1-to-1 any more.
  Callers would have to take a more active role in any serialization they wish to do, as the return type would not be able to just get passed to json.Marshal, for example.

---

I wrote a little bit of Go to play with the idea:

package libindex

import (
	"context"
	"fmt"

	"github.com/quay/claircore"
)

type Reporter interface {
	Digest(context.Context) (claircore.Digest, error)
	State(context.Context) (string, error)
	Packages(context.Context) (Iterator[Package], error)
	Distributions(context.Context) (Iterator[Distribution], error)
	Repositories(context.Context) (Iterator[Repository], error)
	Environments(context.Context) (Iterator[Environment], error)
	Close() error
}

type Iterator[T any] interface {
	All(func(T, error) (cont bool)) error
}

type Package struct {
	Name string
}
type Distribution struct{}
type Repository struct{}
type Environment struct{}

// Iterator implementation to make sure the generics work...

type packageIter struct{}

var _ Iterator[Package] = (*packageIter)(nil)

func (i *packageIter) All(f func(Package, error) bool) error {
	defer func() {
		// clean up
	}()
	var err error
	for n := 0; n < 10; n++ {
		var p Package
		p.Name = fmt.Sprintf("%02x", n)
		if !f(p, err) {
			break
		}
	}
	return err
}

// Reporter implementation to make sure the generics work...

type reporterImpl struct{}

var _ Reporter = (*reporterImpl)(nil)

func (*reporterImpl) Digest(context.Context) (claircore.Digest, error) {
	return claircore.Digest{}, nil
}
func (*reporterImpl) State(context.Context) (string, error) {
	return "", nil
}
func (*reporterImpl) Packages(context.Context) (Iterator[Package], error) {
	return &packageIter{}, nil
}
func (*reporterImpl) Distributions(context.Context) (Iterator[Distribution], error) {
	return nil, nil
}
func (*reporterImpl) Repositories(context.Context) (Iterator[Repository], error) {
	return nil, nil
}
func (*reporterImpl) Environments(context.Context) (Iterator[Environment], error) {
	return nil, nil
}
func (*reporterImpl) Close() error {
	return nil
}

package libindex

import (
	"context"
	"fmt"
)

func Example() {
	ctx := context.Background()
	var r Reporter = new(reporterImpl)
	defer r.Close()
	ps, err := r.Packages(ctx)
	if err != nil {
		panic(err)
	}
	n := 0
	if err := ps.All(func(p Package, err error) bool {
		if err != nil {
			return false
		}
		if n == 8 {
			return false
		}
		fmt.Println("package:", p.Name)
		n++
		return true
	}); err != nil {
		panic(err)
	}
	// Output:
	// package: 00
	// package: 01
	// package: 02
	// package: 03
	// package: 04
	// package: 05
	// package: 06
	// package: 07
}

---

Some additional design considerations:

• Should the return type be an interface that the underlying database driver implements? Or should there be a concrete type returned, with some sort of interface into the database driver?
• Where should the concrete types live? In the libindex/libvuln package, or in a driver-ish package?