The mounts discoverer cache usage is racey

[cpuguy83]

The mount cache is read here without locking:

nvidia-container-toolkit/internal/discover/mounts.go

Lines 59 to 67 in 4604e3b

	func (d *mounts) Mounts() ([]Mount, error) {
	if d.lookup == nil {
	return nil, fmt.Errorf("no lookup defined")
	}
	if d.cache != nil {
	d.logger.Debugf("returning cached mounts")
	return d.cache, nil
	}

But it is set here while locked:

nvidia-container-toolkit/internal/discover/mounts.go

Line 116 in 4604e3b

d.cache = mounts

Best case here is you end up with 2 (or technically possibly more) things running the mounts code needlessly.
Worse cases could be undefined behavior due to the go memory model.

[elezar]

Thanks @cpuguy83. I recently added a WithCache decorator and I think we could use that here instead of managing the locks locally.

I have created #737 to make this change. Please take a look there.