archive version dependencies should be a first-class citizen

[delgadom]

We currently don't support version-specific metadata for archives... just archive-specific metadata.

We should probably think about supporting version-specific metadata. But at the very least we need version-specific dependencies. One way to do this is to allow dependencies directly as version metadata.

[jgerardsimcock]

These are obvious points but I just want to confirm that we understand and are talking about the same things. We need to be able to set and get version metadata. This would be set at archive creation and archive update and gotten at download (for example). Right now we want to get at least dependencies within the version but ultimately, their could conceivably be arbitrary metadata for each version.

[jgerardsimcock]

I am thinking this could be one implementation

metadata =  {
	'archive_name': 'Big Climate Data',
	'source': 'a big data climate instrument',
	'description': 'waves and how you can get stoked and barreled'
	'versions': [('0.0.1', 'NOAA Charts from some time period'), ('0.1.0', 'Later NOAA Charts'), ('0.2.0', 'The most reliable NOAA Charts')]
	}

[delgadom]

Is this for the dependencies? I was thinking more like

{
    '_id': big_climate_data',
    'authority_name': 'osdc',
    'archive_path': '/big/climate/data',
    'versioned': True,
    'metadata': {
        'source': 'a big data climate instrument',
        'description': 'waves and how you can get stoked and barreled'},
    'versions': [
      {'version': 1.0, etc..., 'dependencies': [(arch2, 1.3), (arch7, 1.6.2a1)]},
      {'version': 0.8, etc..., 'dependencies': [(arch2, 1.2), (arch7, 1.4)]}
    ]
}

Does that make sense? Each archive depends on other archives. The version metadata might be extendable beyond just version, checksum, algorithm, time stamp, author, dependencies, but this seems like the minimum requirement.

[delgadom]

On DataArchive, we'll need the following updates:

DataArchive.update

def update(
    self,
    filepath, 
    cache=False, 
    remove=False, 
    bumpversion='patch', 
    prerelease=None, 
    dependencies=None, 
    **kwargs):
    
    ...

    self._update_manager(checksum, kwargs, version=next_version, dependencies=dependencies)

DataArchive.open

def open(
    self, 
    mode='r', 
    version=None, 
    bumpversion='patch', 
    prerelease=None, 
    dependencies=None, 
    *args, 
    **kwargs):
    ...
   
    updater = lambda *args, **kwargs: self._update_manager(
            *args, version=next_version, dependencies=dependencies, **kwargs)
    ...

DataArchive.get_local_path

similar to DataArchive.open

DataArchive._update_manager

def _update_manager(self, checksum, metadata={}, version=None, dependencies=None):

    # by default, dependencies is the last version of dependencies
    if dependencies is None:
        history = self.history
        if len(history) == 0:
            dependencies = []
        else:
            dependencies = history[-1]['dependencies']

    ....

[jgerardsimcock]

#74