Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add logical object renaming feature #71

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Add logical object renaming feature #71

wants to merge 5 commits into from

Conversation

karlcz
Copy link
Contributor

@karlcz karlcz commented Apr 8, 2024

a new POST /path/to/new/object handler takes a JSON command description:

  {
    "command": "rename_from",
    "source_name": "/path/to/orig/object",
    "versions": [ "version1", "version2", ..],
    "copy_acls": true
  }

and mirrors version records under the new object name, implemented as a logical rename (no bulk data copy/movement). The versions field is optional and it processes all versions by default. The copy_acls field is optional and defaults to false, where only basic ownership is set as in normal version uploads.

The implementation uses version.aux fields:

  • rename_to: [ newname, newversion ]
  • hname: original hatrac object name under which data is stored
  • hversion: original hatrac object version under which data is stored

The old record links to the new one and only the new one is processed to serve content. During retrieval, the new record passes the original hatrac name information and aux metadata into the backend storage module.

a new POST /path/to/new/object handler takes a JSON command
description:

  {
    "command": "rename_from",
    "source_name": "/path/to/orig/object",
    "versions": [ "version1", "version2", ..],
    "copy_acls": true
  }

and mirrors version records under the new object name, implemented
as a logical rename (no bulk data copy/movement). The versions
field is optional and processes all versions by default. The
copy_acls field is optional and defaults to false, where only
basic ownership is set as in normal version uploads.

The implementation uses version.aux fields:

- rename_to: [ newname, newversion ]
- name: storage_name
- version: storage_version

so that the old record links to the new one and then the new
one is processed to serve content. The new record has overrides
as needed to address bulk storage via the old info in the backend.
To be more future-proof, distinguish 'hname' and 'hversion' aux
values that represent overrides of URL content from the existing
'version' aux value that overrides backend S3 version addressing.

This way, we do not depend on an implementation characteristic
where the version string embedded in URLs is passed through to
the backend version parameter. This allows for the backend strategy
to change without breaking the rename logic that happens higher
in the stack.
@karlcz karlcz requested a review from mikedarcy April 8, 2024 17:04
@karlcz karlcz self-assigned this Apr 8, 2024
it seems like typing.NamedTuple in some Python versions cannot
tolerate methods alongside the basic typed attributes.
using older (deprecated?) syntax needed for Python 3.8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants