Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CFE-977, CFE-978, CFE-982: Introduce history interface with its Reader and Append implementation #736

Merged
merged 1 commit into from Nov 23, 2023
Merged

CFE-977, CFE-978, CFE-982: Introduce history interface with its Reader and Append implementation #736

merged 1 commit into from Nov 23, 2023

Conversation

aguidirh
Copy link
Contributor

Description

This feature creates and read a history file. This file contains all blobs mirrored in a mirrorToDisk flow.

The Read func will get the latest history file (format is .history-time.RFC3339 example: .history-2023-11-22T11:24:14Z) if no date is specified. In case a date is specified in the NewHistory func, the Read func will return the latest history before the specified date.

The Append func will call the Read func to get the blobs from a previous history file and will generate a new history file with the content of the previous history file + the new blobs mirrored in the current run.

Fixes # CFE-977, CFE-978 and CFE-982

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Unit test under v2/pkg/history/history_test.go

Expected Outcome

Unit tests should pass

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Nov 23, 2023
@openshift-ci-robot
Copy link

openshift-ci-robot commented Nov 23, 2023 •
edited by openshift-ci bot

@aguidirh: This pull request references CFE-982 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "4.15.0" version, but no target version was set.

In response to this:

Description

This feature creates and read a history file. This file contains all blobs mirrored in a mirrorToDisk flow.

The Read func will get the latest history file (format is .history-time.RFC3339 example: .history-2023-11-22T11:24:14Z) if no date is specified. In case a date is specified in the NewHistory func, the Read func will return the latest history before the specified date.

The Append func will call the Read func to get the blobs from a previous history file and will generate a new history file with the content of the previous history file + the new blobs mirrored in the current run.

Fixes # CFE-977, CFE-978 and CFE-982

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Unit test under v2/pkg/history/history_test.go

Expected Outcome

Unit tests should pass

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

sherine-k
sherine-k reviewed Nov 23, 2023
View reviewed changes
Copy link
Contributor

@sherine-k sherine-k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @aguidirh ! great work!
One small question for integration #734: Can the digests included in .history also contain the algorithm (ex: sha256:)?
The tests don't show it but I don't see why it couldn't.

v2/pkg/history/history.go Outdated
}

func isHistoryFile(historyFile fs.DirEntry) bool {
return !historyFile.IsDir() && strings.Contains(historyFile.Name(), historyNamePrefix)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or hasPrefix?

v2/pkg/history/history.go

var log clog.PluggableLoggerInterface

type OSFileCreator struct{}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea!

v2/pkg/history/history_test.go
Comment on lines +21 to +24
func (m MockFileCreator) Create(name string) (io.WriteCloser, error) {
m.Buffer = new(bytes.Buffer)
return nopCloser{m.Buffer}, nil
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea! no tmp folder, no files on disk!
👍

v2/pkg/history/history_test.go
} else {
assert.NoError(t, err)
}
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any way you could also compare the new history content to expectedHist?

lmzuccarelli
lmzuccarelli reviewed Nov 23, 2023
View reviewed changes
Copy link
Contributor

@lmzuccarelli lmzuccarelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aguidirh - agreed great work - nice and clean

@aguidirh
Copy link
Contributor Author

/test sanity

@aguidirh
Copy link
Contributor Author

/retest

1 similar comment
@aguidirh
Copy link
Contributor Author

/retest

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Nov 23, 2023

@aguidirh: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@sherine-k
Copy link
Contributor

/lgtm

@sherine-k
Copy link
Contributor

/approve

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Nov 23, 2023
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Nov 23, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aguidirh, sherine-k

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 23, 2023
@lmzuccarelli
Copy link
Contributor

/label acknowledge-critical-fixes-only

@openshift-ci openshift-ci bot added the acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. label Nov 23, 2023
@openshift-merge-bot openshift-merge-bot bot merged commit ef36dd9 into openshift:main Nov 23, 2023
5 checks passed
@openshift-bot
Copy link
Contributor

[ART PR BUILD NOTIFIER]

This PR has been included in build oc-mirror-plugin-container-v4.15.0-202311232032.p0.gef36dd9.assembly.stream for distgit oc-mirror-plugin.
All builds following this will include this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sherine-k sherine-k sherine-k left review comments

@lmzuccarelli lmzuccarelli lmzuccarelli left review comments

Assignees

@sherine-k sherine-k

Labels
acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@aguidirh @openshift-ci-robot @sherine-k @lmzuccarelli @openshift-bot