Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCI Target #86

Merged
merged 2 commits into from
Jan 24, 2022
Merged

OCI Target #86

merged 2 commits into from
Jan 24, 2022

Conversation

Wwwsylvia
Copy link
Member

Resolves #55

internal/resolver/oci.go Outdated Show resolved Hide resolved
internal/resolver/oci.go Outdated Show resolved Hide resolved
internal/resolver/oci.go Outdated Show resolved Hide resolved
internal/resolver/oci.go Outdated Show resolved Hide resolved
internal/resolver/oci.go Outdated Show resolved Hide resolved
internal/cas/oci.go Outdated Show resolved Hide resolved
internal/cas/oci.go Outdated Show resolved Hide resolved
internal/cas/oci.go Outdated Show resolved Hide resolved
content/oci/oci.go Outdated Show resolved Hide resolved
content/oci/oci.go Outdated Show resolved Hide resolved
ORAS CLI/Go-lib Project Board automation moved this from Backlog to In Progress Dec 27, 2021
@Wwwsylvia Wwwsylvia force-pushed the oci_target branch 2 times, most recently from c6fe769 to 56386c9 Compare December 29, 2021 15:25
internal/ioutil/io.go Outdated Show resolved Hide resolved
internal/ioutil/io.go Outdated Show resolved Hide resolved
content/oci/storage.go Outdated Show resolved Hide resolved
content/oci/storage.go Outdated Show resolved Hide resolved
content/oci/oci.go Outdated Show resolved Hide resolved
content/oci/oci.go Outdated Show resolved Hide resolved
content/oci/oci.go Outdated Show resolved Hide resolved
content/oci/oci.go Outdated Show resolved Hide resolved
Comment on lines 61 to 82
if err := store.validateOCILayoutFile(); err != nil {
return nil, err
}

if err := store.loadIndex(); err != nil {
return nil, err
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What component is supposed to do the initialization to create the layerout and index file?

Copy link
Member Author

@Wwwsylvia Wwwsylvia Jan 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The layout file is initialized in validateOCILayoutFile() when loading. The index file will be initialized on first tag in saveIndex()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. It is not clear from the name that validateOCILayeroutFile creates the layout file if it doesn't exist. Maybe you can rename it something like validateOrCreateOCILayoutFile

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know where I can find a code snippet example of copy an oci artifact from registry to local?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shall update the examples once the implementation is done... Before that, @akashsinghal has written some code to copy some artifacts from OCI target to remote registry, which is helpful - https://github.com/akashsinghal/accelerated-container-image/blob/47ef7a728bed74020ef48814e8efb461f3db0236/cmd/ctr/overlaybd_conv.go#L740-L781

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Linking to #60

content/oci/oci.go Outdated Show resolved Hide resolved
@Wwwsylvia Wwwsylvia force-pushed the oci_target branch 2 times, most recently from 842feab to 463badf Compare January 4, 2022 11:13
copy_test.go Outdated Show resolved Hide resolved
internal/ioutil/io.go Outdated Show resolved Hide resolved
internal/ioutil/io.go Outdated Show resolved Hide resolved
content/memory/memory.go Outdated Show resolved Hide resolved
content/memory/memory.go Outdated Show resolved Hide resolved
content/oci/oci.go Outdated Show resolved Hide resolved
content/oci/oci.go Outdated Show resolved Hide resolved
content/oci/oci.go Outdated Show resolved Hide resolved
content/oci/oci.go Outdated Show resolved Hide resolved
content/oci/oci.go Outdated Show resolved Hide resolved
internal/ioutil/io.go Outdated Show resolved Hide resolved
internal/graph/memory.go Outdated Show resolved Hide resolved
internal/graph/memory.go Outdated Show resolved Hide resolved
content/oci/storage.go Outdated Show resolved Hide resolved
content/oci/storage.go Outdated Show resolved Hide resolved
content/oci/storage.go Outdated Show resolved Hide resolved
content/oci/storage.go Outdated Show resolved Hide resolved
content/oci/storage.go Outdated Show resolved Hide resolved
return s.storage.Exists(ctx, target)
}

// Tag tags a descriptor with a reference string.
Copy link
Contributor

@shizhMSFT shizhMSFT Jan 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the user invokes s.Tag(ctx, desc, "localhost:5000/myrepo:mytag") since it is a common scenario in Copy(). Will you parse it and only store mytag in org.opencontainers.image.ref.name? or consider localhost:5000/myrepo:mytag as an invalid reference?

indexFile, err := os.Open(s.indexPath)
if err != nil {
if !os.IsNotExist(err) {
return fmt.Errorf("failed to open index file: %v", err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: it will be better to use %w instead of %v for wrapping errors. Same comment will be applied to all similar occourances.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see fmt.Errorf() for the documentation of %w.

Copy link
Member Author

@Wwwsylvia Wwwsylvia Jan 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the user invokes s.Tag(ctx, desc, "localhost:5000/myrepo:mytag") since it is a common scenario in Copy(). Will you parse it and only store mytag in org.opencontainers.image.ref.name? or consider localhost:5000/myrepo:mytag as an invalid reference?

We need to store the original user-provided reference in the org.opencontainers.image.ref.name as we use it to construct ref map when loading index. As per the OCI image spec, a reference like "localhost:5000/myrepo:mytag" is valid in grammar.
For the common Copy scenarios, isn't it like

// do something...
ref := "mytag"
err := oci.Push(ctx, desc, content)
err := oci.Tag(ctx, desc, ref)

repo := "localhost:5000/myrepo"
repository, err := remote.NewRepository(repo)
// do something...
root, err := oras.Copy(ctx, oci, ref, repository, ref)

? The user would call oci.Tag(ctx, desc, "mytag") instead of oci.Tag(ctx, desc, "localhost:5000/myrepo:mytag")

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about this scenario?

ref := "localhost:5000/hello:v1"
src, _ := remote.NewRepository(ref)
dst, _ := oci.New("hello")
oras.Copy(context.Background(), src, ref, dst, "")

Copy link
Member Author

@Wwwsylvia Wwwsylvia Jan 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's ok. In this case we will have "org.opencontainers.image.ref.name": "localhost:5000/hello:v1".
The OCI image spec mentions that

No semantic restriction is given for the "org.opencontainers.image.ref.name" annotation of descriptors.

So it makes sense to store FQDN in org.opencontainers.image.ref.name.
After copy, the user can resolve by the same ref.

ref := "localhost:5000/hello:v1"
src, _ := remote.NewRepository(ref)
dst, _ := oci.New("hello")
oras.Copy(context.Background(), src, ref, dst, "")

desc, _ := dst.Resolve(ctx, ref)

Copy link
Contributor

@shizhMSFT shizhMSFT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with suggestions. Please also convert the reference discussion into an issue so that we can address later.

internal/graph/memory.go Outdated Show resolved Hide resolved
internal/graph/memory.go Outdated Show resolved Hide resolved
@Wwwsylvia Wwwsylvia force-pushed the oci_target branch 2 times, most recently from aea9977 to a8079b9 Compare January 18, 2022 06:57
Copy link
Contributor

@shizhMSFT shizhMSFT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sajayantony
Copy link
Contributor

@shizhMSFT please squash merge. Let’s discuss the API usability to consume these and clean up before release.

Copy link
Contributor

@jdolitsky jdolitsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lot to review, but nothing jumps out at me to prevent merge

Signed-off-by: Sylvia Lei <lixia_lei@outlook.com>
Signed-off-by: Sylvia Lei <lixia_lei@outlook.com>
@shizhMSFT shizhMSFT assigned shizhMSFT and unassigned shizhMSFT Jan 24, 2022
@shizhMSFT shizhMSFT merged commit 24c5637 into oras-project:main Jan 24, 2022
ORAS CLI/Go-lib Project Board automation moved this from In Progress to Done Jan 24, 2022
@Wwwsylvia Wwwsylvia deleted the oci_target branch January 25, 2022 01:59
@shizhMSFT shizhMSFT mentioned this pull request Apr 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

OCI Target
6 participants