Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support private repo #41

Merged
merged 2 commits into from
Feb 11, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
56 changes: 54 additions & 2 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions Gopkg.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,7 @@
[[constraint]]
name = "github.com/openfaas/faas-provider"
version = "0.14.0"

[[constraint]]
name = "github.com/docker/cli"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I looked at the containerd resolve it seemed cleaner / more lightweight without depending on Docker itself, like this - https://github.com/containerd/containerd/blob/master/cmd/ctr/commands/resolver.go

Do you think it can be made to work without the Docker CLI?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

like "github.com/containerd/containerd/remotes/docker" using NewDockerAuthorizer?

Even it (https://github.com/containerd/containerd/blob/4f348c5b5b53a70b282ab5b81b2cc8f03bf6a338/remotes/docker/authorizer.go#L90) didn't seem to use Docker itself.

What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't easily remove dependency on docker/cli, mostly because I believe we want to support cred helper binaries, though I think I've heard that there is an alternative go pkg for that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should open docker/cli PR to reduce dependencies

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed to support the helper binaries then? Does containerd/cri not support pulling from AWS ECR then?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder what @ibuildthecloud did in k3c? https://github.com/rancher/k3c/blob/master/pkg/daemon/image.go

I can see a reference to docker/distribution there actually

version = "19.3.5"
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,10 @@ You can run this tutorial on your Raspberry Pi, or adapt the steps for a regular

* [faasd - lightweight Serverless for your Raspberry Pi](https://blog.alexellis.io/faasd-for-lightweight-serverless/)

### Using private repos

To use private image repos, `~/.docker/config.json` needs to be copied to `/var/lib/faasd/.docker/config.json`.

### Manual / developer instructions

See [here for manual / developer instructions](docs/DEV.md)
Expand Down
14 changes: 7 additions & 7 deletions pkg/provider/handlers/deploy.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,17 @@ import (
"net/http"
"os"
"path"
"strings"

cninetwork "github.com/openfaas/faasd/pkg/cninetwork"
"github.com/openfaas/faasd/pkg/service"
"github.com/containerd/containerd"
"github.com/containerd/containerd/cio"
"github.com/containerd/containerd/namespaces"
"github.com/containerd/containerd/oci"
gocni "github.com/containerd/go-cni"
"github.com/docker/distribution/reference"
"github.com/opencontainers/runtime-spec/specs-go"
"github.com/openfaas/faas-provider/types"
cninetwork "github.com/openfaas/faasd/pkg/cninetwork"
"github.com/openfaas/faasd/pkg/service"
"github.com/pkg/errors"
)

Expand Down Expand Up @@ -64,11 +64,11 @@ func MakeDeployHandler(client *containerd.Client, cni gocni.CNI, secretMountPath
}

func deploy(ctx context.Context, req types.FunctionDeployment, client *containerd.Client, cni gocni.CNI, secretMountPath string) error {

imgRef := "docker.io/" + req.Image
if strings.Index(req.Image, ":") == -1 {
imgRef = imgRef + ":latest"
r, err := reference.ParseNormalizedNamed(req.Image)
if err != nil {
return err
}
imgRef := reference.TagNameOnly(r).String()

snapshotter := ""
if val, ok := os.LookupEnv("snapshotter"); ok {
Expand Down
58 changes: 55 additions & 3 deletions pkg/service/service.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,23 @@ import (
"context"
"fmt"
"log"
"os"
"path/filepath"
"sync"
"time"

"github.com/containerd/containerd"
"github.com/containerd/containerd/errdefs"
"github.com/containerd/containerd/remotes"
"github.com/containerd/containerd/remotes/docker"
"github.com/docker/cli/cli/config"
"github.com/docker/cli/cli/config/configfile"
"golang.org/x/sys/unix"
)

// dockerConfigDir contains "config.json"
const dockerConfigDir = "/var/lib/faasd/.docker/"

// Remove removes a container
func Remove(ctx context.Context, client *containerd.Client, name string) error {

Expand Down Expand Up @@ -90,16 +99,59 @@ func killTask(ctx context.Context, task containerd.Task) error {
return err
}

func getResolver(ctx context.Context, configFile *configfile.ConfigFile) (remotes.Resolver, error) {
// credsFunc is based on https://github.com/moby/buildkit/blob/0b130cca040246d2ddf55117eeff34f546417e40/session/auth/authprovider/authprovider.go#L35
credFunc := func(host string) (string, string, error) {
if host == "registry-1.docker.io" {
host = "https://index.docker.io/v1/"
}
ac, err := configFile.GetAuthConfig(host)
if err != nil {
return "", "", err
}
if ac.IdentityToken != "" {
return "", ac.IdentityToken, nil
}
return ac.Username, ac.Password, nil
}
authOpts := []docker.AuthorizerOpt{docker.WithAuthCreds(credFunc)}
authorizer := docker.NewDockerAuthorizer(authOpts...)
opts := docker.ResolverOptions{
Hosts: docker.ConfigureDefaultRegistries(docker.WithAuthorizer(authorizer)),
}
return docker.NewResolver(opts), nil
}

func PrepareImage(ctx context.Context, client *containerd.Client, imageName, snapshotter string) (containerd.Image, error) {
var (
empty containerd.Image
resolver remotes.Resolver
)
if _, stErr := os.Stat(filepath.Join(dockerConfigDir, config.ConfigFileName)); stErr == nil {
configFile, err := config.Load(dockerConfigDir)
if err != nil {
return nil, err
}
resolver, err = getResolver(ctx, configFile)
if err != nil {
return empty, err
}
} else if !os.IsNotExist(stErr) {
return empty, stErr
}

var empty containerd.Image
image, err := client.GetImage(ctx, imageName)
if err != nil {
if !errdefs.IsNotFound(err) {
return empty, err
}

img, err := client.Pull(ctx, imageName, containerd.WithPullUnpack)
rOpts := []containerd.RemoteOpt{
containerd.WithPullUnpack,
}
if resolver != nil {
rOpts = append(rOpts, containerd.WithResolver(resolver))
}
img, err := client.Pull(ctx, imageName, rOpts...)
if err != nil {
return empty, fmt.Errorf("cannot pull: %s", err)
}
Expand Down
57 changes: 57 additions & 0 deletions vendor/github.com/Microsoft/hcsshim/osversion/osversion_windows.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

23 changes: 23 additions & 0 deletions vendor/github.com/Microsoft/hcsshim/osversion/windowsbuilds.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.