Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deploy and populate a local container registry #165

Closed
4 tasks
ustiugov opened this issue Mar 11, 2021 · 10 comments · Fixed by #188
Closed
4 tasks

Deploy and populate a local container registry #165

ustiugov opened this issue Mar 11, 2021 · 10 comments · Fixed by #188
Labels
enhancement New feature or request
Milestone
Benchmarking at s...

Comments

@ustiugov
Copy link
Member

ustiugov commented Mar 11, 2021 •
edited by HongyuHe
Loading

For the FaaS benchmarking at scale, we would like to avoid the obvious bottleneck of a remote registry, like DockerHub.

Required changes:

  • the ctriface module should not default to docker.io/
  • YAML files should specify the registry when specifying each container image
  • need to add a script that would deploy by URL and populate the local registry in a vHive cluster, by copying the images from remote registries, e.g., DockerHub, to the cluster-local registry
  • tests need to be updated accordingly

We welcome a contribution from @amohoste on this issue.
@ustiugov ustiugov added the enhancement New feature or request label Mar 11, 2021
@ustiugov ustiugov added this to the Benchmarking at scale milestone Mar 11, 2021
@amohoste
Copy link
Contributor

amohoste commented Mar 16, 2021 •
edited
Loading

How would you like the local registry population to work? Do you want it to be automated such that upon calling the deployer, images are first pulled to the local registry or did you have a separate script in mind.

Also, there should be a way for the script to know which global registry it can fetch the images from. A straightforward way to do this would be to map the local url to the global url of a specified registry. For example if a user enters docker-registry.registry.svc.cluster.local:5000/vhiveease/pyaes:var_workload in the yaml file, the script tries to pull the image from docker.io/vhiveease/pyaes:var_workload.

Regarding the test cases, we could either extend the image names with docker.io and add some separate testcases for the local registry, or we could let all testcases default to the local registry in which case we would have to first call a function which pulls the images to the local registry.

Please feel free to give other suggestions.

@ustiugov
Copy link
Member Author

@amohoste sorry for the late response.

I would prefer to give complete flexibility of using different registries, incl. dockerhub, AWS ecr, vHive local registry, etc., to the user who specifies the YAML files. The script can take a list of images and two arguments: source registry and destination registry.

@amohoste
Copy link
Contributor

amohoste commented Mar 16, 2021 •
edited
Loading

I suppose a separate script it is then?

Regarding the testcases, should I make them point to docker.io and add some for the local registry or should I port all of them to the local registry?

@ustiugov
Copy link
Member Author

yes, separate. In the unit tests as well as the integration tests except in the cri/ module, you can use docker.io. In cri, we test integration with kubernetes so there it would be good to use the local registry instead (e.g., for one of the functions). The CRI tests are not 100% stable as of now so please let us know in case of any weird failures.

@amohoste
Copy link
Contributor

Could you point me to where the test images for the CRI tests get deployed? cri_test.go is calling the following hardcoded url: helloworld.default.192.168.1.240.xip.io:80 but I'm not 100% sure where this is getting deployed exactly. If we would want to test a function deployed from the local registry, I think we also would also need to deploy a different image than helloworld since we can't have 2 images with the same name.

@ustiugov
Copy link
Member Author

@amohoste these are three different entities.

  • helloworld.default.192.168.1.240.xip.io:80 is a URL of a deployed function that is exposed by the ingress (i.e., istio load balancer).
  • Images are root filesystems (think of docker images).
  • The registry is just a database of these images that need to be pulled by workers (hosts) before starting function instances.

Different functions can use the same images the same way as you can spawn many instances of docker containers on one or different nodes.

@amohoste
Copy link
Contributor

amohoste commented Mar 18, 2021 •
edited
Loading

I am indeed aware of this distinction. I was assuming the helloworld part in the deployed function url was derived from the imagename which would mean that deploying the same image twice from different registries wouldn’t work because it would result in the same url. It indeed makes sense that’s it’s possible to be able to specify the name of the deployed function url.

Regarding the other part of my question, where do the functions for the tests get deployed exactly? I would also have to pull am image to the local registry and deploy it somewhere in the test codebase if I would want to test the local registry functions in the CRI module.

@ustiugov
Copy link
Member Author

Functions deployment is cluster-wide. k8s decides when and on which host to starts an instance of a function. During deployment, it spawns one instance, by default. k8s decides on the worker placement, then the worker fetches the image from the given registry and stores the image locally in the devmapper (think of docker pull). A user doesn't need to do anything except specify a registry -- that must be reachable from a service within the vHive cluster or over the Internet-- as a part of the image name in the corresponding YAML.

@amohoste
Copy link
Contributor

Sorry, I should have made my question more clear. I meant to say in which part of the testing code the deployment of the functions is being done. I saw the github workflow contains some code to deploy the functions but the makefile also contains some deployment functionality. Is it sufficient to pull the image to a local registry and deploy it in the makefile?

@ustiugov
Copy link
Member Author

you are looking at the wrong test (the one for the stock knative mode). For the vHive default mode with Firecracker, look at this test. You can augment this test.

@amohoste amohoste mentioned this issue Mar 20, 2021
Merged
@ustiugov ustiugov linked a pull request Mar 24, 2021 that will close this issue
Deployment of a local registry #188
Merged
HermioneKT added a commit that referenced this issue Jan 31, 2024
@HermioneKT
# This is a combination of 252 commits.
3c6919f 
parent 6674807
author HermioneKT <hermionegrangerkt@gmail.com> 1706694164 +0800
committer HermioneKT <hermionegrangerkt@gmail.com> 1706694164 +0800

test

# This is the commit message #157:

test

# This is the commit message #159:

test

# This is the commit message #160:

test

# This is the commit message #161:

test

# This is the commit message #162:

test

# This is the commit message #163:

test

# This is the commit message #164:

tesT

# This is the commit message #165:

test

# This is the commit message #166:

test

# This is the commit message #167:

test

# This is the commit message #168:

test

# This is the commit message #169:

test

# This is the commit message #170:

test

# This is the commit message #171:

Test

# This is the commit message #172:

test

# This is the commit message #173:

test

# This is the commit message #174:

test

# This is the commit message #175:

test

# This is the commit message #176:

test

# This is the commit message #177:

test

# This is the commit message #178:

test

# This is the commit message #179:

test

# This is the commit message #180:

test

# This is the commit message #181:

test

# This is the commit message #182:

test

# This is the commit message #183:

test

# This is the commit message #184:

test

# This is the commit message #185:

test

# This is the commit message #186:

test

# This is the commit message #187:

test

# This is the commit message #188:

test

# This is the commit message #189:

test

# This is the commit message #190:

Test

# This is the commit message #191:

Test

# This is the commit message #192:

test

# This is the commit message #193:

Test

# This is the commit message #194:

test

# This is the commit message #195:

test

# This is the commit message #196:

test

# This is the commit message #197:

test

# This is the commit message #198:

test

# This is the commit message #199:

Test

# This is the commit message #200:

test

# This is the commit message #201:

test

# This is the commit message #202:

Test

# This is the commit message #203:

test

# This is the commit message #204:

test

# This is the commit message #205:

test

# This is the commit message #206:

test

# This is the commit message #207:

test

# This is the commit message #208:

test

# This is the commit message #209:

test

# This is the commit message #210:

test

# This is the commit message #211:

Test

# This is the commit message #212:

test

# This is the commit message #213:

Test

# This is the commit message #214:

Test

# This is the commit message #215:

Test

# This is the commit message #216:

test

# This is the commit message #217:

Test

# This is the commit message #218:

test

# This is the commit message #219:

test

# This is the commit message #220:

test

# This is the commit message #221:

test

# This is the commit message #222:

test

# This is the commit message #223:

test

# This is the commit message #224:

test

# This is the commit message #225:

test

# This is the commit message #226:

test

# This is the commit message #227:

test

# This is the commit message #228:

test

# This is the commit message #229:

Test

# This is the commit message #230:

test

# This is the commit message #231:

test

# This is the commit message #232:

test

# This is the commit message #233:

test

# This is the commit message #234:

Test

# This is the commit message #235:

test

# This is the commit message #236:

test

# This is the commit message #237:

test

# This is the commit message #238:

test

# This is the commit message #239:

test

# This is the commit message #240:

test

# This is the commit message #241:

test

# This is the commit message #242:

test

# This is the commit message #243:

test

# This is the commit message #244:

test

# This is the commit message #245:

test

# This is the commit message #246:

test

# This is the commit message #247:

test

# This is the commit message #248:

test

# This is the commit message #249:

test

# This is the commit message #250:

test

# This is the commit message #251:

test

# This is the commit message #252:

test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
Benchmarking at scale
Development

Successfully merging a pull request may close this issue.

Deployment of a local registry
2 participants
@ustiugov @amohoste