Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Node Conformance Test: Package Node Conformance Test #30174

Closed
9 of 10 tasks
Random-Liu opened this issue Aug 6, 2016 · 4 comments
Closed
9 of 10 tasks

Node Conformance Test: Package Node Conformance Test #30174

Random-Liu opened this issue Aug 6, 2016 · 4 comments
Assignees
@Random-Liu
Labels
area/node-e2e area/test sig/node Categorizes an issue or PR as relevant to SIG Node.
Milestone
v1.5

Comments

@Random-Liu
Copy link
Member

Random-Liu commented Aug 6, 2016 •
edited
Loading

For #30122.
In 1.4, we plan to provide part of the node e2e test to the user as the node conformance test.

For user adoption, We should package the node conformance test well to make it easy to ship and for the user to use.

After discussion with @dchen1107 and @vishh, here is the current plan:

  • Node Conformance Test: Remove unnecessary binary copy #30348 Node Conformance Test: Refactor node e2e framework #30540 Separate node conformance test and its dependencies from node e2e framework. To package node conformance test, we should firstly figure out what should be included in node conformance test and separate them out. According to the UX defined in Node Conformance Test Planning #30122, the node conformance test should include:
    • [Required] Ginkgo test suite
    • [Required] Necessary kubernetes services:
    • [Nice-to-have] Ginkgo. The binary of test suite already has enough functionality to run the test sequentially. However, ginkgo is more powerful, e.g. we still need ginkgo to run test in parallel. So it is nice to have.
    • [Optional] Move kubelet start logic out of test suite. Currently in node e2e, we start kubelet in the test suite, which is not needed in the node conformance test. We should separate the logic out of the test suite into the node e2e framework.
    • [Optional] Node E2E: Remove kubelet related flags #36919 Make the current kubelet flags as test suite input and make it accessible by all tests, so that we could:
      • Filter test based on kubelet configuration.
      • Run test against current kubelet configuration.
  • Make the node conformance test one binary. To make it easier to ship and use, we decided to statically link all required services into a single test binary. To achieve this, we need to:
  • Node Conformance Test: Containerize the node e2e test #31093 Package the node conformance test as a docker image. In fact, container runtime itself is one of the test targets of the node conformance test. We should not rely on it to run the test. However, container image is actually the easiest way to package and ship your program to different platforms. To compromise, we decided to:
    • Ship a docker image version for most users.
    • Build the test suite into a single binary to make it easy for users who don't like the container runtime dependency to build and run the test themselves.

@dchen1107 @vishh
/cc @kubernetes/sig-node
@Random-Liu Random-Liu added area/test sig/node Categorizes an issue or PR as relevant to SIG Node. area/node-e2e labels Aug 6, 2016
@Random-Liu Random-Liu added this to the v1.4 milestone Aug 6, 2016
@Random-Liu Random-Liu self-assigned this Aug 6, 2016
@Random-Liu Random-Liu mentioned this issue Aug 6, 2016
Closed
8 tasks
@vishh
Copy link
Contributor

vishh commented Aug 6, 2016

I'd prefer including kubelet also in the static binary and unifying conformance testing and node e2e. We can run kubelet as part of testing in node e2e and not run in the case of conformance testing. The default can be that of not running kubelet.

This was referenced Aug 6, 2016
Merged
Merged
Merged
Merged
@dchen1107 dchen1107 mentioned this issue Aug 8, 2016
Closed
@Random-Liu
Copy link
Member Author

Random-Liu commented Aug 9, 2016 •
edited
Loading

I'd prefer including kubelet also in the static binary and unifying conformance testing and node e2e. We can run kubelet as part of testing in node e2e and not run in the case of conformance testing. The default can be that of not running kubelet.

There are 2 questions about this:
1) Should we statically link in kubelet?
I prefer not.

  • For node conformance test, we assume user should have set up kubelet on the node, so no need to carry one ourselves.
  • For node e2e test, I don't think test statically linked kubelet is testing "real kubelet". We should test kubelet binary generated by our build process, started by our cmd/kubelet, deploy it and test it, which I think is "real kubelet". We can statically link etcd, apiserver and namespace controller, because our purpose is not testing them, they are just part of test framework. However, kubelet is different.

If the theory makes sense, then both node conformance test and node e2e test don't need to link in kubelet. :)

2) Should we start kubelet in the test suite?
One problem of starting kubelet in the test suite is that we plan to run node conformance test in container. However, currently kubelet has to run on the host. There is no good way to start a native process from inside a container (maybe ssh or something). So we can't start kubelet in test suite for node conformance test.
For node e2e test, we can start kubelet in test suite, if we don't containerize the test. However, in this way node conformance and e2e test are not unified, because we've to start kubelet in one and not in another.

If we start kubelet outside of the test suite, the framework could be shared in node conformance test and node e2e test.

For node conformance test:
Step 1: User set up the node, including kubelet, cni etc. (In the future, using installer)
------ Shared from here ------
Step 2: Run node test script.

  • If kubelet is not started, the script starts kubelet with default flags.
  • If kubelet is running but is pointed to local apiserver, the script restarts kubelet keeping all flags except apiserver address.

Step 3: The node test script starts node test container with current kubelet flag as one of the arguments.

For node e2e test:
Step 1: Node e2e framework ssh and set up the node, including kubelet, cni etc. (In the future, using installer)
Step 2: Node e2e framework ssh and run kubelet with flags we want to test with.
------ Shared from here ------
Step 3: Run the same node test script.
Step 4: The node test script starts node test container with current kubelet flag as one of the arguments.

In this way, we can:

  1. Make sure kubelet is installed in the same way with user in the future.
  2. Unify node conformance test and node e2e test.

@vishh @dchen1107 WDYT?

This was referenced Aug 10, 2016
Merged
Merged
k8s-github-robot pushed a commit that referenced this issue Aug 17, 2016
Merge pull request #30116 from Random-Liu/start-e2e-services-in-separ…
457a151 
…ate-process

Automatic merge from submit-queue

Node Conformance Test: Start e2e services in a separate process

For #30122.

This is the first step of #30174.

This PR added a `start-services-only` mode in the test suite, and start all e2e services in a separate process by run the test binary again in `start-services-only` mode.
k8s-github-robot pushed a commit that referenced this issue Aug 18, 2016
Merge pull request #30198 from Random-Liu/statically-link-etcd
6faa91d 
Automatic merge from submit-queue

Node Conformance Test: Statically link etcd

For #30122, #30174.

This PR is part of our roadmap to package node conformance test.

It statically linked etcd into the node e2e framework. In the future all e2e services will be linked in, and print log into the same log file `services.log`.

@dchen1107 @vishh 
/cc @kubernetes/sig-node
k8s-github-robot pushed a commit that referenced this issue Aug 20, 2016
Merge pull request #30200 from Random-Liu/move-namespace-controller-t…
98c4029 
…o-services

Automatic merge from submit-queue

Node Conformance Test: Move namespace controller to services

For #30122, #30174.
Based on #30116, #30198.

**Please only review the 3rd PR.**

This PR is part of our roadmap to package node conformance test.
The 1st commit is from #30116, which started e2e services in a separate process.
The 2nd commit is from #30198, it statically linked etcd into the node e2e framework.

The 3rd commit is new, it moved namespace controller into e2e services.

@dchen1107 @vishh 
/cc @kubernetes/sig-node @kubernetes/sig-testing
k8s-github-robot pushed a commit that referenced this issue Aug 21, 2016
Merge pull request #30211 from Random-Liu/statically-link-apiserver
dbfad78 
Automatic merge from submit-queue

Node Conformance Test: Statically link apiserver

For #30122, #30174.
Based on #30116, #30198, #30200.

**Please only review the 4th PR.**

This PR is part of our roadmap to package node conformance test.
The 1st commit is from #30116, which started e2e services in a separate process.
The 2nd commit is from #30198, it statically linked etcd into the node e2e framework.
The 3rd commit is from #30200, it moved namespace controller into e2e services.

The 4th commit is new, it statically linked apiserver into the node e2e framework.

@dchen1107 @vishh 
/cc @kubernetes/sig-node @kubernetes/sig-testing
@Random-Liu Random-Liu mentioned this issue Aug 22, 2016
Merged
k8s-github-robot pushed a commit that referenced this issue Aug 24, 2016
Merge pull request #30348 from Random-Liu/remove-unnecessary-binary-copy
1952986 
Automatic merge from submit-queue

Node Conformance Test: Remove unnecessary binary copy

For #30122, #30174.

This PR removed unnecessary dependencies in the node e2e test framework, because we've statically linked these dependencies.

@dchen1107 @vishh 
/cc @kubernetes/sig-node @kubernetes/sig-testing
k8s-github-robot pushed a commit that referenced this issue Aug 25, 2016
Merge pull request #30540 from Random-Liu/refactor-node-e2e-framework
863dd10 
Automatic merge from submit-queue

Node Conformance Test: Refactor node e2e framework

For #30122, #30174.
Based on #30348.

**Please only review the last 3 commits.**

This PR is part of our roadmap to package node conformance test.
The 1st commit is from #30348, it removed unnecessary dependencies in the node e2e test framework, because we've statically linked these dependencies.

The PR refactored the node e2e framework. Moving different utilities into different packages under `pkg/`.

We need to do this because:
1) Files like e2e_remote.go and e2e_build.go should only be used by runner, but they were compiled into the test suite because they were placed in the same package. The worst thing is that it will introduce some never used flags in the test suite binary.
2) Make the directory structure more clear. Only test should be placed in `test/e2e_node`, other utilities should be placed in different packages in `pkg/`.

@dchen1107 @vishh 
/cc @kubernetes/sig-node @kubernetes/sig-testing
@dchen1107 dchen1107 mentioned this issue Aug 26, 2016
Closed
18 tasks
@goltermann goltermann modified the milestones: v1.5, v1.4 Sep 6, 2016
k8s-github-robot pushed a commit that referenced this issue Nov 8, 2016
Merge pull request #31093 from Random-Liu/containerize-node-e2e-test
0df6384 
Automatic merge from submit-queue

Node Conformance Test: Containerize the node e2e test

For #30122, #30174.
Based on #32427, #32454.

**Please only review the last 3 commits.**

This PR packages the node e2e test into a docker image:
- 1st commit: Add `NodeConformance` flag in the node e2e framework to avoid starting kubelet and collecting system logs. We do this because:
  - There are all kinds of ways to manage kubelet and system logs, for different situation we need to mount different things into the container, run different commands. It is hard and unnecessary to handle the complexity inside the test suite.
- 2nd commit: Remove all `sudo` in the test container. We do this because:
  - In most container, there is no `sudo` command, and there is no need to use `sudo` inside the container.
  - It introduces some complexity to use `sudo` inside the test. (#29211, #26748) In fact we just need to run the test suite with `sudo`.
- 3rd commit: Package the test into a docker container with corresponding `Makefile` and `Dockerfile`. We also added a `run_test.sh` script to start kubelet and run the test container. The script is only for demonstration purpose and we'll also use the script in our node e2e framework. In the future, we should update the script to start kubelet in production way (maybe with `systemd` or `supervisord`).

@dchen1107 @vishh 
/cc @kubernetes/sig-node @kubernetes/sig-testing



**Release note**:

<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->

``` release-note
Release alpha version node test container gcr.io/google_containers/node-test-ARCH:0.1 for users to verify their node setup.
```
@dims
Copy link
Member

dims commented Nov 17, 2016 •
edited
Loading

@Random-Liu all issues must be labeled either release blocker or non release blocking by end of day 18 November 2016 PST. (or please move it to 1.6) cc @kubernetes/sig-node

@dchen1107
Copy link
Member

All work items listed in this issue were addressed. There are several enhancement related to the node conformance test being tracked through #30122.

@mmerkes mmerkes mentioned this issue Sep 14, 2020
Closed
@hasheddan hasheddan mentioned this issue Mar 6, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Random-Liu Random-Liu

Labels
area/node-e2e area/test sig/node Categorizes an issue or PR as relevant to SIG Node.
Projects
None yet
Milestone
v1.5
Development

No branches or pull requests
5 participants
@dims @Random-Liu @vishh @dchen1107 @goltermann