Skip to content

Conversation

@asm582
Copy link
Member

@asm582 asm582 commented May 2, 2023

Merge quota management functionality

dmatch01 and others added 30 commits December 14, 2022 11:59
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
…n to happen.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
dmatch01 and others added 11 commits March 1, 2023 18:44
…ng running Travis job.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
* Move POC rest impl. to quota plubins.

* Initial checkin of new quota management.

* Partial update to e2e script to enable instatiate 2 differently configured mcad controllers.

* Added kuttl installation to e2e test.

* Synced version file.

* Modified kuttl installation to include sudo for commands.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Added startup of MCAD with Quota Management enabled

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Added kuttl test command to e2e testing.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Provided an alternative to the travis_wait command.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Added full e2e testing back.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Added explicit kuttl config file.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Set e2e test to above if kuttl test fails.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Uptick of release number.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Fix syntax error.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Added creation of test namespace to kuttl testing steps.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Test kuttl failure in Travis run.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Test fix to make e2e test failure in background mode.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Test fix to make e2e test failure in background mode.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Test fix to make e2e test kuttle success run in background mode.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Fix job names to match file names and increased timeout for preemption to happen.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Increase kuttle timeout.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Test fix to make e2e test failure in background mode.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Test fix to make e2e test failure in background mode.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Test fix to make e2e test kuttle success run in background mode.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Use different color code for e2e PID message.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Debugging kuttl failure.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Rearrange setup of kuttl test for more stability of test.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Introduced forced error test to validate new Travis workaround for long running Travis job.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Force final failure test for e2e kuttl test.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Removed forced failure for Travis testing.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Added delay step 3 to give time for MCAD to process subtrees.

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

* Updated QM test to use Helm3

Signed-off-by: dmatch01 <darroyo@us.ibm.com>

---------

Signed-off-by: dmatch01 <darroyo@us.ibm.com>
Signed-off-by: dmatch01 <darroyo@us.ibm.com>
* update (#298)

* Upgraded the golang version
Updated travis config for the new version
Updated build.sh scripts for the new version

* Updates to go.mod
Updates to travis.yml, to set dist
Added the generated.deepcopy files

* Removed unused package.

* e2e test script updates

* Queue.go updates
Other fixes to the makefile and e2e test

* Updates to the e2e script
Updates to util.go to allow for different images to be used.

* Updates to to local host setup

* Changes to the run-e2e-kind.sh locally (#290)

* Changes to the run-e2e-kind.sh to

* Changed out echo server image, small edits

* Backed out change in the clean up code.

* Updated the makefile to build the executable

* Address code review comments.

* * Small changes to the e2e script
* Build documentation updates.

* Documentation updates per code review
Fixed errors in the e2e script to load ubuntu:latest image correctly.

* Cleaned up false errors
Rectified use of undefined variable.

* Downgraded go to version 1.18 to allow for use of ubi images.

* Downgraded travis ubuntu distro

* Added use of docker builder using ubi8 images
Makefile updates

* Cleaned up the obsolete build scripts
Updated documentation
Minor changes to makefile

* avoid preempt of failed co-schd pods (#317)

* Update CONTROLLER_VERSION

* Updates to travis and devcontainer

* Small updates to the run-e2e test

* Updated go-restful dependency

* More go.mod updates
debug messages for the e2e script.

* Added back in the removal of taints from head node
Cleanup of junk files.

* Remove whitesapce

* Small corrections.

* Refactoring of env-setup

* moe arch issues with the kuttl binnary

* Updated the go version in go.mod (#316)

* Updated the go version in go.mod
Ran `go mod tidy`
Updated travis.yml
Deleted staled code, disabled the quota manager in the controller.

* Added travis changes.

* Removed linter install
Ported changes to .travis.yml from quota-management-branch

* Removed linter install
Ported changes to .travis.yml from quota-management-branch

* Downgraded go to version 1.18 to allow for use of ubi images.

* Added use of docker builder using ubi8 images
Makefile updates

* Cleaned up the obsolete build scripts
Updated documentation
Minor changes to makefile

* Small fix to the docker file.

* More updates to go.mod to fix vulnerabilities

* Removed text files.

* Yet another text file removal

* Small fix to address PR review comments.

* Create 1.29.57 release

Signed-off-by: Anish Asthana <anishasthana1@gmail.com>

* Adding Performance test script (#289)

* Adding Performance test script

Signed-off-by: James Busche <jbusche@us.ibm.com>

* red hat image and more checking

Signed-off-by: James Busche <jbusche@us.ibm.com>

* Check for mcad install

Signed-off-by: James Busche <jbusche@us.ibm.com>

* remove oc and formatting

Signed-off-by: James Busche <jbusche@us.ibm.com>

* fix to run from any namespace

Signed-off-by: James Busche <jbusche@us.ibm.com>

---------

Signed-off-by: James Busche <jbusche@us.ibm.com>

* fix typo (#326)

* Document updates.

* remove invalid characters from branch name an replace with - (#328)

Signed-off-by: Kevin <kpostlet@redhat.com>

* Bump golang.org/x/text from 0.3.4 to 0.3.8 (#321)

Bumps [golang.org/x/text](https://github.com/golang/text) from 0.3.4 to 0.3.8.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](golang/text@v0.3.4...v0.3.8)

---
updated-dependencies:
- dependency-name: golang.org/x/text
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump golang.org/x/net from 0.0.0-20210405180319-a5a99cb37ef4 to 0.7.0 (#324)

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.0.0-20210405180319-a5a99cb37ef4 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/commits/v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/gogo/protobuf from 1.3.1 to 1.3.2 (#323)

Bumps [github.com/gogo/protobuf](https://github.com/gogo/protobuf) from 1.3.1 to 1.3.2.
- [Release notes](https://github.com/gogo/protobuf/releases)
- [Commits](gogo/protobuf@v1.3.1...v1.3.2)

---
updated-dependencies:
- dependency-name: github.com/gogo/protobuf
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Added .devcontainer to gitignore
Removed  directory from commit.

* Codereview comments.

* Bump golang.org/x/crypto from 0.0.0-20201002170205-7f63de1d35b0 to 0.1.0 (#322)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.0.0-20201002170205-7f63de1d35b0 to 0.1.0.
- [Release notes](https://github.com/golang/crypto/releases)
- [Commits](https://github.com/golang/crypto/commits/v0.1.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fixes to the setup part of the script.

* change schedSpec to schedulingSpec and note M1 image for echo server in tutorial (#333)

Signed-off-by: Sunyanan Choochotkaew <sunyanan.choochotkaew1@ibm.com>

* Changes user to 1000 (#329)

* non-root 1000 user

Signed-off-by: James Busche <jbusche@us.ibm.com>

* cleanup dockerfile

Signed-off-by: James Busche <jbusche@us.ibm.com>

* moving curl into builder

Signed-off-by: James Busche <jbusche@us.ibm.com>

---------

Signed-off-by: James Busche <jbusche@us.ibm.com>

* Address  race condition in XController variable (#330)

* Update Dockerfile:
*  To not to use root user to build
*  To allow for passing of golang build args
Updated Makefile:
* To selectively run the generate code
* To allow for passing of golang build args

* Updates to ensure that docker builds flags are used
Update to the e2e script to allow for not caching test runs and cluster setup

* Updates to e2e script and travis for log level
Fixed 1 race condition issue.

* Reverted needless changes.

* Documentation updates.
Minor fixes to make file.

* Fixed bug in docker file
Reduced noise in e2e test.

* Fixed potential regression.

* Added back Dockerfile

* Fix bad merge.

---------

Signed-off-by: Anish Asthana <anishasthana1@gmail.com>
Signed-off-by: James Busche <jbusche@us.ibm.com>
Signed-off-by: Kevin <kpostlet@redhat.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Sunyanan Choochotkaew <sunyanan.choochotkaew1@ibm.com>
Co-authored-by: Kai-Hsun Chen <kaihsun@apache.org>
Co-authored-by: Abhishek Malvankar <asm582@users.noreply.github.com>
Co-authored-by: Mustafa Eyceoz <maxusmusti@gmail.com>
Co-authored-by: Anish Asthana <anishasthana1@gmail.com>
Co-authored-by: James Busche <101845348+jbusche@users.noreply.github.com>
Co-authored-by: Lionel Villard <villard@us.ibm.com>
Co-authored-by: Kevin Postlethwait <k.postlethwait24@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sunyanan Choochotkaew <sunya-ch@ist.osaka-u.ac.jp>
z103cb
z103cb previously approved these changes May 3, 2023
Copy link
Contributor

@z103cb z103cb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@asm582
Copy link
Member Author

asm582 commented May 4, 2023

Testing this PR to merge quota management into the base. Tests focus was on testing existing features with the quota management feature turned off and looking for regressions if any.

  • All scale tests and manual tests pass.
  • Two test cases that are failing in Travis are mostly due to Travis VM resource jitter, these tests need to be modified to not rely on VM resources down to the millicores to pass. On local env, one out of the two of the test passed, and one fails due to a mismatch in resource assignment.

I think we can merge this PR into the main with the quota management feature turned off.

@asm582
Copy link
Member Author

asm582 commented May 4, 2023

@z103cb Can you help this PR get reviewed and approved?

Copy link
Contributor

@z103cb z103cb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is Ok to merge.

@asm582 asm582 merged commit e391803 into main May 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants