Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixes 'Add kustomize support to kfctl' #2548

Closed
wants to merge 183 commits into from

Conversation

kkasravi
Copy link
Contributor

@kkasravi kkasravi commented Feb 25, 2019

fixes


This change is Reviewable

Kam D Kasravi added 30 commits June 29, 2018 18:13
@kkasravi kkasravi changed the title [WIP] fixes 'Add kustomize support to kfctl' fixes 'Add kustomize support to kfctl' Mar 8, 2019
@kkasravi
Copy link
Contributor Author

kkasravi commented Mar 8, 2019

additional functionality will follow

Kam D Kasravi added 5 commits March 7, 2019 22:56
* Getting Started card content added, and un-hidden. Windows support added in webpack to run dev builds

* Getting Started card fully implemented from mocks

* Spaced out cards a little more

* Got eslint working! And fixed eslint issues in webpack
Kam D Kasravi added 5 commits March 8, 2019 10:40
* Add Azure support for kfctl.sh

* pass in sku

* update ds
…forms' (kubeflow#2547)

* revert to prior version

* fixes kfctl (golang) rename 'ks' directory and app to ksonnet and ksonnet.go respectively

* fixes 'kfctl - Fetch registry automatically'

* fixes 'refactor gcp, minikube, docker-for-desktop, ack to be kfctl plugins'

* change DefaultDevRepo to point to just the repo not repo/kubeflow

* fixes 'refactor gcp, minikube, docker-for-desktop, ack to be kfctl plugins'

* plugins for all existing KfApp instances {ksonnet, minikube, foo}.

* update golang version

* fixes 'docker-for-desktop'

* delete meta-controller-cluster-role-binding

* update README.md

* remove unused var DEBUG

* remove bootstrap/cmd/plugins/foo.go

* fixes kfctl (golang) rename 'ks' directory and app to ksonnet and ksonnet.go respectively

* fixes 'add the gcp platform to kfctl (golang)'

* fix merge conflict

* cli option --project is an init flag

* fix to write out project to app.yaml

* fix for nil project check

* include fix for kubeflow#2367

* added updateDM, createSecrets to Apply

* fixes to macros

* merge changes from upstream master

* remove old files, update README.md

* fix for init app-name where app-name is not a path

* fix for gcp apply

* code to create a Gcp Secret, doesn't do correct auth right now

* snapshot

* update on downloadK8sManifests, gcpInitProject, added TODO's

* additional work on gcpInitProject

* gcpInitProject now works but requires GOOGLE_APPLICATION_CREDENTIALS env var

* initial pass on updateDeployment

* update README.md to described static vs dynamic platforms

* updates to Gcp.updateDM, added GOOGLE_APPLICATION_CREDENTIALS to the Makefile tests

* minor changes on init, generate usage

* snapshot

* additional global resources identified

* do not delete scalingpolicies

* change default platform to ksonnet

* default --version to master until 0.5.0

* remove verbose from spark-operator apply

* address comments from Jeremy

* address comments from Jeremy

* fixes kubeflow#2537, refer to no platform as 'none'

* add general test for kfctl in Makefile that also tests the plugin capability

* remove kftcl config since we now have bootstrap config, fix for DefaultPlatform="none"

* change test of platform none to create directory none, updated README

* remove reference to docker-for-desktop

* update README.md, remove --debug flag (not used)

* fixes 'kfctl (golang) Simplify the child/parent relationships in platforms'

* fix backendconfig name (kubeflow#2542)

* fix backendconfig name

* fix test

* pin to the latest verified tf-operator commit in order to enable changes in tf-operator HEAD  whcih likely break kubeflow checkout tests

* fix (kubeflow#2506)

* Added unit tests for GetUpdatedPolicy at gcpUtils.go - bootstrap/app (kubeflow#2516)

* Added tests for 'GetUpdatedPolicy' - bootstrap/app

* Fix typo - gcpUtils_test.go

* Added a windows equivalent script for the makefile for central-dashboard (kubeflow#2518)

* update katib component (kubeflow#2553)

* Refactor for Express + Webpack build stack. (kubeflow#2572)

* Refactor for Express + Webpack build stack.

- Add webpack to process front-end code in public/
- Add TS and Express for server backend
- Update Dockerfile to run the server as Node.
- Fix issue with external links.

* Updates based on feedback

* address review comments

* move tests under test directory

* Add ESLint to enforce Google JS style in public/ (kubeflow#2579)

* Add ESLint to enforce Google JS style in public/

* Switch to 4 space tabs

* Updating python package imports related to tf-operator (kubeflow#2581)

* updated tf-operator imports

* fixed a typo

* fixed a typo

* Profiles e2e test (kubeflow#2404)

* profiles e2e test

* fix

* review comment

* fix

* fix

* fix

* fix

* add retry

* fix

* Update the jupyter-web-app Image (kubeflow#2589)

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* Golang profile controller  (kubeflow#2533)

* init

* create api

* wip

* fix

* fix

* fix

* fix test

* address review

* fix test

* make it cluster scope

* fix test

* fix test

* fix

* turnoff jupyter images authentication (kubeflow#2591)

* Updated Jupyter image paths for using in notebook controller (kubeflow#2552)

* updated jupyter image paths for using in notebook conroller

* removed 1.4.1 image from the list of options

* updated jupyter image tags to point to the latest verified ones

* updated jupyter images to allow any origin (kubeflow#2606)

* Removing prune for Subresource field in CRD (kubeflow#2598)

* Adding backOffLimit field to metric collector spec (kubeflow#2597)

* remove global resources by label rather than listing all and looking for the label

* mv client to coordinator

* Fixed Dashboard v0.0.1 w/ Polymer3 + Webpack (kubeflow#2576)

* Multiple issues tackled:
- Fixed webpack for windows (components/ is not a valid path in windows)
- Spacing and semicolons matched to old-codebase
- Added cross-platform support to npm run targets
- Added missing dep (babel/polyfill)
- Dashboard build version is now corrected and hover will reveal kubeflow vs Dashboard versions
- Pug fixed to match syntax for empty-value attributes
- Updated webpack for readability

* Added babel-polyfill to runtime deps rather than dev deps

* [Windows-only] Color coded all make output, Tag check added, failures now have messages, also dirty git output now has a warning during build

* Updated new dashboard reference :v20190227-v0.4.0-rc.1-176-g738de035

* Fixed Multiple changes:
- Server port fixed to 8082 which is what is needed for GKE deployment
- Refactored some code

* Final working state for new image

* TS Changes

* Build updated to match latest :v20190301-v0.4.0-rc.1-183-g07eecd2f

* Migrated dev dependency to correct location

* Uncommented linter, fixed PR comments from @prodonjs

* updated jupyter images based on postsubmit results of kubeflow@97b94c0 (kubeflow#2611)

* Adding additional printer columns to display status (kubeflow#2609)

* Adding additional printer columns to display status

* Fixing tests

* Semi-working kfctl go binary (kubeflow#2587)

* Add flag --skip-init-gcp-project

* fix gcpInitProject

* update error msg

* add deployment creation logic

* fix bugs on generateDMData

* fix bugs

* add default config to ipName and hostName

* fix bugs on app.yaml

* add Fingerprint to update deployment

* make network and gcfs as DM optional

* add todos

* add iam policy handlers

* finish patch iam policy

* adding poll op to dm api calls

* add gke api version

* temp

* change mysqlPd/nfsPd names

* remove

* temp

* add config types

* add basic auth flag

* move config to gcp

* link use basic auth flag to generate

* use basic auth to determine config path

* change to gabrielwen repo for now

* change config on the fly

* change repo path

* create new Genereate method

* add required packages

* implement NewGenerate

* clean up default config and add on-the-fly configs

* move config handling to ksonnet.go

* add ks param set

* add application as component

* fix typo

* add simple TargetConfiguration factory

* add fixed backof

* temp

* fix from merge

* make namespace in gke

* bind user as default-admin in gke

* finish createSecrets

* write oauth secret

* uncomment ks apply

* add ks show

* add ks apply logic to k8sutils

* uncomment deployment

* build rest.Config from remote

* impersonate as admin for rbac

* use remote config as REST client config

* update default.yaml path

* complete apply logic

* add retry

* add ignore

* merge

* remove temp comment

* use backoff in gcp

* remove progress as it is not updating

* use backoff

* use backoff

*  use default zone

* fix zone config

* add doc

* add doc

* update Makefile

* fix flag doc

* fix

* doc

* doc

* add doc

* sort packages and component names

* shell out to gcloud and kubectl for now

* remove some configs

* fix

* remove GkeApiVersion

* add todo

* add notebook-controller back

* change names

* updates

* add field InitRequired

* remove KsInitParams

* migrate

* Add status to notebook (kubeflow#2558)

* wip

* wip

* update test to check status condition

* fix

* Add prodonjs to OWNERS (kubeflow#2613)

* Changed sidebar tab from JupyterHub to Notebooks (kubeflow#2619)

* fix (kubeflow#2620)

* pf gcb (kubeflow#2603)

* Setting the base_url flag of Jupyter images (kubeflow#2627)

* set base_url of jupyter

* change the env base_url to NB_PREFIX to sync with kubeflow#2620

* fix (kubeflow#2631)

* resolve merge conflicts from upstream/master, fix error on master when --platform=none or not specified

* store non exist parameter differently (kubeflow#2625)

* fix (kubeflow#2632)

* update pipeline system images to 0.1.12 release (kubeflow#2637)

* updated jupyter image paths to point to the latest ones (kubeflow#2638)

* add retry around ks init (kubeflow#2643)

* add retry around ks init

* update comments

* merge errors

* Getting started card completed (kubeflow#2651)

* Getting Started card content added, and un-hidden. Windows support added in webpack to run dev builds

* Getting Started card fully implemented from mocks

* Spaced out cards a little more

* Got eslint working! And fixed eslint issues in webpack
…low#2626)

* Separate front-end components and add mock activities service.

- Adds deep-link routing behavior for all front-end pages.
  Iframed pages reflect a URL to the browser that makes them
  linkable but hides the internal page.
- Seperates the front-end into subcomponents, including an
  activities view.
- Fix iframe view behavior to have it cover the space voided
  by the layout header.
- Adds /api/activities endpoint to the Express server for
  activities to eventually be replaced with real data.
- Sets the _devMode flag to true based on the webpack build env.

* Address PR comments

* Merge changes from PR kubeflow#2651
@googlebot
Copy link

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

@googlebot googlebot added cla: yes and removed cla: no labels Mar 8, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers: ironpan, kunmingg

If they are not already assigned, you can assign the PR to them by writing /assign @ironpan @kunmingg in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kkasravi
Copy link
Contributor Author

kkasravi commented Mar 8, 2019

@jlewi high number of files is because I had to rewrite author on an early commit I did so it needs to do it from one commit before to HEAD. I think that caused git to think merge from master files are part of this commit.

@jlewi
Copy link
Contributor

jlewi commented Mar 9, 2019

What if you rebase on master and squash all the commit? Does that get rid of unnecessary changes?

Copy link
Contributor

@jlewi jlewi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 102 files at r2, 2 of 79 files at r3, 1 of 14 files at r6.
Reviewable status: 3 of 51 files reviewed, 3 unresolved discussions (waiting on @ashahba, @ellis-bigelow, and @kkasravi)


bootstrap/Makefile, line 30 at r6 (raw file):

%.so:
	cd cmd/plugins/$* && \

Should we start thinking about adding a presubmit test?


bootstrap/Makefile, line 113 at r6 (raw file):

	echo created $(IMG):latest

install: build-kfctl dockerfordesktop.so kustomize.so ksonnet.so

Are we not building a single statically linked go binary?


bootstrap/cmd/bootstrap/app/gcpUtils_test.go, line 24 at r4 (raw file):

)

func TestGetUpdatedPolicy(t *testing.T) {

Did you actually modify this file?

If this is a spurious change maybe try rebasing on master?

@kkasravi
Copy link
Contributor Author

This has too many spurious changes. See #2673

@kkasravi kkasravi closed this Mar 10, 2019
@kkasravi kkasravi deleted the kfctl_kustomize branch April 3, 2019 09:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants