Add Proposal of Refactor prepare_script #22

ninjadq · 2018-11-22T10:32:16Z

propose to refactor prepare script

steven-zou · 2019-01-03T06:47:18Z

Please squash your commits and fix the DCO issue.

reasonerjt · 2019-01-04T06:23:43Z

proposals/new/propose_to_refactor_prepare_script.md

+
+
+4. Current config file is `harbor.cfg` which is configparser-style format. In order to describe more complex content, We need replace it using YAML format like `harbor.yml` . current config file may look like this.
+```yaml


Would it be possible to organize the yaml file in a more readable structure?
For example:

core: proxy: .... registry: ...... storage: ...... clair: ...... notary: .....

Additionally, shall we remove the "user settings" from the configuration file?

Agree with you, the final config file should like this. But the first step is compatible with the previous configParser format.

Same comments

reasonerjt · 2019-01-04T06:35:50Z

Two other goals we should consider to achieve in this refactor:

The default data volume /data should be configurable. Particularly, the secretkey should be moved away from the data volume. @ywk253100 knows more background.
The sudo should be removed from entrypoints of the containers in Harbor.

reasonerjt · 2019-01-04T06:37:36Z

We also need a solution for migration from old harbor.cfg to harbor.yaml
I think that's do-able following the way we did, refer to:
https://github.com/goharbor/harbor/tree/master/tools/migration/cfg

steven-zou

Please do changes.

steven-zou · 2019-01-04T08:48:15Z

proposals/new/propose_to_refactor_prepare_script.md

+      ├── Pipfile
+      ├── Pipfile.lock
+      ├── g.py
+      ├── main.py


Suggest adding comments for each file to describe what does the file do.

steven-zou · 2019-01-04T08:50:16Z

proposals/new/propose_to_refactor_prepare_script.md

+   1. ``` 
+      ├── Pipfile
+      ├── Pipfile.lock
+      ├── g.py


Suggest moving all the none global variables out of the g.py as it's used to keep global variables. Or rename it to v.py/values.py.

steven-zou · 2019-01-04T08:52:29Z

proposals/new/propose_to_refactor_prepare_script.md

+
+
+4. Current config file is `harbor.cfg` which is configparser-style format. In order to describe more complex content, We need replace it using YAML format like `harbor.yml` . current config file may look like this.
+```yaml


Same comments

steven-zou · 2019-01-04T09:16:23Z

proposals/new/propose_to_refactor_prepare_script.md

+| uaa_verify_cert         | boolean |
+
+
+5. Packaging all files and its denpendencies into a container.  All these codes and changes is related to python, so we should using official python images as the base image in Dockerfile. But there is a issue that some action need super user to operate. like change the owner and group of an file, create files on host system. I have no proper method to handle this. Maybe mount the directory on host specified in config file to container can solve last question.


Suggest adding an upgrading section to talk about how to upgrade config from old version to new version.

steven-zou · 2019-01-04T09:17:29Z

proposals/new/propose_to_refactor_prepare_script.md

+
+4. Current config file is `harbor.cfg` which is configparser-style format. In order to describe more complex content, We need replace it using YAML format like `harbor.yml` . current config file may look like this.
+```yaml
+## Configuration file of Harbor


Please append the property table copied from daojun's PR to let reviewers know what settings will be removed from harbor.yaml.

nlowe · 2019-01-10T00:39:37Z

I know the existing prepare script is written in python. Have we considered re-writing it in go instead, and using go templates? It drops the dependency on Python at runtime and brings the utility closer in-line with the majority of the codebase.

As far as I can tell, python is only used in some test scripts and the CLI tool which is deprecated.

ninjadq · 2019-01-11T02:59:14Z

I know the existing prepare script is written in python. Have we considered re-writing it in go instead, and using go templates? It drops the dependency on Python at runtime and brings the utility closer in-line with the majority of the codebase.

As far as I can tell, python is only used in some test scripts and the CLI tool which is deprecated.

I think the effort of rewriting it into go and go template is large then refactor in python because we can reuse a lot of current codes and these codes are verified by time. In addition, we wanna put all the dependencies of prepare into a container, so that user does not care about the python dependencies anymore. go is excellent but from my knowledge, python is good at this kind scripts.

ghost · 2019-02-01T16:43:13Z

I have some relatively broad thoughts on the topic of having to run a prepare script in the first place. I'm going to parse those thoughts and eventually submit a fleshed out proposal.

With that said, though, I do agree with @nlowe that having multiple components in disparate languages is something we should eventually aim to address. We should pick one lingua franca and stick with it as much as possible (within reason).

Signed-off-by: Qian Deng <dengq@vmware.com>

…are_script.md Signed-off-by: Qian Deng <dengq@vmware.com>

Signed-off-by: Qian Deng <dengq@vmware.com>

steven-zou · 2019-04-24T08:11:10Z

Per lazy consensus rule, merge proposal to main.

ninjadq force-pushed the ninjadq-patch-1 branch 8 times, most recently from eb9f477 to 1fa9b97 Compare January 4, 2019 06:12

ninjadq changed the title ~~Create propose_to_refactor_prepare_script~~ proposal to refactor prepare_script Jan 4, 2019

ninjadq changed the title ~~proposal to refactor prepare_script~~ Add Proposal of Refactor prepare_script Jan 4, 2019

reasonerjt reviewed Jan 4, 2019

View reviewed changes

steven-zou requested changes Jan 4, 2019

View reviewed changes

ninjadq force-pushed the ninjadq-patch-1 branch from 1fa9b97 to 335ca02 Compare January 7, 2019 05:59

steven-zou added kind/proposal area/installation labels Jan 9, 2019

ninjadq force-pushed the ninjadq-patch-1 branch 2 times, most recently from 7f83531 to 2820575 Compare February 13, 2019 07:14

ninjadq added a commit to ninjadq/harbor that referenced this pull request Feb 14, 2019

Refacotr the prepare script base on the proposal goharbor/community#22

a7abe5f

Signed-off-by: Qian Deng <dengq@vmware.com>

ninjadq added a commit to ninjadq/harbor that referenced this pull request Feb 14, 2019

Refacotr the prepare script base on the proposal goharbor/community#22

2ebcd1d

Signed-off-by: Qian Deng <dengq@vmware.com>

Rename propose_to_refactor_prepare_script to propose_to_refactor_prep…

81c4b1f

…are_script.md Signed-off-by: Qian Deng <dengq@vmware.com>

ninjadq force-pushed the ninjadq-patch-1 branch from 2820575 to 81c4b1f Compare February 18, 2019 02:43

ninjadq added a commit to ninjadq/harbor that referenced this pull request Feb 21, 2019

Refacotr the prepare script base on the proposal goharbor/community#22

ab3e835

Signed-off-by: Qian Deng <dengq@vmware.com>

ninjadq added a commit to ninjadq/harbor that referenced this pull request Feb 21, 2019

Refacotr the prepare script base on the proposal goharbor/community#22

50a5b28

Signed-off-by: Qian Deng <dengq@vmware.com>

ninjadq added a commit to ninjadq/harbor that referenced this pull request Feb 22, 2019

Refacotr the prepare script base on the proposal goharbor/community#22

2f34a35

Signed-off-by: Qian Deng <dengq@vmware.com>

ninjadq added a commit to ninjadq/harbor that referenced this pull request Feb 22, 2019

Refacotr the prepare script base on the proposal goharbor/community#22

8c7b077

Signed-off-by: Qian Deng <dengq@vmware.com>

ninjadq added a commit to ninjadq/harbor that referenced this pull request Feb 26, 2019

Refacotr the prepare script base on the proposal goharbor/community#22

842c6e4

Signed-off-by: Qian Deng <dengq@vmware.com>

ninjadq added a commit to ninjadq/harbor that referenced this pull request Feb 26, 2019

Refacotr the prepare script base on the proposal goharbor/community#22

f7b37f6

Signed-off-by: Qian Deng <dengq@vmware.com>

ninjadq added a commit to ninjadq/harbor that referenced this pull request Mar 11, 2019

Refacotr the prepare script base on the proposal goharbor/community#22

5f80fe7

Signed-off-by: Qian Deng <dengq@vmware.com>

Merge branch 'master' into ninjadq-patch-1

80fb90d

steven-zou merged commit bf8fc22 into master Apr 24, 2019

steven-zou deleted the ninjadq-patch-1 branch April 24, 2019 08:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Proposal of Refactor prepare_script #22

Add Proposal of Refactor prepare_script #22

ninjadq commented Nov 22, 2018

steven-zou commented Jan 3, 2019

reasonerjt Jan 4, 2019 •

edited

ninjadq Jan 4, 2019

steven-zou Jan 4, 2019

reasonerjt commented Jan 4, 2019

reasonerjt commented Jan 4, 2019

steven-zou left a comment

steven-zou Jan 4, 2019

steven-zou Jan 4, 2019

steven-zou Jan 4, 2019

steven-zou Jan 4, 2019

steven-zou Jan 4, 2019

nlowe commented Jan 10, 2019

ninjadq commented Jan 11, 2019

ghost commented Feb 1, 2019

steven-zou commented Apr 24, 2019



		4. Current config file is `harbor.cfg` which is configparser-style format. In order to describe more complex content, We need replace it using YAML format like `harbor.yml` . current config file may look like this.
		```yaml

		\| uaa_verify_cert \| boolean \|


		5. Packaging all files and its denpendencies into a container. All these codes and changes is related to python, so we should using official python images as the base image in Dockerfile. But there is a issue that some action need super user to operate. like change the owner and group of an file, create files on host system. I have no proper method to handle this. Maybe mount the directory on host specified in config file to container can solve last question.

Add Proposal of Refactor prepare_script #22

Add Proposal of Refactor prepare_script #22

Conversation

ninjadq commented Nov 22, 2018

steven-zou commented Jan 3, 2019

reasonerjt Jan 4, 2019 • edited

Choose a reason for hiding this comment

ninjadq Jan 4, 2019

Choose a reason for hiding this comment

steven-zou Jan 4, 2019

Choose a reason for hiding this comment

reasonerjt commented Jan 4, 2019

reasonerjt commented Jan 4, 2019

steven-zou left a comment

Choose a reason for hiding this comment

steven-zou Jan 4, 2019

Choose a reason for hiding this comment

steven-zou Jan 4, 2019

Choose a reason for hiding this comment

steven-zou Jan 4, 2019

Choose a reason for hiding this comment

steven-zou Jan 4, 2019

Choose a reason for hiding this comment

steven-zou Jan 4, 2019

Choose a reason for hiding this comment

nlowe commented Jan 10, 2019

ninjadq commented Jan 11, 2019

ghost commented Feb 1, 2019

steven-zou commented Apr 24, 2019

reasonerjt Jan 4, 2019 •

edited