Add "index" command to Atomic App #750

cdrage · 2016-06-28T15:29:26Z

This adds an index command to list multiple Nulecule's from an external
library that contains an index.yaml file in it's root directory.

Built upon @vpavlin 's work :)

TODO:

atomicapp index list --update command
"official" index docker image (will need to add this to projectatomic/nulecule-library)
script to generate index.yaml
integration tests
docs going to update via Update CLI docs #768

cdrage · 2016-06-28T15:36:16Z

Here's what it looks like right now if you pull the PR and run atomicapp index list:

▶ atomicapp index list
INFO   :: Atomic App: 0.6.0 - Mode: Index
ID                        VER      PROVIDERS  LOCATION                                             
postgresql-atomicapp      1.0.0    {D,O,K}    docker.io/projectatomic/postgresql-centos7-atomicapp 
flask_redis_nulecule      0.0.1    {D,K}      docker.io/projectatomic/flask-redis-centos7-atomicapp
redis-atomicapp           0.0.1    {D,O,K}    docker.io/projectatomic/redis-centos7-atomicapp      
gocounter                 0.0.1    {D,K}      docker.io/projectatomic/gocounter-scratch-atomicapp  
mariadb-atomicapp         1.0.0    {D,O,K}    docker.io/projectatomic/mariadb-centos7-atomicapp    
helloapache-app           0.0.1    {D,K,M}    docker.io/projectatomic/helloapache                  
mongodb-atomicapp         1.0.0    {D,O,K}    docker.io/projectatomic/mongodb-centos7-atomicapp    
etherpad-app              0.0.1    {D,O,K}    docker.io/projectatomic/etherpad-centos7-atomicapp   
apache-centos7-atomicapp  0.0.1    {D,K,M}    docker.io/projectatomic/apache-centos7-atomicapp     
wordpress-atomicapp       2.0.0    {D,O,K}    docker.io/projectatomic/wordpress-centos7-atomicapp  
skydns-atomicapp          0.0.1    {K}        docker.io/projectatomic/skydns-atomicapp             
guestbookgo-atomicapp     0.0.1    {O,K}      docker.io/projectatomic/guestbookgo-atomicapp        
mariadb-app               0.0.1    {D,K}      docker.io/projectatomic/mariadb-fedora-atomicapp     
gitlab-atomicapp          1.2.0    {D,K}      docker.io/projectatomic/gitlab-centos7-atomicapp

Updating:

dropbox/dev/atomicapp  add-index ✔                                                                                                                                                                                                                       0m  ⍉
▶ atomicapp index list --update
INFO   :: Atomic App: 0.6.0 - Mode: Index
INFO   :: Updating the index list
INFO   :: Pulling latest index image...
INFO   :: Skipping pulling docker image: cdrage/nulecule-library
INFO   :: Extracting Nulecule data from image cdrage/nulecule-library to /home/wikus/.atomicapp/index.yaml
INFO   :: Index updated

cdrage · 2016-07-04T15:27:57Z

projectatomic/nulecule-library#96 location metadata has been added for updating the information (including docker.io urls)

cdrage · 2016-07-06T15:31:44Z

Hey guys.

Pretty much done at this point in terms of Atomic App code.

Here are the updates

Not going to include atomicapp index info for now until I research more on how the output format should look like, also whether or not we should include atomicapp index edit
Started work on tests, have two more to go
Need to add docs not only on how to use the command, but also that there will now be an .atomicapp/index on your system
Haven't tested with atomic cli functionality yet

In order to try out this PR, you must:

# with https://github.com/projectatomic/nulecule-library/pull/97 PR patched
cd nulecule-library
docker build -t projectatomic/nulecule-library .
# then
atomicapp index list # should auto-update (put the index.yaml into your .atomicapp/index.yaml

Hope you guys like it :)
@kadel @rtnpro @dustymabe @vpavlin

cdrage · 2016-07-06T15:32:50Z

What I'm waiting for/block with is:

Add a Dockerfile with the nulecule-library index nulecule-library#97 to be merged
hub.docker.com to be updated with projectatomic/nulecule-library docker image for people to pull
code reviews

dustymabe · 2016-07-07T21:20:35Z

atomicapp/index.py

+        # eliminating the INFO :: Atomic App: X.X.X - Mode: Index
+        # logging information. If we passed --debug, ignore.
+        # if not logging.getLogger().getEffectiveLevel() == logging.DEBUG:
+        #     sys.stdout.write("\033[F")


coveralls · 2016-07-14T14:08:56Z

Coverage increased (+1.5%) to 58.497% when pulling 2b882e1 on cdrage:add-index into d853213 on projectatomic:master.

cdrage · 2016-07-14T14:16:51Z

https://hub.docker.com/r/projectatomic/nulecule-library/ has been created with the index.yaml

This has been update and ready for code review + possible merging.

@dustymabe @rtnpro @containscafeine @kadel if ya'll want to try it out / review it :)

cdrage · 2016-07-14T14:17:56Z

I'll be updating the docs in a separate PR since we need to update https://github.com/projectatomic/atomicapp/blob/master/docs/cli.md anyways.
Opened up issue #768 for this.

cdrage · 2016-07-14T14:17:59Z

#dotests

coveralls · 2016-07-14T14:19:07Z

Coverage increased (+1.5%) to 58.497% when pulling e1e5919 on cdrage:add-index into d853213 on projectatomic:master.

dustymabe · 2016-07-14T19:44:42Z

atomicapp/cli/main.py

+
+        index_list = index_action.add_parser("list")
+        index_list.add_argument(
+            "--update",


so what do you think about having this as a separate subcommand rather than an option to --list? seems like atomicapp index update would be more natural than atomicapp index list --update.

How about atomicapp index pull ? (It's a docker pull anyway:) )

That'd be good, what do you think @dustymabe ?

ehh i like "update" but don't really care.. potato potato - go with whatever

coveralls · 2016-07-15T14:56:51Z

Coverage increased (+1.5%) to 58.493% when pulling 61ebdc8 on cdrage:add-index into d853213 on projectatomic:master.

cdrage · 2016-07-15T15:14:05Z

@dustymabe

updated based on your comments. added the atomicapp index update command (i liked your suggestion better). removed the random utils.py stuff I still had there as well.

coveralls · 2016-07-15T15:17:56Z

Coverage increased (+1.5%) to 58.47% when pulling 9f50616 on cdrage:add-index into d853213 on projectatomic:master.

cdrage · 2016-07-15T15:19:40Z

#dotests

dustymabe · 2016-07-15T15:31:30Z

atomicapp/cli/main.py

+        index_generate.add_argument(
+            "location",
+            help=(
+                "Path or Git repository link containing Nulecule applications "


'or Git repository' - is that still true?

coveralls · 2016-07-15T16:03:39Z

Coverage increased (+1.5%) to 58.47% when pulling 2cba0b0 on cdrage:add-index into d853213 on projectatomic:master.

cdrage · 2016-07-20T15:24:53Z

Hey @dustymabe

So the index_location has been updated in the init of the index.py file to the proper location via:

self.index_location = os.path.join(Utils.getUserHome(), INDEX_LOCATION)

Utils.setFileOwnerGroup(dest)

has also been added to the extract_files function so it properly sets the permission when pulling.

One last review? :)

cdrage · 2016-07-20T15:27:12Z

dropbox/dev/atomicapp  add-index ✔                                                                                                                                                                                                                        1m  
▶ atomicapp index list
INFO   :: Atomic App: 0.6.1 - Mode: Index
ID                        VER      PROVIDERS  LOCATION                                             
postgresql-atomicapp      1.0.0    {D,O,K}    docker.io/projectatomic/postgresql-centos7-atomicapp 
flask_redis_nulecule      0.0.1    {D,K}      docker.io/projectatomic/flask-redis-centos7-atomicapp
redis-atomicapp           0.0.1    {D,O,K}    docker.io/projectatomic/redis-centos7-atomicapp      
gocounter                 0.0.1    {D,K}      docker.io/projectatomic/gocounter-scratch-atomicapp  
mariadb-atomicapp         1.0.0    {D,O,K}    docker.io/projectatomic/mariadb-centos7-atomicapp    
helloapache-app           0.0.1    {D,K,M}    docker.io/projectatomic/helloapache                  
mongodb-atomicapp         1.0.0    {D,O,K}    docker.io/projectatomic/mongodb-centos7-atomicapp    
etherpad-app              0.0.1    {D,O,K}    docker.io/projectatomic/etherpad-centos7-atomicapp   
apache-centos7-atomicapp  0.0.1    {D,K,M}    docker.io/projectatomic/apache-centos7-atomicapp     
wordpress-atomicapp       2.0.0    {D,O,K}    docker.io/projectatomic/wordpress-centos7-atomicapp  
skydns-atomicapp          0.0.1    {K}        docker.io/projectatomic/skydns-atomicapp             
guestbookgo-atomicapp     0.0.1    {O,K}      docker.io/projectatomic/guestbookgo-atomicapp        
mariadb-app               0.0.1    {D,K}      docker.io/projectatomic/mariadb-fedora-atomicapp     
gitlab-atomicapp          1.2.0    {D,K}      docker.io/projectatomic/gitlab-centos7-atomicapp     

dropbox/dev/atomicapp  add-index ✔                                                                                                                                                                                                                        1m  
▶ atomicapp index update
INFO   :: Atomic App: 0.6.1 - Mode: Index
INFO   :: Updating the index list
INFO   :: Pulling latest index image...
INFO   :: Skipping pulling docker image: projectatomic/nulecule-library
INFO   :: Copying files from image projectatomic/nulecule-library:/index.yaml to /home/wikus/.atomicapp/index.yaml
INFO   :: Index updated

And the permissions:

▶ tree ~/.atomicapp -ug 
/home/wikus/.atomicapp
└── [wikus    wikus   ]  index.yaml

0 directories, 1 file

coveralls · 2016-07-20T15:27:33Z

Coverage increased (+1.2%) to 58.121% when pulling e51eed5 on cdrage:add-index into 52f96ea on projectatomic:master.

This adds an index command to list multiple Nulecule's from an external library that contains an index.yaml file in it's root directory.

coveralls · 2016-07-20T16:03:18Z

Coverage increased (+1.2%) to 58.121% when pulling 334cafe on cdrage:add-index into 52f96ea on projectatomic:master.

dustymabe · 2016-07-20T17:43:05Z

LGTM

cdrage · 2016-07-20T20:23:18Z

#dotests

cdrage changed the title ~~Add "index" command to Atomic App~~ [WIP] Add "index" command to Atomic App Jun 28, 2016

cdrage force-pushed the add-index branch from 2af4a1a to c404ca3 Compare June 28, 2016 15:33

cdrage force-pushed the add-index branch 5 times, most recently from e984e61 to 38393aa Compare July 4, 2016 15:05

cdrage force-pushed the add-index branch 4 times, most recently from e506577 to b18a177 Compare July 6, 2016 15:27

cdrage changed the title ~~[WIP] Add "index" command to Atomic App~~ Add "index" command to Atomic App Jul 6, 2016

cdrage added the need_review label Jul 6, 2016

cdrage force-pushed the add-index branch 2 times, most recently from 4556605 to 9b3ea0c Compare July 7, 2016 12:09

cdrage mentioned this pull request Jul 7, 2016

[WIP] Add "index" subcommand #379

Closed

cdrage force-pushed the add-index branch from 9b3ea0c to d69246d Compare July 7, 2016 16:20

dustymabe reviewed Jul 7, 2016
View reviewed changes

cdrage force-pushed the add-index branch from 2b882e1 to e1e5919 Compare July 14, 2016 14:13

cdrage removed the work in progress label Jul 14, 2016

dustymabe reviewed Jul 14, 2016
View reviewed changes

cdrage force-pushed the add-index branch from e1e5919 to 61ebdc8 Compare July 15, 2016 14:53

cdrage force-pushed the add-index branch from 61ebdc8 to 9f50616 Compare July 15, 2016 15:13

dustymabe reviewed Jul 15, 2016
View reviewed changes

cdrage force-pushed the add-index branch from 9f50616 to 2cba0b0 Compare July 15, 2016 15:59

cdrage force-pushed the add-index branch from 2cba0b0 to e51eed5 Compare July 20, 2016 15:25

Add "index" command to Atomic App

334cafe

This adds an index command to list multiple Nulecule's from an external library that contains an index.yaml file in it's root directory.

cdrage force-pushed the add-index branch from e51eed5 to 334cafe Compare July 20, 2016 16:01

cdrage merged commit e241be5 into projectatomic:master Jul 20, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add "index" command to Atomic App #750

Add "index" command to Atomic App #750

cdrage commented Jun 28, 2016 •

edited

cdrage commented Jun 28, 2016 •

edited

cdrage commented Jul 4, 2016

cdrage commented Jul 6, 2016

cdrage commented Jul 6, 2016

dustymabe Jul 7, 2016

coveralls commented Jul 14, 2016

cdrage commented Jul 14, 2016 •

edited

cdrage commented Jul 14, 2016

cdrage commented Jul 14, 2016

coveralls commented Jul 14, 2016

dustymabe Jul 14, 2016

vpavlin Jul 14, 2016

cdrage Jul 15, 2016

dustymabe Jul 15, 2016

coveralls commented Jul 15, 2016

cdrage commented Jul 15, 2016

coveralls commented Jul 15, 2016

cdrage commented Jul 15, 2016

dustymabe Jul 15, 2016

coveralls commented Jul 15, 2016

cdrage commented Jul 20, 2016

cdrage commented Jul 20, 2016

coveralls commented Jul 20, 2016

coveralls commented Jul 20, 2016

dustymabe commented Jul 20, 2016

cdrage commented Jul 20, 2016

Add "index" command to Atomic App #750

Add "index" command to Atomic App #750

Conversation

cdrage commented Jun 28, 2016 • edited

cdrage commented Jun 28, 2016 • edited

cdrage commented Jul 4, 2016

cdrage commented Jul 6, 2016

cdrage commented Jul 6, 2016

dustymabe Jul 7, 2016

Choose a reason for hiding this comment

coveralls commented Jul 14, 2016

cdrage commented Jul 14, 2016 • edited

cdrage commented Jul 14, 2016

cdrage commented Jul 14, 2016

coveralls commented Jul 14, 2016

dustymabe Jul 14, 2016

Choose a reason for hiding this comment

vpavlin Jul 14, 2016

Choose a reason for hiding this comment

cdrage Jul 15, 2016

Choose a reason for hiding this comment

dustymabe Jul 15, 2016

Choose a reason for hiding this comment

coveralls commented Jul 15, 2016

cdrage commented Jul 15, 2016

coveralls commented Jul 15, 2016

cdrage commented Jul 15, 2016

dustymabe Jul 15, 2016

Choose a reason for hiding this comment

coveralls commented Jul 15, 2016

cdrage commented Jul 20, 2016

cdrage commented Jul 20, 2016

coveralls commented Jul 20, 2016

coveralls commented Jul 20, 2016

dustymabe commented Jul 20, 2016

cdrage commented Jul 20, 2016

cdrage commented Jun 28, 2016 •

edited

cdrage commented Jun 28, 2016 •

edited

cdrage commented Jul 14, 2016 •

edited