Skip to content
This repository has been archived by the owner on Nov 9, 2020. It is now read-only.

Build and test managed plugin support in CI fixes #1020 #1061

Conversation

shuklanirdesh82
Copy link
Contributor

@shuklanirdesh82 shuklanirdesh82 commented Mar 22, 2017

Build and test managed plugin support in CI
fixes #1020

Testing: Yes, locally and CI too.

CI output:

  1. copying required files
=> Cleaning up older files from root@192.168.31.85 if exists... Wed Mar 22 23:08:39 UTC 2017

=> Copying required files to root@192.168.31.85 ... Wed Mar 22 23:08:39 UTC 2017
  1. make sure make is there, install otherwise
=> installing make on ubuntu machine Wed Mar 22 23:08:40 UTC 2017

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
 linux-headers-4.4.0-31 linux-headers-4.4.0-31-generic linux-headers-4.4.0-59
 linux-headers-4.4.0-59-generic linux-headers-4.4.0-62
 linux-headers-4.4.0-62-generic linux-headers-4.4.0-63
 linux-headers-4.4.0-63-generic linux-headers-4.4.0-64
 linux-headers-4.4.0-64-generic linux-image-4.4.0-31-generic
 linux-image-4.4.0-59-generic linux-image-4.4.0-62-generic
 linux-image-4.4.0-63-generic linux-image-4.4.0-64-generic
 linux-image-extra-4.4.0-31-generic linux-image-extra-4.4.0-59-generic
 linux-image-extra-4.4.0-62-generic linux-image-extra-4.4.0-63-generic
 linux-image-extra-4.4.0-64-generic
Use 'apt autoremove' to remove them.
Suggested packages:
 make-doc
The following NEW packages will be installed:
 make
0 upgraded, 1 newly installed, 0 to remove and 26 not upgraded.
Need to get 151 kB of archives.
After this operation, 365 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 make amd64 4.1-6 [151 kB]
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
dpkg-preconfigure: unable to re-open stdin: 
Fetched 151 kB in 0s (816 kB/s)
Selecting previously unselected package make.\r
(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 285888 files and directories currently installed.)\r
Preparing to unpack .../archives/make_4.1-6_amd64.deb ...\r
Unpacking make (4.1-6) ...\r
Processing triggers for man-db (2.7.5-1) ...\r
Setting up make (4.1-6) ...\r
  1. create/install the plugin
Using the following config:
DOCKER_HUB_REPO root EXTRA_TAG -dev VERSION_TAG 0.1
PLUGIN_NAME root/docker-volume-vsphere:0.1-dev
=== Cleaning work files, images and plugin(s)...
rm -rf ./rootfs
rm -f docker-volume-vsphere
docker plugin rm root/docker-volume-vsphere:0.1-dev -f
Error response from daemon: plugin "root/docker-volume-vsphere:0.1-dev" not found
Makefile:87: recipe for target 'clean' failed
docker rmi root/docker-volume-vsphere:rootfs
make: [clean] Error 1 (ignored)
Error response from daemon: No such image: root/docker-volume-vsphere:rootfs
Makefile:87: recipe for target 'clean' failed
make: [clean] Error 1 (ignored)
== building Docker image, unpacking to ./rootfs and creating plugin...
cp ../build/docker-volume-vsphere .
docker build -q -t root/docker-volume-vsphere:rootfs .
sha256:c950dfeb6e55a67f4207387d3ee8fbff0dbb8254380a403c01fa2742bf26e522
mkdir -p ./rootfs
docker create --name tempContainer root/docker-volume-vsphere:rootfs
e4626599c5db214044ecb65c985ff633750ef23a5bdba8437f119080955ea33c
docker export  tempContainer  | tar -x -C ./rootfs
docker rm -vf  tempContainer
tempContainer
docker rmi root/docker-volume-vsphere:rootfs
Untagged: root/docker-volume-vsphere:rootfs
Deleted: sha256:c950dfeb6e55a67f4207387d3ee8fbff0dbb8254380a403c01fa2742bf26e522
Deleted: sha256:39b893bd9d4a825c9eede502a2e08d99286bfa5524acc157cf357790a195879b
Deleted: sha256:55cad28fdf7678e38e64a7e0c2b7ca7a2379ac70eed8a216770113a177ccadfc
Deleted: sha256:47a013491bc202d6cc7b2d4837cf2f2fc1fa7fb30cfb9d9f32a3f97f7d5875df
Deleted: sha256:46655bfb4b407eee65c2aa5f3d92a83cc94f41c365f65e8561c61b5ed3178470
Deleted: sha256:46f21bbdb7196533cf63fbcc0201b397fed294fec203e55033b67cb304d307d1
Deleted: sha256:6e1298e1c73439e352408ef5524aa7318e744f32ae3ecb119245ba7cd49ca908
-- Creating  plugin root/docker-volume-vsphere:0.1-dev ...
docker plugin create root/docker-volume-vsphere:0.1-dev .
root/docker-volume-vsphere:0.1-dev
  1. sanity check
plugin_sanity_test: [INFO] Running plugin_sanity_test on the clean test setup...
plugin_sanity_test: [INFO] Installed plugin name is:root/docker-volume-vsphere:0.1-dev
plugin_sanity_test: [INFO] Enable plugin and verifies it is enabled or not...
root/docker-volume-vsphere:0.1-dev
plugin_sanity_test: [INFO] try to create volume again and expects volume creation this time as plugin is enabled...
volume_20170322230854
plugin_sanity_test: [INFO] run busybox and write something to this volume
plugin_sanity_test: [INFO] run busybox and check the written stuff
plugin_sanity_test: [INFO] fileExistCount=1
plugin_sanity_test: [INFO] check the volume indeed belongs to the plugin (Not local)...
plugin_sanity_test: [INFO] rm the volume...
volume_20170322230854
plugin_sanity_test: [INFO] disable the plugin...
  1. clean up the plugin installation
root/docker-volume-vsphere:0.1-dev
fatal: Not a git repository (or any of the parent directories): .git
fatal: Not a git repository (or any of the parent directories): .git
=== Cleaning work files, images and plugin(s)...
rm -rf ./rootfs
rm -f docker-volume-vsphere
docker plugin rm root/docker-volume-vsphere:0.1-dev -f
root/docker-volume-vsphere:0.1-dev
docker rmi root/docker-volume-vsphere:rootfs
Error response from daemon: No such image: root/docker-volume-vsphere:rootfs
make: [clean] Error 1 (ignored)
Makefile:87: recipe for target 'clean' failed

/CC @msterin

@shuklanirdesh82 shuklanirdesh82 added this to the 0.13 milestone Mar 22, 2017
@shuklanirdesh82 shuklanirdesh82 self-assigned this Mar 22, 2017
@shuklanirdesh82 shuklanirdesh82 force-pushed the mgdPluginSuppportCI.shuklanirdesh82 branch 3 times, most recently from 17533bc to 6a5af75 Compare March 22, 2017 06:12
@shuklanirdesh82
Copy link
Contributor Author

CI run: https://ci.vmware.run/vmware/docker-volume-vsphere/1759 is showing the test case steps as per planned at #1020

@shuklanirdesh82
Copy link
Contributor Author

@msterin just FYI: CI failure is expected and failing with the known issue not related with this PR. The code is ready for the review.

Copy link
Contributor

@msterin msterin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good change.
My main concern is that the actual test script covers lots of things which seem redundant (like the first 2 checks_ or seem to be Docker test (i.e disable/create volume). I think these parts make the code less maintainable and harder to read, without adding much value. I'd suggest cutting the tests (see comments).
A few more comments/nits, but they are are not blockers

do
TARGET=root@$ip
log "Cleaning up older files from $TARGET if exists..."
$SSH $TARGET rm -fr $TMP_LOC
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a big deal, but it's better to run 1 ssh , not 3.
Something like `ssh $target "rm -fr $TMP_LOC; $MKDIR_P $TMP_LOC $BUILD_LOC; "

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I prefer to do that too. I have seen other functions are doing separately hence this proposal.

Thanks for the comment!

if [ $? -ne 0 ]; then
case $FILE_EXT in
deb)
log "installing make on ubuntu machine"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's easier to install make (if needed) upfront. Something like 'if [ -z "$(which make)" ] l; then ; fi'
instead of trying, installing and trying again

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!


# make sure sanity check is on clean slate and row count should be 2
row_count=`docker plugin ls | wc -l`
if [ $row_count != 2 ]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it'snot clean slate if it's 2 :-)
I am not sure why do we need to check it. the next checl seems to be covering it also

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah! I started with $row_count != 2 and later added -z $pluginName .. I will remove the redundant one (this one).

echo "plugin_sanity_test: [INFO] Installed plugin name is:" $pluginName

# make sure plugin name is not empty
if [ -z $pluginName ]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you may wrap to take $pluginMName in ""

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done!


# check plugin is disabled or not .. by default it should be disabled
echo "plugin_sanity_test: [INFO] check plugin is disabled or not .. by default it should be disabled..."
enabled=`docker plugin inspect $pluginName -f {{.Enabled}}`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is good. Why do we need the first 2 checks though ? They don't seem to add any value. just take space/attention

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done!

# create volume when plugin is disabled state and expects an error
echo "plugin_sanity_test: [INFO] create volume when plugin is disabled state and expects an error..."
docker volume create -d $pluginName --name $volumeName
if [ $? -eq 0 ]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems to be testing Docker internals. I do not think we need that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed from sanity check.


# Enable plugin and verifies it is enabled or not
echo "plugin_sanity_test: [INFO] Enable plugin and verifies it is enabled or not..."
docker plugin enable $pluginName
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here. I understand the test. but it seems to be more for docker functionality. I think we need to minimize cross-testing docker when it is not expected to impact our code

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed redundant part.

# try to create volume again and expects volume creation this time as plugin is enabled
echo "plugin_sanity_test: [INFO] try to create volume again and expects volume creation this time as plugin is enabled..."
docker volume create -d $pluginName --name $volumeName
if [ $? -ne 0 ]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes ! that's he test :-) !

# run busybox and write something to this volume
echo "plugin_sanity_test: [INFO] run busybox and write something to this volume"
docker run --rm -v $volumeName:/vol1 --name=$containerName busybox touch /vol1/$fileName
if [ $? -ne 0 ]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

those 3 lines seem to be everywhere.... I'd consider a functoin for that (this is a nit)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree! Those are kept for customized error message. Please let me know what do you think.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that;s fine

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @msterin !

@@ -56,6 +57,7 @@ AFTER_REMOVE := $(SCRIPTS)/install/after-remove.sh
#
CHECK := $(SCRIPTS)/check.sh
BUILD := $(SCRIPTS)/build.sh
BUILD_TOOLS := $(SCRIPTS)/deploy-tools.sh
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the script is calleed deploy_tools. Why the Makefile vars is called build_tools ? it's confusing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my bad! thanks for catching.

@shuklanirdesh82 shuklanirdesh82 force-pushed the mgdPluginSuppportCI.shuklanirdesh82 branch from 001943d to c526904 Compare March 22, 2017 19:16
Copy link
Contributor

@msterin msterin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice !

@shuklanirdesh82 shuklanirdesh82 merged commit 871ec8a into vmware-archive:master Mar 23, 2017
@shuklanirdesh82 shuklanirdesh82 deleted the mgdPluginSuppportCI.shuklanirdesh82 branch March 23, 2017 05:38
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Build and test managed plugin support in CI
3 participants