third packages smoke tests on running minoca instance

[melezhik]

Hi guys! This is in "a spirit of" - discussion started at https://groups.google.com/forum/#!topic/minoca-dev/vNyrQ3q0F7w

https://github.com/melezhik/minoca-pkg-test

Let me know if sound interesting, so I can go further with other packages.

[evangreen]

Hi Alexey,
Yes, this is pretty cool. I suppose this would be different than the more detailed test infrastructure that runs each package's own tests?

Suppose I don't have anything installed except opkg and git on a fresh Minoca instance. If I wanted to clone this repository and try it out, is there a .sh script I can run that would install all the dependencies I'd need to try this out? That could be helpful for me so I don't have to dig around and figure out how to install all the needed tools.

[melezhik]

I suppose this would be different than the more detailed test infrastructure that runs each package's own tests?

Sure. This is opposite to third packages tests , this is kind of integration/smoke tests, when packages already get built and we need to ensure that they "are good" ( installable and runnable ) for end minoca user.

Suppose I don't have anything installed except opkg and git on a fresh Minoca instance. If I wanted to clone this repository and try it out, is there a .sh script I can run that would install all the dependencies I'd need to try this out? That could be helpful for me so I don't have to dig around and figure out how to install all the needed tools.

Sure. Bootstrap is not that hard. I could create a http endpoint so you can do this:

wget $url | sh

which result in installing sparrow and this plugin ready to run, if this that you want?

[evangreen]

Yes, that would make it easier to try it out. Maybe add a reference to that somewhere in the repository too so people can figure out how to get up and running quickly.

[melezhik]

Hi Evan! This should work:

wget -O - http://sparrowhub.org/minoca.sh | sh
sparrow plg install monoca-pkg-test
sparrow plg run minoca-pkg-test

I have tested this on freshly built minoca instance ( os + gcc_4.9.2 + libgcc_4.9.2+ libgmp_4.3.2 + libmpc_0.8.1 + libmpfr_2.4.2 + libpcre_8.39 + opkg_0.2.4_minoca-i686.ipk + wget_1.15 + libncurses_5.9 )

[melezhik]

BTW - minoca third packages install report ( all the list ) - https://gist.github.com/melezhik/67a6425b2d1f46a5c2c30571727feb0c , interesting ...

[melezhik]

seems like 9 packages failed to install/run smoke tests

$ grep 'NOT OK' /tmp/minoca-pkg-test.report.txt | wc -l
9

[evangreen]

These are great! Thanks for setting this up. Is that script you pasted above in the repository somewhere so people can find it?

Looks like some of the errors are test failures, for example trying to opkg install ca -certificates instead of ca-certificates. But other failures look like bugs. I need to understand what that charset.alias file is and why it seems to conflict in so many packages. We should probably file separate bugs for each of the real failures, maybe batching similar failures into a single bug for the ones that will probably get fixed all at once.

[melezhik]

Is that script you pasted above in the repository somewhere so people can find it?

This script is a sparrow plugin, you may found a useful knowledge about sparrow plugin in general at sparrow docs , a source code of the plugin is here - https://github.com/melezhik/minoca-pkg-test . The plugin gets installed and run via sparrow client. A helpful analogy for sparrow plugins are rpm/deb packages or rubygem/CPAN/pip modules. The difference is that sparrow plugins are more packaged scripts ( optionally with some dependencies ) then just a libraries. A http://sparrowhub.org - a repository of sparrow plugins. Plugins are versioned. Once new version of a plugin is uploaded to sparrowhub the customers could upgrade and run new version on target hosts. Plugins are written by many authors. Currently is me only, by contribution is welcome.

opkg install ca -certificates instead of ca-certificates

Looks like a plugin bug, will take a look.

I need to understand what that charset.alias file

yeah, this is strange, agree ..

We should probably file separate bugs for each of the real failures, maybe batching similar failures into a single bug for the ones that will probably get fixed all at once.

Sure. This issue is about plugin, packages bugs could be described at separated issues. Agree. The report link I added here is just FYI.

[melezhik]

opkg install ca -certificates instead of ca-certificates

found the culprit - https://github.com/melezhik/minoca-pkg-test/blob/master/suite.ini#L14

[melezhik]

BTW, this is how I upgrade sparrow plugin:

[melezhik]

"opkg install ca -certificates instead of ca-certificates" bug should be gone at version 0.0.11 of plugin minoca-pkg-test

[melezhik]

Hi Evan, have you managed to play with minoca-pkg-test ? Do you have any other questions?

[melezhik]

New test report is here - https://gist.github.com/melezhik/032910cf6901a88b6ee7a519b65aaa0c

grep NOT   /tmp/minoca-pkg-test.17.11.2016.report.txt | grep OK | wc -l
7

[melezhik]

btw I just released a new version of sparrow to allow run tests in colorless mode;

export SPARROW_NO_COLOR=1 && sparrow plg run ...

[evangreen]

I was able to get this installed and running myself. Neat! Took a look at your recent paste as well. Here are my thoughts:

• dropbear - This one's tricky because dropbear is a substitute for openSSH. So whichever one is installed first will win, and the other will fail to install. This is in essence a test infrastructure problem, but I'm not sure how to suggest fixing it. Thoughts?
• gnupg - This is a typo bug in package.sh. It's claiming to depend on libcgcrypt, it means libgcrypt.
• libgpg-error - Another test dash typo. Should be libgpg-error, not libgpg -error.

All the other bugs are this colliding charset.alias file. I still haven't researched what that file is, but it's definitely a bug in the packaging that so many packages are trying to install it.

Anyway I like this test. I suppose it's the same set of questions as the other thread we have on Sparrow. I guess with this one you must install Sparrow on the test machine. So far our test infrastructure doesn't have to reach out over the Internet to perform it's nightly duties. I'm nervous that the bandwidth of installing Sparrow on all the machines each night will be a lot. What do you think?

[melezhik]

I was able to get this installed and running myself. Neat! Took a look at your recent paste as well

Great.

dropbear - This one's tricky because dropbear is a substitute for openSSH. ...

The same thoughts for me. We could think about 2 cases here:

• remove openssh if exists, install and test dropbear
• remove dropbear if exists, install and test dropbear

libgpg-error - Another test dash typo. Should be libgpg-error, not libgpg -error.

will take a look

I guess with this one you must install Sparrow on the test machine.

Yes

I'm nervous that the bandwidth of installing Sparrow on all the machines each night will be a lot. What do you think?

Yeah, I understand you, see my estimations on another thread.

[melezhik]

So far our test infrastructure doesn't have to reach out over the Internet to perform it's nightly duties.

Most of dependencies ( gcc, make, curl, perl, bash ) could be installed as third party packages via opkg - could they be installed from local sources ( not Internet ) ? ( already got the answer from other git issue - "The third-party packages we build stay local on the Intranet, so opkg installations are free.")

There are 2 left - Sparrow itself + CPAN modules ( about 10 items ) - their sources are really small and could be placed locally as well. Does it help?

[melezhik]

Finally I do not see any network impact here , around 10 CPAN modules is not a big deal, Sparrow you know is relatively small on dependencies ... But even though you need a pure local install, we could place a source code of related CPAN modules to your local environment and I would rewrite http://sparrowhub.org/minoca.sh so that it fetch things locally not Internet. Does it help?