Debian/Ubuntu packages missing mysql and postgres PMDA

[rlex]

Subj. There is no mysql and postgres "plugins" for pmda daemon.

Verified on 3.11.0 from bintray (jessie and trusty packages)

[kmcdonell]

Que?

$ dpkg -S /var/lib/pcp/pmdas/mysql
pcp: /var/lib/pcp/pmdas/mysql
$ dpkg -S /var/lib/pcp/pmdas/postgresql
pcp: /var/lib/pcp/pmdas/postgresql
$ src/pcp/qa/admin/whatami
bozo 3.11.1 x86_64 Ubuntu 15.10 (wily)

These PMDAs are included in the pcp package. For the Debian-based builds there are not a multiplicity of packages so all the PMDAs are in the pcp package (this is different to the RPM-based builds).

To activate (any) optional PMDA, you'll need to do the PMDA install procedure, which is typically something like ...

$ cd /var/lib/pcp/pmdas/mysql
$ sudo ./Install

Hope that helps.

[rlex]

/var/lib/pcp/pmdas:

activemq/  bonding/  dbping/  ds389log/       gluster/  jbd2/  linux/      lustre/      memcache/  mounts/     news/       nvidia/  perfevent/  proc/      rsyslog/  sendmail/  slurm/    trace/    unbound/  xfs/
apache/    cifs/     dm/      elasticsearch/  gpfs/     json/  lmsensors/  lustrecomm/  mic/       named/      nfsclient/  papi/    pipe/       roomtemp/  samba/    shping/    snmp/     trivial/  vmware/   zimbra/
bash/      cisco/    ds389/   gfs2/           gpsd/     kvm/   logger/     mailq/       mmv/       netfilter/  nginx/      pdns/    pmcd/       root/      sample/   simple/    summary/  txmon/    weblog/   zswap/

Full output: http://pastebin.com/5G7YgpVE
Versions:

lex@midgard ❯ dpkg --list | grep pcp
ii  libpcp-gui2                       3.11.0                               amd64        Performance Co-Pilot graphical client tools library
ii  libpcp-import-perl                3.11.0                               amd64        Performance Co-Pilot log import Perl module
ii  libpcp-import1                    3.11.0                               amd64        Performance Co-Pilot data import library
ii  libpcp-mmv1                       3.11.0                               amd64        Performance Co-Pilot Memory Mapped Value client library
ii  libpcp-pmda-perl                  3.11.0                               amd64        Performance Co-Pilot Domain Agent Perl module
ii  libpcp-pmda3                      3.11.0                               amd64        Performance Co-Pilot Domain Agent library
ii  libpcp-trace2                     3.11.0                               amd64        Performance Co-Pilot application tracing library
ii  libpcp3                           3.11.0                               amd64        Performance Co-Pilot library
ii  pcp                               3.11.0                               amd64        System level performance monitoring and performance management
ii  pcp-conf                          3.11.0                               amd64        Performance Co-Pilot runtime configuration
ii  pcp-doc                           3.11.0                               all          Documentation and tutorial for the Performance Co-Pilot
ii  pcp-export-zabbix-agent           3.11.0                               amd64        Module for exporting from PCP into a Zabbix agent daemon
ii  pcp-gui                           3.11.0                               amd64        Visualisation tools for the Performance Co-Pilot toolkit
ii  python-pcp                        3.11.0                               amd64        Performance Co-Pilot Python PMAPI module

[kmcdonell]

OK, clearly not in the package you installed, so where did your pcp packages come from?

[rlex]

From here: https://bintray.com/pcp (grabbed link from pcp.io page)

[kmcdonell]

OK, that's a problem with the software installation on the build machine used to create these packages ... I'll try and get it sorted.

Thanks for letting us know, and sorry for the inconvenience.

[natoscott]

| OK, that's a problem with the software installation on the build machine used to create these packages ... I'll try and get it sorted.

@kmcdonell I guess its some missing Build-Depends fields in debian/control, now that we have configure.ac checks for more perl/python packages?

[kmcdonell]

@natoscott I think the list of missing build dependencies has grown quite long ... comparing debian/control to the dpkg? lines in qa/admin/check-vm (and ignoring the obvious QA ones), I think the missing list looks like:
libclass-dbi-perl, libdbd-mysql-perl, libdbd-pg-perl, dpkg-dev, build-essential, dh-python, libcairo2-dev, libpapi-dev, libpfm4-dev, g++, libncurses5-dev, python-six, python-json-pointer, libextutils-autoinstall-perl, libxml-tokeparser-perl, librrds-perl, libjson-perl, libwww-perl, libnet-snmp-perl, qt4-qmake, libnss3-tools

Hmm ... may be it is time to figure out how to diff the pkgs I build and test compared to those uploaded to bintray, and I guess those that Debian end up building each time we push a new version.

[natoscott]

@kmcdonell thanks for that list - I've added it to debian/control and checked a build, looks good.

Yep, I would like for us to work on improving the bintray release builds and QA integration. I think what happened here was that even though we're both using qa/admin/check-vm, there's a disconnect between initial VM setup (for bintray builds in this case) and then changes that come along in the interim, like the configure.ac changes adding deps on DBI/DBD and so on (which were reflected in check-vm sometime later).

Perhaps we could set aside some time in the QA-week for this release to explore this some more? I think the work @minnus and @ryandoyle started awhile ago for doing QA in Vagrant VMs (see the Vagrantfile in $TOPDIR and details over at https://www.vagrantup.com) will keep us all on the same page. Then anyone will be able to do both reproducible QA and reproducible releasable builds.

It'll also help, I hope, with the situations where you see QA failures on a local VM that I later cannot reproduce here - like qa/878, argh - we'd both be using the exact same VMs ($TOPDIR/Vagrantfile sets this up and we all use the same configuration, for all the platforms). It could also help get the buildbot setups @lberk manages, and scripts like pcp-qa-summary all able to interoperate on the same VMs.

Sounds like the holy grail for QA and release builds doesn't it? :)

[kmcdonell]

Happy to discuss ... but I think the status quo should be the base.

1. cross-dependencies between the various packaging regimes, check-vm, the VMs we use (including Vagrant created VMs) are a fact of life ... I think we just need some automation to check package contents and reduce the chance of cracks through which artifacts can fall
2. by not using the same VM set ups we increase the genetic diversity in the testing pool and this should reduce the chance of bad things happening (this completely unsubstantiated hypothesis is closer to religion than science)

I don't believe the qa/878 issue relates to VMs ... it probably has more to do with the speed of the real processor under the VM because I see the failures across distros and hence different VMs. If it would help I can make limited access to my VMs available on a case-by-case basis.