Feedback testing debian packages: 9.3.4-rcx

[t-b]

Notes: All java programs were used from the current source distribution from here.

Result:

• Package upgrade worked to 9.3.4~rc2+dfsg2-1~exp3 from the current unstable package

New Version: 9.3.4~rc2+dfsg2-1~exp3
Old Version: 9.2.5a+dfsg1-2+b2

• Jive worked with TangoTest and some of my tango device servers

• Astor worked but showed heartbeat error, see here

• Pogo does not work see here

• Database upgrade did not work. A reinstall of tango-db also did not create the correct database. I've checked with

MariaDB [tango]> SELECT * FROM property_class WHERE value ='EventConfirmSubscription';
Empty set (0.001 sec)

as that database entry does only exist in 9.3.4~rc2. TangoTest works though.

Doing a full reinstall and upgrade via

apt remove --purge "tango*"
apt install tango-db // answering questions regarding automatic management with db-config this yes
apt install -t experimental tango-db // dito

did not change anything.

I'm using mariadb-serverdb. I' attaching a database dump tango-db.tango.2020-01-21-21.13.mysql.guCsZo.zip
which was created by setting export dbc_debug=1 and uninstalling tango-db.

Questions/Suggestions/Bugs:

• Missing dbg package for libtango9 Found it, this is called libtango9-dbgsym and needs the following sources.list http://debug.mirrors.debian.org/debian-debug experimental-debug/main amd64 Packages.
• I think libtango9 should have a versioned dependency on tango-common because it is easily forgotten to upgrade if doing the package upgrade manually from experimental.
• Missing libtango9-doc package It is named libtango-doc and works nicely!
• I had a hard time finding tango_admin, it is in the package libtango-tools which is okay (but nevertheless the package documentation shown with apt show libtango-tools should mention tango_admin), but the file itself should not be in a lib directory but better in bin. The tango device servers are good in /usr/lib/tango. Created https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=949540.
• It would be nice to have a tango-full package to install everything cpptango related. In that way we can just tell users to install tango-full instead of tango-db libtango9 ....

As part of testing the following Issues/PRs have been created:

• 9.3.4-rc2: Does not finish load screen pogo#83

• 9.3.4-rc2: Always shows heartbeat errors for TangoStarter Astor#18

• Update cppTango to latest 9.3-backports on next rc #63

• https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=948496

• https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=949540

Done:

• configure test for java fails #62
• Fix java version detection #64
• Fix database connection testing with mariadb #65
• Default to usr for omni checks #66

@bourtemb @picca

[picca]

Hello, thanks for this test

Result: * Package upgrade worked to 9.3.4~rc2+dfsg2-1~exp3

Can you be more specific, because you explain that the upgrade of the database does not work ?

* Jive worked with TangoTest and some of my tango device servers

I did not yet packages the java part. so whcih one are you using ?

* Astor worked but showed heartbeat error, see here<tango-controls/Astor#18>

idem

* Pogo does not work see here<tango-controls/pogo#83>

idem

* Database upgrade did not work. A reinstall of tango-db also did not create the correct database. I've checked with MariaDB [tango]> SELECT * FROM property_class WHERE value ='EventConfirmSubscription'; Empty set (0.001 sec) as that is only supposed to exist in 9.3.4. TangoTest works though.

I do not understant this last sentance. I packaged the 9.3.4-rc2 so it should work. Is it possible to see the error message

Questions/Suggestions/Bugs: * Missing dbg package for libtango9

This is normal https://wiki.debian.org/DebugPackage now the debug symbols are automatically generated.

* I think libtango9 should have a versioned dependency on tango-common

Can you explain why ?

* Missing libtango9-doc package

In fact the -doc is in libtango-doc. Since I support only one -dev version installed. If we want to have different verion of tango-dev installed this is another story :))

* I had a hard time finding tango_admin, it is in the package libtango-tools which is okay (but nevertheless the package documentation shown with apt show > libtango-tools should mention tango_admin), but the file itself should not be in a lib directory but better in bin. The tango device servers are good in /usr/lib/tango.

can you fill a bug against tango in Debian ?

* It would be nice to have a tango-full package to install everything tango related

Can you explain what should be in this package. For now we only have tango C++ library the (db, starter, accesscontrol test devices), taurus, sardana, avaiulable in Debian. (I do not speak about the java part since it is not possible for now to build it and distribut it in the main distribution. Cheers Frederic

[mliszcz]

Hi, just one question:

I had a hard time finding tango_admin, it is in the package libtango-tools which is okay (but nevertheless the package documentation shown with apt show libtango-tools should mention tango_admin), but the file itself should not be in a lib directory but better in bin. The tango device servers are good in /usr/lib/tango.

What is the motivation behind putting executable applications inside lib? From end-user perspective, I find this quite surprising (I expect apps to be installed to bin, which is usually on system search path).

[picca]

Hi, just one question: I had a hard time finding tango_admin, it is in the package libtango-tools which is okay (but nevertheless the package documentation shown with apt show libtango-tools > should mention tango_admin), but the file itself should not be in a lib directory but better in bin. The tango device servers are good in /usr/lib/tango. What is the motivation behind putting executable applications inside lib? From end-user perspective, I find this quite surprising (I expect apps to be installed to bin, which is usually on system search path).

Hello, Debian maintainer speaking here, I decided to put all devices servers under /usr/lib/tango in order to avoid polluting the PATH with devices which should be managed by the starter device. A bit like all the git command line, where only the git command is in the PATH. For tango_admin, I agreed that it could be in /usr/bin instead of /ust/lib/tango. cheers Fred

[t-b]

@picca I've updated my post and did some more testing.

Regarding the java part of tango. AFAIU there is a licensing issue right? Would we be able to create non-free packages?

[picca]

@picca<https://github.com/picca> I've updated my post and did some more testing.

thanks for the bug report :). I will try to find some time in order to check your post.

Regarding the java part of tango. AFAIU there is a licensing issue right? Would we be able to create non-free packages?

I usually do this, but this time I thnk that there is an issue with java-11 and the removal of CORBA from the jdk... This should be fixed in the .jar provided in the tango source. Cheers Fred

[t-b]

@picca The java packages as they are depend on java-8.

[picca]

@picca<https://github.com/picca> The java packages as they are depend on java-8.

And this is the problem. On Debian Buster (aka 10), there is only java-11.

[t-b]

@picca A new version is out, see https://github.com/tango-controls/TangoSourceDistribution/releases/tag/9.3.4-rc4.

[picca]

uploaded https://tracker.debian.org/news/1140670/accepted-tango-934rc4dfsg1-1exp1-source-into-experimental/

[t-b]

@picca
I did some more testing with rc6:

• https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=949549 is not yet fixed. I'm seeing the lintian error still in https://salsa.debian.org/science-team/tango/-/jobs/808832 and also hardening-check returns the same.
• The database is not upgraded on package upgrade from 9.2.5 to 9.3.4.

I'm seeing

/usr/sbin/dbconfig-generate-include: 246: cannot create /tmp/dbconfig-generate-include.EBasHV: Permission denied

which looks related when doing

apt remove --yes --purge "*tango*" && rm -f /var/tmp/*mysql* && apt install --yes tango-db tango-starter tango-test && apt install --yes -t experimental tango-db tango-starter tango-test

I've verified that my method of checking the database upgrade with

USE tango;
SELECT * FROM property_class WHERE value ='EventConfirmSubscription';

works as I get something when I manually execute update_db_from_9.2.5_to_9.3.4.sql.

• (Just a comment): In the lintian log on salsa I see

$ if lintian --fail-on error --allow-root > /dev/null ; then # collapsed multi-line command
Unknown option: fail-on

General: @bourtemb @mliszcz

• Logviewer is not working -> LogViewer from TangoSourceDistribution 9.3.4-rc6 is not working LogViewer#5
• DataBaseds ipv4 -> DataBaseds: Does not listen on ipv4 port 10000 TangoTickets#40

[picca]

Hello

here the dbconfig-common bug report about this Permission Denied

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=965219

I am also investigating about the hardening. It seems that the [X]FLAGS are not propagated into the Makefile.am.
It seems that it works for log4tango but not for all the tango sources.
I saw in some Makefile.am, some of these fortification harcoded. It seems to me that these flags should not be hardcoded in the Makefile.am but given during the configuration phase.

cheers

here the new build

https://salsa.debian.org/science-team/tango/-/jobs/954274/raw

[picca]

here the relevant part of the build

make[6]: Leaving directory '/builds/science-team/tango/debian/output/tango-9.3.4~rc6+dfsg1/cppserver/AbstractClass'
make[5]: Leaving directory '/builds/science-team/tango/debian/output/tango-9.3.4~rc6+dfsg1/cppserver/AbstractClass'
make[5]: Entering directory '/builds/science-team/tango/debian/output/tango-9.3.4~rc6+dfsg1/cppserver/tangoaccesscontrol'
g++ -DHAVE_CONFIG_H -I. -I../..  -I../../lib/cpp/client -I../../lib/cpp/server -I/usr/include -I../../lib/cpp/log4tango/include -I../../lib/cpp/log4tango/include -I../../lib/cpp/server -I../../cppserver/AbstractClass/AccessControl -I/usr/include/mysql -isystem /usr/include/mit-krb5 -I/usr/include/pgm-5.2 -Wdate-time -D_FORTIFY_SOURCE=2 -I/usr/include   -g -O2 -fdebug-prefix-map=/builds/science-team/tango/debian/output/tango-9.3.4~rc6+dfsg1=. -fstack-protector-strong -Wformat -Werror=format-security   -D_REENTRANT -DOMNI_UNLOADABLE_STUBS -c -o ClassFactory.o ClassFactory.cpp
g++ -DHAVE_CONFIG_H -I. -I../..  -I../../lib/cpp/client -I../../lib/cpp/server -I/usr/include -I../../lib/cpp/log4tango/include -I../../lib/cpp/log4tango/include -I../../lib/cpp/server -I../../cppserver/AbstractClass/AccessControl -I/usr/include/mysql -isystem /usr/include/mit-krb5 -I/usr/include/pgm-5.2 -Wdate-time -D_FORTIFY_SOURCE=2 -I/usr/include   -g -O2 -fdebug-prefix-map=/builds/science-team/tango/debian/output/tango-9.3.4~rc6+dfsg1=. -fstack-protector-strong -Wformat -Werror=format-security   -D_REENTRANT -DOMNI_UNLOADABLE_STUBS -c -o TangoAccessControlClass.o TangoAccessControlClass.cpp
g++ -DHAVE_CONFIG_H -I. -I../..  -I../../lib/cpp/client -I../../lib/cpp/server -I/usr/include -I../../lib/cpp/log4tango/include -I../../lib/cpp/log4tango/include -I../../lib/cpp/server -I../../cppserver/AbstractClass/AccessControl -I/usr/include/mysql -isystem /usr/include/mit-krb5 -I/usr/include/pgm-5.2 -Wdate-time -D_FORTIFY_SOURCE=2 -I/usr/include   -g -O2 -fdebug-prefix-map=/builds/science-team/tango/debian/output/tango-9.3.4~rc6+dfsg1=. -fstack-protector-strong -Wformat -Werror=format-security   -D_REENTRANT -DOMNI_UNLOADABLE_STUBS -c -o TangoAccessControl.o TangoAccessControl.cpp
g++ -DHAVE_CONFIG_H -I. -I../..  -I../../lib/cpp/client -I../../lib/cpp/server -I/usr/include -I../../lib/cpp/log4tango/include -I../../lib/cpp/log4tango/include -I../../lib/cpp/server -I../../cppserver/AbstractClass/AccessControl -I/usr/include/mysql -isystem /usr/include/mit-krb5 -I/usr/include/pgm-5.2 -Wdate-time -D_FORTIFY_SOURCE=2 -I/usr/include   -g -O2 -fdebug-prefix-map=/builds/science-team/tango/debian/output/tango-9.3.4~rc6+dfsg1=. -fstack-protector-strong -Wformat -Werror=format-security   -D_REENTRANT -DOMNI_UNLOADABLE_STUBS -c -o TangoAccessControlStateMachine.o TangoAccessControlStateMachine.cpp
g++ -DHAVE_CONFIG_H -I. -I../..  -I../../lib/cpp/client -I../../lib/cpp/server -I/usr/include -I../../lib/cpp/log4tango/include -I../../lib/cpp/log4tango/include -I../../lib/cpp/server -I../../cppserver/AbstractClass/AccessControl -I/usr/include/mysql -isystem /usr/include/mit-krb5 -I/usr/include/pgm-5.2 -Wdate-time -D_FORTIFY_SOURCE=2 -I/usr/include   -g -O2 -fdebug-prefix-map=/builds/science-team/tango/debian/output/tango-9.3.4~rc6+dfsg1=. -fstack-protector-strong -Wformat -Werror=format-security   -D_REENTRANT -DOMNI_UNLOADABLE_STUBS -c -o DbUtils.o DbUtils.cpp
g++ -DHAVE_CONFIG_H -I. -I../..  -I../../lib/cpp/client -I../../lib/cpp/server -I/usr/include -I../../lib/cpp/log4tango/include -I../../lib/cpp/log4tango/include -I../../lib/cpp/server -I../../cppserver/AbstractClass/AccessControl -I/usr/include/mysql -isystem /usr/include/mit-krb5 -I/usr/include/pgm-5.2 -Wdate-time -D_FORTIFY_SOURCE=2 -I/usr/include   -g -O2 -fdebug-prefix-map=/builds/science-team/tango/debian/output/tango-9.3.4~rc6+dfsg1=. -fstack-protector-strong -Wformat -Werror=format-security   -D_REENTRANT -DOMNI_UNLOADABLE_STUBS -c -o main.o main.cpp
/bin/bash ../../libtool  --tag=CXX   --mode=link g++  -g -O2 -fdebug-prefix-map=/builds/science-team/tango/debian/output/tango-9.3.4~rc6+dfsg1=. -fstack-protector-strong -Wformat -Werror=format-security   -D_REENTRANT -DOMNI_UNLOADABLE_STUBS  -Wl,-z,relro -Wl,-z,now -o TangoAccessControl ClassFactory.o TangoAccessControlClass.o TangoAccessControl.o TangoAccessControlStateMachine.o DbUtils.o main.o -L../../lib/cpp/client -ltango  -lmysqlclient -lzmq ../AbstractClass/AccessControl/libaccesscontrol.la -lpthread -lnsl -ldl -L/usr/lib/bfd-plugins -lomniORB4 -lomniDynamic4 -lCOS4  -lpthread -lomnithread -lnsl -L/usr/lib/bfd-plugins  -lpthread -lomnithread -lpthread
libtool: link: g++ -g -O2 "-fdebug-prefix-map=/builds/science-team/tango/debian/output/tango-9.3.4~rc6+dfsg1=." -fstack-protector-strong -Wformat -Werror=format-security -D_REENTRANT -DOMNI_UNLOADABLE_STUBS -Wl,-z -Wl,relro -Wl,-z -Wl,now -o .libs/TangoAccessControl ClassFactory.o TangoAccessControlClass.o TangoAccessControl.o TangoAccessControlStateMachine.o DbUtils.o main.o  -L../../lib/cpp/client /builds/science-team/tango/debian/output/tango-9.3.4~rc6+dfsg1/lib/cpp/client/.libs/libtango.so -lmysqlclient -lzmq ../AbstractClass/AccessControl/.libs/libaccesscontrol.a -L/usr/lib/bfd-plugins -ldl -lomniORB4 -lomniDynamic4 -lCOS4 -lnsl -lomnithread -lpthread

so it seems that some of the hardening flags are used

[picca]

Here the official flags

picca@sixs3:~/debian/science-team/tango$ dpkg-buildflags 
CFLAGS=-g -O2 -fdebug-prefix-map=/home/experiences/instrumentation/picca/debian/science-team/tango=. -fstack-protector-strong -Wformat -Werror=format-security
CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2
CXXFLAGS=-g -O2 -fdebug-prefix-map=/home/experiences/instrumentation/picca/debian/science-team/tango=. -fstack-protector-strong -Wformat -Werror=format-security
FCFLAGS=-g -O2 -fdebug-prefix-map=/home/experiences/instrumentation/picca/debian/science-team/tango=. -fstack-protector-strong
FFLAGS=-g -O2 -fdebug-prefix-map=/home/experiences/instrumentation/picca/debian/science-team/tango=. -fstack-protector-strong
GCJFLAGS=-g -O2 -fdebug-prefix-map=/home/experiences/instrumentation/picca/debian/science-team/tango=. -fstack-protector-strong
LDFLAGS=-Wl,-z,relro
OBJCFLAGS=-g -O2 -fdebug-prefix-map=/home/experiences/instrumentation/picca/debian/science-team/tango=. -fstack-protector-strong -Wformat -Werror=format-security
OBJCXXFLAGS=-g -O2 -fdebug-prefix-map=/home/experiences/instrumentation/picca/debian/science-team/tango=. -fstack-protector-strong -Wformat -Werror=format-security

[picca]

here the output of hardening-check for the database and the TangoAcessControl

picca@sixs3:~$ hardening-check -v TangoAccessControl 
TangoAccessControl:
 Position Independent Executable: yes
 Stack protected: yes
 Fortify Source functions: no, only unprotected functions found!
	unprotected: memcpy
	unprotected: memmove
	unprotected: strcpy
	unprotected: gethostname
 Read-only relocations: yes
 Immediate binding: yes
 Stack clash protection: unknown, no -fstack-clash-protection instructions found
 Control flow integrity: no, not found!
picca@sixs3:~$ hardening-check -v DataBaseds 
DataBaseds:
 Position Independent Executable: yes
 Stack protected: yes
 Fortify Source functions: yes (some protected functions found)
	unprotected: strcpy
	unprotected: stpcpy
	unprotected: memcpy
	unprotected: gethostname
	unprotected: memmove
	protected: strcpy
	protected: strcat
	protected: sprintf
 Read-only relocations: yes
 Immediate binding: yes
 Stack clash protection: unknown, no -fstack-clash-protection instructions found
 Control flow integrity: no, not found!