From 43edd9c0b3140241ee46806b503b882e8fbe833b Mon Sep 17 00:00:00 2001 From: Ismo Puustinen Date: Mon, 26 Jun 2017 09:54:05 +0300 Subject: [PATCH 1/7] meta-openembedded: update to latest HEAD. * meta-openembedded b063789...d2da8b8 (172): > scsirastools: Upgrade to 1.6.6 > xfsprogs: Remove .la files in ${libdir} > xmlstarlet: Update to 1.6.1 > sqlite: Fix build with hardening flags > bdwgc: add a recipe from oe-core > libdevmapper: sync with lvm2 2.02.171 > kexecboot: update recipe to master on github (4c4f127) > zabbix: 3.0.8 -> 3.0.9 > python-wrapt: adds Python3 support > python-astroid: adds Python3 support > python-lazy-object-proxy: adds Python3 support > python-isort: adds Python3 support > python-pylint: adds Python3 support > makedumpfile: Fix build with hardening > gitver: skip packages instead of panic()ing if ${GITVER} fails to expand > python-setuptools-scm: update to version 1.15.6 > tiobench: Fix build with hardening flags > openocd: Fix build with gcc7 > ssiapi: Fix build with gcc7 and musl > libvncserver: upgrade to 0.9.11 > nginx: update to version 1.13.1 > python-paho-mqtt: update to version 1.2.3 > python-pytest: update to version 3.1.2 > python-prompt-toolkit: update to version 1.0.14 > python-parse: update to version 1.8.2 > python-mprpc: update to version 0.1.14 > python-lazy-object-proxy: update to version 1.3.1 > python-flask-pymongo: update to version 0.5.1 > python-flask: update to version 0.12.2 > python-pybind11: update to version 2.1.1 > python3-multidict: update to version 2.1.6 > python-cryptography: update to version 1.9 > python-cryptography-vectors: update to version 1.9 > python-urllib3: update to version 1.21.1 > python-zopeinterface: update to version 4.4.2 > python-gsocketpool: update to version 0.1.6 > python-unidiff: update to version 0.5.4 > python-isort: update to version 4.2.15 > python-py: update to version 1.4.34 > python-selectors34: update to version 1.2 > python-serpent: update to version 1.22 > python-gevent: update to version 1.2.2 > python-sqlalchemy: update to version 1.1.10 > python-monotonic: update to version 1.3 > python-jsonpatch: update to version 1.16 > python-pbr: update to version 3.0.1 > python-wrapt: update to version 1.10.10 > python-lxml: update to version 3.8.0 > python-cmd2: update to version 0.7.2 > python-chardet: update to version 3.0.4 > python-astroid: update to version 1.5.3 > kconfig-frontends: Update to 4.11.0.1 > kconfig-frontends: Add recipe > libfile-slurp-perl: add new recipe > passwdqc: add new recipe and replace pam-passwdqc > rrdtool: allow configuration to disable nls > networkmanager: add support for systemd-resolved > fwts: Bump SRCREV to latest past 17.03.00 release > mercurial: CVE-2017-9462 > fuse: fix fetcher error. > gitver: fix broken ${GITSHA} and recursion in S when PV="${GITVER}" > gitver: extend class to use EXTERNALSRC if set > libdc1394: Add X11 and opengl deps if distro has them in policy > hiredis: Add recipe > debsums: New recipe > libfile-fnmatch-perl: New recipe > librcf: fix a clang compiling issue > librcf: convert CRLF to LF > dos2unix.bbclass: add recipe > dos2unix: add recipe > po4a: add recipe > gflags: change download name. > esound: Upgrade to 0.2.41 > logwarn: add new recipe > enscript: add new recipe > agent-proxy: add new recipe > python3-django: add python3 recipe and update to 1.11.2 > python3-django-south: add new recipe > openldap: Fix CVE-2017-9287 > espeak: Fix FTBFS due to narrowing warnings on 32bit x86 > meson: add LDFLAGS to bbclass C/C++ link arguments > fuse: let overriding base_sbindir have the proper effect. > meson: fix build/host confusion for bbclass > meson: update Meson devtool to 0.40.1 > opencv: fix runtime dependencies. > opencv: fix compilation error when OpenCL is enabled. > inotify-tools: fix __NR_inotify_add_watch system call number on _MIPS_SIM_ABI64 > tbb: Update to 2017 Update 6 > konkretcmpi: Update and fix build with latest oe-core > srecord: add collection of tools for manipulating EPROM load files > zsh: update to 5.3.1 > xterm: 325 -> 327 > openipmi: Fix build with musl > python-numeric: Fix build with musl > caps: Fix buld with -pie > umip: fix EXTRA_OECONF and add RRECOMMENDS > umip: add systemd and sysvinit support > umip: fix for parallel compilation > konkretcmpi: Update and fix build with latest oe-core > leveldb: Update to 1.20 > onboard: Upgrade to 1.4.1 > python-cffi: Upgrade to 1.10.0 > audiofile: Inherit pkgconfig instead of binconfig > fuse: update 2.9.4 to 2.9.7 > python-pyroute: Add support for python3 > python-netaddr: Add support for python3 > python-mprpc: Add support for python3 > python-gsocketpool: Add support for python3 > python-gevent: Add support for python3 > python-msgpack: Add support for python3 > python-greenlet: Add support for python3 > poco: make cmake packages relocatable > nginx: depend on zlib instead of gzip > libmicrohttpd: update to version 0.9.55 > libupnp: Upgrade 1.6.19 -> 1.6.21 > gpsd: update to 3.16 > gpsd: don't compile with gold > Revert "rrdtool: add gettext-native dependency" > meta-perl: Make glibc-gconv deps apply for glibc based systems alone > lvm2/libdevmapper: fix invoking lvcreate failed > python3-pyudev: add a python3 recipe > python3-pyparted: add a python3 recipe > gnome-themes: remove PNBLACKLIST > recipes-support: Move libgpiod to meta-oe directory > nodejs: update to 4.8.3 > python-typing: update to version 3.6.1 > python-pysocks: update to version 1.6.7 > python-progress: update to version 1.3 and add RDEPENDS > python-babel: update to version 2.4.0 > libteam: Initial support > opencv: fix building with PACKAGECONFIG "dnn" enabled. > libxml-libxml-perl: update to version 2.0129 > libcapture-tiny-perl: update to version 0.46 > libdbd-sqlite-perl: update to version 1.54 > libdevel-globaldestruction-perl: update to version 0.14 > libextutils-helpers-perl: update to version 0.026 > meson: Use pkgconfig-native for the native tools > meson: Suport for c++ cross-compilation > ssiapi: add new recipe > xf86-video-mga: add new recipe > gd : CVE-2016-6906 > dbus-daemon-proxy: inherit pkgconfig, unblacklist and set package version > xfce4-vala: update vala api to 0.36 > python-sdnotify/python3-sdnotify: Add new recipe > python-cython: mangle scripts to use /usr/bin/env python > libnet-dns-perl: update to version 1.10 > libcgi-perl: update to version 4.36 > libcurses-perl: update to version 1.36 > libstrictures-perl: update to version 2.000003 > libsub-exporter-progressive-perl: update to version 0.001013 > libmodule-build-tiny-perl: update to version 0.039 > python-psutil: update to version 5.2.2 > python-pyro4: update to version 4.59 > python-certifi: update to version 2017.4.17 > python-pymisp: update to version 2.4.71 > python-markupsafe: update to version 1.0 > python-cryptography: update to version 1.8.1 > python-cryptography-vectors: update to version 1.8.1 > python-backports-functools-lru-cache: update to version 1.4 > python-jinja: update to version 2.9.6 > python3-zopeinterface: add python3 packaging > python-zopeinterface: fixup test package > python-werkzeug: update to version 0.12.2 > python-pbr: update to version 3.0.0 > nginx: update stable version to 1.12.0 > nginx: update development version to 1.13.0 > hwloc: Add > opencv: Make opencv-ts create share library intead of static > rygel: Update to 0.34.0 > espeak, espeak-data: Update to 1.48.04 release > xstdcmap: Change libxmu from RDEPENDS -> DEPENDS and unblacklist xstdcmap > rrdtool: add gettext-native dependency Signed-off-by: Ismo Puustinen --- meta-openembedded | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta-openembedded b/meta-openembedded index b063789560..d2da8b82cf 160000 --- a/meta-openembedded +++ b/meta-openembedded @@ -1 +1 @@ -Subproject commit b063789560bfb9c60a7a15277b5b3a9839b5ba74 +Subproject commit d2da8b82cfb2f8e55236751c1c56c9e7f3b5f68d From 7d61cac79b3e43ded1a74b7754bbba73c70cd212 Mon Sep 17 00:00:00 2001 From: Ismo Puustinen Date: Tue, 13 Jun 2017 23:51:43 +0300 Subject: [PATCH 2/7] refkit-config.inc: add PACKAGECONFIG "dnn" to OpenCV. This add support for "dnn" (Deep Neural Network) OpenCV module. The module makes it possible to run a limited set of Caffe (and other) DNN models without having the full DNN framework installed. Signed-off-by: Ismo Puustinen --- meta-refkit-core/conf/distro/include/refkit-config.inc | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/meta-refkit-core/conf/distro/include/refkit-config.inc b/meta-refkit-core/conf/distro/include/refkit-config.inc index ba9cc320b7..c7c147219a 100644 --- a/meta-refkit-core/conf/distro/include/refkit-config.inc +++ b/meta-refkit-core/conf/distro/include/refkit-config.inc @@ -135,8 +135,14 @@ PACKAGECONFIG_append_pn-ovmf_df-refkit-config = " secureboot" # only build minimal gstreamer1.0-plugins-bad PACKAGECONFIG_pn-gstreamer1.0-plugins-bad_df-refkit-config ?= "" -# Enable OpenCL. -PACKAGECONFIG_append_pn-opencv_df-refkit-config = " opencl" +# Enable OpenCL and DNN modules. +PACKAGECONFIG_append_pn-opencv_df-refkit-config = " opencl dnn" + +######################################################################### +# Changes that normally are better suited for a .bbappend have +# to be done here if the corresponding .bb file is not guaranteed +# to be present (i.e. outside of OE-core). +######################################################################### # Use nftables instead of iptables. PACKAGECONFIG_remove_pn-connman_df-refkit-config = "iptables" From 736b86f2f365801281474059a6af87aaeedba8e0 Mon Sep 17 00:00:00 2001 From: Ismo Puustinen Date: Thu, 15 Jun 2017 15:40:04 +0300 Subject: [PATCH 3/7] caffe-bvlc-reference: add new package. The recipe downloads a caffe network and some test data for testing OpenCV DNN module (and possibly Caffe itself). Signed-off-by: Ismo Puustinen --- .../caffe-bvlc-reference_0.1.bb | 60 +++++++++++++++++++ .../files/change_input_format.patch | 23 +++++++ .../caffe-bvlc-reference/files/dnn-test.py | 34 +++++++++++ 3 files changed, 117 insertions(+) create mode 100644 meta-refkit-computervision/recipes-computervision/caffe-bvlc-reference/caffe-bvlc-reference_0.1.bb create mode 100644 meta-refkit-computervision/recipes-computervision/caffe-bvlc-reference/files/change_input_format.patch create mode 100755 meta-refkit-computervision/recipes-computervision/caffe-bvlc-reference/files/dnn-test.py diff --git a/meta-refkit-computervision/recipes-computervision/caffe-bvlc-reference/caffe-bvlc-reference_0.1.bb b/meta-refkit-computervision/recipes-computervision/caffe-bvlc-reference/caffe-bvlc-reference_0.1.bb new file mode 100644 index 0000000000..0ccc619570 --- /dev/null +++ b/meta-refkit-computervision/recipes-computervision/caffe-bvlc-reference/caffe-bvlc-reference_0.1.bb @@ -0,0 +1,60 @@ +# Copyright (C) 2017 Ismo Puustinen +# Released under the MIT license (see COPYING.MIT for the terms) + +DESCRIPTION = "Download the BVLC reference network as test data." +HOMEPAGE = "https://github.com/BVLC/caffe/tree/master/models/bvlc_reference_caffenet" +LICENSE = "BVLC-model-license & CC-BY-2.5 & BSD-3-Clause" +SECTION = "tests" +DEPENDS = "" + +# Image by Lilly M # +# (https://en.wikipedia.org/wiki/Saluki#/media/File:Chart_perski_0002.jpg) +# using CC BY 2.5 license (https://creativecommons.org/licenses/by/2.5/) +LIC_FILES_CHKSUM = " \ + file://${COMMON_LICENSE_DIR}/CC-BY-SA-3.0;md5=3248afbd148270ac7337a6f3e2558be5 \ + file://${WORKDIR}/LICENSE;md5=650b869bd8ff2aed59c62bad2a22a821 \ +" + +SRC_URI = " \ + http://dl.caffe.berkeleyvision.org/caffe_ilsvrc12.tar.gz;name=ilsvrc12 \ + http://dl.caffe.berkeleyvision.org/bvlc_reference_caffenet.caffemodel;name=caffenet \ + https://raw.githubusercontent.com/BVLC/caffe/f16b5f2eb96cbb97d9a4b2b7312a23cb16f43dac/models/bvlc_reference_caffenet/deploy.prototxt;name=deploy \ + https://upload.wikimedia.org/wikipedia/commons/2/27/Chart_perski_0002.jpg;downloadfilename=dog.jpg;name=dog \ + https://raw.githubusercontent.com/opencv/opencv_contrib/009d2efb75fbb0eded127864cb1ca932d58d1738/LICENSE;name=bsd3 \ + file://dnn-test.py \ + file://change_input_format.patch \ +" + +SRC_URI[ilsvrc12.md5sum] = "f963098ea0e785a968ca1eb634003a90" +SRC_URI[ilsvrc12.sha256sum] = "e35c0c1994a21f7d8ed49d01881ce17ab766743d3b0372cdc0183ff4d0dfc491" + +SRC_URI[caffenet.md5sum] = "af678f0bd3cdd2437e35679d88665170" +SRC_URI[caffenet.sha256sum] = "472d4a06035497b180636d8a82667129960371375bd10fcb6df5c6c7631f25e0" + +SRC_URI[deploy.md5sum] = "955051d11e44bd29dd87a25dd766ec23" +SRC_URI[deploy.sha256sum] = "922248a4d2f6aac1cc8e7e5dbd996cc2ecd3356480d67c198f6cf96b12311a04" + +SRC_URI[dog.md5sum] = "76efa2a64d2c78078166f8f4ff375682" +SRC_URI[dog.sha256sum] = "f163822499bdd03a3bf4d3cb437d52ab5082d51edbcb7c98f1c42101d6358c70" + +SRC_URI[bsd3.md5sum] = "650b869bd8ff2aed59c62bad2a22a821" +SRC_URI[bsd3.sha256sum] = "7c34d28e784b202aa4998f477fd0aa9773146952d7f6fa5971369fcdda59cf48" + +do_patch_prepend() { + bb.utils.movefile(os.path.join(d.getVar("WORKDIR"), "deploy.prototxt"), d.getVar("S")) +} + +do_install() { + install -d ${D}${datadir}/Caffe/models/bvlc_reference_caffenet/ + install -d ${D}${datadir}/Caffe/data/ilsvrc12 + install -d ${D}${bindir} + + install ${WORKDIR}/synset_words.txt ${D}${datadir}/Caffe/data/ilsvrc12 + install ${WORKDIR}/bvlc_reference_caffenet.caffemodel ${D}${datadir}/Caffe/models/bvlc_reference_caffenet/ + install ${S}/deploy.prototxt ${D}${datadir}/Caffe/data/ + install ${WORKDIR}/dog.jpg ${D}${datadir}/Caffe/data/ + install -m 755 ${WORKDIR}/dnn-test.py ${D}${bindir} +} + +FILES_${PN} += "${datadir}/Caffe/*" + diff --git a/meta-refkit-computervision/recipes-computervision/caffe-bvlc-reference/files/change_input_format.patch b/meta-refkit-computervision/recipes-computervision/caffe-bvlc-reference/files/change_input_format.patch new file mode 100644 index 0000000000..bebd463a40 --- /dev/null +++ b/meta-refkit-computervision/recipes-computervision/caffe-bvlc-reference/files/change_input_format.patch @@ -0,0 +1,23 @@ +diff --git a/deploy.prototxt b/deploy.prototxt +index 907116e..a3de789 100644 +--- a/deploy.prototxt ++++ b/deploy.prototxt +@@ -1,10 +1,11 @@ + name: "CaffeNet" +-layer { +- name: "data" +- type: "Input" +- top: "data" +- input_param { shape: { dim: 10 dim: 3 dim: 227 dim: 227 } } +-} ++ ++input: "data" ++input_dim: 10 ++input_dim: 3 ++input_dim: 224 ++input_dim: 224 ++ + layer { + name: "conv1" + type: "Convolution" + diff --git a/meta-refkit-computervision/recipes-computervision/caffe-bvlc-reference/files/dnn-test.py b/meta-refkit-computervision/recipes-computervision/caffe-bvlc-reference/files/dnn-test.py new file mode 100755 index 0000000000..db0fabcfef --- /dev/null +++ b/meta-refkit-computervision/recipes-computervision/caffe-bvlc-reference/files/dnn-test.py @@ -0,0 +1,34 @@ +#!/usr/bin/env python3 + +# Classify an image using a suitable model. The image conversion magic +# is from +# https://github.com/opencv/opencv_contrib/blob/master/modules/dnn/samples/googlenet_python.py +# (3-clause BSD license). + +import numpy as np +import cv2 +import sys + +if len(sys.argv) != 4: + print("Usage: dnn.py ") + sys.exit(1) + +cv2.ocl.setUseOpenCL(False) + +# read the image +test_img = cv2.imread(sys.argv[3]) + +# resize +resized = cv2.resize(test_img, (224,224)) +converted = np.moveaxis(resized, 2, 0) +data = np.reshape(converted.astype(np.float32), (-1, 3, 224, 224)) + +# initialize network +net = cv2.dnn.readNetFromCaffe(sys.argv[1], sys.argv[2]) +net.setBlob(".data", data) + +# run the network +net.forward() + +# print the class +print(str(net.getBlob("prob").argmax())) From 1f81c9923209692a725a0512ff5a5055545a6949 Mon Sep 17 00:00:00 2001 From: Ismo Puustinen Date: Thu, 15 Jun 2017 16:08:02 +0300 Subject: [PATCH 4/7] supported recipes: added caffe-bvlc-reference. Signed-off-by: Ismo Puustinen --- meta-refkit/conf/distro/include/refkit-supported-recipes.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/meta-refkit/conf/distro/include/refkit-supported-recipes.txt b/meta-refkit/conf/distro/include/refkit-supported-recipes.txt index 3c5c68aef2..3fac6632c5 100644 --- a/meta-refkit/conf/distro/include/refkit-supported-recipes.txt +++ b/meta-refkit/conf/distro/include/refkit-supported-recipes.txt @@ -64,6 +64,7 @@ bubblewrap@flatpak-layer busybox@core bzip2@core ca-certificates@core +caffe-bvlc-reference@refkit-computervision cairo@core cantarell-fonts@core catkin-runtime@ros-layer From 1e517829221073e109944de6ca74532d2516eee2 Mon Sep 17 00:00:00 2001 From: Ismo Puustinen Date: Thu, 15 Jun 2017 16:25:19 +0300 Subject: [PATCH 5/7] tests: added a test for OpenCV DNN module. Signed-off-by: Ismo Puustinen --- .../conf/test/refkit-image-computervision.manifest | 1 + .../oeqa/runtime/multimedia/opencv/opencv_dnn_1.py | 11 +++++++++++ 2 files changed, 12 insertions(+) create mode 100644 meta-iotqa/lib/oeqa/runtime/multimedia/opencv/opencv_dnn_1.py diff --git a/meta-iotqa/conf/test/refkit-image-computervision.manifest b/meta-iotqa/conf/test/refkit-image-computervision.manifest index bd5a1a7c0f..21e04ae503 100644 --- a/meta-iotqa/conf/test/refkit-image-computervision.manifest +++ b/meta-iotqa/conf/test/refkit-image-computervision.manifest @@ -1,3 +1,4 @@ # Tests for computervision profile oeqa.runtime.multimedia.realsense.realsense_headless oeqa.runtime.multimedia.opencl.opencl_viennacl_1 +oeqa.runtime.multimedia.opencv.opencv_dnn_1 diff --git a/meta-iotqa/lib/oeqa/runtime/multimedia/opencv/opencv_dnn_1.py b/meta-iotqa/lib/oeqa/runtime/multimedia/opencv/opencv_dnn_1.py new file mode 100644 index 0000000000..08208f4057 --- /dev/null +++ b/meta-iotqa/lib/oeqa/runtime/multimedia/opencv/opencv_dnn_1.py @@ -0,0 +1,11 @@ +from oeqa.oetest import oeRuntimeTest + +class OpenCVDNN1Test(oeRuntimeTest): + def test_opencv_dnn_1(self): + # Classify an image using opencv-dnn + (status, output) = self.target.run('dnn-test.py /usr/share/Caffe/data/deploy.prototxt /usr/share/Caffe/models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel /usr/share/Caffe/data/dog.jpg') + # The dog must be identified as Saluki, gazelle hound. It's + # index 176. + self.assertEqual(status, 0, msg="Error messages: %s" % output) + lines = output.split("\n") + self.assertEqual(lines[-1], "176", msg="Misclassified image(%s): %s " % (lines[-1], output)) From 17dc85179719560758d0194d04ea9cdc37a247e7 Mon Sep 17 00:00:00 2001 From: Ismo Puustinen Date: Thu, 15 Jun 2017 21:07:17 +0300 Subject: [PATCH 6/7] packagegroup-computervision-test: add caffe-bvlc-reference. Add a Caffe test network and some test data. This is needed for testing OpenCV DNN module. It can also be used for demos and it's compatible with regular Caffe. Signed-off-by: Ismo Puustinen --- .../packagegroups/packagegroup-computervision-test.bb | 1 + 1 file changed, 1 insertion(+) diff --git a/meta-refkit-computervision/recipes-computervision/packagegroups/packagegroup-computervision-test.bb b/meta-refkit-computervision/recipes-computervision/packagegroups/packagegroup-computervision-test.bb index 8568ee9b6a..cd2d37b9c0 100644 --- a/meta-refkit-computervision/recipes-computervision/packagegroups/packagegroup-computervision-test.bb +++ b/meta-refkit-computervision/recipes-computervision/packagegroups/packagegroup-computervision-test.bb @@ -8,4 +8,5 @@ RDEPENDS_${PN} = " \ python3-opencv \ librealsense-examples \ viennacl-examples \ + caffe-bvlc-reference \ " From b4ae8afe3d97c005215559794bdb64db8f910e55 Mon Sep 17 00:00:00 2001 From: Ismo Puustinen Date: Wed, 21 Jun 2017 13:51:22 +0300 Subject: [PATCH 7/7] lvm2: fix lvm2-udevrules runtime dependency. Scripts in lvm2-udevrules use dmsetup. The command is nowadays provided by lvm2. Add runtime dependency to the package. See http://lists.openembedded.org/pipermail/openembedded-devel/2017-June/113291.html for proposed upstream change. Signed-off-by: Ismo Puustinen --- .../meta-oe/recipes-support/lvm2/lvm2_%.bbappend | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 meta-refkit-core/bbappends/meta-openembedded/meta-oe/recipes-support/lvm2/lvm2_%.bbappend diff --git a/meta-refkit-core/bbappends/meta-openembedded/meta-oe/recipes-support/lvm2/lvm2_%.bbappend b/meta-refkit-core/bbappends/meta-openembedded/meta-oe/recipes-support/lvm2/lvm2_%.bbappend new file mode 100644 index 0000000000..a0ecb5799a --- /dev/null +++ b/meta-refkit-core/bbappends/meta-openembedded/meta-oe/recipes-support/lvm2/lvm2_%.bbappend @@ -0,0 +1,2 @@ +# Add missing runtime dependency to udev rule files. +RDEPENDS_${PN}-udevrules_append_class-target = " lvm2"