[C++] Provide .deb and .rpm packages #16224
Comments
|
Wes McKinney / @wesm: @kou do you have the expertise to write the package recipes? I can try to help |
|
Kouhei Sutou / @kou:
Yes. We need to decide our package repository location before we create the package recipes. For example, if we choose packagecloud, we should not use CMake's CPack. We should use debian/ directory and .spec file. |
|
Kouhei Sutou / @kou: I used the package repository server that is maintained by me because I didn't know about Apache Arrow's policy. |
|
Wes McKinney / @wesm: |
|
Wes McKinney / @wesm: |
|
Kouhei Sutou / @kou: |
|
Wes McKinney / @wesm: |
|
Wes McKinney / @wesm: |
|
Kouhei Sutou / @kou: |
|
Kouhei Sutou / @kou: |
|
Wes McKinney / @wesm: |
|
Kouhei Sutou / @kou: |
|
Kouhei Sutou / @kou: |
|
Rares Vernica / @rvernica: arrow-dist/cpp-linux# rake --trace apt
rake aborted!
undefined local variable or method `__dir__' for #<ApacheArrowPackageTask:0x00000000a7de60>
/arrow-dist/cpp-linux/Rakefile:31:in `arrow_source_dir'
/arrow-dist/cpp-linux/Rakefile:24:in `initialize'
/arrow-dist/cpp-linux/Rakefile:110:in `new'
/arrow-dist/cpp-linux/Rakefile:110:in `<top (required)>'
/usr/lib/ruby/vendor_ruby/rake/rake_module.rb:25:in `load'
/usr/lib/ruby/vendor_ruby/rake/rake_module.rb:25:in `load_rakefile'
/usr/lib/ruby/vendor_ruby/rake/application.rb:589:in `raw_load_rakefile'
/usr/lib/ruby/vendor_ruby/rake/application.rb:89:in `block in load_rakefile'
/usr/lib/ruby/vendor_ruby/rake/application.rb:160:in `standard_exception_handling'
/usr/lib/ruby/vendor_ruby/rake/application.rb:88:in `load_rakefile'
/usr/lib/ruby/vendor_ruby/rake/application.rb:72:in `block in run'
/usr/lib/ruby/vendor_ruby/rake/application.rb:160:in `standard_exception_handling'
/usr/lib/ruby/vendor_ruby/rake/application.rb:70:in `run'
/usr/bin/rake:27:in `<main>'Am I using it the wrong way? |
|
Kouhei Sutou / @kou: % ruby -v |
# ruby -v
ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]I am trying to get the |
|
Kouhei Sutou / @kou: This patch will help you: diff --git a/cpp-linux/Rakefile b/cpp-linux/Rakefile
index c0e5963..d612423 100644
--- a/cpp-linux/Rakefile
+++ b/cpp-linux/Rakefile
@@ -28,7 +28,7 @@ class ApacheArrowPackageTask < PackageTask
private
def arrow_source_dir
- File.join(__dir__, "..", "arrow")
+ File.join(File.dirname(__FILE__), "..", "arrow")
end
def detect_version(release_time)
diff --git a/cpp-linux/apt/ubuntu-14.04/Dockerfile b/cpp-linux/apt/ubuntu-14.04/Dockerfile
new file mode 100644
index 0000000..f993458
--- /dev/null
+++ b/cpp-linux/apt/ubuntu-14.04/Dockerfile
@@ -0,0 +1,42 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+FROM ubuntu:14.04
+
+ENV DEBIAN_FRONTEND noninteractive
+
+ARG DEBUG
+
+RUN \
+ quiet=$([ "${DEBUG}" = "yes" ] || echo "-qq") && \
+ apt update ${quiet} && \
+ apt install -y -V ${quiet} \
+ build-essential \
+ cmake3 \
+ debhelper\
+ devscripts \
+ git \
+ gtk-doc-tools \
+ libboost-filesystem-dev \
+ libboost-system-dev \
+ libgirepository1.0-dev \
+ libglib2.0-doc \
+ libjemalloc-dev \
+ lsb-release \
+ pkg-config && \
+ apt-get clean && \
+ rm -rf /var/lib/apt/lists/*
diff --git a/cpp-linux/debian/control b/cpp-linux/debian/control
index 147a73c..aa0571d 100644
--- a/cpp-linux/debian/control
+++ b/cpp-linux/debian/control
@@ -10,8 +10,6 @@ Build-Depends:
libboost-system-dev,
libboost-filesystem-dev,
libjemalloc-dev,
- python3-dev,
- python3-numpy,
libgirepository1.0-dev,
gtk-doc-tools,
gobject-introspection
@@ -32,21 +30,6 @@ Description: Apache Arrow is a data processing library for analysis
.
This package provides library files.
-Package: libarrow-python0
-Section: libs
-Architecture: any
-Multi-Arch: same
-Pre-Depends: ${misc:Pre-Depends}
-Depends:
- ${misc:Depends},
- ${shlibs:Depends},
- libarrow0 (= ${binary:Version}),
- python3,
- python3-numpy
-Description: Apache Arrow is a data processing library for analysis
- .
- This package provides library files for Python.
-
Package: libarrow-dev
Section: libdevel
Architecture: any
diff --git a/cpp-linux/debian/libarrow-dev.install b/cpp-linux/debian/libarrow-dev.install
index 92b5574..67c6cdc 100644
--- a/cpp-linux/debian/libarrow-dev.install
+++ b/cpp-linux/debian/libarrow-dev.install
@@ -1,6 +1,4 @@
usr/include/arrow/
usr/lib/*/libarrow.a
-usr/lib/*/libarrow_*.a
usr/lib/*/libarrow.so
-usr/lib/*/libarrow_*.so
usr/lib/*/pkgconfig/arrow.pc
diff --git a/cpp-linux/debian/rules b/cpp-linux/debian/rules
index 641e07c..f92b589 100755
--- a/cpp-linux/debian/rules
+++ b/cpp-linux/debian/rules
@@ -17,10 +17,7 @@ override_dh_auto_configure:
--builddirectory=cpp_build \
-- \
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE) \
- -DARROW_BUILD_TESTS=OFF \
- -DARROW_PYTHON=ON \
- -DPythonInterp_FIND_VERSION=ON \
- -DPythonInterp_FIND_VERSION_MAJOR=3
+ -DARROW_BUILD_TESTS=OFF
dh_auto_configure \
--sourcedirectory=c_glib \
--builddirectory=c_glib_build \
diff --git a/cpp-linux/package-task.rb b/cpp-linux/package-task.rb
index c9a0755..ed115bd 100644
--- a/cpp-linux/package-task.rb
+++ b/cpp-linux/package-task.rb
@@ -180,9 +180,10 @@ VERSION=#{@version}
def define_apt_task
namespace :apt do
code_names = [
- ["debian", "stretch"],
- ["ubuntu", "16.04"],
- ["ubuntu", "17.04"],
+ # ["debian", "stretch"],
+ ["ubuntu", "14.04"],
+ # ["ubuntu", "16.04"],
+ # ["ubuntu", "17.04"],
]
debian_dir = "debian"
apt_dir = "apt"You can build .deb for Ubuntu 14.04 by Ruby 1.9 with this patch. |
|
Rares Vernica / @rvernica: # docker run --rm --tty --interactive --volume `pwd`:/host:rw --env DEBUG=yes apache-arrow-ubuntu-14.04 bash -x /host/build.sh
+ LANG=C
+ . /host/env.sh
++ PACKAGE=apache-arrow
++ VERSION=0.7.0.20170827
++ lsb_release --id --short
++ tr A-Z a-z
+ distribution=ubuntu
++ lsb_release --codename --short
+ code_name=trusty
+ case "${distribution}" in
+ component=universe
+ run mkdir -p build
+ mkdir -p build
+ test 0 -ne 0
+ run cp /host/tmp/apache-arrow-0.7.0.20170827.tar.gz build/apache-arrow_0.7.0.20170827.orig.tar.gz
+ cp /host/tmp/apache-arrow-0.7.0.20170827.tar.gz build/apache-arrow_0.7.0.20170827.orig.tar.gz
+ test 0 -ne 0
+ run cd build
+ cd build
+ test 0 -ne 0
+ run tar xfz apache-arrow_0.7.0.20170827.orig.tar.gz
+ tar xfz apache-arrow_0.7.0.20170827.orig.tar.gz
+ test 0 -ne 0
+ run cd apache-arrow-0.7.0.20170827/
+ cd apache-arrow-0.7.0.20170827/
+ test 0 -ne 0
+ run cp -rp /host/tmp/debian debian
+ cp -rp /host/tmp/debian debian
+ test 0 -ne 0
+ '[' yes = yes ']'
+ run debuild -us -uc
+ debuild -us -uc
This package has a Debian revision number but there does not seem to be
an appropriate original tar file or .orig directory in the parent directory;
(expected one of apache-arrow_0.6.0.orig.tar.gz, apache-arrow_0.6.0.orig.tar.bz2,
apache-arrow_0.6.0.orig.tar.lzma, apache-arrow_0.6.0.orig.tar.xz or apache-arrow-0.7.0.20170827.orig)
continue anyway? (y/n) y
dpkg-buildpackage -rfakeroot -D -us -uc
dpkg-buildpackage: warning: using a gain-root-command while being root
dpkg-buildpackage: source package apache-arrow
dpkg-buildpackage: source version 0.6.0-1
dpkg-buildpackage: source distribution unstable
dpkg-buildpackage: source changed by Kouhei Sutou <kou@clear-code.com>
dpkg-source --before-build apache-arrow-0.7.0.20170827
dpkg-buildpackage: host architecture amd64
fakeroot debian/rules clean
dh clean --with gir
dh_testdir
dh_auto_clean
dh_clean
dpkg-source -b apache-arrow-0.7.0.20170827
dpkg-source: error: can't build with source format '3.0 (quilt)': no upstream tarball found at ../apache-arrow_0.6.0.orig.tar.{bz2,gz,lzma,xz}
dpkg-buildpackage: error: dpkg-source -b apache-arrow-0.7.0.20170827 gave error exit status 255
debuild: fatal error at line 1364:
dpkg-buildpackage -rfakeroot -D -us -uc failed
+ test 29 -ne 0
+ echo 'Failed debuild' -us -uc
Failed debuild -us -uc
+ exit 1Here are the root@1cbdea4671f5:/# find . -name "apache-arrow*"
./host/tmp/apache-arrow-0.7.0.20170827.tar.gz
./build/apache-arrow_0.7.0.20170827.orig.tar.gz
./build/apache-arrow-0.7.0.20170827
./build/apache-arrow_0.6.0-1_amd64.build |
|
Rares Vernica / @rvernica: |
|
Kouhei Sutou / @kou: |
|
Rares Vernica / @rvernica: I also needed Ubuntu 14.04 packages and I was able to build them myself for now. If it is not too much trouble, it would be nice to have Ubuntu 14.04 packages for Arrow 0.7 available as well when time comes. |
|
Wes McKinney / @wesm: |
|
Rares Vernica / @rvernica: |
|
Wes McKinney / @wesm: |
If we provide .deb and .rpm packages for C++ Arrow, users can install it easily. (At least, I'm happy as an user.)
Is there any location to provide .deb and .rpm packages? If it doesn't exist, how about using https://packagecloud.io/ with "Open Source plan"? We can find "Open Source plan" by clicking "Looking for free or open-source plans" at https://packagecloud.io/pricing .
Environment: GNU/Linux
Reporter: Kouhei Sutou / @kou
Assignee: Kouhei Sutou / @kou
Related issues:
Note: This issue was originally created as ARROW-592. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: