New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compressor/zstd: add zstd compression plugin #13075

Merged
merged 2 commits into from Jan 26, 2017

Conversation

Projects
None yet
3 participants
@tchaikov
Contributor

tchaikov commented Jan 24, 2017

No description provided.

@liewegas

This comment has been minimized.

Member

liewegas commented Jan 24, 2017

retest this please

@tchaikov

This comment has been minimized.

Contributor

tchaikov commented Jan 24, 2017

/home/jenkins-build/build/workspace/ceph-pull-requests/qa/workunits/cephtool/test.sh:32: expect_false:  ceph osd pool set real-tier cache_target_dirty_ratio -.2
Traceback (most recent call last):
  File "/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/ceph", line 106, in <module>
    import rados
ImportError: libceph-common.so: cannot map zero-fill pages: Cannot allocate memory

retest this please.

@liewegas

This comment has been minimized.

Member

liewegas commented Jan 24, 2017

@tchaikov

This comment has been minimized.

Contributor

tchaikov commented Jan 24, 2017

@liewegas

This comment has been minimized.

Member

liewegas commented Jan 24, 2017

the debian build fails:

-- Installing: /build/ceph-11.1.0-6995-gf8ef7f6/debian/tmp/usr/lib/ceph/compressor/libceph_zlib.so.2.0.0
-- Installing: /build/ceph-11.1.0-6995-gf8ef7f6/debian/tmp/usr/lib/ceph/compressor/libceph_zlib.so.2
-- Installing: /build/ceph-11.1.0-6995-gf8ef7f6/debian/tmp/usr/lib/ceph/compressor/libceph_zlib.so
-- Installing: /build/ceph-11.1.0-6995-gf8ef7f6/debian/tmp/usr/local/include/zstd.h
-- Installing: /build/ceph-11.1.0-6995-gf8ef7f6/debian/tmp/usr/local/include/zbuff.h
-- Installing: /build/ceph-11.1.0-6995-gf8ef7f6/debian/tmp/usr/local/include/zdict.h
-- Installing: /build/ceph-11.1.0-6995-gf8ef7f6/debian/tmp/usr/local/lib/libzstd.a
CMake Error at src/compressor/zstd/lib/cmake_install.cmake:78 (file):
  file INSTALL cannot find
  "/build/ceph-11.1.0-6995-gf8ef7f6/obj-x86_64-linux-gnu/src/compressor/zstd/lib/libzstd.1.1.1024.so".
Call Stack (most recent call first):
  src/compressor/zstd/cmake_install.cmake:37 (include)
  src/compressor/cmake_install.cmake:39 (include)
  src/cmake_install.cmake:387 (include)
  cmake_install.cmake:37 (include)
  

@tchaikov tchaikov self-assigned this Jan 24, 2017

@tchaikov tchaikov changed the title from compressor/zstd: add zstd compression plugin to [DNM] compressor/zstd: add zstd compression plugin Jan 24, 2017

@tchaikov tchaikov removed the needs-qa label Jan 24, 2017

# libzstd - build it statically
list(APPEND ZSTD_CMAKE_ARGS
-DCMAKE_C_COMPILER_ARGS="-fPIC -Wno-unused-variable -O3")

This comment has been minimized.

@cbodley

cbodley Jan 24, 2017

Contributor

from the build log:

CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_C_COMPILER_ARGS

This comment has been minimized.

@tchaikov

tchaikov Jan 24, 2017

Contributor

@cbodley fixed and repushed.

@tchaikov tchaikov changed the title from [DNM] compressor/zstd: add zstd compression plugin to compressor/zstd: add zstd compression plugin Jan 24, 2017

@tchaikov

This comment has been minimized.

Contributor

tchaikov commented Jan 25, 2017

changelog

  • do not set the build directory to "current binary dir" of "src/compressor/zstd". by sharing the build directory, "make install" will always tries to use the libzstd's Makefile created by its CMakeLists.txt. this prevents the INSTALL_COMMAND "true" overriding from working.

pushed to https://shaman.ceph.com/builds/ceph/wip-zstd/, let's see if it builds.

@tchaikov

This comment has been minimized.

Contributor

tchaikov commented Jan 25, 2017

/home/jenkins-build/build/workspace/ceph-pull-requests/qa/workunits/ceph-helpers.sh:1151: wait_for_clean:  return 1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/erasure-code/test-erasure-eio.sh:137: rados_put_get:  return 1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/erasure-code/test-erasure-eio.sh:172: rados_get_data_eio:  return 1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/erasure-code/test-erasure-eio.sh:313: TEST_rados_get_with_subreadall_eio_shard_0:  return 1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/erasure-code/test-erasure-eio.sh:38: run:  return 1

https://jenkins.ceph.com/job/ceph-pull-requests/17475/consoleFull#27421202540526d21-3511-427d-909c-dd086c0d1034
retest this please.

@tchaikov

This comment has been minimized.

Contributor

tchaikov commented Jan 25, 2017

Assert because (miter != peer_missing.end())
Indication 2 

test failed due to a known issue already got fixed on master.

liewegas added some commits Sep 20, 2016

add src/zstd submodule
Signed-off-by: Sage Weil <sage@redhat.com>
compressor/zstd: add zstd compressor
Build/link of zstd itself is maybe not ideal, but it works fine.

Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
@tchaikov

This comment has been minimized.

Contributor

tchaikov commented Jan 25, 2017

rebased against master.

@tchaikov tchaikov added the needs-qa label Jan 25, 2017

@tchaikov tchaikov merged commit c443c2c into ceph:master Jan 26, 2017

3 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodifed Submodules submodules for project are unmodified
Details
default Build finished.
Details

@tchaikov tchaikov deleted the tchaikov:wip-zstd branch Jan 26, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment