Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

compressor/zstd: add zstd compression plugin #13075

Merged
merged 2 commits into from Jan 26, 2017
Merged

compressor/zstd: add zstd compression plugin #13075

merged 2 commits into from Jan 26, 2017

Conversation

tchaikov
Copy link
Contributor

No description provided.

@tchaikov tchaikov mentioned this pull request Jan 24, 2017
Closed
@liewegas
Copy link
Member

retest this please

@tchaikov
Copy link
Contributor Author 

/home/jenkins-build/build/workspace/ceph-pull-requests/qa/workunits/cephtool/test.sh:32: expect_false:  ceph osd pool set real-tier cache_target_dirty_ratio -.2
Traceback (most recent call last):
  File "/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/ceph", line 106, in <module>
    import rados
ImportError: libceph-common.so: cannot map zero-fill pages: Cannot allocate memory

retest this please.

@liewegas
Copy link
Member

liewegas commented Jan 24, 2017 via email

@tchaikov
Copy link
Contributor Author

tchaikov commented Jan 24, 2017 •
edited

@liewegas included ceph/ceph-ci@f8ef7f6 .

@liewegas
Copy link
Member

the debian build fails:

-- Installing: /build/ceph-11.1.0-6995-gf8ef7f6/debian/tmp/usr/lib/ceph/compressor/libceph_zlib.so.2.0.0
-- Installing: /build/ceph-11.1.0-6995-gf8ef7f6/debian/tmp/usr/lib/ceph/compressor/libceph_zlib.so.2
-- Installing: /build/ceph-11.1.0-6995-gf8ef7f6/debian/tmp/usr/lib/ceph/compressor/libceph_zlib.so
-- Installing: /build/ceph-11.1.0-6995-gf8ef7f6/debian/tmp/usr/local/include/zstd.h
-- Installing: /build/ceph-11.1.0-6995-gf8ef7f6/debian/tmp/usr/local/include/zbuff.h
-- Installing: /build/ceph-11.1.0-6995-gf8ef7f6/debian/tmp/usr/local/include/zdict.h
-- Installing: /build/ceph-11.1.0-6995-gf8ef7f6/debian/tmp/usr/local/lib/libzstd.a
CMake Error at src/compressor/zstd/lib/cmake_install.cmake:78 (file):
  file INSTALL cannot find
  "/build/ceph-11.1.0-6995-gf8ef7f6/obj-x86_64-linux-gnu/src/compressor/zstd/lib/libzstd.1.1.1024.so".
Call Stack (most recent call first):
  src/compressor/zstd/cmake_install.cmake:37 (include)
  src/compressor/cmake_install.cmake:39 (include)
  src/cmake_install.cmake:387 (include)
  cmake_install.cmake:37 (include)

@tchaikov tchaikov self-assigned this Jan 24, 2017
@tchaikov tchaikov changed the title compressor/zstd: add zstd compression plugin [DNM] compressor/zstd: add zstd compression plugin Jan 24, 2017
@tchaikov tchaikov removed the needs-qa label Jan 24, 2017
cbodley
cbodley reviewed Jan 24, 2017
View reviewed changes
src/compressor/zstd/CMakeLists.txt

# libzstd - build it statically
list(APPEND ZSTD_CMAKE_ARGS
-DCMAKE_C_COMPILER_ARGS="-fPIC -Wno-unused-variable -O3")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from the build log:

CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_C_COMPILER_ARGS

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cbodley fixed and repushed.

@tchaikov tchaikov changed the title [DNM] compressor/zstd: add zstd compression plugin compressor/zstd: add zstd compression plugin Jan 24, 2017
@tchaikov
Copy link
Contributor Author

tchaikov commented Jan 25, 2017 •
edited

changelog

  • do not set the build directory to "current binary dir" of "src/compressor/zstd". by sharing the build directory, "make install" will always tries to use the libzstd's Makefile created by its CMakeLists.txt. this prevents the INSTALL_COMMAND "true" overriding from working.

pushed to https://shaman.ceph.com/builds/ceph/wip-zstd/, let's see if it builds.

@tchaikov
Copy link
Contributor Author 

/home/jenkins-build/build/workspace/ceph-pull-requests/qa/workunits/ceph-helpers.sh:1151: wait_for_clean:  return 1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/erasure-code/test-erasure-eio.sh:137: rados_put_get:  return 1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/erasure-code/test-erasure-eio.sh:172: rados_get_data_eio:  return 1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/erasure-code/test-erasure-eio.sh:313: TEST_rados_get_with_subreadall_eio_shard_0:  return 1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/erasure-code/test-erasure-eio.sh:38: run:  return 1

https://jenkins.ceph.com/job/ceph-pull-requests/17475/consoleFull#27421202540526d21-3511-427d-909c-dd086c0d1034
retest this please.

@tchaikov
Copy link
Contributor Author 

Assert because (miter != peer_missing.end())
Indication 2

test failed due to a known issue already got fixed on master.

@liewegas @tchaikov
add src/zstd submodule
46b3849 
Signed-off-by: Sage Weil <sage@redhat.com>
@liewegas @tchaikov
compressor/zstd: add zstd compressor
4dd38b9 
Build/link of zstd itself is maybe not ideal, but it works fine.

Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
@tchaikov
Copy link
Contributor Author

rebased against master.

@tchaikov tchaikov merged commit c443c2c into ceph:master Jan 26, 2017
@tchaikov tchaikov deleted the wip-zstd branch January 26, 2017 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@liewegas liewegas liewegas approved these changes

@cbodley cbodley cbodley approved these changes

Assignees

@tchaikov tchaikov

Labels
common feature needs-doc needs-qa
Projects
None yet
Milestone
No milestone
3 participants
@tchaikov @liewegas @cbodley