New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setting /etc/security/limits.conf per FAQ leads to LVM problem (and what is utility of mmap?) #514

Closed
davclark opened this Issue Dec 12, 2017 · 1 comment

Comments

Projects
None yet
2 participants
@davclark

davclark commented Dec 12, 2017

I updated limits.conf per the FAQ, with the lines:

* soft memlock 262144
* hard memlock 262144

But then when I logged out and tried to log in again, I got an lvmedad error and no X session. I also tried setting use_lvmedad to 0, but then I simply got a non-functional X session explaining that it couldn't detect card or peripherals. I couldn't even respond to dialogs. Commenting these lines got rid of the problem. And also, if I just use my username instead of * in limits.conf, this gets rid of the problem.

Based on this stackoverflow question, I'm guessing this is just general bugginess in LVM, but it seems worth reporting in case other users run into this issue. It's rather debilitating! (note, this is NOT the same problem, and the fix on stack overflow - which I describe trying above - doesn't work.)

In any case, restricting to my username seems to safely up my memory limits and the mmap errors go away. However, it's not clear it helps performance much! On my CPU even when I restrict to the first four cores, and all report successfully pinning memory, I only get about 77 H/s per core. Memory does not get pinned for (logical) cores 4-7. I do understand that using all cores cripples performance (using 15 of 16 I only had ~15 H/s per core). Similarly, when I only use (logical) cores 0-4 and 8-11, memory gets pinned for all of them, but performance is still ~15 H/s.

I'm curious if I can get better performance out of this threadripper. In particular, if I should be messing with NUMA or not.

In any case, thanks for the tool! It's sending a little hashing your way ;)

Basic information

  • Threadripper 1900x, 16GB RAM
  • RX 580 (Red Devil in "silent" mode)

Compile issues

  • Ubuntu 16.04.3
add **all** commands you used and the **full** compile output here

sudo sysctl -w vm.nr_hugepages=128
sudo vim /etc/security/limits.conf # Add suggested lines from FAQ

run `cmake -LA .` in the build folder and add the output here

CMAKE_AR:FILEPATH=/usr/bin/ar
CMAKE_BUILD_TYPE:STRING=Release
CMAKE_COLOR_MAKEFILE:BOOL=ON
CMAKE_CXX_COMPILER:FILEPATH=/usr/bin/c++
CMAKE_CXX_FLAGS:STRING=
CMAKE_CXX_FLAGS_DEBUG:STRING=-g
CMAKE_CXX_FLAGS_MINSIZEREL:STRING=-Os -DNDEBUG
CMAKE_CXX_FLAGS_RELEASE:STRING=-O3 -DNDEBUG
CMAKE_CXX_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG
CMAKE_C_COMPILER:FILEPATH=/usr/bin/cc
CMAKE_C_FLAGS:STRING=
CMAKE_C_FLAGS_DEBUG:STRING=-g
CMAKE_C_FLAGS_MINSIZEREL:STRING=-Os -DNDEBUG
CMAKE_C_FLAGS_RELEASE:STRING=-O3 -DNDEBUG
CMAKE_C_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG
CMAKE_EXE_LINKER_FLAGS:STRING=
CMAKE_EXE_LINKER_FLAGS_DEBUG:STRING=
CMAKE_EXE_LINKER_FLAGS_MINSIZEREL:STRING=
CMAKE_EXE_LINKER_FLAGS_RELEASE:STRING=
CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO:STRING=
CMAKE_EXPORT_COMPILE_COMMANDS:BOOL=OFF
CMAKE_INSTALL_PREFIX:PATH=/home/dav/Desktop/xmr-stak/build
CMAKE_LINKER:FILEPATH=/usr/bin/ld
CMAKE_LINK_STATIC:BOOL=OFF
CMAKE_MAKE_PROGRAM:FILEPATH=/usr/bin/make
CMAKE_MODULE_LINKER_FLAGS:STRING=
CMAKE_MODULE_LINKER_FLAGS_DEBUG:STRING=
CMAKE_MODULE_LINKER_FLAGS_MINSIZEREL:STRING=
CMAKE_MODULE_LINKER_FLAGS_RELEASE:STRING=
CMAKE_MODULE_LINKER_FLAGS_RELWITHDEBINFO:STRING=
CMAKE_NM:FILEPATH=/usr/bin/nm
CMAKE_OBJCOPY:FILEPATH=/usr/bin/objcopy
CMAKE_OBJDUMP:FILEPATH=/usr/bin/objdump
CMAKE_RANLIB:FILEPATH=/usr/bin/ranlib
CMAKE_SHARED_LINKER_FLAGS:STRING=
CMAKE_SHARED_LINKER_FLAGS_DEBUG:STRING=
CMAKE_SHARED_LINKER_FLAGS_MINSIZEREL:STRING=
CMAKE_SHARED_LINKER_FLAGS_RELEASE:STRING=
CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO:STRING=
CMAKE_SKIP_INSTALL_RPATH:BOOL=NO
CMAKE_SKIP_RPATH:BOOL=NO
CMAKE_STATIC_LINKER_FLAGS:STRING=
CMAKE_STATIC_LINKER_FLAGS_DEBUG:STRING=
CMAKE_STATIC_LINKER_FLAGS_MINSIZEREL:STRING=
CMAKE_STATIC_LINKER_FLAGS_RELEASE:STRING=
CMAKE_STATIC_LINKER_FLAGS_RELWITHDEBINFO:STRING=
CMAKE_STRIP:FILEPATH=/usr/bin/strip
CMAKE_VERBOSE_MAKEFILE:BOOL=FALSE
CPU_ENABLE:BOOL=ON
CUDA_ENABLE:BOOL=OFF
HWLOC:FILEPATH=/usr/lib/x86_64-linux-gnu/libhwloc.so
HWLOC_ENABLE:BOOL=ON
HWLOC_INCLUDE_DIR:PATH=/usr/include
MHTD:FILEPATH=/usr/lib/x86_64-linux-gnu/libmicrohttpd.so
MICROHTTPD_ENABLE:BOOL=ON
MTHD_INCLUDE_DIR:PATH=/usr/include
OPENSSL_CRYPTO_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libcrypto.so
OPENSSL_INCLUDE_DIR:PATH=/usr/include
OPENSSL_SSL_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libssl.so
OpenCL_ENABLE:BOOL=ON
OpenCL_INCLUDE_DIR:PATH=/usr/include
OpenCL_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libOpenCL.so
OpenSSL_ENABLE:BOOL=ON
PKG_CONFIG_EXECUTABLE:FILEPATH=/usr/bin/pkg-config
XMR-STAK_COMPILE:STRING=native
XMR-STAK_CURRENCY:STRING=all

Issue with the execution

  • Do you compiled the miner by our own? YES
run `./xmr-stak --version-long` and add the output here

xmr-stak/2.1.0/31a0784/master/lin/amd-cpu/aeon-monero/10

AMD OpenCl issue

run `clinfo` and add the output here

CMAKE_AR:FILEPATH=/usr/bin/ar
CMAKE_BUILD_TYPE:STRING=Release
CMAKE_COLOR_MAKEFILE:BOOL=ON
CMAKE_CXX_COMPILER:FILEPATH=/usr/bin/c++
CMAKE_CXX_FLAGS:STRING=
CMAKE_CXX_FLAGS_DEBUG:STRING=-g
CMAKE_CXX_FLAGS_MINSIZEREL:STRING=-Os -DNDEBUG
CMAKE_CXX_FLAGS_RELEASE:STRING=-O3 -DNDEBUG
CMAKE_CXX_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG
CMAKE_C_COMPILER:FILEPATH=/usr/bin/cc
CMAKE_C_FLAGS:STRING=
CMAKE_C_FLAGS_DEBUG:STRING=-g
CMAKE_C_FLAGS_MINSIZEREL:STRING=-Os -DNDEBUG
CMAKE_C_FLAGS_RELEASE:STRING=-O3 -DNDEBUG
CMAKE_C_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG
CMAKE_EXE_LINKER_FLAGS:STRING=
CMAKE_EXE_LINKER_FLAGS_DEBUG:STRING=
CMAKE_EXE_LINKER_FLAGS_MINSIZEREL:STRING=
CMAKE_EXE_LINKER_FLAGS_RELEASE:STRING=
CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO:STRING=
CMAKE_EXPORT_COMPILE_COMMANDS:BOOL=OFF
CMAKE_INSTALL_PREFIX:PATH=/home/dav/Desktop/xmr-stak/build
CMAKE_LINKER:FILEPATH=/usr/bin/ld
CMAKE_LINK_STATIC:BOOL=OFF
CMAKE_MAKE_PROGRAM:FILEPATH=/usr/bin/make
CMAKE_MODULE_LINKER_FLAGS:STRING=
CMAKE_MODULE_LINKER_FLAGS_DEBUG:STRING=
CMAKE_MODULE_LINKER_FLAGS_MINSIZEREL:STRING=
CMAKE_MODULE_LINKER_FLAGS_RELEASE:STRING=
CMAKE_MODULE_LINKER_FLAGS_RELWITHDEBINFO:STRING=
CMAKE_NM:FILEPATH=/usr/bin/nm
CMAKE_OBJCOPY:FILEPATH=/usr/bin/objcopy
CMAKE_OBJDUMP:FILEPATH=/usr/bin/objdump
CMAKE_RANLIB:FILEPATH=/usr/bin/ranlib
CMAKE_SHARED_LINKER_FLAGS:STRING=
CMAKE_SHARED_LINKER_FLAGS_DEBUG:STRING=
CMAKE_SHARED_LINKER_FLAGS_MINSIZEREL:STRING=
CMAKE_SHARED_LINKER_FLAGS_RELEASE:STRING=
CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO:STRING=
CMAKE_SKIP_INSTALL_RPATH:BOOL=NO
CMAKE_SKIP_RPATH:BOOL=NO
CMAKE_STATIC_LINKER_FLAGS:STRING=
CMAKE_STATIC_LINKER_FLAGS_DEBUG:STRING=
CMAKE_STATIC_LINKER_FLAGS_MINSIZEREL:STRING=
CMAKE_STATIC_LINKER_FLAGS_RELEASE:STRING=
CMAKE_STATIC_LINKER_FLAGS_RELWITHDEBINFO:STRING=
CMAKE_STRIP:FILEPATH=/usr/bin/strip
CMAKE_VERBOSE_MAKEFILE:BOOL=FALSE
CPU_ENABLE:BOOL=ON
CUDA_ENABLE:BOOL=OFF
HWLOC:FILEPATH=/usr/lib/x86_64-linux-gnu/libhwloc.so
HWLOC_ENABLE:BOOL=ON
HWLOC_INCLUDE_DIR:PATH=/usr/include
MHTD:FILEPATH=/usr/lib/x86_64-linux-gnu/libmicrohttpd.so
MICROHTTPD_ENABLE:BOOL=ON
MTHD_INCLUDE_DIR:PATH=/usr/include
OPENSSL_CRYPTO_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libcrypto.so
OPENSSL_INCLUDE_DIR:PATH=/usr/include
OPENSSL_SSL_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libssl.so
OpenCL_ENABLE:BOOL=ON
OpenCL_INCLUDE_DIR:PATH=/usr/include
OpenCL_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libOpenCL.so
OpenSSL_ENABLE:BOOL=ON
PKG_CONFIG_EXECUTABLE:FILEPATH=/usr/bin/pkg-config
XMR-STAK_COMPILE:STRING=native
XMR-STAK_CURRENCY:STRING=all
dav@monolith:~/D/x/build$ bin/xmr-stak --version-long
Version: xmr-stak/2.1.0/31a0784/master/lin/amd-cpu/aeon-monero/10
dav@monolith:~/D/x/build$ clinfo
Number of platforms                               1
Platform Name                                   AMD Accelerated Parallel Processing
Platform Vendor                                 Advanced Micro Devices, Inc.
Platform Version                                OpenCL 2.0 AMD-APP (2482.3)
Platform Profile                                FULL_PROFILE
Platform Extensions                             cl_khr_icd cl_amd_event_callback cl_amd_offline_devices 
Platform Extensions function suffix             AMD

Platform Name                                   AMD Accelerated Parallel Processing
Number of devices                                 1
Device Name                                     Ellesmere
Device Vendor                                   Advanced Micro Devices, Inc.
Device Vendor ID                                0x1002
Device Version                                  OpenCL 1.2 AMD-APP (2482.3)
Driver Version                                  2482.3
Device OpenCL C Version                         OpenCL C 1.2 
Device Type                                     GPU
Device Profile                                  FULL_PROFILE
Device Board Name (AMD)                         Radeon RX 580 Series
Device Topology (AMD)                           PCI-E, 42:00.0
Max compute units                               36
SIMD per compute unit (AMD)                     4
SIMD width (AMD)                                16
SIMD instruction width (AMD)                    1
Max clock frequency                             1355MHz
Graphics IP (AMD)                               8.0
Device Partition                                (core)
  Max number of sub-devices                     36
  Supported partition types                     none specified
Max work item dimensions                        3
Max work item sizes                             256x256x256
Max work group size                             256
Preferred work group size multiple              64
Wavefront width (AMD)                           64
Preferred / native vector sizes                 
  char                                                 4 / 4       
  short                                                2 / 2       
  int                                                  1 / 1       
  long                                                 1 / 1       
  half                                                 1 / 1        (cl_khr_fp16)
  float                                                1 / 1       
  double                                               1 / 1        (cl_khr_fp64)
Half-precision Floating-point support           (cl_khr_fp16)
  Denormals                                     No
  Infinity and NANs                             No
  Round to nearest                              No
  Round to zero                                 No
  Round to infinity                             No
  IEEE754-2008 fused multiply-add               No
  Support is emulated in software               No
  Correctly-rounded divide and sqrt operations  No
Single-precision Floating-point support         (core)
  Denormals                                     No
  Infinity and NANs                             Yes
  Round to nearest                              Yes
  Round to zero                                 Yes
  Round to infinity                             Yes
  IEEE754-2008 fused multiply-add               Yes
  Support is emulated in software               No
  Correctly-rounded divide and sqrt operations  Yes
Double-precision Floating-point support         (cl_khr_fp64)
  Denormals                                     Yes
  Infinity and NANs                             Yes
  Round to nearest                              Yes
  Round to zero                                 Yes
  Round to infinity                             Yes
  IEEE754-2008 fused multiply-add               Yes
  Support is emulated in software               No
  Correctly-rounded divide and sqrt operations  No
Address bits                                    64, Little-Endian
Global memory size                              6114410496 (5.694GiB)
Global free memory (AMD)                        5952912 (5.677GiB)
Global memory channels (AMD)                    8
Global memory banks per channel (AMD)           16
Global memory bank width (AMD)                  256 bytes
Error Correction support                        No
Max memory allocation                           4244635648 (3.953GiB)
Unified memory for Host and Device              No
Minimum alignment for any data type             128 bytes
Alignment of base address                       2048 bits (256 bytes)
Global Memory cache type                        Read/Write
Global Memory cache size                        16384
Global Memory cache line                        64 bytes
Image support                                   Yes
  Max number of samplers per kernel             16
  Max size for 1D images from buffer            134217728 pixels
  Max 1D or 2D image array size                 2048 images
  Base address alignment for 2D image buffers   256 bytes
  Pitch alignment for 2D image buffers          256 bytes
  Max 2D image size                             16384x16384 pixels
  Max 3D image size                             2048x2048x2048 pixels
  Max number of read image args                 128
  Max number of write image args                8
Local memory type                               Local
Local memory size                               32768 (32KiB)
Local memory syze per CU (AMD)                  65536 (64KiB)
Local memory banks (AMD)                        32
Max constant buffer size                        4244635648 (3.953GiB)
Max number of constant args                     8
Max size of kernel argument                     1024
Queue properties                                
  Out-of-order execution                        No
  Profiling                                     Yes
Prefer user sync for interop                    Yes
Profiling timer resolution                      1ns
Profiling timer offset since Epoch (AMD)        1513064698294324448ns (Tue Dec 12 02:44:58 2017)
Execution capabilities                          
  Run OpenCL kernels                            Yes
  Run native kernels                            No
  Thread trace supported (AMD)                  Yes
  SPIR versions                                 1.2
printf() buffer size                            1048576 (1024KiB)
Built-in kernels                                
Device Available                                Yes
Compiler Available                              Yes
Linker Available                                Yes
Device Extensions                               cl_khr_fp64 cl_amd_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_fp16 cl_khr_gl_sharing cl_amd_device_attribute_query cl_amd_vec3 cl_amd_printf cl_amd_media_ops cl_amd_media_ops2 cl_amd_popcnt cl_khr_image2d_from_buffer cl_khr_spir cl_khr_gl_event 

NULL platform behavior
clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...)  No platform
clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...)   No platform
clCreateContext(NULL, ...) [default]            No platform
clCreateContext(NULL, ...) [other]              Success [AMD]
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU)  No platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU)  No platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR)  No platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM)  No platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL)  No platform

Stability issue

  • Is the CPU or GPU overclocked? NO (except stock overclock on GPU)
  • Is the Main memory of the CPU or GPU undervolted? NO
@fireice-uk

This comment has been minimized.

Show comment
Hide comment
@fireice-uk

fireice-uk Jan 3, 2018

Owner

Try "no_mlck" setting - performance should be identical to "never" unless you are on a low memory system.

Owner

fireice-uk commented Jan 3, 2018

Try "no_mlck" setting - performance should be identical to "never" unless you are on a low memory system.

@fireice-uk fireice-uk closed this Jan 8, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment