Server crashed at startup in CentOS 7.6 #981

lale314 · 2020-01-09T10:37:55Z

Welcome to Milvus!
Milvus Release version: v0.6.0, built at 2020-01-09 18:06.24, with OpenBLAS library.
You are using Milvus CPU edition
Last commit id: 4a116a6

Loading configuration from: conf/server_config.yaml
Initializing log config from: conf/log_config.conf
MinIO DISABLED !
Milvus server is going to shutdown ...
Milvus server exit...

The text was updated successfully, but these errors were encountered:

yhmo · 2020-01-09T10:53:28Z

Could you print the content of milvus-20-01-09-18:17-error.log and milvus-20-01-09-18:17-debug.log?

lale314 · 2020-01-09T12:28:24Z

Could you print the content of milvus-20-01-09-18:17-error.log and milvus-20-01-09-18:17-debug.log?

error log no content
tail -f logs/milvus-20-01-09-18:17-error.log

yhmo · 2020-01-10T01:42:05Z

Is seems crash at startup during initialize. Could you print the content of server_config.yaml?

lale314 · 2020-01-10T03:38:24Z

Is seems crash at startup during initialize. Could you print the content of server_config.yaml?

I just configured backend_url.

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.

version: 0.1

#----------------------+------------------------------------------------------------+------------+-----------------+
# Server Config        | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# address              | IP address that Milvus server monitors.                    | IP         | 0.0.0.0         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# port                 | Port that Milvus server monitors. Port range (1024, 65535) | Integer    | 19530           |
#----------------------+------------------------------------------------------------+------------+-----------------+
# deploy_mode          | Milvus deployment type:                                    | DeployMode | single          |
#                      |   single, cluster_readonly, cluster_writable               |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# time_zone            | Use UTC-x or UTC+x to specify a time zone.                 | Timezone   | UTC+8           |
#----------------------+------------------------------------------------------------+------------+-----------------+
server_config:
  address: 0.0.0.0
  port: 19530
  deploy_mode: single
  time_zone: UTC+8

#----------------------+------------------------------------------------------------+------------+-----------------+
# DataBase Config      | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# backend_url          | URL for metadata storage, using SQLite (for single server  | URL        | sqlite://:@:/   |
#                      | Milvus) or MySQL (for distributed cluster Milvus).         |            |                 |
#                      | Format: dialect://username:password@host:port/database     |            |                 |
#                      | Keep 'dialect://:@:/', 'dialect' can be either 'sqlite' or |            |                 |
#                      | 'mysql', replace other texts with real values.             |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# insert_buffer_size   | Buffer size used for data insertion.                       | Integer    | 1 (GB)          |
#                      | The sum of 'insert_buffer_size' and 'cpu_cache_capacity'   |            |                 |
#                      | must be less than system memory size.                      |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# preload_table        | A comma-separated list of table names that need to be pre- | StringList |                 |
#                      | loaded when Milvus server starts up.                       |            |                 |
#                      | '*' means preload all existing tables.                     |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
db_config:
  backend_url: mysql://milvus:1I3olSbg8UJNp7YGsOOr@192.168.31.56:3306/milvus
  insert_buffer_size: 1
  preload_table:

#----------------------+------------------------------------------------------------+------------+-----------------+
# Storage Config       | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# primary_path         | Primary directory used to save meta data, vector data and  | Path       | /var/lib/milvus |
#                      | index data.                                                |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# secondary_path       | A semicolon-separated list of secondary directories used   | Path       |                 |
#                      | to save vector data and index data.                        |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# minio_enable         | Enable MinIO storage or not.                               | Boolean    | false           |
#----------------------+------------------------------------------------------------+------------+-----------------+
# minio_address        | MinIO storage service IP address.                          | IP         | 127.0.0.1       |
#----------------------+------------------------------------------------------------+------------+-----------------+
# minio_port           | MinIO storage service port. Port range (1024, 65535)       | Integer    | 9000            |
#----------------------+------------------------------------------------------------+------------+-----------------+
# minio_access_key     | MinIO storage service access key.                          | String     | minioadmin      |
#----------------------+------------------------------------------------------------+------------+-----------------+
# minio_secret_key     | MinIO storage service secret key.                          | String     | minioadmin      |
#----------------------+------------------------------------------------------------+------------+-----------------+
# minio_bucket         | MinIO storage service bucket name.                         | String     | milvus-bucket   |
#----------------------+------------------------------------------------------------+------------+-----------------+
storage_config:
  primary_path: /tmp/milvus
  secondary_path:
  minio_enable: false
  minio_address: 127.0.0.1
  minio_port: 9000
  minio_access_key: minioadmin
  minio_secret_key: minioadmin
  minio_bucket: milvus-bucket

#----------------------+------------------------------------------------------------+------------+-----------------+
# Metric Config        | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# enable_monitor       | Enable monitoring function or not.                         | Boolean    | false           |
#----------------------+------------------------------------------------------------+------------+-----------------+
# address              | Pushgateway address                                        | IP         | 127.0.0.1       +
#----------------------+------------------------------------------------------------+------------+-----------------+
# port                 | Pushgateway port, port range (1024, 65535)                 | Integer    | 9091            |
#----------------------+------------------------------------------------------------+------------+-----------------+
metric_config:
  enable_monitor: false
  address: 127.0.0.1
  port: 9091

#----------------------+------------------------------------------------------------+------------+-----------------+
# Cache Config         | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# cpu_cache_capacity   | The size of CPU memory used for caching data for faster    | Integer    | 4 (GB)          |
#                      | query. The sum of 'cpu_cache_capacity' and                 |            |                 |
#                      | 'insert_buffer_size' must be less than system memory size. |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# cache_insert_data    | Whether to load inserted data into cache immediately for   | Boolean    | false           |
#                      | hot query. If want to simultaneously insert and query      |            |                 |
#                      | vectors, it's recommended to enable this config.           |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
cache_config:
  cpu_cache_capacity: 4
  cache_insert_data: false

#----------------------+------------------------------------------------------------+------------+-----------------+
# Engine Config        | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# use_blas_threshold   | A Milvus performance tuning parameter. This value will be  | Integer    | 1100            |
#                      | compared with 'nq' to decide if OpenBLAS should be used.   |            |                 |
#                      | If nq >= use_blas_threshold, OpenBLAS will be used, search |            |                 |
#                      | response times will be stable but the search speed will be |            |                 |
#                      | slower; if nq < use_blas_threshold, SSE will be used,      |            |                 |
#                      | search speed will be faster but search response times will |            |                 |
#                      | fluctuate.                                                 |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# gpu_search_threshold | A Milvus performance tuning parameter. This value will be  | Integer    | 1000            |
#                      | compared with 'nq' to decide if the search computation will|            |                 |
#                      | be executed on GPUs only.                                  |            |                 |
#                      | If nq >= gpu_search_threshold, the search computation will |            |                 |
#                      | be executed on GPUs only;                                  |            |                 |
#                      | if nq < gpu_search_threshold, the search computation will  |            |                 |
#                      | be executed on both CPUs and GPUs.                         |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
engine_config:
  use_blas_threshold: 1100
  gpu_search_threshold: 1000

#----------------------+------------------------------------------------------------+------------+-----------------+
# GPU Resource Config  | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# enable               | Enable GPU resources or not.                               | Boolean    | false           |
#----------------------+------------------------------------------------------------+------------+-----------------+
# cache_capacity       | The size of GPU memory per card used for cache.            | Integer    | 1 (GB)          |
#----------------------+------------------------------------------------------------+------------+-----------------+
# search_resources     | The list of GPU devices used for search computation.       | DeviceList | gpu0            |
#                      | Must be in format gpux.                                    |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# build_index_resources| The list of GPU devices used for index building.           | DeviceList | gpu0            |
#                      | Must be in format gpux.                                    |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
gpu_resource_config:
  enable: false
  cache_capacity: 1
  search_resources:
    - gpu0
  build_index_resources:
    - gpu0

#----------------------+------------------------------------------------------------+------------+-----------------+
# Tracing Config       | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# json_config_path     | Absolute path for tracing config file.                     | Path       |                 |
#                      | Leave it empty, a no-op tracer will be created.            |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
tracing_config:
  json_config_path:

yhmo · 2020-01-10T06:43:46Z

The global.log shows you are "Using SQLite", but the server_config.yaml backend_url is mysql, it is weird.
You compile milvus from source code right? Then you run milvus by the "start_server.sh" script from source code directory, the script use the server_config.yaml under the /milvus/conf folder. Could you double-check the /milvus/conf/server_config.yaml ?

lale314 · 2020-01-10T09:40:24Z

The global.log shows you are "Using SQLite", but the server_config.yaml backend_url is mysql, it is weird.
You compile milvus from source code right? Then you run milvus by the "start_server.sh" script from source code directory, the script use the server_config.yaml under the /milvus/conf folder. Could you double-check the /milvus/conf/server_config.yaml ?

I tried it twice, the first time using sqlite and the second time using mysql. The configuration using sqlite is the default. All failed. I recompiled the source from git, and the compilation was successful, but the startup still failed.

yhmo · 2020-01-11T03:36:23Z

What is your operating system? We recommend Ubuntu18.04. Before compiling the code, please run milvus/core/ubuntu_build_deps.sh firstly.
Also you can use C++ IDE(such as CLion) or GDB to debug milvus at startup, to investigate at which line it crashed.
I guess it may relate to your develop environment, our developers could not reproduce the crash.

lale314 · 2020-01-11T10:04:53Z

What is your operating system? We recommend Ubuntu18.04. Before compiling the code, please run milvus/core/ubuntu_build_deps.sh firstly.
Also you can use C++ IDE(such as CLion) or GDB to debug milvus at startup, to investigate at which line it crashed.
I guess it may relate to your develop environment, our developers could not reproduce the crash.

My operating system is CentOS Linux release 7.6.1810,and I ran core / centos7_build_deps.sh before compiling. I will try to debug it using gdb.

lale314 · 2020-01-12T03:21:17Z

What is your operating system? We recommend Ubuntu18.04. Before compiling the code, please run milvus/core/ubuntu_build_deps.sh firstly.
Also you can use C++ IDE(such as CLion) or GDB to debug milvus at startup, to investigate at which line it crashed.
I guess it may relate to your develop environment, our developers could not reproduce the crash.

GNU gdb (GDB) Red Hat Enterprise Linux 8.0.1-36.el7
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ../bin/milvus_server...done.
(gdb) r -c ../conf/server_config.yaml -l ../conf/log_config.conf
Starting program: milvus/core/milvus/bin/milvus_server -c ../conf/server_config.yaml -l ../conf/log_config.conf
Missing separate debuginfos, use: debuginfo-install glibc-2.17-292.el7.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

__  _________ _   ____  ______

/ |/ / / /| | / / / / / __/
/ /|/ // // /| |/ / // /\ \
// /////_/__/

Welcome to Milvus!
Milvus Release version: v0.6.0, built at 2020-01-10 17:17.26, with OpenBLAS library.
You are using Milvus CPU edition
Last commit id: 8acdb49

Loading configuration from: ../conf/server_config.yaml
Initializing log config from: ../conf/log_config.conf
MinIO DISABLED !
[New Thread 0x7fffef264700 (LWP 174308)]
[New Thread 0x7fffeea63700 (LWP 174309)]
[New Thread 0x7fffee262700 (LWP 174310)]
[New Thread 0x7fffe7fff700 (LWP 174311)]
[New Thread 0x7fffeda61700 (LWP 174312)]
[New Thread 0x7fffed260700 (LWP 174313)]
[New Thread 0x7fffeca5f700 (LWP 174314)]
[New Thread 0x7fffe77fe700 (LWP 174315)]

Thread 1 "milvus_server" received signal SIGSEGV, Segmentation fault.
0x0000000000596a9f in std::string::_Rep::_M_dispose(std::allocator const&) [clone .isra.121] [clone .part.122] ()
Missing separate debuginfos, use: debuginfo-install cyrus-sasl-lib-2.1.26-23.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-34.el7.x86_64 libcom_err-1.42.9-13.el7.x86_64 libcurl-7.29.0-54.el7_7.1.x86_64 libgcc-4.8.5-39.el7.x86_64 libgfortran-4.8.5-39.el7.x86_64 libgfortran4-8.3.1-2.1.1.el7.x86_64 libgomp-4.8.5-39.el7.x86_64 libidn-1.28-4.el7.x86_64 libquadmath-4.8.5-39.el7.x86_64 libselinux-2.5-14.1.el7.x86_64 libssh2-1.8.0-3.el7.x86_64 libstdc++-4.8.5-39.el7.x86_64 mariadb-libs-5.5.64-1.el7.x86_64 nspr-4.21.0-1.el7.x86_64 nss-3.36.0-7.el7_5.x86_64 nss-softokn-freebl-3.44.0-5.el7.x86_64 nss-util-3.44.0-3.el7.x86_64 openblas-serial-0.3.3-2.el7.x86_64 openldap-2.4.44-21.el7_6.x86_64 openssl-libs-1.0.2k-19.el7.x86_64 pcre-8.32-17.el7.x86_64 zlib-1.2.7-18.el7.x86_64
(gdb) bt
#0 0x0000000000596a9f in std::string::_Rep::_M_dispose(std::allocator const&) [clone .isra.121] [clone .part.122] ()
#1 0x000000000059a945 in milvus::engine::meta::MySQLMetaImpl::Initialize() ()
#2 0x000000000059b9dd in milvus::engine::meta::MySQLMetaImpl::MySQLMetaImpl(milvus::engine::DBMetaOptions const&, int const&) ()
#3 0x0000000000594f42 in milvus::engine::MetaFactory::Build(milvus::engine::DBMetaOptions const&, int const&) ()
#4 0x0000000000571dff in milvus::engine::DBImpl::DBImpl(milvus::engine::DBOptions const&) ()
#5 0x000000000056b47f in milvus::engine::DBFactory::Build(milvus::engine::DBOptions const&) ()
#6 0x00000000004ae0cb in milvus::server::DBWrapper::StartService() ()
#7 0x00000000004b0383 in milvus::server::Server::StartService() ()
#8 0x00000000004b1627 in milvus::server::Server::Start() ()
#9 0x000000000042356f in main ()
(gdb) c
Continuing.
Milvus server is going to shutdown ...
[Thread 0x7fffed260700 (LWP 174313) exited]
[Thread 0x7fffeda61700 (LWP 174312) exited]
[Thread 0x7fffe7fff700 (LWP 174311) exited]
[Thread 0x7fffef264700 (LWP 174308) exited]
[Thread 0x7fffeea63700 (LWP 174309) exited]
Milvus server exit...[Thread 0x7fffee262700 (LWP 174310) exited]

[Thread 0x7fffeca5f700 (LWP 174314) exited]
[Thread 0x7ffff7fc8980 (LWP 174304) exited]
[Inferior 1 (process 174304) exited with code 01]

yhmo · 2020-01-13T06:49:49Z

I do some test on CentOS docker container(We have public docker images for milvus build environment). Firstly pull CentOS images:

docker pull milvusdb/milvus-cpu-build-env:v0.7.0-centos7

then start a container from this image, and download milvus source code to compile. I use sqlite to run milvus, seems everything is ok.
The CentOS version is

CentOS Linux release 7.7.1908 (Core)

yhmo · 2020-01-13T07:04:40Z

Your crash point is in MySQl initialize phase. From the call stack I could not get the root cause of the crash.
I suggest you use MySQL docker image to run mysql service, firstly download image:

docker pull mysql:latest

Start MySQL container:

docker run -p 3306:3306 -e MYSQL_ROOT_PASSWORD=123456 -d mysql:latest

Login and create database for milvus:

mysql -h127.0.0.1 -uroot -p123456
create database milvus;

Change server_config.yaml to use mysql:

backend_url: mysql://root:123456@127.0.0.1:3306/milvus

Start milvus to test.

lale314 · 2020-01-13T10:13:58Z

Your crash point is in MySQl initialize phase. From the call stack I could not get the root cause of the crash.
I suggest you use MySQL docker image to run mysql service, firstly download image:
docker pull mysql:latest
Start MySQL container:
docker run -p 3306:3306 -e MYSQL_ROOT_PASSWORD=123456 -d mysql:latest
Login and create database for milvus:
mysql -h127.0.0.1 -uroot -p123456
create database milvus;
Change server_config.yaml to use mysql:
backend_url: mysql://root:123456@127.0.0.1:3306/milvus
Start milvus to test.

I tried and failed. Then I used the debug parameter to compile the source, and debugged with gdb.

Thread 1 "milvus_server" received signal SIGSEGV, Segmentation fault.
0x0000000000426464 in __gnu_cxx::__exchange_and_add (__mem=0xfffffffffffffffa, __val=-1) at /opt/rh/devtoolset-7/root/usr/include/c++/7/ext/atomicity.h:49
49 { return __atomic_fetch_add(__mem, __val, __ATOMIC_ACQ_REL); }
Missing separate debuginfos, use: debuginfo-install cyrus-sasl-lib-2.1.26-23.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-34.el7.x86_64 libcom_err-1.42.9-13.el7.x86_64 libcurl-7.29.0-54.el7_7.1.x86_64 libgcc-4.8.5-39.el7.x86_64 libgfortran-4.8.5-39.el7.x86_64 libgfortran4-8.3.1-2.1.1.el7.x86_64 libgomp-4.8.5-39.el7.x86_64 libidn-1.28-4.el7.x86_64 libquadmath-4.8.5-39.el7.x86_64 libselinux-2.5-14.1.el7.x86_64 libssh2-1.8.0-3.el7.x86_64 libstdc++-4.8.5-39.el7.x86_64 mariadb-libs-5.5.64-1.el7.x86_64 nspr-4.21.0-1.el7.x86_64 nss-3.36.0-7.el7_5.x86_64 nss-softokn-freebl-3.44.0-5.el7.x86_64 nss-util-3.44.0-3.el7.x86_64 openblas-serial-0.3.3-2.el7.x86_64 openldap-2.4.44-21.el7_6.x86_64 openssl-libs-1.0.2k-19.el7.x86_64 pcre-8.32-17.el7.x86_64 zlib-1.2.7-18.el7.x86_64
(gdb) bt
#0 0x0000000000426464 in __gnu_cxx::__exchange_and_add (__mem=0xfffffffffffffffa, __val=-1) at /opt/rh/devtoolset-7/root/usr/include/c++/7/ext/atomicity.h:49
#1 0x00000000004264fb in __gnu_cxx::__exchange_and_add_dispatch (__mem=0xfffffffffffffffa, __val=-1) at /opt/rh/devtoolset-7/root/usr/include/c++/7/ext/atomicity.h:82
#2 0x000000000042a63c in std::string::_Rep::_M_dispose (this=0xffffffffffffffea, __a=...) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/basic_string.h:3247
#3 0x0000000000428c70 in std::basic_string<char, std::char_traits, std::allocator >::~basic_string (this=0x7fffffffc890, __in_chrg=)
at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/basic_string.h:3621
#4 0x0000000000579634 in boost::filesystem::path::~path (this=0x7fffffffc890, __in_chrg=) at /usr/local/include/boost/filesystem/path.hpp:56
#5 0x00000000005f84a2 in milvus::engine::meta::MySQLMetaImpl::Initialize (this=0x13f8160) at /home/itouchtv/install_src/milvus/core/src/db/meta/MySQLMetaImpl.cpp:252
#6 0x00000000005f79cb in milvus::engine::meta::MySQLMetaImpl::MySQLMetaImpl (this=0x13f8160, options=..., mode=@0x7fffffffd838: 0) at /home/itouchtv/install_src/milvus/core/src/db/meta/MySQLMetaImpl.cpp:174
#7 0x00000000005f692b in __gnu_cxx::new_allocatormilvus::engine::meta::MySQLMetaImpl::construct<milvus::engine::meta::MySQLMetaImpl, milvus::engine::DBMetaOptions const&, int const&> (this=0x7fffffffcf97, __p=0x13f8160,
__args#0=..., __args#1=@0x7fffffffd838: 0) at /opt/rh/devtoolset-7/root/usr/include/c++/7/ext/new_allocator.h:136
#8 0x00000000005f66c2 in std::allocator_traits<std::allocatormilvus::engine::meta::MySQLMetaImpl >::construct<milvus::engine::meta::MySQLMetaImpl, milvus::engine::DBMetaOptions const&, int const&> (__a=...,
__p=0x13f8160, __args#0=..., __args#1=@0x7fffffffd838: 0) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/alloc_traits.h:475
#9 0x00000000005f62f0 in std::_Sp_counted_ptr_inplace<milvus::engine::meta::MySQLMetaImpl, std::allocatormilvus::engine::meta::MySQLMetaImpl, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<milvus::engine::DBMetaOptions const&, int const&> (this=0x13f8150, __a=...) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/shared_ptr_base.h:526
#10 0x00000000005f5efd in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<milvus::engine::meta::MySQLMetaImpl, std::allocatormilvus::engine::meta::MySQLMetaImpl, milvus::engine::DBMetaOptions const&, int const&> (this=0x7fffffffd2d8, __a=...) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/shared_ptr_base.h:637
#11 0x00000000005f5d3e in std::__shared_ptr<milvus::engine::meta::MySQLMetaImpl, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocatormilvus::engine::meta::MySQLMetaImpl, milvus::engine::DBMetaOptions const&, int const&> (this=0x7fffffffd2d0, __tag=..., __a=...) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/shared_ptr_base.h:1295
#12 0x00000000005f5c59 in std::shared_ptrmilvus::engine::meta::MySQLMetaImpl::shared_ptr<std::allocatormilvus::engine::meta::MySQLMetaImpl, milvus::engine::DBMetaOptions const&, int const&> (this=0x7fffffffd2d0,
__tag=..., __a=...) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/shared_ptr.h:344
#13 0x00000000005f5a8b in std::allocate_shared<milvus::engine::meta::MySQLMetaImpl, std::allocatormilvus::engine::meta::MySQLMetaImpl, milvus::engine::DBMetaOptions const&, int const&> (__a=..., __args#0=...,
__args#1=@0x7fffffffd838: 0) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/shared_ptr.h:691
#14 0x00000000005f58d2 in std::make_shared<milvus::engine::meta::MySQLMetaImpl, milvus::engine::DBMetaOptions const&, int const&> (__args#0=..., __args#1=@0x7fffffffd838: 0)
at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/shared_ptr.h:707
#15 0x00000000005f5242 in milvus::engine::MetaFactory::Build (metaOptions=..., mode=@0x7fffffffd838: 0) at /home/itouchtv/install_src/milvus/core/src/db/meta/MetaFactory.cpp:65
#16 0x00000000005c22ba in milvus::engine::DBImpl::DBImpl (this=0x13f6de0, options=...) at /home/itouchtv/install_src/milvus/core/src/db/DBImpl.cpp:72
#17 0x00000000005c1bbf in __gnu_cxx::new_allocatormilvus::engine::DBImpl::construct<milvus::engine::DBImpl, milvus::engine::DBOptions const&> (this=0x7fffffffd587, __p=0x13f6de0, __args#0=...)
at /opt/rh/devtoolset-7/root/usr/include/c++/7/ext/new_allocator.h:136
#18 0x00000000005c1ab8 in std::allocator_traits<std::allocatormilvus::engine::DBImpl >::construct<milvus::engine::DBImpl, milvus::engine::DBOptions const&> (__a=..., __p=0x13f6de0, __args#0=...)
at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/alloc_traits.h:475
#19 0x00000000005c1925 in std::_Sp_counted_ptr_inplace<milvus::engine::DBImpl, std::allocatormilvus::engine::DBImpl, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<milvus::engine::DBOptions const&> (this=0x13f6dd0,
__a=...) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/shared_ptr_base.h:526
#20 0x00000000005c1595 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<milvus::engine::DBImpl, std::allocatormilvus::engine::DBImpl, milvus::engine::DBOptions const&> (this=0x7fffffffd718, __a=...)
at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/shared_ptr_base.h:637
#21 0x00000000005c1402 in std::__shared_ptr<milvus::engine::DBImpl, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocatormilvus::engine::DBImpl, milvus::engine::DBOptions const&> (this=0x7fffffffd710, __tag=...,
__a=...) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/shared_ptr_base.h:1295
#22 0x00000000005c133d in std::shared_ptrmilvus::engine::DBImpl::shared_ptr<std::allocatormilvus::engine::DBImpl, milvus::engine::DBOptions const&> (this=0x7fffffffd710, __tag=..., __a=...)
at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/shared_ptr.h:344
#23 0x00000000005c10f9 in std::allocate_shared<milvus::engine::DBImpl, std::allocatormilvus::engine::DBImpl, milvus::engine::DBOptions const&> (__a=..., __args#0=...)
at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/shared_ptr.h:691
#24 0x00000000005c0e1b in std::make_shared<milvus::engine::DBImpl, milvus::engine::DBOptions const&> (__args#0=...) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/shared_ptr.h:707
#25 0x00000000005c0973 in milvus::engine::DBFactory::Build (options=...) at /home/itouchtv/install_src/milvus/core/src/db/DBFactory.cpp:44
---Type to continue, or q to quit---
#26 0x00000000004c8d7f in milvus::server::DBWrapper::StartService (this=0x13953c0 milvus::server::DBWrapper::GetInstance()::wrapper) at /home/itouchtv/install_src/milvus/core/src/server/DBWrapper.cpp:165
#27 0x00000000004cc652 in milvus::server::Server::StartService (this=0x1395320 milvus::server::Server::GetInstance()::server) at /home/itouchtv/install_src/milvus/core/src/server/Server.cpp:265
#28 0x00000000004cbf7a in milvus::server::Server::Start (this=0x1395320 milvus::server::Server::GetInstance()::server) at /home/itouchtv/install_src/milvus/core/src/server/Server.cpp:204
#29 0x00000000005b956a in main (argc=5, argv=0x7fffffffddd8) at /home/itouchtv/install_src/milvus/core/src/main.cpp:142

yhmo · 2020-01-14T02:32:19Z

Please check your boost library version. Typically the boost library is installed in /usr/lib64 for CentOS. We require boost library version not less than 1.53.
To install boost 1.53 in CentOS:

yum install boost
yum install boost-devel

lale314 · 2020-01-14T07:36:20Z

Please check your boost library version. Typically the boost library is installed in /usr/lib64 for CentOS. We require boost library version not less than 1.53.
To install boost 1.53 in CentOS:
yum install boost
yum install boost-devel

I found two boost installed on my machine, 1.53 on / usr / lib64 and 1.67 on / usr / local / lib, I moved that 1.67 and recompiled the source code, finally run successfully.Thanks!

yhmo · 2020-01-14T08:12:31Z

Great!

lale314 changed the title ~~服务一启动一挂了~~ 服务一启动就挂了 Jan 9, 2020

yhmo added the help wanted Extra attention is needed label Jan 10, 2020

yhmo changed the title ~~服务一启动就挂了~~ Server crashed at startup Jan 10, 2020

yhmo changed the title ~~Server crashed at startup~~ Server crashed at startup in CentOS 7.6 Jan 13, 2020

yhmo closed this as completed Jan 14, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Server crashed at startup in CentOS 7.6 #981

Server crashed at startup in CentOS 7.6 #981

lale314 commented Jan 9, 2020

yhmo commented Jan 9, 2020

lale314 commented Jan 9, 2020

yhmo commented Jan 10, 2020

lale314 commented Jan 10, 2020 •

edited by yhmo

yhmo commented Jan 10, 2020 •

edited

lale314 commented Jan 10, 2020

yhmo commented Jan 11, 2020 •

edited

lale314 commented Jan 11, 2020

lale314 commented Jan 12, 2020

yhmo commented Jan 13, 2020

yhmo commented Jan 13, 2020 •

edited

lale314 commented Jan 13, 2020

yhmo commented Jan 14, 2020 •

edited

lale314 commented Jan 14, 2020

yhmo commented Jan 14, 2020

Server crashed at startup in CentOS 7.6 #981

Server crashed at startup in CentOS 7.6 #981

Comments

lale314 commented Jan 9, 2020

yhmo commented Jan 9, 2020

lale314 commented Jan 9, 2020

yhmo commented Jan 10, 2020

lale314 commented Jan 10, 2020 • edited by yhmo

yhmo commented Jan 10, 2020 • edited

lale314 commented Jan 10, 2020

yhmo commented Jan 11, 2020 • edited

lale314 commented Jan 11, 2020

lale314 commented Jan 12, 2020

yhmo commented Jan 13, 2020

yhmo commented Jan 13, 2020 • edited

lale314 commented Jan 13, 2020

yhmo commented Jan 14, 2020 • edited

lale314 commented Jan 14, 2020

yhmo commented Jan 14, 2020

lale314 commented Jan 10, 2020 •

edited by yhmo

yhmo commented Jan 10, 2020 •

edited

yhmo commented Jan 11, 2020 •

edited

yhmo commented Jan 13, 2020 •

edited

yhmo commented Jan 14, 2020 •

edited