Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RPM M3] RPM package requirements #1548

Closed
Tracked by #27
peterzhuamazon opened this issue Jan 25, 2022 · 12 comments
Closed
Tracked by #27

[RPM M3] RPM package requirements #1548

peterzhuamazon opened this issue Jan 25, 2022 · 12 comments
Assignees
@peterzhuamazon
Labels
rpm

Comments

@peterzhuamazon
Copy link
Member

peterzhuamazon commented Jan 25, 2022 •
edited

Tasks Estimate Status Notes Dependencies
RPM package name should follow the binary file naming scheme %{name}-%{version}-%{release}.%{architecture}.rpm so we have a clear path for upgrade/re-release (e.g. opensearch-1.2.0-1.x86_64.rpm) 1 Complete The logic will be handled by Jenkinsfile, the assemble workflow will still output opensearch-1.2.0-linux-x64.rpm for now  Need Jenkinsfile to handle it during the build process, not handled by the assemble_workflow code base
RPM package should be set to be a un-relocatable package, so that users will not get confused and move the content to a different directory, which then cause the software to be unfunctional 2 Complete $ rpm -qip ~/sandbox/opensearch-1.2.0-linux-x64-20210110.rpmName : opensearchVersion : 1.2.0Release : 1Architecture: x86_64Install Date: (not installed)Group : defaultSize : 645433774License : ASL 2.0Signature : (none)Source RPM : opensearch-1.2.0-1.src.rpmBuild Date : Mon 10 Jan 2022 07:09:50 PM ESTBuild Host : dev-dsk-zhujiaxi-2a-5c9b3e5e.us-west-2.amazon.comRelocations : /Packager : OpenSearchVendor : OpenSearchURL : https://opensearch.org/Summary : opensearch rpm 1.2.0Description :opensearch rpm 1.2.0  As of 2017, FPM is having conflict to setup --prefix vs --config-files: jordansissel/fpm#931.

After talking to @spotaws I then manually remove PREFIX to set package to unlocatable. However, this requires --edit and manually tweak file. Will need to get a fork of FPM and remove the default pending of PREFIX then send a PR.

I find a workaround to provide "" to --prefix and now the package is not relocatable. Code in comments.
RPM package should have its dependencies pulled upon installation, without bundling them in the package 5 Complete This is a more detailed version of M1 task about dependencies, see comments  
RPM package will have pre/post install/remove scripts to handle min and plugin specific configurations 3 Complete    
RPM package should let rpm itself handle the creation and removal of user/group/dir during installation / removal, and remove any of these logic from the pre/post scripts 2 Complete Need to figure out what resource is needed and what is to be removed, see comments  
RPM package should be as slim as possible, without duplicate libs and related dependencies 3 Complete   Dependencies setup needs to function before we can remove duplicate libs

20220209 Update: Able to get dependency requirements running in rpm spec file

20220301: More issues debating whether make JDK a dependency a good idea, let alone nodejs is very hard to treat as a dependency due to node10 too old: #1682 & opensearch-project/OpenSearch#2302
RPM package should be installed in /opt directory to follow the LINUX standard, and add a symlink to the old directory presented in ODFE for backward compatibility 1 No Plan After evaluating, there are many dependencies to the folder location such as security plugin demo installer. We will keep using /usr/share/<> for now.   
RPM package should have no reference to Elasticsearch or Kibana 1 Complete    
RPM package should redirect its log to /var/log location per LINUX standard 1 Complete    
RPM package removal will not remove user data from the LINUX host 2 Complete   Depend on what resources are to be removed when uninstalling the package
RPM package should have a clear description of its content in metadata 1 Complete "--rpm-summary", "Open source distributed and RESTful search engine", "--description", "OpenSearch makes it easy to ingest, search, visualize, and analyze your data.\nReference document can be found at https://opensearch.org/docs/latest",  
RPM package should have a clear label of LICENSE file location, as well as the LICENSE type in its metadata. The LICENSE file should always present on the host once installed 1 Complete license will always present on the server  
RPM package should have its documentation files labeled as doc, so that users can choose whether they want to install the docs to save space 1 Complete docs mark for not install on server  
RPM package should contain process manager related service file, so user can manage the software with ease 3 Complete 1 for OpenSearch, 1 for Dashboards, 1 for Performance Analyzer

20220208 new issue: opensearch-project/OpenSearch#2073		 System must have systemd (standard for all the RH flavored distros we support now)
RPM package should not couple the opensearch/dashboards process with any additonal plugin services to reduce the complexity and improve the management 1 Complete PA can be started by user without starting OpenSearch  
RPM package should handle the initialization of certain plugins automatically 2 Complete   Pre/Post scripts need to be ready for this

Additional information:
@peterzhuamazon peterzhuamazon mentioned this issue Jan 25, 2022
Closed
22 tasks
@bbarani bbarani changed the title The RPM package should adhere to below standards and requirements RPM package requirements Jan 31, 2022
@peterzhuamazon
Copy link
Member Author

Need to set the package to not relocatable:

Relocations : (not relocatable)

@peterzhuamazon
Copy link
Member Author 

<% if !prefix.nil? and !prefix.empty? %>
Prefix: <%= prefix %>
<% end -%>

https://github.com/jordansissel/fpm/blob/master/templates/rpm.erb#L64-L66

Supply --prefix "" will allow package to be not relocatable.

@peterzhuamazon
Copy link
Member Author

In research on testing remove the post_remove.sh and verify whether the dir are removed.
Also needs to test post_install.sh once post_remove.sh removal does not affect dir removal.

@peterzhuamazon
Copy link
Member Author

Dependency runs:

Installing:
 opensearch                                                                         x86_64                                                       1.2.0-1                                                                             /opensearch-1.2.0-1.x86_64                                                       364 M
Installing for dependencies:
 java-latest-openjdk                                                                x86_64                                                       1:17.0.1.0.12-1.rolling.el7                                                         epel                                                                             210 k
 java-latest-openjdk-headless                                                       x86_64                                                       1:17.0.1.0.12-1.rolling.el7                                                         epel                                                                              40 M

@peterzhuamazon
Copy link
Member Author

peterzhuamazon commented Feb 8, 2022 •
edited 


Name        : opensearch
Version     : 1.2.0
Release     : 1
Architecture: x86_64
Install Date: (not installed)
Group       : Application/Internet
Size        : 381234594
License     : Apache-2.0
Signature   : (none)
Source RPM  : opensearch-1.2.0-1.src.rpm
Build Date  : Tue 08 Feb 2022 12:06:32 AM UTC
Build Host  : <>
Relocations : (not relocatable)
URL         : https://www.opensearch.org/
Summary     : An open source distributed and RESTful search engine
Description :
OpenSearch makes it easy to ingest, search, visualize, and analyze your data.
For more information, see: https://www.opensearch.org/

@peterzhuamazon
Copy link
Member Author 


{
  "name" : "",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "HQCFfEehRTeqUKY2gxxhjQ",
  "version" : {
    "distribution" : "opensearch",
    "number" : "1.2.0",
    "build_type" : "tar",
    "build_hash" : "c459282fd67ddb17dcc545ec9bcdc805880bcbec",
    "build_date" : "2021-11-22T16:57:18.360386Z",
    "build_snapshot" : false,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}


● opensearch.service - OpenSearch
   Loaded: loaded (/etc/systemd/system/opensearch.service; disabled; vendor preset: disabled)
   Active: active (running) since Wed 2022-02-09 01:24:21 UTC; 26s ago
     Docs: https://opensearch.org/
 Main PID: 12342 (java)
    Tasks: 80
   Memory: 1.3G
   CGroup: /system.slice/opensearch.service
           └─12342 /usr/share/opensearch/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMe...

@peterzhuamazon
Copy link
Member Author

Still need to find a better way to handle third party JDKs, as of now only bundled JDK run.

@peterzhuamazon
Copy link
Member Author

peterzhuamazon commented Feb 10, 2022 •
edited

some tests 

Is this ok [y/d/N]: y
Downloading packages:
java-15-amazon-corretto-devel-15.0.2.7-1.x86_64.rpm                                                                                                                                                                                                                                                  | 192 MB  00:00:03
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : 1:java-15-amazon-corretto-devel-15.0.2.7-1.x86_64                                                                                                                                                                                                                                                        1/2
  Installing : opensearch-1.2.0-1.x86_64                                                                                                                                                                                                                                                                                2/2
  Verifying  : 1:java-15-amazon-corretto-devel-15.0.2.7-1.x86_64                                                                                                                                                                                                                                                        1/2
  Verifying  : opensearch-1.2.0-1.x86_64                                                                                                                                                                                                                                                                                2/2

Installed:
  opensearch.x86_64 0:1.2.0-1

Dependency Installed:
  java-15-amazon-corretto-devel.x86_64 1:15.0.2.7-1

Complete!

% ll /var/log/opensearch/install_demo_configuration.log
-rw-r--r-- 1 opensearch opensearch 1385 Feb 10 00:10 /var/log/opensearch/install_demo_configuration.log

% cat /var/log/opensearch/install_demo_configuration.log
OpenSearch Security Demo Installer
 ** Warning: Do not use on production or public reachable systems **
Basedir: /usr/share/opensearch
This script maybe require your root password for 'sudo' privileges
OpenSearch install type: rpm/deb on NAME="Amazon Linux"
OpenSearch config dir: /etc/opensearch
OpenSearch config file: /etc/opensearch/opensearch.yml
OpenSearch bin dir: /usr/share/opensearch/bin
OpenSearch plugins dir: /usr/share/opensearch/plugins
OpenSearch lib dir: /usr/share/opensearch/lib
Detected OpenSearch Version: x-content-1.2.0
Detected OpenSearch Security Version: 1.2.0.0

### Success
### Execute this script now on all your nodes and then start all nodes
### OpenSearch Security will be automatically initialized.
### If you like to change the runtime configuration
### change the files in ../securityconfig and execute:
sudo "/usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh" -cd "/usr/share/opensearch/plugins/opensearch-security/securityconfig" -icl -key "/etc/opensearch/kirk-key.pem" -cert "/etc/opensearch/kirk.pem" -cacert "/etc/opensearch/root-ca.pem" -nhnv
### or run ./securityadmin_demo.sh
### To use the Security Plugin ConfigurationGUI
### To access your secured cluster open https://<hostname>:<HTTP port> and log in with admin/admin.
### (Ignore the SSL certificate warning because we installed self-signed demo certificates)

% sudo systemctl start opensearch

% sudo systemctl status opensearch
● opensearch.service - OpenSearch
   Loaded: loaded (/etc/systemd/system/opensearch.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2022-02-10 00:15:17 UTC; 4s ago
     Docs: https://opensearch.org/
 Main PID: 21236 (java)
    Tasks: 28
   Memory: 1.1G
   CGroup: /system.slice/opensearch.service
           └─21236 /usr/lib/jvm/java-15-amazon-corretto/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsIn...

Feb 10 00:15:17 <> systemd[1]: Started OpenSearch.
Feb 10 00:15:18 <> systemd-entrypoint[21236]: No passphrase file found, start opensearch without importing the passphrase

% ps -ef | grep java
opensea+ 21236     1 99 00:15 ?        00:00:26 /usr/lib/jvm/java-15-amazon-corretto/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms1g -Xmx1g -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-2127011330754106921 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/opensearch/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=536870912 -Dopensearch.path.home=/usr/share/opensearch -Dopensearch.path.conf=/etc/opensearch -Dopensearch.distribution.type=tar -Dopensearch.bundled_jdk=true -cp /usr/share/opensearch/lib/* org.opensearch.bootstrap.OpenSearch -p /var/run/opensearch/opensearch.pid --quiet


% curl https://localhost:9200 -u admin:admin --insecure
{
  "name" : "",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "86RCc961S6CIGc0H8MzY_w",
  "version" : {
    "distribution" : "opensearch",
    "number" : "1.2.0",
    "build_type" : "tar",
    "build_hash" : "c459282fd67ddb17dcc545ec9bcdc805880bcbec",
    "build_date" : "2021-11-22T16:57:18.360386Z",
    "build_snapshot" : false,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

% ll /var/log/opensearch
total 164
-rw-r--r-- 1 opensearch opensearch 45338 Feb 10 00:17 gc.log
-rw-r--r-- 1 opensearch opensearch  2041 Feb 10 00:15 gc.log.00
-rw-r--r-- 1 opensearch opensearch  1385 Feb 10 00:10 install_demo_configuration.log
-rw-r--r-- 1 opensearch opensearch   411 Feb 10 00:15 opensearch_deprecation.json
-rw-r--r-- 1 opensearch opensearch   588 Feb 10 00:15 opensearch_deprecation.log
-rw-r--r-- 1 opensearch opensearch     0 Feb 10 00:15 opensearch_index_indexing_slowlog.json
-rw-r--r-- 1 opensearch opensearch     0 Feb 10 00:15 opensearch_index_indexing_slowlog.log
-rw-r--r-- 1 opensearch opensearch     0 Feb 10 00:15 opensearch_index_search_slowlog.json
-rw-r--r-- 1 opensearch opensearch     0 Feb 10 00:15 opensearch_index_search_slowlog.log
-rw-r--r-- 1 opensearch opensearch 34731 Feb 10 00:16 opensearch.log
-rw-r--r-- 1 opensearch opensearch 61588 Feb 10 00:16 opensearch_server.json

% ll /var/run/opensearch
total 4
-rw-r--r-- 1 opensearch opensearch 5 Feb 10 00:15 opensearch.pid

% ll /var/lib/opensearch
total 20
-rw-r--r-- 1 opensearch opensearch    5 Feb 10 00:15 batch_metrics_enabled.conf
-rw-r--r-- 1 opensearch opensearch    5 Feb 10 00:15 logging_enabled.conf
drwxr-xr-x 3 opensearch opensearch 4096 Feb 10 00:15 nodes
-rw-r--r-- 1 opensearch opensearch    5 Feb 10 00:15 performance_analyzer_enabled.conf
-rw-r--r-- 1 opensearch opensearch    5 Feb 10 00:15 rca_enabled.conf

% ll /usr/share/opensearch
total 252
drwxr-xr-x  2 opensearch opensearch   4096 Feb 10 00:09 bin
lrwxrwxrwx  1 opensearch opensearch     19 Feb 10 00:09 data -> /var/lib/opensearch
drwxr-xr-x  3 opensearch opensearch   4096 Feb 10 00:09 lib
-rw-r--r--  1 opensearch opensearch  11358 Nov 22 16:55 LICENSE.txt
lrwxrwxrwx  1 opensearch opensearch     19 Feb 10 00:09 logs -> /var/log/opensearch
-rw-r--r--  1 opensearch opensearch   4043 Nov 22 17:13 manifest.yml
drwxr-xr-x 19 opensearch opensearch   4096 Feb 10 00:09 modules
-rw-r--r--  1 opensearch opensearch 215355 Nov 22 17:00 NOTICE.txt
drwxr-xr-x  6 opensearch opensearch   4096 Feb 10 00:09 performance-analyzer-rca
drwxr-xr-x 14 opensearch opensearch   4096 Feb 10 00:10 plugins
-rw-r--r--  1 opensearch opensearch   1761 Nov 22 16:55 README.md

@peterzhuamazon
Copy link
Member Author

20220218: PA issue block 1.3.0 build now.
opensearch-project/performance-analyzer#140

@peterzhuamazon
Copy link
Member Author

peterzhuamazon commented Feb 20, 2022 •
edited

20220219: RPM core extracted + install zips + rpmbuild to repack with all the requirements:

  • Notify runs well + rpm in the API responses textsa
  • Auto-redirect to /var/lib/opensearch and /var/log/opensearch
  • Using exact same file structure of the core rpm based setups
Installation + Verification Logs 

(22-02-20 2:26:27) <0> [~]
% rpm -qip opensearch-1.3.0-1.x86_64.rpm
Name        : opensearch
Version     : 1.3.0
Release     : 1
Architecture: x86_64
Install Date: (not installed)
Group       : Application/Internet
Size        : 332697768
License     : Apache-2.0
Signature   : (none)
Source RPM  : opensearch-1.3.0-1.src.rpm
Build Date  : Sun 20 Feb 2022 02:23:59 AM UTC
Build Host  : dev-dsk-zhujiaxi-2a-5c9b3e5e.us-west-2.amazon.com
Relocations : (not relocatable)
URL         : https://www.opensearch.org/
Summary     : An open source distributed and RESTful search engine
Description :
OpenSearch makes it easy to ingest, search, visualize, and analyze your data.
For more information, see: https://www.opensearch.org/

(22-02-20 2:26:31) <0> [~]
% sudo yum install opensearch-1.3.0-1.x86_64.rpm
Loaded plugins: priorities, remove-with-leaves
Examining opensearch-1.3.0-1.x86_64.rpm: opensearch-1.3.0-1.x86_64
Marking opensearch-1.3.0-1.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package opensearch.x86_64 0:1.3.0-1 will be installed
--> Processing Dependency: java-15-amazon-corretto-devel for package: opensearch-1.3.0-1.x86_64
amzn2-amazon                                                                                                                                                                                                                                                                                       | 3.1 kB  00:00:00
amzn2-core                                                                                                                                                                                                                                                                                         | 3.8 kB  00:00:00
amzn2-kernel                                                                                                                                                                                                                                                                                       | 3.8 kB  00:00:00
amzn2-mate                                                                                                                                                                                                                                                                                         | 2.7 kB  00:00:00
cloud-dev-dsk                                                                                                                                                                                                                                                                                      | 3.0 kB  00:00:00
dcv                                                                                                                                                                                                                                                                                                | 3.0 kB  00:00:00
firefox                                                                                                                                                                                                                                                                                            | 2.7 kB  00:00:00
1160 packages excluded due to repository priority protections
--> Running transaction check
---> Package java-15-amazon-corretto-devel.x86_64 1:15.0.2.7-1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==========================================================================================================================================================================================================================================================================================================================
 Package                                                                                Arch                                                            Version                                                                 Repository                                                                           Size
==========================================================================================================================================================================================================================================================================================================================
Installing:
 opensearch                                                                             x86_64                                                          1.3.0-1                                                                 /opensearch-1.3.0-1.x86_64                                                          317 M
Installing for dependencies:
 java-15-amazon-corretto-devel                                                          x86_64                                                          1:15.0.2.7-1                                                            AmazonCorretto                                                                      192 M

Transaction Summary
==========================================================================================================================================================================================================================================================================================================================
Install  1 Package (+1 Dependent package)

Total size: 510 M
Total download size: 192 M
Installed size: 649 M
Is this ok [y/d/N]: y
Downloading packages:
java-15-amazon-corretto-devel-15.0.2.7-1.x86_64.rpm                                                                                                                                                                                                                                                | 192 MB  00:00:04
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : 1:java-15-amazon-corretto-devel-15.0.2.7-1.x86_64                                                                                                                                                                                                                                                      1/2
  Installing : opensearch-1.3.0-1.x86_64                                                                                                                                                                                                                                                                              2/2
### NOT starting on installation, please execute the following statements to configure opensearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable opensearch.service
### You can start opensearch service by executing
 sudo systemctl start opensearch.service
### Created opensearch demo certificates in /etc/opensearch/certs
 See certs creation log in /var/log/opensearch/install_demo_configuration.log
  Verifying  : 1:java-15-amazon-corretto-devel-15.0.2.7-1.x86_64                                                                                                                                                                                                                                                      1/2
  Verifying  : opensearch-1.3.0-1.x86_64                                                                                                                                                                                                                                                                              2/2

Installed:
  opensearch.x86_64 0:1.3.0-1

Dependency Installed:
  java-15-amazon-corretto-devel.x86_64 1:15.0.2.7-1

Complete!

(22-02-20 2:27:17) <0> [~]
% ll /etc/opensearch/certs
total 28
-rw-r--r-- 1 opensearch opensearch 1704 Feb 20 02:27 esnode-key.pem
-rw-r--r-- 1 opensearch opensearch 1168 Feb 20 02:27 esnode.pem
-rw-r--r-- 1 opensearch opensearch 1704 Feb 20 02:27 kirk-key.pem
-rw-r--r-- 1 opensearch opensearch 1164 Feb 20 02:27 kirk.pem
-rw-r--r-- 1 opensearch opensearch 1675 Feb 20 02:27 root-ca-key.pem
-rw-r--r-- 1 opensearch opensearch 1281 Feb 20 02:27 root-ca.pem
-rw-r--r-- 1 opensearch opensearch   17 Feb 20 02:27 root-ca.srl

(22-02-20 2:27:24) <0> [~]
% cat /var/log/opensearch/install_demo_configuration.log
=========== OpenSearch Security Dev Configurations Installer ============
This script maybe require your root password for 'sudo' privileges

Checking if certificates exist in /etc/opensearch/certs/

#############################
Step 1: Generating self-signed certificates using OpenSSL
#############################

Creating certs/ directory...
Root certificates created at /etc/opensearch/certs/certs/root-ca.pem
Admin certificates created at /etc/opensearch/certs/certs/kirk.pem
Node certificates created at /etc/opensearch/certs/certs/esnode.pem

#############################
Step 2: Installing self-signed certificates
#############################


Certificates installed successfully! See updated settings in config/opensearch.yml

#############################
Step 3: Initialize security
#############################

OpenSearch Security will be automatically initialized.
If you like to change the runtime configuration
change the files in ../securityconfig and execute:
sudo "/usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh" -cd "/usr/share/opensearch/plugins/opensearch-security/securityconfig" -icl -key "/etc/opensearch/certs/kirk-key.pem" -cert "/etc/opensearch/certs/kirk.pem" -cacert "/etc/opensearch/certs/root-ca.pem" -nhnv
or run ./securityadmin_demo.sh
To use the Security Plugin ConfigurationGUI

#############################
Setup complete!
#############################

Single node setup successful! You can now start the cluster with bin/opensearch

To access your secured cluster open https://<hostname>:<HTTP port> and log in with admin/admin. Example
curl -k https://localhost:9200 -u admin:admin
(Ignore the SSL certificate warning because we installed self-signed certificates)

Follow the steps below to setup multi-node cluster
1. Execute this script now on all your nodes
2. Replace certificates on all other nodes with generated certificates on this node. The certificates can be found on path - /etc/opensearch/certs/
3. Start all nodes with bin/opensearch

For configuring language clients like Java, Python, Go, and tools like Logstash, Grafana, etc.,
please refer to the CLIENTS AND TOOLS documentation on https://opensearch.org/docs/latest

#############################
Disclaimer!
#############################

 ** Warning: Do not use on production or public reachable systems **
Self signed certificates installed via this tool are good for development or use within a trusted private network.
For any production use, we recommend using trusted CA provided certificates.

(22-02-20 2:27:29) <0> [~]
% sudo systemctl start opensearch.service

<Note: Notify runs correctly now and wait for around 20s for opensearch to properly starts up>

(22-02-20 2:27:50) <0> [~]
% sudo systemctl status opensearch.service
● opensearch.service - OpenSearch
   Loaded: loaded (/usr/lib/systemd/system/opensearch.service; disabled; vendor preset: disabled)
   Active: active (running) since Sun 2022-02-20 02:27:50 UTC; 9s ago
     Docs: https://opensearch.org/
 Main PID: 15212 (java)
    Tasks: 78
   Memory: 1.3G
   CGroup: /system.slice/opensearch.service
           └─15212 /usr/lib/jvm/java-15-amazon-corretto/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetails...

Feb 20 02:27:37  systemd[1]: Starting OpenSearch...
Feb 20 02:27:50  systemd[1]: Started OpenSearch.
Feb 20 02:27:50  systemd-entrypoint[15212]: Exception in thread "Attach Listener" Agent failed to start!

(22-02-20 2:27:59) <0> [~]
% curl https://localhost:9200 -u admin:admin --insecure
{
  "name" : "",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "YT9MkKnCQdiML4rlIyjAcQ",
  "version" : {
    "distribution" : "opensearch",
    "number" : "1.3.0",
    "build_type" : "rpm",
    "build_hash" : "54c7206d8cba6973d19a814f2705e35ae5cc66db",
    "build_date" : "2022-02-19T17:16:28.462002Z",
    "build_snapshot" : false,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

(22-02-20 2:28:31) <1> [~]
% curl https://localhost:9200/_cat/plugins -u admin:admin --insecure
 opensearch-alerting                  1.3.0.0
 opensearch-asynchronous-search       1.3.0.0
 opensearch-cross-cluster-replication 1.3.0.0
 opensearch-index-management          1.3.0.0
 opensearch-job-scheduler             1.3.0.0
 opensearch-knn                       1.3.0.0
 opensearch-observability             1.3.0.0
 opensearch-performance-analyzer      1.3.0.0
 opensearch-reports-scheduler         1.3.0.0
 opensearch-security                  1.3.0.0
 opensearch-sql                       1.3.0.0

(22-02-20 2:28:51) <1> [~]
% curl "https://localhost:9200/_cluster/health?pretty" -u admin:admin --insecure
{
  "cluster_name" : "opensearch",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "discovered_master" : true,
  "active_primary_shards" : 1,
  "active_shards" : 1,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

@peterzhuamazon peterzhuamazon changed the title RPM package requirements [RPM M3] RPM package requirements Feb 21, 2022
@peterzhuamazon peterzhuamazon mentioned this issue Feb 22, 2022
Closed
1 task
@peterzhuamazon
Copy link
Member Author

20220301: More issues debating whether make JDK a dependency a good idea, let alone nodejs is very hard to treat as a dependency due to node10 too old: #1682 & opensearch-project/OpenSearch#2302

@peterzhuamazon
Copy link
Member Author

Resolved by #1726

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@peterzhuamazon peterzhuamazon

Labels
rpm
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@peterzhuamazon