The LME ./deploy.sh script hangs on "Waiting for Elasticsearch to be ready" #33

aubenasa · 2023-11-13T21:39:52Z

🐛 Summary

What's wrong? Please be specific.
The execution of the script "deploy.sh" does not go to the end and hangs on the error message " Waiting for Elasticsearch to be ready"

To reproduce

Steps to reproduce the behavior:

Do this: install a Linux Ubuntu 22. 04. 3 LTS (Codename: Jammy)
configure a static IP address on the server
Open the appropriate ports on the server with ufw : Inbound TCP 5044; Inbound TCP 443; Inbound TCP 22 for SSH management
Then this: run the script provided for the LME implementation:

Install Git client to be able to clone the LME repository

sudo apt update
sudo apt install git -y

Download a copy of the LME files

sudo git clone https://github.com/cisagov/lme.git /opt/lme/

Change to the LME directory containing files for the Linux server

cd /opt/lme/Chapter\ 3\ Files/

Execute script with root privileges

sudo ./deploy.sh install

Expected behavior

What did you expect to happen that didn't? I was expecting the script to run till the end and give me a screen like this after successful installation of Elasticsearch:
##################################################################################

Kibana/Elasticsearch Credentials are (these will not be accessible again!!!!)

Web Interface login:

elastic:

System Credentials

kibana:

logstash_system:

logstash_writer:

dashboard_update:

##################################################################################

Any helpful log output or screenshots

Paste the results here:
Creating network lme_esnet
Creating service lme_elasticsearch
Creating service lme_kibana
Creating service lme_logstash
Waiting for elasticsearch to be ready

Add any screenshots of the problem here.

Add logging to indicate the script's progress and where it might be failing + introduce a maximum number of 60 attempts to check for Elasticsearch readiness, preventing the script from hanging indefinitely.

Update deploy.sh to debug issue #33

llwaterhouse · 2023-11-20T21:26:23Z

The update to stop the script from hanging has been included in release v 1.1.0

* Adding updates to troubleshooting to address the latest issues. * Added Filtering.md to documents to discuss how to filter out unnessecary logs * Change "activate selected" to "Enable" * Changed "New - User Security" to "User Security" to reflect current dashboard name * Updated dashboard count and location * Adds a script to export dashboards * Adding Compute Software Overview dashboard * User HR Dashboard Ready for Review and Release * Bump Elasticsearch Version * Bump version in readme * adding alert dashboard (#46) Co-authored-by: Diabe <0743724407@HQ.DHS.GOV> Co-authored-by: Michael Reeves <147089975+mreeve-snl@users.noreply.github.com> * Add a command to allow the execution of the winlogbeat.exe file (#38) Co-authored-by: Clint Baxley <clint.baxley-ctr@ecstech.com> * add process_explorer.ndjson file (#37) Co-authored-by: root <root@LS1.524zunprk23u3ery524odj1xdc.dx.internal.cloudapp.net> Co-authored-by: Connor <107427279+causand22@users.noreply.github.com> * Creating Initial Draft of issue templates (#34) * Creating Initial Draft of issue templates Issue Templates to aid with docs_update * Update bug-or-error-report.md * Update bug-or-error-report.md Minor typos * Proofread bug-or-error-report.md, updated phrasing in some places --------- Co-authored-by: Chad Poland <128160399+Chad-CISA@users.noreply.github.com> Co-authored-by: Linda Waterhouse <82845774+llwaterhouse@users.noreply.github.com> Co-authored-by: mitchelbaker-cisa <mitchel.baker@cisa.dhs.gov> * remove input controls and update filtering with Kibana Control filters for (hostname, process exe, process pid) * Alert Dashboard review (#49) * adding alert dashboard * Create Alerting_dashboard.ndjson * Rename Alerting_dasboard.ndjson to Alerting_dashboard.ndjson * Rename Alerting_dashboard.ndjson to alert_dashboard.ndjson * Delete dasboards directory --------- Co-authored-by: Diabe <0743724407@HQ.DHS.GOV> Co-authored-by: Michael Reeves <147089975+mreeve-snl@users.noreply.github.com> * Delete dasboards directory (#50) * Update deploy.sh to debug issue #33 Add logging to indicate the script's progress and where it might be failing + introduce a maximum number of 60 attempts to check for Elasticsearch readiness, preventing the script from hanging indefinitely. * Updates the dashboard menu and all of the dashboards that use it. (#53) * Change the navigation menu to exclude the old home page and include the new dashboards. * Delete the security dashboard home --------- Co-authored-by: Clint Baxley <clint.baxley-ctr@ecstech.com> * Lme update functionality (#30) * adding updates to chapter3 for deploy.sh changes * adding updates to dashboard and lme_update to log and run as better cron jobs * adding in more notes to chapter3 on update functionality * Added the following features to deploy.sh: - update function to add lme_upadte.sh and dashboard_update.sh to root's crontab - fixed final permissions so that /opt/lme is readable by `sudo` group - y/n on the uninstall options fixed - upgrade function updated to check for 1.0 version and only remove crontab in upgrading from 0.5.1 - usage function to print the usage * fixing read/write on the files_for_windows.zip * fixing backups permissions * Update chapter3.md (#29) * Update chapter3.md Changed winlogbeat 8.5.0 link to one, that allows user to download not only zip, but also sha512 control sum and also choose between zip and MSI. * Update chapter3.md Changed Winlogbeat to 8.11.1 * Update the readmes to delete old dashboards and import new ones. (#54) Co-authored-by: Clint Baxley <clint.baxley-ctr@ecstech.com> * Update Uninstall_Sysmon64.ps1 (#27) Check if Sysmon is installed, run the uninstall command with elevated privileges, and handle potential errors. Remove the Sysmon executable if the uninstallation is successful. * Deploy upgrade 1.1.0 (#58) * adding in upgrade command to go from 1.0 -> 1.1.0 * pushing upgrade notes * adding updates to deploy.sh for upgrading 1.0 -> 1.1.0 * adding CONTRIBUTING.md,RELEASES.md, and Custom PR-Template (#41) * adding Contribution and release documentation to help standardize these processes * Update CONTRIBUTING.md fixed typos. * documenting PR template to standardize and streamline Pull Requests * adding a few more changes * adding formatting changes * Rename pull_request_template.md to pull_request_template.md Actually renamed directory PULL_REUQEST_TEMPLATE to PULL_REQUEST_TEMPLATE --------- Co-authored-by: Linda Waterhouse <82845774+llwaterhouse@users.noreply.github.com> * remove updates that break the installation process, need more refactoring/testing before we can push these changes * Release 1.1.0 small updates (#61) * updating deploy.sh with fixes that solve permissions issues and still provide security for files with plaintext passwords * updating docs to state more accurate required disk sizes --------- Co-authored-by: Alden Hilton <106177711+adhilto@users.noreply.github.com> Co-authored-by: Clint Baxley <clint.baxley-ctr@ecstech.com> Co-authored-by: Connor Aubry <caubry@sandia.gov> Co-authored-by: Grant (SNL) <108766839+rgbrow1949@users.noreply.github.com> Co-authored-by: Clint Baxley <c.baxley-ctr@ecstech.com> Co-authored-by: ddiabe <133152385+ddiabe@users.noreply.github.com> Co-authored-by: Diabe <0743724407@HQ.DHS.GOV> Co-authored-by: mitchelbaker-cisa <149098823+mitchelbaker-cisa@users.noreply.github.com> Co-authored-by: root <root@LS1.524zunprk23u3ery524odj1xdc.dx.internal.cloudapp.net> Co-authored-by: Connor <107427279+causand22@users.noreply.github.com> Co-authored-by: Chad Poland <128160399+Chad-CISA@users.noreply.github.com> Co-authored-by: Linda Waterhouse <82845774+llwaterhouse@users.noreply.github.com> Co-authored-by: mitchelbaker-cisa <mitchel.baker@cisa.dhs.gov> Co-authored-by: Dmytro Korzhevin <dkorzhevin@gmail.com>

aubenasa · 2023-11-27T17:08:17Z

Hello, Thank for the update regarding the fix of the LME script hanging. I tried again many times to rerun the script but still I'm getting the same error "Waiting for Elasticsearch to be ready". I may be submitting a new issue ticket. Sincerely, ***@***.***<http://www.dallascitynews.net/> Aubenas D Akpaca AOT Senior Security Analyst City of Dallas|DallasCityNews.net Department of Aviation 8008 Herb Kelleher Way/Rm 4021A Dallas, TX 75235 Cell: 214.677.6851 ***@***.******@***.***> [twitter icon] <https://twitter.com/CityofDallas> [facebook icon] <https://www.facebook.com/DallasCityHall/> [youtube icon] <https://www.youtube.com/user/dmcclel> ***@***.*** From: Linda Waterhouse ***@***.***> Sent: Monday, November 20, 2023 3:27 PM To: cisagov/LME ***@***.***> Cc: Akpaca, Aubenas ***@***.***>; Author ***@***.***> Subject: Re: [cisagov/LME] The LME ./deploy.sh script hangs on "Waiting for Elasticsearch to be ready" (Issue #33) External Email! The update to stop the script from hanging has been included in release v 1.1.0 - Reply to this email directly, view it on GitHub<#33 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BD6AROUKGGRMPOP5MRFVNDLYFPDITAVCNFSM6AAAAAA7JZ2EEWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJZHAZDOOJRHA>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>> CAUTION: This email originated from outside of the organization. Please, do not click links or open attachments unless you recognize the sender and know the content is safe.

llwaterhouse · 2023-11-27T17:13:44Z

The release will be coming out tomorrow. The script will exit after a certain number of attempts.

However, just because the script exits doesn't mean there aren't underlying issues that still need to be resolved. Please reference our troubleshooting.md doc.

llwaterhouse · 2023-11-30T22:37:10Z

If you are still having issues after upgrading to the new release, please open a new issue. Thank you.

dkorzhevin mentioned this issue Nov 17, 2023

Update deploy.sh to debug issue #33 #52

Merged

12 tasks

mreeve-snl added a commit that referenced this issue Nov 20, 2023

Merge pull request #52 from dkorzhevin/patch-4

b28983b

Update deploy.sh to debug issue #33

llwaterhouse closed this as completed Nov 20, 2023

llwaterhouse reopened this Nov 27, 2023

llwaterhouse closed this as completed Nov 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The LME ./deploy.sh script hangs on "Waiting for Elasticsearch to be ready" #33

The LME ./deploy.sh script hangs on "Waiting for Elasticsearch to be ready" #33

aubenasa commented Nov 13, 2023

llwaterhouse commented Nov 20, 2023

aubenasa commented Nov 27, 2023 via email

llwaterhouse commented Nov 27, 2023

llwaterhouse commented Nov 30, 2023

The LME ./deploy.sh script hangs on "Waiting for Elasticsearch to be ready" #33

The LME ./deploy.sh script hangs on "Waiting for Elasticsearch to be ready" #33

Comments

aubenasa commented Nov 13, 2023

🐛 Summary

To reproduce

Install Git client to be able to clone the LME repository

Download a copy of the LME files

Change to the LME directory containing files for the Linux server

Execute script with root privileges

Expected behavior

Kibana/Elasticsearch Credentials are (these will not be accessible again!!!!)

Web Interface login:

elastic:

System Credentials

kibana:

logstash_system:

logstash_writer:

dashboard_update:

Any helpful log output or screenshots

llwaterhouse commented Nov 20, 2023

aubenasa commented Nov 27, 2023 via email

llwaterhouse commented Nov 27, 2023

llwaterhouse commented Nov 30, 2023