Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

arkime start log #555

Closed
alleniverson33 opened this issue Aug 29, 2024 · 5 comments
Closed

arkime start log #555

alleniverson33 opened this issue Aug 29, 2024 · 5 comments
Labels
bug Something isn't working

Comments

@alleniverson33
Copy link

Describe the bug
arkime start log no have Express server listening on port 8005 in development mode why?

To Reproduce
Steps to reproduce the behavior:

  1. ./script/start -f ../.kube/config

**Screenshots and/or Logs **
arkimr container log
Waiting for WISE to start
WISE is running!

Initializing OpenSearch database...
WARNING OpenSearch/Elasticsearch health is 'yellow' instead of 'green', things may be broken

There is 1 OpenSearch/Elasticsearch data node, if you expect more please fix first before proceeding.

This is a fresh Arkime install
Erasing

Malcolm Version:

  • Version [e.g. v23.08.1]

How are you running Malcolm?
k8s
@alleniverson33 alleniverson33 added the bug Something isn't working label Aug 29, 2024
@mmguero
Copy link
Collaborator

mmguero commented Aug 29, 2024

I've just tested this on two kubernetes instances and one ISO-installed version of Malcolm:

arkime | usermod: no changes
arkime | root
arkime | uid=0(root) gid=0(root) groups=0(root)
arkime | 2024-08-29 13:41:16,083 INFO Set uid to user 0 succeeded
arkime | 2024-08-29 13:41:16,089 INFO RPC interface 'supervisor' initialized
arkime | 2024-08-29 13:41:16,089 CRIT Server 'unix_http_server' running without any HTTP authentication checking
arkime | 2024-08-29 13:41:16,090 INFO supervisord started with pid 1065
arkime | 2024-08-29 13:41:17,093 INFO spawned: 'initialize' with pid 1101
arkime | 2024-08-29 13:41:17,095 INFO spawned: 'pcap-arkime' with pid 1102
arkime | 2024-08-29 13:41:17,097 INFO spawned: 'viewer' with pid 1103
arkime | 2024-08-29 13:41:17,099 INFO spawned: 'wise' with pid 1104
arkime | 2024-08-29 13:41:18,101 INFO success: initialize entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
arkime | 2024-08-29 13:41:18,101 INFO success: viewer entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
arkime | 2024-08-29 13:41:18,101 INFO success: wise entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
arkime | 2024-08-29 13:41:32,117 INFO success: pcap-arkime entered RUNNING state, process has stayed up for > than 15 seconds (startsecs)
arkime | 2024-08-29 13:42:26 URL:https://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.csv [23323/23323] -> "ipv4-address-space.csv_new" [1]
arkime | 2024-08-29 13:42:27 URL:https://www.wireshark.org/download/automated/data/manuf [2812753/2812753] -> "oui.txt_new" [1]
arkime | Giving opensearch-local time to start...
arkime | opensearch-local is up and healthy at http://opensearch:9200
arkime | opensearch-local is running!
arkime | Giving WISE time to start...
arkime | Launch wise...
arkime | curl: (7) Failed to connect to 127.0.0.1 port 8081 after 0 ms: Couldn't connect to server
arkime | Waiting for WISE to start
arkime | curl: (7) Failed to connect to 127.0.0.1 port 8081 after 0 ms: Couldn't connect to server
arkime | Waiting for WISE to start
arkime | [[13:42:34.566]] [LOG]   /opt/arkime/wiseService/wiseService.js listening on host 0.0.0.0 port 8081 in development mode
arkime | WISE is running!
arkime |
arkime | Initializing opensearch-local database...
arkime | This is a fresh Arkime install
arkime | Erasing
arkime | Creating
arkime | Finished
arkime | Creating default user...
arkime | WARNING - Using authMode=header since not set, add to config file to silence this warning.
arkime | Added
arkime | Initializing fields...
arkime | Initializing views...
arkime | Creating view "Arkime Sessions"
arkime | Creating view "Public IP Addresses"
arkime | Creating view "Suricata Alerts"
arkime | Creating view "Suricata Logs"
arkime | Creating view "Uninventoried Internal Assets"
arkime | Creating view "Uninventoried Observed Services"
arkime | Creating view "Zeek conn.log"
arkime | Creating view "Zeek Exclude conn.log"
arkime | Creating view "Zeek Logs"
arkime | Setting defaults...
arkime |
arkime | opensearch-local database initialized!
arkime |
arkime | {"_shards":{"total":17,"successful":17,"failed":0}}2024-08-29 13:43:32,762 INFO exited: initialize (exit status 0; expected)
arkime | Launch viewer...
arkime | WARNING - Using authMode=header since not set, add to config file to silence this warning.
arkime | SECURITY WARNING - when userNameHeader is set, viewHost should be localhost or use iptables
arkime | /opt/arkime/viewer/viewer.js listening on host 0.0.0.0 port 8005 in development mode
arkime | This node will process Periodic Queries (CRON) & Hunts, delayed by 85 seconds

And have shelled into the container to verify that the API port 8005 is actually responding:

root@arkime-deployment-8548c658bf-htzkq:/opt/arkime# curl -sSLk -XGET -H 'Content-type: application/json' -H "http_auth_http_user: user" -H "Authorization:" -H "Accept: application/json; indent=4" https://localhost:8005/api/eshealth | jq
{
  "cluster_name": "docker-cluster",
  "status": "green",
  "timed_out": false,
  "number_of_nodes": 1,
  "number_of_data_nodes": 1,
  "discovered_master": true,
  "discovered_cluster_manager": true,
  "active_primary_shards": 18,
  "active_shards": 18,
  "relocating_shards": 0,
  "initializing_shards": 0,
  "unassigned_shards": 0,
  "delayed_unassigned_shards": 0,
  "number_of_pending_tasks": 0,
  "number_of_in_flight_fetch": 0,
  "task_max_waiting_in_queue_millis": 0,
  "active_shards_percent_as_number": 100,
  "version": "2.16.0",
  "molochDbVersion": 80
}

This is all unchanged from the way it's been in the previous version.

@mmguero mmguero closed this as not planned Won't fix, can't repro, duplicate, stale Aug 29, 2024
@alleniverson33
Copy link
Author

I've just tested this on two kubernetes instances and one ISO-installed version of Malcolm:

arkime | usermod: no changes
arkime | root
arkime | uid=0(root) gid=0(root) groups=0(root)
arkime | 2024-08-29 13:41:16,083 INFO Set uid to user 0 succeeded
arkime | 2024-08-29 13:41:16,089 INFO RPC interface 'supervisor' initialized
arkime | 2024-08-29 13:41:16,089 CRIT Server 'unix_http_server' running without any HTTP authentication checking
arkime | 2024-08-29 13:41:16,090 INFO supervisord started with pid 1065
arkime | 2024-08-29 13:41:17,093 INFO spawned: 'initialize' with pid 1101
arkime | 2024-08-29 13:41:17,095 INFO spawned: 'pcap-arkime' with pid 1102
arkime | 2024-08-29 13:41:17,097 INFO spawned: 'viewer' with pid 1103
arkime | 2024-08-29 13:41:17,099 INFO spawned: 'wise' with pid 1104
arkime | 2024-08-29 13:41:18,101 INFO success: initialize entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
arkime | 2024-08-29 13:41:18,101 INFO success: viewer entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
arkime | 2024-08-29 13:41:18,101 INFO success: wise entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
arkime | 2024-08-29 13:41:32,117 INFO success: pcap-arkime entered RUNNING state, process has stayed up for > than 15 seconds (startsecs)
arkime | 2024-08-29 13:42:26 URL:https://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.csv [23323/23323] -> "ipv4-address-space.csv_new" [1]
arkime | 2024-08-29 13:42:27 URL:https://www.wireshark.org/download/automated/data/manuf [2812753/2812753] -> "oui.txt_new" [1]
arkime | Giving opensearch-local time to start...
arkime | opensearch-local is up and healthy at http://opensearch:9200
arkime | opensearch-local is running!
arkime | Giving WISE time to start...
arkime | Launch wise...
arkime | curl: (7) Failed to connect to 127.0.0.1 port 8081 after 0 ms: Couldn't connect to server
arkime | Waiting for WISE to start
arkime | curl: (7) Failed to connect to 127.0.0.1 port 8081 after 0 ms: Couldn't connect to server
arkime | Waiting for WISE to start
arkime | [[13:42:34.566]] [LOG]   /opt/arkime/wiseService/wiseService.js listening on host 0.0.0.0 port 8081 in development mode
arkime | WISE is running!
arkime |
arkime | Initializing opensearch-local database...
arkime | This is a fresh Arkime install
arkime | Erasing
arkime | Creating
arkime | Finished
arkime | Creating default user...
arkime | WARNING - Using authMode=header since not set, add to config file to silence this warning.
arkime | Added
arkime | Initializing fields...
arkime | Initializing views...
arkime | Creating view "Arkime Sessions"
arkime | Creating view "Public IP Addresses"
arkime | Creating view "Suricata Alerts"
arkime | Creating view "Suricata Logs"
arkime | Creating view "Uninventoried Internal Assets"
arkime | Creating view "Uninventoried Observed Services"
arkime | Creating view "Zeek conn.log"
arkime | Creating view "Zeek Exclude conn.log"
arkime | Creating view "Zeek Logs"
arkime | Setting defaults...
arkime |
arkime | opensearch-local database initialized!
arkime |
arkime | {"_shards":{"total":17,"successful":17,"failed":0}}2024-08-29 13:43:32,762 INFO exited: initialize (exit status 0; expected)
arkime | Launch viewer...
arkime | WARNING - Using authMode=header since not set, add to config file to silence this warning.
arkime | SECURITY WARNING - when userNameHeader is set, viewHost should be localhost or use iptables
arkime | /opt/arkime/viewer/viewer.js listening on host 0.0.0.0 port 8005 in development mode
arkime | This node will process Periodic Queries (CRON) & Hunts, delayed by 85 seconds

And have shelled into the container to verify that the API port 8005 is actually responding:

root@arkime-deployment-8548c658bf-htzkq:/opt/arkime# curl -sSLk -XGET -H 'Content-type: application/json' -H "http_auth_http_user: user" -H "Authorization:" -H "Accept: application/json; indent=4" https://localhost:8005/api/eshealth | jq
{
  "cluster_name": "docker-cluster",
  "status": "green",
  "timed_out": false,
  "number_of_nodes": 1,
  "number_of_data_nodes": 1,
  "discovered_master": true,
  "discovered_cluster_manager": true,
  "active_primary_shards": 18,
  "active_shards": 18,
  "relocating_shards": 0,
  "initializing_shards": 0,
  "unassigned_shards": 0,
  "delayed_unassigned_shards": 0,
  "number_of_pending_tasks": 0,
  "number_of_in_flight_fetch": 0,
  "task_max_waiting_in_queue_millis": 0,
  "active_shards_percent_as_number": 100,
  "version": "2.16.0",
  "molochDbVersion": 80
}

This is all unchanged from the way it's been in the previous version.

Yes, I have successfully deployed it before, but when I encountered this issue with stopping and restarting, how many attempts did I make to stop and restart before being able to recover

@mmguero
Copy link
Collaborator

mmguero commented Aug 30, 2024

If that's the case, I guess check your Kubernetes events and debug it on that side. Nothing changed in this last release of Malcolm with regards to how it deploys with Kubernetes.

@alleniverson33
Copy link
Author

If that's the case, I guess check your Kubernetes events and debug it on that side. Nothing changed in this last release of Malcolm with regards to how it deploys with Kubernetes.

I'll try again, thank you

@alleniverson33
Copy link
Author

If that's the case, I guess check your Kubernetes events and debug it on that side. Nothing changed in this last release of Malcolm with regards to how it deploys with Kubernetes.

There is another issue that I couldn't find the index 'arkime_stissions3-240830' in opensearch today after starting yesterday. This may be due to some reasons？

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
Malcolm
Status: Invalid
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mmguero @alleniverson33 and others