Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

don't automatically expose elasticsearch port (9200) unless explicitly configured to do so #38

Closed
mmguero opened this issue Mar 1, 2021 · 0 comments
Closed

don't automatically expose elasticsearch port (9200) unless explicitly configured to do so #38

mmguero opened this issue Mar 1, 2021 · 0 comments
Assignees
@mmguero
Labels
docker Relating to docker and docker-compose as used by Malcolm install.py Relating to the install.py configuration script nginx Relating to Malcolm's use of nginx opensearch Relating to Malcolm's use of OpenSearch

Comments

@mmguero
Copy link
Collaborator

mmguero commented Mar 1, 2021 •
edited
Loading

I'm implementing a change so that port 9200 isn't exposed automatically in docker-compose.yml, unless prompted to do so during configuration of install.py --configure.
@mmguero mmguero added docker Relating to docker and docker-compose as used by Malcolm opensearch Relating to Malcolm's use of OpenSearch install.py Relating to the install.py configuration script nginx Relating to Malcolm's use of nginx labels Mar 1, 2021
@mmguero mmguero self-assigned this Mar 1, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Mar 1, 2021
@mmguero
for idaholab#38, don't automatically open port 9200 for elasticsearch…
77050c1 
… unless configured to do so
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Mar 1, 2021
@mmguero
related to idaholab#38, whitelist htpdate
f89bc04
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Mar 1, 2021
@mmguero
related to idaholab#38, whitelist test connection
28ff966
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Mar 2, 2021
@mmguero
related to idaholab#38, remove user agent check. it's currently not s…
d3ef853 
…omething that elasticsearch.js (used by arkime viewer) sets and I'm not sure it's worth the effort at the moment
@mmguero mmguero closed this as completed Mar 2, 2021
@mmguero mmguero mentioned this issue Mar 2, 2021
Merged
mmguero added a commit to cisagov/Malcolm that referenced this issue Mar 3, 2021
@mmguero
v3.0.1 development (#161)
3589754 
Malcolm v3.0.1

- Version bumps
  - Open Distro for Elastic ([v1.13.0](https://github.com/opendistro-for-elasticsearch/opendistro-build/blob/main/release-notes/opendistro-for-elasticsearch-release-notes-1.13.0.md)), which adds the following functionality over the previous release
    - [Reporting](https://opendistro.github.io/for-elasticsearch-docs/docs/kibana/reporting/)
    - [Historical data anomaly detection](https://opendistro.github.io/for-elasticsearch-docs/docs/ad/#step-6-analyze-historical-data)
  - ODFE v1.13.0 is based on the Elastic components 7.10.2 ([elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/release-notes-7.10.2.html), [kibana](https://www.elastic.co/guide/en/kibana/current/release-notes-7.10.2.html), [logstash](https://www.elastic.co/guide/en/logstash/current/logstash-7-10-2.html), [beats](https://www.elastic.co/guide/en/beats/libbeat/master/release-notes-7.10.2.html))
  - Zeek [3.0.13](https://github.com/zeek/zeek/releases/tag/v3.0.13)
  - NGINX [1.19.7](https://nginx.org/en/CHANGES)
  - Alpine Linux [3.13](https://alpinelinux.org/posts/Alpine-3.13.0-released.html) Docker base layer
  - docker-compose [1.28.5](https://docs.docker.com/compose/release-notes/) in Malcolm installable ISO version
- Restored the [sankey visualization](https://github.com/uniberg/kbn_sankey_vis) which was temporarily removed in Malcolm v3.0.0 (although there are still a few minor cosmetic [issues](uniberg/kbn_sankey_vis#15) with it)
- Removed port 8443 for upload (now just use /upload over the regular HTTPS port)
- Fixed issue with ODFE email alerts not being able to use self-signed SMTP certificates by importing CA certs in `nginx/ca-trust` into the JDK trust store for Elasticsearch and Logstash (see idaholab#37)
- Don't expose the Elasticsearch 9200 by default, it must now be explicitly be enabled during `install.py -c` (see idaholab#38)
- For ISO-installed versions of Malcolm and Hedgehog Linux, populate `/etc/os-release` with information about the build/release version
- Populate user-agent for a few clients ([Arkime's moloch-capture](arkime/arkime#1615), some hedgehog test connection processes) so they're not just sent as blank when communicating with Malcolm
- Added Arkime link to Kibana dashboards' navigation pane
- Fix some issues in control script with older python3 versions (3.6.x) with `contextlib.nullcontext` not being available
- Fix suggestion for yum-based distributions to install python 3 requests via pip
mmguero added a commit that referenced this issue Mar 3, 2021
@mmguero
v3.0.1 development (#161)
0379153 
Malcolm v3.0.1

- Version bumps
  - Open Distro for Elastic ([v1.13.0](https://github.com/opendistro-for-elasticsearch/opendistro-build/blob/main/release-notes/opendistro-for-elasticsearch-release-notes-1.13.0.md)), which adds the following functionality over the previous release
    - [Reporting](https://opendistro.github.io/for-elasticsearch-docs/docs/kibana/reporting/)
    - [Historical data anomaly detection](https://opendistro.github.io/for-elasticsearch-docs/docs/ad/#step-6-analyze-historical-data)
  - ODFE v1.13.0 is based on the Elastic components 7.10.2 ([elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/release-notes-7.10.2.html), [kibana](https://www.elastic.co/guide/en/kibana/current/release-notes-7.10.2.html), [logstash](https://www.elastic.co/guide/en/logstash/current/logstash-7-10-2.html), [beats](https://www.elastic.co/guide/en/beats/libbeat/master/release-notes-7.10.2.html))
  - Zeek [3.0.13](https://github.com/zeek/zeek/releases/tag/v3.0.13)
  - NGINX [1.19.7](https://nginx.org/en/CHANGES)
  - Alpine Linux [3.13](https://alpinelinux.org/posts/Alpine-3.13.0-released.html) Docker base layer
  - docker-compose [1.28.5](https://docs.docker.com/compose/release-notes/) in Malcolm installable ISO version
- Restored the [sankey visualization](https://github.com/uniberg/kbn_sankey_vis) which was temporarily removed in Malcolm v3.0.0 (although there are still a few minor cosmetic [issues](uniberg/kbn_sankey_vis#15) with it)
- Removed port 8443 for upload (now just use /upload over the regular HTTPS port)
- Fixed issue with ODFE email alerts not being able to use self-signed SMTP certificates by importing CA certs in `nginx/ca-trust` into the JDK trust store for Elasticsearch and Logstash (see #37)
- Don't expose the Elasticsearch 9200 by default, it must now be explicitly be enabled during `install.py -c` (see #38)
- For ISO-installed versions of Malcolm and Hedgehog Linux, populate `/etc/os-release` with information about the build/release version
- Populate user-agent for a few clients ([Arkime's moloch-capture](arkime/arkime#1615), some hedgehog test connection processes) so they're not just sent as blank when communicating with Malcolm
- Added Arkime link to Kibana dashboards' navigation pane
- Fix some issues in control script with older python3 versions (3.6.x) with `contextlib.nullcontext` not being available
- Fix suggestion for yum-based distributions to install python 3 requests via pip
@mmguero mmguero mentioned this issue May 13, 2021
Merged
mmguero added a commit to cisagov/Malcolm that referenced this issue May 13, 2021
@mmguero
Malcolm v3.1.0 development (#165)
c09ce35 
* [Network analyzers](https://github.com/cisagov/malcolm#Protocols)
    - Added support for [EtherCAT](https://en.wikipedia.org/wiki/EtherCAT) ([ICS protocol](https://github.com/cisagov/icsnpp-ethercat))
    - Fixed and improved Spicy-based [LDAP analyzer](zeek/spicy-analyzers#56)
    - Detect VPN [protocols](https://github.com/zeek/spicy-analyzers/tree/main/analyzer/protocol) IPsec, OpenVPN and WireGuard

* New or improved
    - Updated many Kibana dashboards and added dashbaords for newly-supported network protocols
    - Improved output of debug logs from docker images
    - Many minor improvements to underlying system for ISO installations
    - **Massively** cut build time for Hedgehog ISO and Zeek Docker container by using .deb packages from released versions rather than building from source
    - During build, [install all Zeek plugins](https://github.com/cisagov/Malcolm/blob/master/shared/bin/zeek_install_plugins.sh) via zkg

* Version updates
    - **[Zeek](https://github.com/zeek/zeek/releases) v4.0.1**
    - [Spicy](https://github.com/zeek/spicy) v1.0.0
    - [Open Distro For Elasticsearch](https://opendistro.github.io/for-elasticsearch-docs/version-history/) v1.13.2
    - [Yara](https://github.com/VirusTotal/yara/releases) v4.1.0
    - [Capa](https://github.com/fireeye/capa/releases) v1.6.3
    - switch from centos:7 to [amazonlinux:2](https://hub.docker.com/_/amazonlinux) for base Docker image to build Kibana plugins
    - [stunnel](https://www.stunnel.org/NEWS.html) v5.59
    - [NGINX](https://nginx.org/) v1.20.0
    - [LLVM/clang](https://releases.llvm.org/11.0.1/docs/ReleaseNotes.html) toolchain v11
    - Flask-Cors v3.0.9 for Hedgehog kiosk interface (dependabot-flagged [security alert](https://nvd.nist.gov/vuln/detail/CVE-2020-25032))
    - latest updates of various Zeek plugins, system and python packages, etc.
    - all Python scripts updated to Python 3

* Bugs fixed
    - When LDAP authentication is used instead of BASIC authentication, show a landing page rather than a server error when attempting to browse to the local authentication management interface
    - Fixed a [regression bug](idaholab#42) where Malcolm fails to start correctly if not using UID/GID 1000:1000
    - [Don't automatically expose](idaholab#38) elasticsearch (and logstash) ports unless explicitly configured to do so
    - freshclam should update the clamav database [during docker image build](idaholab#39)
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mmguero mmguero

Labels
docker Relating to docker and docker-compose as used by Malcolm install.py Relating to the install.py configuration script nginx Relating to Malcolm's use of nginx opensearch Relating to Malcolm's use of OpenSearch
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mmguero