Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Malcolm instance landing page #252

Closed
mmguero opened this issue Sep 5, 2023 · 9 comments
Closed

Malcolm instance landing page #252

mmguero opened this issue Sep 5, 2023 · 9 comments
Assignees
Labels
enhancement New feature or request nginx Relating to Malcolm's use of nginx UI Relating to general UI experience
Milestone

Comments

@mmguero
Copy link
Collaborator

mmguero commented Sep 5, 2023

In previous version of Malcolm hitting the "root" URI of the instance (e.g., https://localhost/ if Malcolm is running locally) takes you to Arkime (the /sessions page, specifically).

I think it would be a better idea to design a landing page for Malcolm that is a jumping-off place for the various components of Malcolm. This could be a simple static page, but should look attractive and modern and ought to fit in as best as possible other elements of the Malcolm UI. The links it ought to contain include:

  • local Malcolm components
    • README (/readme)
    • Arkime (/sessions)
    • Dashboards (/dashboards)
    • Artifact Upload (/upload)
    • NetBox (/netbox)
    • Local Account Management (/auth)
    • CyberChef (/cyberchef/)
    • API (/mapi, not sure if we care to include this or not?)
  • external links
    • GitHub project
    • GitHub Issue tracker

Pretty much all the work for this is going to be in nginx.conf, where you can see the Arkime stuff is currently the default. It'll be changing the default / location to point to the new landing page and identifying all of the arkime locations (sessions, connections, spiview, etc.) and making sure they all go to Arkime as planned. Or, alternately, change it so there's an /arkime/ part of the UI and have everything under there (/arkime/sessions, /arkime/spiview, etc.).

We also have to determine what's hosting the page. Rather than another docker container, it seems to me it would make sense just to have the nginx-proxy container do it like it's doing for the readme.

@mmguero mmguero added enhancement New feature or request nginx Relating to Malcolm's use of nginx UI Relating to general UI experience labels Sep 5, 2023
@n8hacks
Copy link
Collaborator

n8hacks commented Sep 25, 2023

The best solution seems to have this sit within the Nginx container itself.

@n8hacks n8hacks self-assigned this Sep 25, 2023
@n8hacks
Copy link
Collaborator

n8hacks commented Oct 5, 2023

Here is what I am thinking. I'd love feedback. It is dark mode by default. Also when you hover over a tile it goes opaque.

Image

@mmguero
Copy link
Collaborator Author

mmguero commented Oct 6, 2023

I like the aesthetic of it but I think it's going to need to have some text describing what things are. If this is where people are first going to land the first time they run Malcolm they're going to look at it and be like "what is all of this."

Maybe those same icons, but like, in rows with some text describing each thing? Sort or like:

  • Arkime - Visualizations of Zeek logs, Suricata alerts, and Arkime sessions including access to full packet payloads
  • Dashboards - Prebuilt and user-definable dashboards for Zeek logs, Suricata alerts, and third-party logs, as well as anomaly detection, alerting, and reporting.
  • Upload - ...

etc.

@n8hacks
Copy link
Collaborator

n8hacks commented Oct 16, 2023

That's a great idea! What do you think of this. It pops up with the additional text when you hover over the box. If we want to move towards rows, we can do that too.

Image

@e101sg
Copy link

e101sg commented Oct 17, 2023

Looks good! Wish there is FAQ as well. Some of the Do's /Do not or common doubts can be there in a Text file.

@mmguero
Copy link
Collaborator Author

mmguero commented Oct 23, 2023

I think that's going in the right direction, @n8hacks . Let's keep on that track and see how it's received and we can tweak in the future if needed.

@mmguero mmguero added the CISA label Nov 13, 2023
@mmguero mmguero added this to the v23.12.0 milestone Nov 14, 2023
@mmguero mmguero modified the milestones: v23.12.1, v24.01.0 Dec 5, 2023
@mmguero mmguero self-assigned this Jan 2, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 2, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 2, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 2, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 2, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 2, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 2, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 2, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 2, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 2, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 2, 2024
@mmguero
Copy link
Collaborator Author

mmguero commented Jan 4, 2024

Screenshot 2024-01-04 at 10-44-21 Landing Page - Start Bootstrap Theme

working on a new design

mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 4, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 4, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 4, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 4, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 4, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 5, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 5, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 5, 2024
@mmguero
Copy link
Collaborator Author

mmguero commented Jan 9, 2024

TODO: add the malcolm version/build information to the landing page

mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 9, 2024
@mmguero
Copy link
Collaborator Author

mmguero commented Jan 10, 2024

Image

@mmguero mmguero closed this as completed Jan 10, 2024
This was referenced Jan 17, 2024
mmguero added a commit that referenced this issue Jan 17, 2024
Malcolm v24.01.0 contains new features, improvements, bug fixes and component version updates.

v23.12.1...v24.0.1

* Features and enhancements
    + new Malcolm instance landing page (#252)
    + file carve download with password-protected .zip file (#288)
    + new "all files exept common plain text files" option for Malcolm's file carving to match Hedgehog capability (#290)
    + allow customizing indexes for logs written to OpenSearch/Elasticsearch (#313)
    + more consistently differentiate between uploaded and live-captured traffic (#321)
    + make download extracted file context item from Arkime smarter (#330)
    + improve netbox device type library import by using "official" import script (#384)
* Component version updates
    + Alpine Linux to [v3.19](https://alpinelinux.org/posts/Alpine-3.19.0-released.html) as the base for some Docker images
    + Fluent Bit to [v2.2.2](https://github.com/fluent/fluent-bit/releases/tag/v2.2.2)
    + Beats to [v8.11.4](https://www.elastic.co/guide/en/beats/libbeat/8.11/release-notes-8.11.4.html)
    + LogStash to [v8.11.4](https://www.elastic.co/guide/en/logstash/current/logstash-8-11-4.html)
* Bug fixes
    + Suricata Alerts dashboard "Alerts - Tags" visualization is useless (#314)
    + third party logs are not parsed correctly from fluentbit -> fluentd aggregator -> Malcolm (#318)
    + update document lookup APIs to search either network or host data (#322)
    + suricata rule update is broken (#323)
    + time sync from hedgehog to Malcolm opensearch instance not working (#324)
    + fix issue specifying database mode via command-line
    + have pruning of OpenSearch indices (based on size) include "other" Malcolm indices as well (e.g., nginx logs, system resources, third-party logs, etc.)
* Configuration changes (in [environment variables](https://idaholab.github.io/Malcolm/docs/malcolm-config.html#MalcolmConfigEnvVars) in [`./config/`](https://github.com/idaholab/Malcolm/tree/v24.0.1/config))
    + added the following variables with relation to #313
        - added `ARKIME_ROTATE_INDEX` to [`arkime.env`](https://github.com/idaholab/Malcolm/tree/v24.0.1/arkime.env.example) with default value of `daily` (see [Arkime docs on rotateIndex](https://arkime.com/settings#rotateIndex))
        - added the following variables and defaults to [`opensearch.env`](https://github.com/idaholab/Malcolm/tree/v24.0.1/opensearch.env.example):
        ```
        # OpenSearch index patterns and timestamp fields
        # Index pattern for network traffic logs written via Logstash (e.g., Zeek logs, Suricata alerts)
        MALCOLM_NETWORK_INDEX_PATTERN=arkime_sessions3-*
        # Default time field to use for network traffic logs in Logstash and Dashboards
        MALCOLM_NETWORK_INDEX_TIME_FIELD=firstPacket
        # Suffix used to create index to which network traffic logs are written (supports Ruby strftime strings in %{})
        MALCOLM_NETWORK_INDEX_SUFFIX=%{%y%m%d}
        # Index pattern for other logs written via Logstash (e.g., nginx, beats, fluent-bit, etc.)
        MALCOLM_OTHER_INDEX_PATTERN=malcolm_beats_*
        # Default time field to use for other logs in Logstash and Dashboards
        MALCOLM_OTHER_INDEX_TIME_FIELD=@timestamp
        # Suffix used to create index to which other logs are written (supports Ruby strftime strings in %{})
        MALCOLM_OTHER_INDEX_SUFFIX=%{%y%m%d}
        # Index pattern used specifically by Arkime (will probably match MALCOLM_NETWORK_INDEX_PATTERN, should probably be arkime_sessions3-*)
        ARKIME_NETWORK_INDEX_PATTERN=arkime_sessions3-*
        # Default time field used by for sessions in Arkime viewer
        ARKIME_NETWORK_INDEX_TIME_FIELD=firstPacket
        ```
    + changed default for `EXTRACTED_FILE_HTTP_SERVER_KEY` to `infected` in [`zeek-secret.env`](https://github.com/idaholab/Malcolm/tree/v24.0.1/zeek-secret.env.example)
    + added `EXTRACTED_FILE_HTTP_SERVER_ZIP` with default value of `false` in [`zeek.env`](https://github.com/idaholab/Malcolm/tree/v24.0.1/zeek.env.example), see (#288)
mmguero added a commit to cisagov/Malcolm that referenced this issue Jan 17, 2024
Malcolm v24.01.0 contains new features, improvements, bug fixes and component version updates.

v23.12.1...v24.0.1

* Features and enhancements
    + new Malcolm instance landing page (idaholab#252)
    + file carve download with password-protected .zip file (idaholab#288)
    + new "all files exept common plain text files" option for Malcolm's file carving to match Hedgehog capability (idaholab#290)
    + allow customizing indexes for logs written to OpenSearch/Elasticsearch (idaholab#313)
    + more consistently differentiate between uploaded and live-captured traffic (idaholab#321)
    + make download extracted file context item from Arkime smarter (idaholab#330)
    + improve netbox device type library import by using "official" import script (idaholab#384)
* Component version updates
    + Alpine Linux to [v3.19](https://alpinelinux.org/posts/Alpine-3.19.0-released.html) as the base for some Docker images
    + Fluent Bit to [v2.2.2](https://github.com/fluent/fluent-bit/releases/tag/v2.2.2)
    + Beats to [v8.11.4](https://www.elastic.co/guide/en/beats/libbeat/8.11/release-notes-8.11.4.html)
    + LogStash to [v8.11.4](https://www.elastic.co/guide/en/logstash/current/logstash-8-11-4.html)
* Bug fixes
    + Suricata Alerts dashboard "Alerts - Tags" visualization is useless (idaholab#314)
    + third party logs are not parsed correctly from fluentbit -> fluentd aggregator -> Malcolm (idaholab#318)
    + update document lookup APIs to search either network or host data (idaholab#322)
    + suricata rule update is broken (idaholab#323)
    + time sync from hedgehog to Malcolm opensearch instance not working (idaholab#324)
    + fix issue specifying database mode via command-line
    + have pruning of OpenSearch indices (based on size) include "other" Malcolm indices as well (e.g., nginx logs, system resources, third-party logs, etc.)
* Configuration changes (in [environment variables](https://idaholab.github.io/Malcolm/docs/malcolm-config.html#MalcolmConfigEnvVars) in [`./config/`](https://github.com/cisagov/Malcolm/tree/v24.0.1/config))
    + added the following variables with relation to idaholab#313
        - added `ARKIME_ROTATE_INDEX` to [`arkime.env`](https://github.com/cisagov/Malcolm/tree/v24.0.1/arkime.env.example) with default value of `daily` (see [Arkime docs on rotateIndex](https://arkime.com/settings#rotateIndex))
        - added the following variables and defaults to [`opensearch.env`](https://github.com/cisagov/Malcolm/tree/v24.0.1/opensearch.env.example):
        ```
        # OpenSearch index patterns and timestamp fields
        # Index pattern for network traffic logs written via Logstash (e.g., Zeek logs, Suricata alerts)
        MALCOLM_NETWORK_INDEX_PATTERN=arkime_sessions3-*
        # Default time field to use for network traffic logs in Logstash and Dashboards
        MALCOLM_NETWORK_INDEX_TIME_FIELD=firstPacket
        # Suffix used to create index to which network traffic logs are written (supports Ruby strftime strings in %{})
        MALCOLM_NETWORK_INDEX_SUFFIX=%{%y%m%d}
        # Index pattern for other logs written via Logstash (e.g., nginx, beats, fluent-bit, etc.)
        MALCOLM_OTHER_INDEX_PATTERN=malcolm_beats_*
        # Default time field to use for other logs in Logstash and Dashboards
        MALCOLM_OTHER_INDEX_TIME_FIELD=@timestamp
        # Suffix used to create index to which other logs are written (supports Ruby strftime strings in %{})
        MALCOLM_OTHER_INDEX_SUFFIX=%{%y%m%d}
        # Index pattern used specifically by Arkime (will probably match MALCOLM_NETWORK_INDEX_PATTERN, should probably be arkime_sessions3-*)
        ARKIME_NETWORK_INDEX_PATTERN=arkime_sessions3-*
        # Default time field used by for sessions in Arkime viewer
        ARKIME_NETWORK_INDEX_TIME_FIELD=firstPacket
        ```
    + changed default for `EXTRACTED_FILE_HTTP_SERVER_KEY` to `infected` in [`zeek-secret.env`](https://github.com/cisagov/Malcolm/tree/v24.0.1/zeek-secret.env.example)
    + added `EXTRACTED_FILE_HTTP_SERVER_ZIP` with default value of `false` in [`zeek.env`](https://github.com/cisagov/Malcolm/tree/v24.0.1/zeek.env.example), see (idaholab#288)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request nginx Relating to Malcolm's use of nginx UI Relating to general UI experience
Projects
Status: Released
Development

No branches or pull requests

3 participants