v3.7.0

v3.7.0

This release adds bulk creation and bulk actions across the whole platform: paste many items at once, or select rows and edit, delete or export them in a single step. It also fixes the SMTP variables of the example .env, refreshes the README, and adds per-app tests for all of it.

Update Procedure

This release does not include a database migration.

Run every command below from the deployment/ directory, in this order:

1. Pull the new images: make pull
2. Stop the stack: make down
3. Start the stack on the new images: make up. Wait until the watcher container is healthy.

Configuration to review in the .env at the root of the repository (the one read by the watcher container):

• New, optional: BULK_CREATE_MAX_ITEMS=200: maximum number of lines accepted by one bulk add request.
• New, optional: BULK_ACTION_MAX_ITEMS=200: maximum number of selected rows accepted by one bulk update / delete / export request.
  Both default to 200 when missing, so existing installations keep working without any change. A request above the limit is rejected as a whole and nothing is created or changed.
• Fix to apply to your own .env if you copied the previous example, the SMTP lines were swapped (#308). Replace

  EMAIL_HOST=
  SMTP_SERVER=25
  

  with

  EMAIL_PORT=25
  SMTP_SERVER=<your SMTP server>
  

  Watcher reads the mail server from SMTP_SERVER and the port from EMAIL_PORT; EMAIL_HOST was never read. If your mails already worked, your .env is already correct.

What's Changed

Bulk Add ("One item per line")

• The add forms of Legitimate Domains, Suspicious Domains (Website Monitoring), DNS Threats Monitored (domains and keywords), Data Leak search patterns, Banned Words and Monitored Keywords now have a One item per line toggle: paste one value per line and every other field is applied to all of them.
• A Preview button shows what would happen without saving anything.
• Every item is processed independently: existing or invalid values never block the batch. A per-item summary (created / skipped / error, with the reason) is shown afterwards.
• Domains are normalized: lowercase, scheme / path / port removed, trailing dot removed, defanged hxxps://example[.]com restored. The same normalization now also applies when creating a single domain.
• Keywords keep their case. An invalid regex on a Data Leak pattern is reported on its own line only.
• A domain that is already a Legitimate Domain is refused for Website Monitoring, as with a single creation. New sites of a batch are checked in the background one after the other, and their rows reload while that happens.

Bulk Actions on existing rows

• Tick rows in any table (domains, keywords, alerts...) to change several fields at once or delete them. Editable fields: legitimacy, takedown / legal team / blocking requests (suspicious domains), alert level (monitored keywords), status (alerts of Website Monitoring, DNS Threats Monitored and Data Leak).
• On Suspicious Domains, selected rows can be exported to Legitimate Domains (the exported sites are removed from Website Monitoring) or to MISP (one event per site, explained in Need help?).
• Every item goes through the same validation and permissions as a single action: the model permissions required for a single create, edit or delete are required here too, and exporting to Legitimate Domains also requires the permission to delete sites.
• Lists and dashboard statistics refresh without reloading the page.

API

• New endpoints on each module: POST <endpoint>/bulk_create/, bulk_update/ and bulk_delete/, plus bulk_export_legitimate/ and bulk_export_misp/ on /api/site_monitoring/site/.
• All of them return the same per-item report (created / skipped / errors, one result per item).

Tests

• Bulk tests are now written in each app's own tests.py (Legitimate Domains and normalization in common; Website Monitoring, DNS Threats Monitored, Data Leak and Threats Watcher in their apps). The full Django suite passes (411 tests).
• Fixed the DNS Threats Monitored Cypress test that read the Source column by a fixed position, which broke when the selection checkbox column is displayed.

Full Changelog: v3.6.1...v3.7.0