Alert Preferences System - #322
Conversation
Signed-off-by: kunalsz <kunalavengers@gmail.com>
|
I think you should mock the various alerts objects you want to call and check if the mock function is being actually called. |
Signed-off-by: kunalsz <kunalavengers@gmail.com>
|
@ManofWax I have added tests, they work fine. But during testing I noticed that when an alert is notified through an alerter , the alert is marked |
|
I think that, at this point, the In this way, each alerter can set Let's waiting for the @ManofWax's feedback, too. |
|
Yeah I think the JSON field is a good solution |
Signed-off-by: kunalsz <kunalavengers@gmail.com>
* 2.4.0 (#287) * feat:frontend- Alert details page (#282) * Opensearch testing (#264) * first commit to rebase opensearch-testing with develop * tOpensearch testing added * Delete .idea directory * Delete .DS_Store * Delete buffalogs/celerybeat-schedule * Opensearch testing added * Opensearch testing added * Opensearch testing added * Delete .idea directory * updates made to opensearch testing * updates made to opensearch testing * Fix exception handling in OpenSearch ingestion and test assertions * Fix: Apply Black formatting and update test logic * Fix: Apply ISORT formatting and update test logic * fixed the example template json for opensearch * solved the put_index_template issue * Update README.md (#283) * Add support for alert filtering in UI/backend (#284) Signed-off-by: drona-gyawali <dronarajgyawali@gmail.com> * version 2.4.0 --------- Signed-off-by: drona-gyawali <dronarajgyawali@gmail.com> Co-authored-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> Co-authored-by: sofie204 <sophiaizokun@gmail.com> * Comprehensive Testing (#309) * Comprehensive Testing Signed-off-by: kunalsz <kunalavengers@gmail.com> * Use exception instead of error while logging Signed-off-by: kunalsz <kunalavengers@gmail.com> * Moves read_config to base_alerting.py Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * Version 2.5.0 * Alert Message Formatter (#315) Signed-off-by: kunalsz <kunalavengers@gmail.com> * Added ingestion api enpoint (#324) * Added `__custom_fields__` to ingestion configuration * Created ingestion view function * Added ingestion api urls to url path * Moved documentation from wiki to the project docs folder (#329) * Converted Elasticsearch fake data generator script into a Django Mgmt… (#331) * Converted Elasticsearch fake data generator script into a Django Mgmt Command * Removed the old script * Added healthcheck to the Postgres Docker container (#332) * Update python dependencies (#325) * Added FawltyDeps linter for finding undeclared and/or unused 3rd-party dependencies * Removed useless python libraries * Set requests as an optional lib because it's used only for the alerting * Linters * Updated python libraries * Updated CHANGELOG.md * Update * Moved django-cors-headers from dev requirements to requirements * Added splunk and opensearch lib in the requirements_dev because used in the tests * Updated elasticsearch and Kibana docker image versions to 7.17.28 * Clean django reusable app dependencies from 2.6.0 version * Refactored requirements * Updated elasticsearch CI version to 7.17.28 * fix * typo * CI change: read requirements_path as a list * Added requirements-ci.txt file in order to include requirements-opt.txt in the CI * typo * fix * Removed buffalogs.egg-info/ folder * Remove egg-info files from repository and ignore them * Removed all egg-info files from repo * Added PyYAML requirement (#337) * Email Integration in Users Model (#320) * Email Integration in Users Model Signed-off-by: kunalsz <kunalavengers@gmail.com> * Add recipient_list_users in alerting.json Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * version 2.6.0 * fix * Alert Preferences System (#322) * Alert Preferences System Signed-off-by: kunalsz <kunalavengers@gmail.com> * Add tests for sending actual alerts and mock alerts Signed-off-by: kunalsz <kunalavengers@gmail.com> * Update notified field in alerts with notified_status Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * Customizable templates (#350) * Customizable templates Signed-off-by: kunalsz <kunalavengers@gmail.com> * Add Jinja2 to requirements.txt Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * Feature [Buffalogs] Alerter Api Endpoints (#347) * Feature [Buffalogs] Alerter Api Endpoints * Restructured test codes into modules * Created new view functions * Updated url list * * Added views module to tests * * Updated active_alerter view function to handle multiple active alerters * * Created validator, detection and task test modules * Updated to Elasticsearch 9 (#357) * Updated to Elasticsearch 9 * Updated Django minor to 4.2.23 * fix * Fix CI elastic_search container creation * Version 2.7.0 (#359) * Retry mechanism with exponential backoff (#348) * Implement retry mechanism with exponential backoff Signed-off-by: kunalsz <kunalavengers@gmail.com> * Implement backoff for exponential retrying Signed-off-by: kunalsz <kunalavengers@gmail.com> * Implement backoff for the rest of the alerters Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * feat:Add country_name validator (#342) * feat:Add country_name validator Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * Add migrations for config cleaning & update countries.json path - Added migration 0015 to clean invalid country codes from existing data - Added migration 0016 to alter the allowed_countries field with validation - Refactored code and change code path as mentioned Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * solve linter issues * solve isort issue Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * refactor validators to accept name Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * deleted unwanted/dump file * resolve migrations conflict * checking ci testout * Refactor migration files --------- Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * Remove `test_send_actual_alert` from alerter tests (#378) * Remove test_send_actual_alert from alerters tests Signed-off-by: kunalsz <kunalavengers@gmail.com> * Add `test_alert_network_failure` back to test_alert_discord.py --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * Update contributing file and linters versions in CI (#377) * Updated linters versions in CI * Updated CONTRIBUTING file * Moved test_alert_filter.py into the tests/detection folder (#370) * 336 feature backend integrate filters into setup config mgmt command (#338) * Integrated all fields and checks to the setup_config mgmt command * Added tests * Added mgmt command doc * removed useless function * Updated doc * Updated CHANGELOG.md * Fix * Refactor utils functions (#361) * Centralized the load_test_data function * Centralized the load_ingestion_config_data function * Centralized the load_index_template function * Centralized the load_data function * litners * Updated CHANGELOG.md * Fix import * typo * Fixed the `TypeError: string indices must be integers, not 'str'` in the `ingestion.get_ingestion_sources` view * Updated CHANGELOG.md * Moved the countries list file in the config/buffalogs folder (#385) * Upgraded Django to 5.2 (#386) * Updated .gitignore * Updated CHANGELOG.md * Updated the licence classifier because deprecated (#387) * Update .gitignore * Fix alerts view read config (#391) * Fix impossible_travel.views.alerts * Removed get_config_read_write dependency * Added missing filename to `read_config` and `write_config` calls * removed get_config_read_write from imports * fixed keyerror * Refactor(tests): improve Testalerter performance using setUpTestData (#382) Replaced setUp with setUpTestData to avoid redundant DB operations across tests. This reduces test execution time and improves readability. Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * Version 2.8.1 * Develop updated - rebase (#395) * Update .gitignore * Fix alerts view read config (#391) * Fix impossible_travel.views.alerts * Removed get_config_read_write dependency * Added missing filename to `read_config` and `write_config` calls * removed get_config_read_write from imports * fixed keyerror * Refactor(tests): improve Testalerter performance using setUpTestData (#382) Replaced setUp with setUpTestData to avoid redundant DB operations across tests. This reduces test execution time and improves readability. Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * Version 2.8.1 --------- Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> Co-authored-by: noble47 <noblebest56@gmail.com> Co-authored-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * Removed deprecated reviewers field from dependabot (#398) * Feature enable querying alerts by multiple keys (#381) * Feature enable querying alerts by multiple keys * Removed `notified` from alerts.json fixture * Fixed bug in charts.py * Moved alert query logic to Alert model * Enabled query by multiple keys via the alert_api endpoint * Added serialize method to Alert object for uniform dictionary representation of alerts * * Refactored filtering logic for risk_score to filter by both string (high, low and medium) and integers (1-8) * zipped alerts.json fixture * * Aligned test_views expected json response to Alerts serialized value * Refactor test setup to use setUpTestData for improved efficiency and consistency (#399) * Fix frontend login ERR_CONNECTION_REFUSED by exposing backend on port 8000 (#403) * Add port mapping for buffalogs_nginx to expose port 8000 * Add uWSGI application server requirement to requirements.txt * Refactor Ingestions tests for performance and fixed Opensearch data structure in tests (#400) * Refactor SplunkIngestionTestCase to use setUpTestData * Refactor OpensearchIngestionTestCase to use class-level setup * Refactor import statements due to ci/cd issue * use mocks in opensearch_test * Refactor Api testcases (#405) Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * Updated CHANGELOG.md * Feature add log query (#411) * Feature enable login query * removed .env file * [P3] Configurable thresholds and aggregation options (#402) * Club similar alerts together Signed-off-by: kunalsz <kunalavengers@gmail.com> * Track notify_alerts within the TaskSettings model Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * version 2.9.0 * Add requests library for HTTP operations in requirements (#416) * Slack Integration Enhancements (#372) * Slack Integration Enhancements Signed-off-by: kunalsz <kunalavengers@gmail.com> * Fix errors by updating test_alert_factory and add alert_template_slack.jinja Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * prevent Worldmap crash on empty data and ensure map renders (#425) * Update Worldmap component to use new geojson data source and optimize data fetching logic * Add node_modules in .gitignore * update DateContextProvider to set initial date range based on current date (#427) * Refactor restful API (#424) * Added model serializer module for consistent representation of model instance. * * fixed errors in serializer class * resolved user template views * updated alerts endpoint in `alerts.json` * * updated view function, templates and javascripts * Scheduled Digests (#419) * Scheduled Digests Signed-off-by: kunalsz <kunalavengers@gmail.com> * Add tests for ScheduledAlertSummaryTask and Refactor slack_alerting Signed-off-by: kunalsz <kunalavengers@gmail.com> * Extend to the rest of the Alerters Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * Add imp travel config filter (#421) * Added ignored_impossible_travel_countries_couples and ignored_impossible_travel_all_same_country fields in the Config model * Added logic * added IGNORED_IMP_TRAVEL_ALL_SAME_COUNTRY and IGNORED_IMP_TRAVEL_COUNTRIES_COUPLE constants * Added migration * Fixed settings path in debug env * Fixed impossible travel fields enrichment to alert * Alligned old tests * Logic refactoring * Added tests * Added field validator * Moved CERTEGO_REPO_DIR variable definition * Added doc for new AlertFilterType fields * Updated CHANGELOG.md and created package 2.10.0 (#428) * Documentation Uniformity (#408) * Standardize and Improve docs related to alerting Signed-off-by: kunalsz <kunalavengers@gmail.com> * Add docs for devs on how to setup alerters Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * Updated elastic to 9.1.0 (#432) * Updated elastic to 9.1.0 * Updated to 9.1.3 * elastic 9.1.2 * Removed settings variables about ingestion (duplication otherwise (#433) * Updated docker images (#431) * Added BuffalogsCleanModelsPeriodicallyTask in TaskSettings model (#423) * Added intelligence_category field mapping in the ingestion config file (#434) * Bump actions/checkout from 4 to 5 (#414) Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v5) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Changed default Config filter values (#439) * Bump actions/setup-node from 4 to 5 (#436) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4 to 5. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Lorygold <lory.goldoni@gmail.com> * Updated python dependencies (#440) * Bump actions/setup-python from 4 to 6 (#437) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 6. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v6) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Lorygold <lory.goldoni@gmail.com> * Added tests for setup_config mgmt command (#441) * Added test * Fixed setup_config mgmt command for multiple fields * Version 2.11.0 * install_requires fix * Fixed sed command in Dockerfile that updated /etc/apt/sources.list, preventing Docker build failure with exit code 2 (#442) * Added new Config fields in the django admin interface (#444) * Added Package Release CI workflow (#445) * Added Package Release CI workflow * Fixed branch name * Updated CHANGELOG.md * Refactor/test: centeralize load ingestion func (#447) * Refactor/test: centeralize load ingestion func * refactor/added url config back in the test * lint issue: resolved * deleted dump file * Added possibility to run setup_config --set-default-values mgmt command in safe or force modes (#449) * Added Pagination to Alerts API using `limit` and `offset` as optional… (#448) * Added Pagination to Alerts API using `limit` and `offset` as optional request payload * * Added validators test * Added serializers test * Improve xor readability * improved user_agent query * Bump actions/setup-node from 5 to 6 (#450) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 5 to 6. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v5...v6) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Added intelligence_category field in mgmt command fake_data_generation (#451) * Fixed UnboundLocalError for impossible_travel mgmt command when launched manually (#453) * Added user_learning_period as configurable field (default: changed to 14 days) (#454) * Version 2.12.0 * update * feat/Management command reset_user_risk_score (#459) * feat/Management command reset_user_risk_score * refactor: reset user risk score * refactor: removed the sqlite file and added the test cases * fix:lint issue * fix: run the linter to fix the issue * fix:removed extra test database from seetings.py * fix: fixed the failing test case for the test managements commands to reset user * docs(management):added the docs for the reset_user command * fix(dockerfile):fixed the docker file for the frontend (#467) * Fixed that if the alert is filtered, the user.risk_score is not updated (#472) * docs: replace wiki links with docs/ equivalents and add starter docs … (#473) * docs: replace wiki links with docs/ equivalents and add starter docs (fixes #464) * docs: add starter docs (about, example, rest-apis, troubleshooting) - fixes #464 * Added tests for models (#474) * Added Support for Tags in Alert Model (#465) * Added Support for Tags in Alert Model * undo migrations * fix validators issue and refactored choices * Revert migration 0015 file to match upstream/develop * Add test cases for Alert model * feat: add task execution logging (#471) * feat: add task execution logging * resolve linter issue * resolve linter error * resolve linter * Add testcases for models - Login, Config, & UserIP (#476) * Add testcases for models - Login, Config, & UserIP * fix linters issues * Fixed username lowercase (#479) * Fixed username lowercase * Added patched_components utility for tests * Fix/unboundlocalerror homepage (#482) * Fix UnboundLocalError on /homepage/ (#285): ensure variable initialization before use * Fix UnboundLocalError on /homepage/ (#285) * fix: connect networks using buffalogs-network (#484) * fix: update Elasticsearch template endpoint to use index template (#483) * fix: update Elasticsearch template endpoint to use index template * chore: remove lh * Bump actions/checkout from 5 to 6 (#485) Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 6. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v5...v6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Version 2.13.0 * Implement session timeout with idle detection, warning modal, and auto-logout (#488) * Added versions to buffacli dependencies * Revert "Merged buffacli" This reverts commit 390d307, reversing changes made to 9cd6a00. * Added version number to dependencies --------- Signed-off-by: drona-gyawali <dronarajgyawali@gmail.com> Signed-off-by: kunalsz <kunalavengers@gmail.com> Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Lorena Goldoni <33703137+Lorygold@users.noreply.github.com> Co-authored-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> Co-authored-by: sofie204 <sophiaizokun@gmail.com> Co-authored-by: zeit <75166996+kunalsz@users.noreply.github.com> Co-authored-by: Lorygold <lory.goldoni@gmail.com> Co-authored-by: Rupam Golui <142140947+Itz-Agasta@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Saurabh Yadav <116506457+saurabh007007@users.noreply.github.com> Co-authored-by: NobleCoder69 <2802chandanyadav@gmail.com> Co-authored-by: Prafull Pandey <81959129+prafullpandey614@users.noreply.github.com> Co-authored-by: Varun Tomar <varunrajput8077@gmail.com> Co-authored-by: icarus77 <113765142+mrtaz77@users.noreply.github.com> Co-authored-by: srijan <141031709+srijan2607@users.noreply.github.com>
* Project Setup And Draft Documentation (#295) * Created project directory * Created project README.md * Added docker file to build * Added docker compose file to root folder * Trying adding buffacli to github actions * momentarely reverted pull request * Added test_buffacli.py file (#311) * Added test_buffacli.py file * Fixed black linter * Generalized buffacli CI test path * Fixed tests module buffacli * Feature add setup and show commands (#313) * * Renamed buffa to buffacli * Comprehensive Testing (#309) * Comprehensive Testing Signed-off-by: kunalsz <kunalavengers@gmail.com> * Use exception instead of error while logging Signed-off-by: kunalsz <kunalavengers@gmail.com> * Moves read_config to base_alerting.py Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * feat(command-show): add alert-types and setup command * Added command to display buffalogs supported alert types * Enabled formatting of output as table, markdown or json * Included configuration of buffacli via command `setup` and ini config file * Modified README.md file * Modified README.md * Project Setup And Draft Documentation * Created project directory * Created project README.md * Added docker file to build * Added docker compose file to root folder * Bug Fixes * Corrected typo in README.md file * Fix formatoption default value bug in `show/app.py` * Changed default `buffalogs_url` config value to `http` protocol from `https` --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> Co-authored-by: zeit <75166996+kunalsz@users.noreply.github.com> * Fixed BuffaCLI CI for tests with pytest (#355) * Fixed BuffaCLI CI for tests with pytest * merge * Added __init__.py file into src/buffacli (#363) * Feature add show alerters command (#352) * Refactor [BuffaCLI] * Restructured code base * Added command to print out ingestion sources, configuration and active ingestion * * Removed redundant arguments from Ingesiton model * Reformated formatters function doc string * * Added tests * Feature (BuffaCLI) New Command - show alerters * Added Alerters model * Added request functions to retrieve alerters * Added command function in `show.py` * Added tests * Installed buffacli requriements (#373) * Refactor [BuffaCLI] (#340) * Refactor [BuffaCLI] * Restructured code base * Added command to print out ingestion sources, configuration and active ingestion * * Removed redundant arguments from Ingesiton model * Reformated formatters function doc string * * Added tests * Formatted code with linters * Added pytest ini config * Modified linters version to fit CI * Feature [buffacli] Added Generic Request Exception Handling (#393) * Feature add alert query commands (#384) * Refactor [BuffaCLI] * Restructured code base * Added command to print out ingestion sources, configuration and active ingestion * * Removed redundant arguments from Ingesiton model * Reformated formatters function doc string * * Added tests * Feature (BuffaCLI) New Command - show alerters * Added Alerters model * Added request functions to retrieve alerters * Added command function in `show.py` * Added tests * Initial setup for querying functionality * Feature [Buffacli] add query alert command * Created AlertQuery model * Defined alery query request method * Created command function for querying alerts * Moved printing responsiblity to render module * Updated linters (#337) * Added test for AlertQuery model * removed debug print statement * Feature add verbose printing (#415) * Feature (Buffacli) Login query command * Feature (BuffaCLI) Enabled query export * Added export mechanism for JSON and CSV file * Modified render module to account for output option * Added output option to query command * removed ideas.txt file * Bug fix * Feature (BuffaCLI) Added verbose printing * Feature enable query export (#413) * Feature (Buffacli) Login query command * Feature (BuffaCLI) Enabled query export * Added export mechanism for JSON and CSV file * Modified render module to account for output option * Added output option to query command * removed ideas.txt file * Bug fix * Feature (Buffacli) Login query command (#412) * Feature (Buffacli) Login query command * removed ideas.txt file * Added pip installation package check in the CI (#463) * Fix buffacli missing config resource (#462) * Fixed missing config.json file after installation * Removed redundant setup.py. `pyproject.toml` handles all package metadata and build configuration (PEP 621), making `setup.py` unnecessary. * Fixed linting issue * Updated pip installation (from git) (#458) * Fixed conflict between Buffacli and Develop (#492) * 2.4.0 (#287) * feat:frontend- Alert details page (#282) * Opensearch testing (#264) * first commit to rebase opensearch-testing with develop * tOpensearch testing added * Delete .idea directory * Delete .DS_Store * Delete buffalogs/celerybeat-schedule * Opensearch testing added * Opensearch testing added * Opensearch testing added * Delete .idea directory * updates made to opensearch testing * updates made to opensearch testing * Fix exception handling in OpenSearch ingestion and test assertions * Fix: Apply Black formatting and update test logic * Fix: Apply ISORT formatting and update test logic * fixed the example template json for opensearch * solved the put_index_template issue * Update README.md (#283) * Add support for alert filtering in UI/backend (#284) Signed-off-by: drona-gyawali <dronarajgyawali@gmail.com> * version 2.4.0 --------- Signed-off-by: drona-gyawali <dronarajgyawali@gmail.com> Co-authored-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> Co-authored-by: sofie204 <sophiaizokun@gmail.com> * Comprehensive Testing (#309) * Comprehensive Testing Signed-off-by: kunalsz <kunalavengers@gmail.com> * Use exception instead of error while logging Signed-off-by: kunalsz <kunalavengers@gmail.com> * Moves read_config to base_alerting.py Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * Version 2.5.0 * Alert Message Formatter (#315) Signed-off-by: kunalsz <kunalavengers@gmail.com> * Added ingestion api enpoint (#324) * Added `__custom_fields__` to ingestion configuration * Created ingestion view function * Added ingestion api urls to url path * Moved documentation from wiki to the project docs folder (#329) * Converted Elasticsearch fake data generator script into a Django Mgmt… (#331) * Converted Elasticsearch fake data generator script into a Django Mgmt Command * Removed the old script * Added healthcheck to the Postgres Docker container (#332) * Update python dependencies (#325) * Added FawltyDeps linter for finding undeclared and/or unused 3rd-party dependencies * Removed useless python libraries * Set requests as an optional lib because it's used only for the alerting * Linters * Updated python libraries * Updated CHANGELOG.md * Update * Moved django-cors-headers from dev requirements to requirements * Added splunk and opensearch lib in the requirements_dev because used in the tests * Updated elasticsearch and Kibana docker image versions to 7.17.28 * Clean django reusable app dependencies from 2.6.0 version * Refactored requirements * Updated elasticsearch CI version to 7.17.28 * fix * typo * CI change: read requirements_path as a list * Added requirements-ci.txt file in order to include requirements-opt.txt in the CI * typo * fix * Removed buffalogs.egg-info/ folder * Remove egg-info files from repository and ignore them * Removed all egg-info files from repo * Added PyYAML requirement (#337) * Email Integration in Users Model (#320) * Email Integration in Users Model Signed-off-by: kunalsz <kunalavengers@gmail.com> * Add recipient_list_users in alerting.json Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * version 2.6.0 * fix * Alert Preferences System (#322) * Alert Preferences System Signed-off-by: kunalsz <kunalavengers@gmail.com> * Add tests for sending actual alerts and mock alerts Signed-off-by: kunalsz <kunalavengers@gmail.com> * Update notified field in alerts with notified_status Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * Customizable templates (#350) * Customizable templates Signed-off-by: kunalsz <kunalavengers@gmail.com> * Add Jinja2 to requirements.txt Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * Feature [Buffalogs] Alerter Api Endpoints (#347) * Feature [Buffalogs] Alerter Api Endpoints * Restructured test codes into modules * Created new view functions * Updated url list * * Added views module to tests * * Updated active_alerter view function to handle multiple active alerters * * Created validator, detection and task test modules * Updated to Elasticsearch 9 (#357) * Updated to Elasticsearch 9 * Updated Django minor to 4.2.23 * fix * Fix CI elastic_search container creation * Version 2.7.0 (#359) * Retry mechanism with exponential backoff (#348) * Implement retry mechanism with exponential backoff Signed-off-by: kunalsz <kunalavengers@gmail.com> * Implement backoff for exponential retrying Signed-off-by: kunalsz <kunalavengers@gmail.com> * Implement backoff for the rest of the alerters Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * feat:Add country_name validator (#342) * feat:Add country_name validator Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * Add migrations for config cleaning & update countries.json path - Added migration 0015 to clean invalid country codes from existing data - Added migration 0016 to alter the allowed_countries field with validation - Refactored code and change code path as mentioned Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * solve linter issues * solve isort issue Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * refactor validators to accept name Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * deleted unwanted/dump file * resolve migrations conflict * checking ci testout * Refactor migration files --------- Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * Remove `test_send_actual_alert` from alerter tests (#378) * Remove test_send_actual_alert from alerters tests Signed-off-by: kunalsz <kunalavengers@gmail.com> * Add `test_alert_network_failure` back to test_alert_discord.py --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * Update contributing file and linters versions in CI (#377) * Updated linters versions in CI * Updated CONTRIBUTING file * Moved test_alert_filter.py into the tests/detection folder (#370) * 336 feature backend integrate filters into setup config mgmt command (#338) * Integrated all fields and checks to the setup_config mgmt command * Added tests * Added mgmt command doc * removed useless function * Updated doc * Updated CHANGELOG.md * Fix * Refactor utils functions (#361) * Centralized the load_test_data function * Centralized the load_ingestion_config_data function * Centralized the load_index_template function * Centralized the load_data function * litners * Updated CHANGELOG.md * Fix import * typo * Fixed the `TypeError: string indices must be integers, not 'str'` in the `ingestion.get_ingestion_sources` view * Updated CHANGELOG.md * Moved the countries list file in the config/buffalogs folder (#385) * Upgraded Django to 5.2 (#386) * Updated .gitignore * Updated CHANGELOG.md * Updated the licence classifier because deprecated (#387) * Update .gitignore * Fix alerts view read config (#391) * Fix impossible_travel.views.alerts * Removed get_config_read_write dependency * Added missing filename to `read_config` and `write_config` calls * removed get_config_read_write from imports * fixed keyerror * Refactor(tests): improve Testalerter performance using setUpTestData (#382) Replaced setUp with setUpTestData to avoid redundant DB operations across tests. This reduces test execution time and improves readability. Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * Version 2.8.1 * Develop updated - rebase (#395) * Update .gitignore * Fix alerts view read config (#391) * Fix impossible_travel.views.alerts * Removed get_config_read_write dependency * Added missing filename to `read_config` and `write_config` calls * removed get_config_read_write from imports * fixed keyerror * Refactor(tests): improve Testalerter performance using setUpTestData (#382) Replaced setUp with setUpTestData to avoid redundant DB operations across tests. This reduces test execution time and improves readability. Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * Version 2.8.1 --------- Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> Co-authored-by: noble47 <noblebest56@gmail.com> Co-authored-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * Removed deprecated reviewers field from dependabot (#398) * Feature enable querying alerts by multiple keys (#381) * Feature enable querying alerts by multiple keys * Removed `notified` from alerts.json fixture * Fixed bug in charts.py * Moved alert query logic to Alert model * Enabled query by multiple keys via the alert_api endpoint * Added serialize method to Alert object for uniform dictionary representation of alerts * * Refactored filtering logic for risk_score to filter by both string (high, low and medium) and integers (1-8) * zipped alerts.json fixture * * Aligned test_views expected json response to Alerts serialized value * Refactor test setup to use setUpTestData for improved efficiency and consistency (#399) * Fix frontend login ERR_CONNECTION_REFUSED by exposing backend on port 8000 (#403) * Add port mapping for buffalogs_nginx to expose port 8000 * Add uWSGI application server requirement to requirements.txt * Refactor Ingestions tests for performance and fixed Opensearch data structure in tests (#400) * Refactor SplunkIngestionTestCase to use setUpTestData * Refactor OpensearchIngestionTestCase to use class-level setup * Refactor import statements due to ci/cd issue * use mocks in opensearch_test * Refactor Api testcases (#405) Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> * Updated CHANGELOG.md * Feature add log query (#411) * Feature enable login query * removed .env file * [P3] Configurable thresholds and aggregation options (#402) * Club similar alerts together Signed-off-by: kunalsz <kunalavengers@gmail.com> * Track notify_alerts within the TaskSettings model Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * version 2.9.0 * Add requests library for HTTP operations in requirements (#416) * Slack Integration Enhancements (#372) * Slack Integration Enhancements Signed-off-by: kunalsz <kunalavengers@gmail.com> * Fix errors by updating test_alert_factory and add alert_template_slack.jinja Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * prevent Worldmap crash on empty data and ensure map renders (#425) * Update Worldmap component to use new geojson data source and optimize data fetching logic * Add node_modules in .gitignore * update DateContextProvider to set initial date range based on current date (#427) * Refactor restful API (#424) * Added model serializer module for consistent representation of model instance. * * fixed errors in serializer class * resolved user template views * updated alerts endpoint in `alerts.json` * * updated view function, templates and javascripts * Scheduled Digests (#419) * Scheduled Digests Signed-off-by: kunalsz <kunalavengers@gmail.com> * Add tests for ScheduledAlertSummaryTask and Refactor slack_alerting Signed-off-by: kunalsz <kunalavengers@gmail.com> * Extend to the rest of the Alerters Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * Add imp travel config filter (#421) * Added ignored_impossible_travel_countries_couples and ignored_impossible_travel_all_same_country fields in the Config model * Added logic * added IGNORED_IMP_TRAVEL_ALL_SAME_COUNTRY and IGNORED_IMP_TRAVEL_COUNTRIES_COUPLE constants * Added migration * Fixed settings path in debug env * Fixed impossible travel fields enrichment to alert * Alligned old tests * Logic refactoring * Added tests * Added field validator * Moved CERTEGO_REPO_DIR variable definition * Added doc for new AlertFilterType fields * Updated CHANGELOG.md and created package 2.10.0 (#428) * Documentation Uniformity (#408) * Standardize and Improve docs related to alerting Signed-off-by: kunalsz <kunalavengers@gmail.com> * Add docs for devs on how to setup alerters Signed-off-by: kunalsz <kunalavengers@gmail.com> --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> * Updated elastic to 9.1.0 (#432) * Updated elastic to 9.1.0 * Updated to 9.1.3 * elastic 9.1.2 * Removed settings variables about ingestion (duplication otherwise (#433) * Updated docker images (#431) * Added BuffalogsCleanModelsPeriodicallyTask in TaskSettings model (#423) * Added intelligence_category field mapping in the ingestion config file (#434) * Bump actions/checkout from 4 to 5 (#414) Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v5) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Changed default Config filter values (#439) * Bump actions/setup-node from 4 to 5 (#436) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4 to 5. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Lorygold <lory.goldoni@gmail.com> * Updated python dependencies (#440) * Bump actions/setup-python from 4 to 6 (#437) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 6. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v6) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Lorygold <lory.goldoni@gmail.com> * Added tests for setup_config mgmt command (#441) * Added test * Fixed setup_config mgmt command for multiple fields * Version 2.11.0 * install_requires fix * Fixed sed command in Dockerfile that updated /etc/apt/sources.list, preventing Docker build failure with exit code 2 (#442) * Added new Config fields in the django admin interface (#444) * Added Package Release CI workflow (#445) * Added Package Release CI workflow * Fixed branch name * Updated CHANGELOG.md * Refactor/test: centeralize load ingestion func (#447) * Refactor/test: centeralize load ingestion func * refactor/added url config back in the test * lint issue: resolved * deleted dump file * Added possibility to run setup_config --set-default-values mgmt command in safe or force modes (#449) * Added Pagination to Alerts API using `limit` and `offset` as optional… (#448) * Added Pagination to Alerts API using `limit` and `offset` as optional request payload * * Added validators test * Added serializers test * Improve xor readability * improved user_agent query * Bump actions/setup-node from 5 to 6 (#450) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 5 to 6. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v5...v6) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Added intelligence_category field in mgmt command fake_data_generation (#451) * Fixed UnboundLocalError for impossible_travel mgmt command when launched manually (#453) * Added user_learning_period as configurable field (default: changed to 14 days) (#454) * Version 2.12.0 * update * feat/Management command reset_user_risk_score (#459) * feat/Management command reset_user_risk_score * refactor: reset user risk score * refactor: removed the sqlite file and added the test cases * fix:lint issue * fix: run the linter to fix the issue * fix:removed extra test database from seetings.py * fix: fixed the failing test case for the test managements commands to reset user * docs(management):added the docs for the reset_user command * fix(dockerfile):fixed the docker file for the frontend (#467) * Fixed that if the alert is filtered, the user.risk_score is not updated (#472) * docs: replace wiki links with docs/ equivalents and add starter docs … (#473) * docs: replace wiki links with docs/ equivalents and add starter docs (fixes #464) * docs: add starter docs (about, example, rest-apis, troubleshooting) - fixes #464 * Added tests for models (#474) * Added Support for Tags in Alert Model (#465) * Added Support for Tags in Alert Model * undo migrations * fix validators issue and refactored choices * Revert migration 0015 file to match upstream/develop * Add test cases for Alert model * feat: add task execution logging (#471) * feat: add task execution logging * resolve linter issue * resolve linter error * resolve linter * Add testcases for models - Login, Config, & UserIP (#476) * Add testcases for models - Login, Config, & UserIP * fix linters issues * Fixed username lowercase (#479) * Fixed username lowercase * Added patched_components utility for tests * Fix/unboundlocalerror homepage (#482) * Fix UnboundLocalError on /homepage/ (#285): ensure variable initialization before use * Fix UnboundLocalError on /homepage/ (#285) * fix: connect networks using buffalogs-network (#484) * fix: update Elasticsearch template endpoint to use index template (#483) * fix: update Elasticsearch template endpoint to use index template * chore: remove lh * Bump actions/checkout from 5 to 6 (#485) Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 6. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v5...v6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Version 2.13.0 * Implement session timeout with idle detection, warning modal, and auto-logout (#488) * Added versions to buffacli dependencies * Revert "Merged buffacli" This reverts commit 390d307, reversing changes made to 9cd6a00. * Added version number to dependencies --------- Signed-off-by: drona-gyawali <dronarajgyawali@gmail.com> Signed-off-by: kunalsz <kunalavengers@gmail.com> Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Lorena Goldoni <33703137+Lorygold@users.noreply.github.com> Co-authored-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> Co-authored-by: sofie204 <sophiaizokun@gmail.com> Co-authored-by: zeit <75166996+kunalsz@users.noreply.github.com> Co-authored-by: Lorygold <lory.goldoni@gmail.com> Co-authored-by: Rupam Golui <142140947+Itz-Agasta@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Saurabh Yadav <116506457+saurabh007007@users.noreply.github.com> Co-authored-by: NobleCoder69 <2802chandanyadav@gmail.com> Co-authored-by: Prafull Pandey <81959129+prafullpandey614@users.noreply.github.com> Co-authored-by: Varun Tomar <varunrajput8077@gmail.com> Co-authored-by: icarus77 <113765142+mrtaz77@users.noreply.github.com> Co-authored-by: srijan <141031709+srijan2607@users.noreply.github.com> * fixed conflicts in base_alerting file * modified charts and discord alerting to match develop branch version (#513) * fixed repeated variables because of merge --------- Signed-off-by: kunalsz <kunalavengers@gmail.com> Signed-off-by: drona-gyawali <dronarajgyawali@gmail.com> Signed-off-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: noble47 <noblebest56@gmail.com> Co-authored-by: Federico Foschini <ffoschini@certego.net> Co-authored-by: zeit <75166996+kunalsz@users.noreply.github.com> Co-authored-by: Dorna Raj Gyawali <dronarajgyawali@gmail.com> Co-authored-by: sofie204 <sophiaizokun@gmail.com> Co-authored-by: Rupam Golui <142140947+Itz-Agasta@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Saurabh Yadav <116506457+saurabh007007@users.noreply.github.com> Co-authored-by: NobleCoder69 <2802chandanyadav@gmail.com> Co-authored-by: Prafull Pandey <81959129+prafullpandey614@users.noreply.github.com> Co-authored-by: Varun Tomar <varunrajput8077@gmail.com> Co-authored-by: icarus77 <113765142+mrtaz77@users.noreply.github.com> Co-authored-by: srijan <141031709+srijan2607@users.noreply.github.com>
Resolves #321
Changes Made:
active_alerterchanged toactive_alerters