v1.16.0

https://docs.openkat.nl/release-notes/1.16.html

What's Changed

• Upgrade GitHub actions by @ammar92 in #2235
• Add note about Debian packages to 1.14 release notes by @dekkers in #2234
• Bump msgpackr from 1.6.1 to 1.10.1 in /rocky by @dependabot in #2219
• Updated deploy-pages action by @ammar92 in #2251
• Fix environment page in docs by @ammar92 in #2257
• Fix export buttons report by @Rieven in #2259
• Translations update from Hosted Weblate by @weblate in #2261
• Fixed invalid type usage in get_rabbit_channel and close_rabbit_channel by @ammar92 in #2280
• Translations update from Hosted Weblate by @weblate in #2279
• List item behaviour by @HeleenSG in #2281
• fix zero division by @noamblitz in #2298
• Updated template file to respect environment prefixes in docs by @ammar92 in #2317
• Remove preselection from multireport flow by @noamblitz in #2318
• Fix/upgrade jinja2 by @ammar92 in #2326
• Fix multiple Debian issues by @dekkers in #2283
• Add max fds ulimit to octopoes api worker by @originalsouth in #2327
• add meta / cache hash for rpki boefje to raw output by @underdarknl in #2255
• Remove smartphone from bug report template by @dekkers in #2334
• More ulimits for buggy celery by @originalsouth in #2338
• Remove icons from compliance issue table by @madelondohmen in #2340
• Sector report summary - Best and worst scoring security checks by @madelondohmen in #2312
• Chore/update pr template with comments by @Donnype in #2305
• Translations update from Hosted Weblate by @weblate in #2311
• Remove IPs with zero vulnerabilities by @madelondohmen in #2319
• Add IPv6 config to docker-compose.yml by @stephanie0x00 in #2256
• Render dicts and list ooi attrs as jsonfield by @noamblitz in #2355
• Add hrefs to Basic Security overview by @madelondohmen in #2330
• Feat/normalizer mimetype upload deeplink by @underdarknl in #2220
• Check for sudo in install and update script by @dekkers in #2360
• Adds CAA records to the model, boefje, normalizer, adds a check bit and a finding by @underdarknl in #2315
• Add token authentication by @dekkers in #2349
• Update dependencies by @ammar92 in #2348
• Error handling for Generate Report by @madelondohmen in #2274
• Remove debian11 packages by @dekkers in #2358
• Fix WEASYPRINT_BASEURL default value and change ports in docker-compose.yml by @dekkers in #2373
• Prevent double github actions by @dekkers in #2374
• Remove uWSGI by @dekkers in #2366
• Convert docker-compose to docker compose by @originalsouth in #2341
• Rename invalid rpki finding to expired by @noamblitz in #2377
• Show created at and data from in reports by @noamblitz in #2370
• Update RabbitMQ to the latest version by @dekkers in #2392
• Fix/394 Introduce clearance level control for objects imported by CSV by @originalsouth in #2390
• Update dependencies by @ammar92 in #2396
• Use --diff-filter=U instead of --staged in fix-poetry-merge-conflict by @dekkers in #2398
• Feat/more csp checks by @underdarknl in #2025
• Remove everything related to old crux by @dekkers in #2403
• Fix normalizer filtering on tasks endpoint by @jpbruinsslot in #2414
• Bump actions/cache from 3 to 4 by @dependabot in #2417
• Bump dorny/paths-filter from 2 to 3 by @dependabot in #2418
• Fix usage of SPAN_EXPORT_GRPC_ENDPOINT variable by @ammar92 in #2420
• Fix mail report json by @noamblitz in #2426
• Fix open ports report by @noamblitz in #2430
• minor speedup, Return immediately on the first disabled plugin for report check by @underdarknl in #2434
• use oois instead of references by @noamblitz in #2433
• Update fastapi by @ammar92 in #2444
• fix wordpress check by @noamblitz in #2445
• Allow creation of declared scan profiles through normalizers by @Donnype in #2428
• Update DNS report by @madelondohmen in #2413
• upgrade forcediphttpsadapter by @noamblitz in #2453
• Fix argument order and hence avoid the octopoes factory by @Donnype in #2454
• Hotfix for default arg by @Donnype in #2458
• Feature/report benchmarks by @Donnype in #2447
• Add traces for pop and push to queue in scheduler by @jpbruinsslot in #2467
• Add mypy and application bootstrap test to boefjes by @Donnype in #2460
• Fix/normalizer produces, list was empty. is now populated and links to object page by @underdarknl in #2411
• Add timeouts to CVE API downloader by @dekkers in #2455
• Select all OOIS for Aggregate Reports by @Rieven in #2222
• Update Starlette, FastAPI and Django by @dekkers in #2480
• Do not log an error on token refresh in bytes client by @dekkers in #2469
• Enable ruff bandit checks by @dekkers in #2465
• Fix kat_dns settings by @originalsouth in #2459
• Ignore certificate errors in security txt boefje by @dekkers in #2487
• Add unit tests for vulnerability report and fix first_seen by @dekkers in #2462
• minimal / maximal number of required checboxes checked functionality by @underdarknl in #2375
• Remove support for Python 3.8 and 3.9 by @dekkers in #2470
• Update scheduler architecture documentation by @jpbruinsslot in #2387
• Revert fastapi and starlette downgrade by @dekkers in #2489
• Do not show an error in tasks stats if no tasks have run yet by @dekkers in #2486
• Make rtest fail less than 1 in 20 by @originalsouth in #2441
• Bump python-multipart from 0.0.6 to 0.0.7 in /bytes by @dependabot in #2494
• Health page accessable during onboarding by @Rieven in #2499
• Fix/systems report domain count by @noamblitz in #2490
• Skip or hide section on Plugin Page by @madelondohmen in #2461
• remove duplicate oois from report by @noamblitz in #2504
• Rename list method to prevent conflict with builtin by @dekkers in #2498
• Fix bug get_selection that is now an imported method by @Rieven in #2509
• Align primary button to the left by @Rieven in #2500
• Update typing to Python 3.10 style by @dekkers in #2491
• Open ports report unit tests by @noamblitz in #2514
• Update and remove dependencies by @ammar92 in #2517
• Add pyupgrade and django-upgrade to pre-commit by @dekkers in #2523
• Ipv6 report unit tests by @noamblitz in #2512
• Add documentation about yielding declared scan profiles in normalizers by @Donnype in #2501
• Fix/case insensitive hsts by @noamblitz in #2505
• Checks for future dates (observed_at) by @Rieven in #2023
• Mail Report unit tests by @madelondohmen in #2513
• Fix typing in rocky and few other places by @dekkers in #2519
• Fix/1963 nxdomain keeps findings in kat by @originalsouth in #2310
• Add VLSM settings for Nmap IP-range by @zcrt in #1378
• Safe Connections Report unit tests by @madelondohmen in #2515
• Bump settings-doc from 3.0.0 to 4.0.0 by @dependabot in #2529
• unit tests rpki report by @noamblitz in #2511
• Translations update from Hosted Weblate by @weblate in #2546
• Remove FMEA by @ammar92 in #2539
• Update packages by @ammar92 in #2552
• Fix missing fmea init.py by @dekkers in #2553
• Unit tests to Name Server Report by @madelondohmen in #2542
• Make valid time required parameter in the octopoes API by @dekkers in #2543
• Remove unnecessary toplevel dependencies by @dekkers in #2554
• Select all oois triggers toggle all by @Rieven in #2536
• Disable ruff split-on-trailing-comma and update ruff by @dekkers in #2544
• Sort vulnerabilities in vulnerability report by @noamblitz in #2378
• Add return typing to report test fixtures by @noamblitz in #2557
• Fix/2527 octopoes unicode by @originalsouth in #2558
• Quick fix for PDF table overflow by @madelondohmen in #2562
• Add pool size config and logs by @zcrt in #2541
• add unit test for web report by @noamblitz in #2528
• Updated findings database. Removed old findings, added Impact, Source… by @stephanie0x00 in #2569
• Feature/efficient reporting by @Donnype in #2516
• Set a timeout on hanging test ssl container by @noamblitz in #2560
• Raise exception if boefje input OOI has been deleted by @dekkers in #2573
• Create findings report by @madelondohmen in #2393
• Fix octopoes typing by @dekkers in #2555
• Update django by @dekkers in #2587
• Hotfix for where_in queries for abstract types by @Donnype in #2577
• Add metrics collection for scheduler using prometheus by @jpbruinsslot in #2468
• Fix wrong solving of merge conflict by @dekkers in #2585
• Feature/efficient reporting for all reports by @Donnype in #2586
• Filter out undeserializable objects from xtdb query in construct_neighbour_query_multi by @originalsouth in #2592
• TLS Report unit tests by @madelondohmen in #2593
• Translations update from Hosted Weblate by @weblate in #2594
• Fix missing finding_type table by @madelondohmen in #2596
• add extra checks for findings to dns report by @underdarknl in #2506
• fix deprecated warning due to old env in .env-defaults by @underdarknl in #2597
• Translations update from Hosted Weblate by @weblate in #2614
• Fix bug in Generate Report by @madelondohmen in #2616
• Fix HTML in DNS report by @madelondohmen in #2617
• DNS Report unit tests by @madelondohmen in #2602
• Fix architecture check for Wappalyzer boefje by @noamblitz in #2620
• Filter invalid mimetypes in boefjes output. by @underdarknl in #2563
• split output into two raw files, and fix no-output mimetype in main.py by @underdarknl in #2604
• fix output mimetypes for no-action runs by @underdarknl in #2606
• Add 1.15 release notes by @dekkers in #2605
• Bump weasyprint from 61.0 to 61.2 in /rocky by @dependabot in #2625
• Fix setting clearence level on plugin detail page by @noamblitz in #2623
• Refactor onboarding wizard to use Generate Report flow to create a DNS report by @Rieven in #2561
• Update tabler icons to v3.01 by @Rieven in #2640
• New Rocky Paginator by @Rieven in #2627
• Makefile .env-default typo by @noamblitz in #2647
• Replace requests with httpx by @ammar92 in #2576
• Fix missing apt update in keiko github action by @dekkers in #2668
• Textual changes by @HeleenSG in #2676
• Remove unnecessary loop in FilterRequest in scheduler by @jpbruinsslot in #2684
• fix: openssl boefje stuck on port 80 by @tobiasBDO in #2600
• Remove superfluous curly bracket open from graph view template by @originalsouth in #2700
• Improvements of Aggregate Report by @madelondohmen in #2643
• refactor the leakix normalizer to be less one giant method. by @underdarknl in #2363
• Add fix-byte-order-marker and pretty-format-json to pre-commit by @dekkers in #2634
• Fix pdf alignment by @HeleenSG in #2674
• Fix critical vulnerability counter by @madelondohmen in #2712
• Improve generate report by @madelondohmen in #2633
• Fix #1739 by @originalsouth in #2705
• Upgrade pre-commit hooks by @ammar92 in #2729
• Remove Docker Compose: "version" by @originalsouth in #2718
• Undo project-directory in Rocky by @originalsouth in #2734
• Feat stepper design v2 by @HeleenSG in #2704
• Plugins overview in appendix not showing any plugins by @Rieven in #2694
• Fix in System Specific by @madelondohmen in #2732
• Query non-reference fields and subclass-specific fields through path queries by @Donnype in #2662
• Feature/boefjes to oci images by @Donnype in #2709
• Dont report vulnerabilites without version info of the software for snyk by @noamblitz in #2730
• Add xtdb-cli tool to Octopoes by @originalsouth in #2733
• Bump actions/configure-pages from 4 to 5 by @dependabot in #2745
• Remove octopoes coverage workflow by @dekkers in #2755
• Updated phonenumbers and django-phonenumber-field by @ammar92 in #2757
• fix schema errors on empty / missing schemas by @underdarknl in #2744
• OOI selection at Aggregate report does not remember changed selection by @Rieven in #2619
• Fix static files for container images/Debian packages when DEBUG is on by @dekkers in #2742
• Upgrade pillow by @ammar92 in #2783
• Replace black with ruffs formatter by @ammar92 in #2762
• Feature/disallowed domains in csp by @noamblitz in #2624
• Add 127.0.0.1 to allowed hosts in Debian package by @dekkers in #2758
• Replace Wappalyzer by @ammar92 in #2727
• Add why container stdout/stderr can't be used for boefjes output by @dekkers in #2673
• Fix/2721 improve error handling by better exception aggregation v2 by @originalsouth in #2795
• Set katalogus limit to 200 by @dekkers in #2798
• Use public cryptography API in SSL certificate normalizer by @dekkers in #2796
• More improvements for reports by @madelondohmen in #2722
• Prevent confusing errors from leaking into the general logs by @originalsouth in #2815
• Fix wrong type in save_raw and list_origin_parameters httpx params by @dekkers in #2819
• Update performance of Findings Report by @madelondohmen in #2799
• Change titles to meet configuration page design by @madelondohmen in #2818
• Updated idna package by @ammar92 in #2845
• Fix missing cipher csv in Debian package by @dekkers in #2850
• Add 1.15 release name by @dekkers in #2854
• Bump sqlparse from 0.4.4 to 0.5.0 in /rocky by @dependabot in #2856
• Update dnspython by @ammar92 in #2861
• Bump aiohttp from 3.9.3 to 3.9.4 in /boefjes by @dependabot in #2867
• Add new boefjes and normalizers cover images to KAT-alogus by @Rieven in #2859
• Update and fix nuclei by @noamblitz in #2865
• Add some additional exception handling to the scheduler by @jpbruinsslot in #2814
• Use qualified image URLs in docker-compose by @dekkers in #2869
• Introduce importing/exporting capabilities in Octopoes/xtdb-multinode-tool (addressing #2761) by @originalsouth in #2855
• Don't show records of the nameservers of chosen hostnames in DNS report by @noamblitz in #2809
• Styling Report Types names by @Rieven in #2791
• Backup scripts from TobiasBDO by @ring-ring-ring in #1794
• Plugin overview table by @madelondohmen in #2804
• Add some additional exception handling to the scheduler by @jpbruinsslot in #2878
• Fixed image reference in pdio_subfinder plugin by @ammar92 in #2860
• Implement Octopoes Models documentation by @ammar92 in #2858
• Updated some packages by @ammar92 in #2891
• Add stepper to report configuration by @madelondohmen in #2868
• Translations update from Hosted Weblate by @weblate in #2792
• Build nmap OCI image that handles both nmap-tcp and nmap-udp using a new oci_arguments field by @Donnype in #2832
• Fix and improve running boefjes/normalizer by hand by @dekkers in #2802
• Docs update on OCI image building from current Python/Docker boefjes by @Donnype in #2827
• Specify bit in the answer raw file instead of as mimetype by @dekkers in #2900
• Add a warning to the CSP validator for 'self' on script-src directives. by @underdarknl in #2672
• Replace old plugin in Vulnerability Report by @madelondohmen in #2910
• Update jinja2 and tqdm by @ammar92 in #2919
• Add wappalyzer software to weburl instead of hostname by @noamblitz in #2912
• Set no limit by default in the KATalogus plugin API by @Donnype in #2921
• Replace python-jose with pyjwt by @Donnype in #2925
• Fixes in OOI Add/ Edit form by @ammar92 in #2906
• Update required and suggested plugin section in report configuration by @madelondohmen in #2897
• Remove many new ports open boefje/normalizer by @dekkers in #2932
• Add workaround for Granian HTTP/2 bug by @dekkers in #2931
• Add prettier pre-commit hook by @dekkers in #2928
• Cleanup and squash migrations by @dekkers in #2934
• Update cve_2023_35078_not_vulnerable.html by @underdarknl in #2935
• Install CA root keys as package by @originalsouth in #2917
• Developer documentation about reports by @madelondohmen in #2908
• Change normalizer function signature by @dekkers in #2927
• Adding 'Checked logs for warnings/errors' to QA checklist by @stephanie0x00 in #2951
• fix version handling when no version is present. by @underdarknl in #2962
• Fix aggregate plugin overview table by @madelondohmen in #2896
• Fix typing in boefjes/normalizers by @dekkers in #2933
• Update granian and remove workaround for fixed bug by @dekkers in #2980
• Updated packages by @ammar92 in #2972
• Add drill trace option in dnssec boefje by @dekkers in #2979
• Fix task api status code response for malformed id in the scheduler by @jpbruinsslot in #2953
• User documentation for reports by @stephanie0x00 in #2898
• Translations update from Hosted Weblate by @weblate in #2930
• Adding IPv6 support to documentation for Docker setups by @stephanie0x00 in #2813
• Reports: Fix select all OOIs by @Rieven in #2909
• Translations update from Hosted Weblate by @weblate in #2996
• Fix merge conflicts in weblate by @dekkers in #3007
• Phase out the Repository model from the KATalogus by @Donnype in #2984
• Rewrite xtdb-cli.py with "click" by @originalsouth in #2957
• Translations update from Hosted Weblate by @weblate in #3012
• Add raw AuthToken SQL migration by @Donnype in #3009
• chore: Resolves css-issues found by sonarcloud by @HeleenSG in #3034
• Fixes text in secondary menu on scan profile detail page by @TwistMeister in #3035
• Fix empty consumes of boefjes will trigger tasks in scheduler by @jpbruinsslot in #3017
• Translations update from Hosted Weblate by @weblate in #3018
• Translations update from Hosted Weblate by @weblate in #3048
• fix: 🔧 update db normalize setting by @zcrt in #2777
• Documentation - developer and helper functionality documentation for xtdb-cli tool by @stephanie0x00 in #3023
• Fixed 2 small mistakes in documentation by @Souf149 in #3089
• feat: 📝 add API titles by @zcrt in #3055
• Translations update from Hosted Weblate by @weblate in #3091
• Add Frysk to our selectable list of languages by @underdarknl in #3090
• Updated urllib3 by @ammar92 in #3098
• Update README.rst, add some more details ad links to conform to iReal… by @underdarknl in #3049
• Add Boefje and Normalizer models in the KATalogus by @Donnype in #3011
• Update feature and PR templates by @madelondohmen in #3063
• Add burp normalizer docs by @stephanie0x00 in #3101
• Bump braces from 3.0.2 to 3.0.3 in /rocky by @dependabot in #3100
• feat: 📝 add human readable hostnames to vulnerability report by @zcrt in #2952
• fix consistent form width by @HeleenSG in #3104
• Translations update from Hosted Weblate by @weblate in #3115
• Match frontend with design - part 1 by @madelondohmen in #3109
• Update report overview table in report manual by @madelondohmen in #3106
• Translations update from Hosted Weblate by @weblate in #3125
• Match frontend with design - part 2 by @madelondohmen in #3132
• Update helper text for nmap-ports boefje by @stephanie0x00 in #3097
• RFD 0001: Add RFD for Requests for Discussion by @jpbruinsslot in #3002
• Fix pydantic serialization of enum on PATCH of tasks by @jpbruinsslot in #2944
• Documentation on how to make a boefje, normalizer, model, bit and report with examples. by @Souf149 in #2967
• Translations update from Hosted Weblate by @weblate in #3154
• Fix Snyk boefje creating empty CVE ids by @dekkers in #3144
• Fix rabbtimq exception handling in scheduler by @jpbruinsslot in #3092
• Bump djangorestframework from 3.14.0 to 3.15.2 in /rocky by @dependabot in #3157
• Add workflow that builds containerized images on ghcr.io by @Donnype in #3148
• Optimize queries executed when running bits by @originalsouth in #3024
• Saving reports by @noamblitz in #2904
• Ignore missing csp if page is not xss capable by @noamblitz in #3126
• fix: Sidemenu by @HeleenSG in #2821
• Added reports.md to the documentation tree by @Souf149 in #3112
• Update renderNormalizerOutputOOIs.js by @underdarknl in #3142
• Translations update from Hosted Weblate by @weblate in #3172
• Add pluginToggler.js to Aggregate Report (1.16) by @dekkers in #3203
• Updated certifi (1.16) #3209 by @ammar92 in #3212
• Updated Django (1.16) by @ammar92 in #3218
• Backport: update nmap udp image as well by @Donnype in #3233
• cve-2024-6387 from RickGeex (#3194) by @noamblitz in #3237
• Fix: add related objects crash (1.16) by @dekkers in #3283
• [Backport] RFC3161HashRepository accepts rfc3161_provider only as a string and Pydantic URLs are not strings anymore by @Donnype in #3284
• Update Django (1.16) by @ammar92 in #3325
• Fix CSRF error in API with token auth (1.16) by @dekkers in #3328
• Fix new boefjes issue for scheduler (1.16) by @dekkers in #3329
• Fix broken token auth when 2FA is enabled (1.16) by @dekkers in #3327

New Contributors

• @Souf149 made their first contribution in #3089

Full Changelog: v1.14.0...v1.16.0