0.5.0

What's Changed

• EUCC scraping improvements by @tkachyna in #558
• CC/EUCC fix cert metadata extraction (wrongly assigned with report metadata) by @adamjanovsky in #621
• Scrape FIPS algorithm data by @adamjanovsky in #616
• Bump pypdf from 6.9.1 to 6.9.2 by @dependabot[bot] in #613
• Bump requests from 2.32.5 to 2.33.0 by @dependabot[bot] in #614
• Bump cryptography from 46.0.5 to 46.0.7 by @dependabot[bot] in #623
• Bump aiohttp from 3.13.3 to 3.13.4 by @dependabot[bot] in #619
• Bump pygments from 2.19.2 to 2.20.0 by @dependabot[bot] in #617
• Imrove EUCC scraping: Add missing product description; Improve scraping of contact information by @tkachyna in #622
• Bump pytest from 9.0.2 to 9.0.3 by @dependabot[bot] in #628
• Bump pillow from 12.1.1 to 12.2.0 by @dependabot[bot] in #627
• Bump pypdf from 6.9.2 to 6.10.0 by @dependabot[bot] in #626
• Bump pypdf from 6.10.0 to 6.10.2 by @dependabot[bot] in #633
• Bump mako from 1.3.10 to 1.3.11 by @dependabot[bot] in #634
• Introduce DocumentState for FIPSCertificate class by @adamjanovsky in #629
• Bump lxml from 6.0.2 to 6.1.0 by @dependabot[bot] in #637
• Bump pip from 26.0.1 to 26.1 by @dependabot[bot] in #640
• Bump mako from 1.3.11 to 1.3.12 by @dependabot[bot] in #641
• Bump urllib3 from 2.6.3 to 2.7.0 by @dependabot[bot] in #644
• ci: bump actions to latest versions by @jborsky in #645
• Bump idna from 3.11 to 3.15 in the uv group across 1 directory by @dependabot[bot] in #647
• Add tests + scraping changes for EUCC by @tkachyna in #638
• Bump docling from 2.80.0 to 2.91.0 in the uv group across 1 directory by @dependabot[bot] in #649
• Bump the uv group across 1 directory with 2 updates by @dependabot[bot] in #655
• fix: make test_process_parallel_callback order-independent by @m-mikulica in #657
• Fix EUCC date parsing by @jborsky in #648
• fix: set convert_ok before extraction by @jborsky in #650
• fix: handle missing package key by @jborsky in #661
• Bump pypdf from 6.10.2 to 6.12.0 in the uv group across 1 directory by @dependabot[bot] in #658
• build(deps): bump the uv group across 1 directory with 8 updates by @dependabot[bot] in #664
• 2.1. Scrape data about certified PPs from NIAP portal by @m-mikulica in #659
• Modified certificate regex for the Netherlands; add certificate regex for Sweden by @tkachyna in #676
• build(deps): bump pip from 26.1 to 26.1.2 in the uv group across 1 directory by @dependabot[bot] in #678
• Fix: EUCC cert_id bug by @m-mikulica in #673
• Update EUCC regexes by @tkachyna in #686
• build(deps): bump the uv group across 1 directory with 3 updates by @dependabot[bot] in #683
• build(deps): bump the uv group across 1 directory with 5 updates by @dependabot[bot] in #687
• build(deps): bump pypdf from 6.14.2 to 6.15.0 in the uv group across 1 directory by @dependabot[bot] in #688
• Add documentation for automated analysis (heuristics) by @jborsky in #696
• build(deps): bump nltk from 3.10.0 to 3.10.3 in the uv group across 1 directory by @dependabot[bot] in #697
• build(deps): bump the uv group across 1 directory with 4 updates by @dependabot[bot] in #698
• Incremental updating by @jborsky in #690
• Add a document layer by @jborsky in #691
• feat: allow finer control of aux processing modes by @jborsky in #700
• build(deps): bump accelerate from 1.13.0 to 1.15.0 in the uv group across 1 directory by @dependabot[bot] in #706
• build(deps): bump the uv group across 1 directory with 3 updates by @dependabot[bot] in #707
• 2.7. Improve accuracy of scraping NSCIB Common Criteria report frontpages by @tkachyna in #689
• fix: quote null key in rules.yaml by @m-mikulica in #709
• Extract algorithms from policy tables using Docling by @jborsky in #692
• feat: automatically install spacy model by @jborsky in #711
• Br1 extraction by @ndavidova in #537
• fix: scrape all EUCC pages by @jborsky in #715

New Contributors

• @ndavidova made their first contribution in #537

Full Changelog: 0.4.1...0.5.0

0.4.1

What's Changed

• Fix dockerbuild by @jborsky in #611
• Accept CVEs with status: 'Deferred' into the dataset by @adamjanovsky in #610
• Fix json_path not being set before serialization in CVEDatasetHandler/CPEDatasetHandler by @jborsky in #612

Full Changelog: 0.4.0...0.4.1

0.4.0

What's Changed

• Use tmpdir with enough free space. by @J08nY in #476
• Get rid of RETURNCODE_OK. This is not C99. by @J08nY in #477
• Fix mutable default state. by @J08nY in #478
• Get rid of DUMMY_NONEXISTING_PATH by @J08nY in #479
• Fix ReferenceSegmentExtractor. by @J08nY in #486
• Move flaky remote tests into "scheduled" workflow. by @J08nY in #487
• Fix aux dset dir warning. by @J08nY in #488
• Fix cicrular import issue. by @J08nY in #489
• Demonstration notebook of annotation of references with LLMs by @fedorst in #481
• chore(deps): bump jinja2 from 3.1.5 to 3.1.6 in /requirements by @dependabot[bot] in #492
• Semantic search demo by @fedorst in #491
• Improve BSI cert_id regex. by @J08nY in #493
• Add third paper to README. by @J08nY in #494
• Fix/schemes again2 by @J08nY in #495
• chore(deps): bump h11 from 0.14.0 to 0.16.0 in /requirements by @dependabot[bot] in #497
• Fix tqdm ignoring settings. by @J08nY in #496
• chore(deps): bump transformers from 4.49.0 to 4.50.0 in /requirements by @dependabot[bot] in #498
• chore(deps): bump tornado from 6.4.2 to 6.5.1 in /requirements by @dependabot[bot] in #500
• chore(deps): bump requests from 2.32.3 to 2.32.4 in /requirements by @dependabot[bot] in #503
• Add version warning for dataset compatibility by @J08nY in #505
• chore(deps): bump urllib3 from 2.3.0 to 2.5.0 in /requirements by @dependabot[bot] in #507
• chore(deps): bump aiohttp from 3.11.13 to 3.12.14 in /requirements by @dependabot[bot] in #510
• Run CI on various Python versions. by @J08nY in #511
• Make a JSON schema for CC Certificate and Dataset objects. by @J08nY in #516
• feat: add EUCC regexes for certificate IDs by @tkachyna in #524
• chore(deps): bump torch from 2.6.0 to 2.7.1 in /requirements by @dependabot[bot] in #504
• chore(deps): bump transformers from 4.50.0 to 4.53.0 in /requirements by @dependabot[bot] in #512
• WIP: add MLKEM into PQC keywords by @petrs in #467
• Add Copilot instructions (main) by @Copilot in #520
• Ignore CVSS V4.0 metrics in CVE loading. by @J08nY in #529
• Move to uv by @J08nY in #530
• Fix bad list join in CVSS parsing. by @J08nY in #534
• Fix the scheme download again. by @J08nY in #535
• Fix CC scheme mapping for NIAP certs. by @J08nY in #538
• Docling conversion by @jborsky in #525
• Fix SECCERTS-21H. by @J08nY in #541
• Preparing for EUCC: Moving heuristics functions to a common file by @tkachyna in #536
• Defer docling imports by @adamjanovsky in #548
• refactor: add CVSS v4.0 support to CVE sample by @adamjanovsky in #549
• Preparing for EUCC: Moving convert and extraction functions to common files by @tkachyna in #550
• Fix failing docling tests due to different outputs by @jborsky in #551
• Add fetching and processing for EUCC Certificates + refactoring the rest of the pipeline by @tkachyna in #552
• Fix compatibility with pandas 3.0 by @jborsky in #563
• Refine regex patterns for post-quantum crypto by @J08nY in #594
• build/update deps by @jborsky in #595
• Bump nltk from 3.9.1 to 3.9.3 by @dependabot[bot] in #596
• Bump tornado from 6.5.1 to 6.5.5 by @dependabot[bot] in #597
• Bump pypdf from 5.7.0 to 6.9.1 by @dependabot[bot] in #598
• build: update uv.lock by @jborsky in #607
• New MIP statuses by @jborsky in #565

Full Changelog: 0.3.2...0.4.0

0.3.2

What's Changed

• Fix dockerhub release by @adamjanovsky in #475

Full Changelog: 0.3.1...0.3.2

0.3.1

What's Changed

• replace seccerts -> sec-certs by @adamjanovsky in #403
• Artifacts for chain-of-trust paper by @adamjanovsky in #404
• Add paper preprint links to readme and citation file by @adamjanovsky in #407
• add final cose citation by @adamjanovsky in #410
• Fix/cc link compat by @J08nY in #412
• Allow for download through our caching proxy at sec-certs.org by @J08nY in #413
• Fix NVD tests. by @J08nY in #414
• Fix FIPS pipeline for new certs. by @J08nY in #417
• Try to fix FIPS alg search. by @J08nY in #418
• Fix FIPS aplgo download column parse. by @J08nY in #421
• New digests by @J08nY in #424
• Update chain-of-trust citation in readme by @adamjanovsky in #426
• Prefer links from HTML in CC. by @J08nY in #427
• Adding additional regexes for PQC based on recent certificates by @petrs in #433
• Add more PQC rules. by @J08nY in #434
• Fix NVD api download. by @J08nY in #436
• Bump deprecated CI action version. by @J08nY in #437
• Notebook with icmc 2024 presentation plots by @adamjanovsky in #439
• Chain of trust plots extended version by @adamjanovsky in #438
• Fix CC scheme downloads by @J08nY in #442
• Fix mapping of certs to PPs broken due to CC URL change. by @J08nY in #444
• Add a way to download full dataset archive (including PDFs) from the web. by @J08nY in #446
• Avoid double serialization in CVE and CPE dataset load. by @J08nY in #447
• Add Qatar to cert ID rules. by @J08nY in #448
• Fix Qatar really! by @J08nY in #449
• Awesome curated searches v2 by @J08nY in #450
• Fix/schemes again by @J08nY in #456
• Add computation of previous and next certificate versions based on ID. by @J08nY in #459
• Add PP processing (rebased) by @J08nY in #469
• Pre-release fixes by @adamjanovsky in #473

Full Changelog: 0.2.1...0.3.1

0.2.1

Important changes

• Cleanup cert-id rules by @J08nY in #386
• Add extraction of certificate data. by @J08nY in #389
• Reference notebook by @adamjanovsky in #392

Small changes

• Fix a bunch of deprecation warnings by @J08nY in #385
• Add France archived list. by @J08nY in #384
• introduce dockerhub environment by @adamjanovsky in #377
• document reference context inference by @adamjanovsky in #394

Dependabot updates

• chore(deps): bump cryptography from 41.0.5 to 41.0.6 in /requirements by @dependabot in #378
• chore(deps): bump transformers from 4.35.2 to 4.36.0 in /requirements by @dependabot in #380
• chore(deps): bump jinja2 from 3.1.2 to 3.1.3 in /requirements by @dependabot in #381
• chore(deps): bump pillow from 10.1.0 to 10.2.0 in /requirements by @dependabot in #382
• chore(deps): bump aiohttp from 3.9.0 to 3.9.2 in /requirements by @dependabot in #383
• chore(deps): bump cryptography from 41.0.6 to 42.0.0 in /requirements by @dependabot in #387
• chore(deps): bump cryptography from 42.0.0 to 42.0.2 in /requirements by @dependabot in #391

Full Changelog: 0.1.6...0.2.0

0.1.6

What's Changed

• add some latex commands previously missed from notebooks by @adamjanovsky in #341
• new LaTeX commands vuln. notebook by @adamjanovsky in #342
• Bump requests from 2.28.1 to 2.31.0 in /requirements by @dependabot in #340
• fix typo in docs by @adamjanovsky in #344
• Bump tornado from 6.2 to 6.3.2 in /requirements by @dependabot in #343
• bump action for dockerhub upload by @adamjanovsky in #347
• Bump scipy from 1.9.3 to 1.10.0 in /requirements by @dependabot in #346
• don't clone in dockerfile, copy local repo contents by @adamjanovsky in #348
• Bump pygments from 2.13.0 to 2.15.0 in /requirements by @dependabot in #350
• Bump certifi from 2022.12.7 to 2023.7.22 in /requirements by @dependabot in #352
• Bump tornado from 6.3.2 to 6.3.3 in /requirements by @dependabot in #354
• anonymize the docs by @adamjanovsky in #355
• Add profiling by @J08nY in #353
• Analysis of FIPS references by @GeorgeFI in #351
• chore(deps): bump urllib3 from 1.26.13 to 1.26.17 in /requirements by @dependabot in #359
• chore(deps): bump pillow from 9.3.0 to 10.0.1 in /requirements by @dependabot in #362
• feat: Added pytesseract wrapper by @GeorgeFI in #357
• chore(deps): bump urllib3 from 1.26.17 to 1.26.18 in /requirements by @dependabot in #366
• Wrap CC scheme download in try-except and continue on fail. by @J08nY in #371
• Fix/better log filename and requirements file by @adamjanovsky in #369
• switch black -> ruff format by @adamjanovsky in #373
• Bump req python to 3 10 by @adamjanovsky in #374
• Reference analysis by @adamjanovsky in #322
• Add multi-architecture docker builds by @adamjanovsky in #376

Full Changelog: 0.1.5...0.1.6

0.1.5

What's Changed

• coerce problematic datetime values in cert csvs/dfs/htmls by @adamjanovsky in #334
• pypi: switch to trusted publishing by @adamjanovsky in #336

Full Changelog: 0.1.4...0.1.5

0.1.4

What's Changed

• Replace linters with ruff by @adamjanovsky in #308
• Deanonymize docs and ack IGA by @adamjanovsky in #309
• Pin linter versions by @adamjanovsky in #311
• fix json deserialization due to cert path handling by @adamjanovsky in #312
• add EU/MSMT logo by @adamjanovsky in #313
• train/valid/test split for reference annotations by @adamjanovsky in #314
• build(deps): bump ipython from 8.7.0 to 8.10.0 in /requirements by @dependabot in #315
• bump mypy to 1.0.0 by @adamjanovsky in #316
• Introduce-reference-bool by @adamjanovsky in #317
• Feat: add info retrieval split including labeling metadata by @dmacko232 in #318
• build(deps-dev): bump markdown-it-py from 2.1.0 to 2.2.0 in /requirements by @dependabot in #319
• Change sampled images to lower amount of test images. by @dmacko232 in #320
• Feat cpe configurations by @GeorgeFI in #300
• delay and prune look-up dict build in CVEDataset by @adamjanovsky in #321
• Fix #323. by @J08nY in #325
• switch to pydantic in settings management by @adamjanovsky in #326
• Add CC scheme and FIPS IUT/MIP matching by @J08nY in #329
• Switch from NVD json feeds to API by @adamjanovsky in #328
• Fix path fixtures yielding. by @J08nY in #330
• poc notebook by @adamjanovsky in #332
• release demo notebook in docs by @adamjanovsky in #333

New Contributors

• @dmacko232 made their first contribution in #318

Full Changelog: 0.1.3...0.1.4

0.1.3

What's Changed

• Issue/298 ocr processing fails, delete draft release action, download spacy model during test pipeline by @adamjanovsky in #299
• fix broken badges by @adamjanovsky in #301
• Fix/badge targets by @adamjanovsky in #302
• Fix/cli arguments by @adamjanovsky in #303
• Fix #304. by @J08nY in #305
• Fix/pypdf and cli fixes by @adamjanovsky in #306
• jsonschema: set min. value of n_threads = -1 by @adamjanovsky in #307

Full Changelog: 0.1.2...0.1.3