Skip to content

Commit

Permalink
Merge pull request #403 from crocs-muni/400-standardize-project-namin…
Browse files Browse the repository at this point in the history
…g-sec-certs

replace seccerts -> sec-certs
  • Loading branch information
J08nY committed Apr 29, 2024
2 parents 7fba82c + bec611d commit a4aa019
Show file tree
Hide file tree
Showing 20 changed files with 494 additions and 217 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,4 +48,4 @@ jobs:
zip -r docs.zip *
- name: Push docs to website
run: |
curl -F data=@docs/docs.zip https://seccerts.org/docs/upload?token=${{ secrets.DOCS_AUTH_TOKEN }}
curl -F data=@docs/docs.zip https://sec-certs.org/docs/upload?token=${{ secrets.DOCS_AUTH_TOKEN }}
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ You contribution is warmly welcomed. You can help by:

## Dependencies

For complete list of system dependencies, see [docs/installation](https://seccerts.org/docs/installation.html).
For complete list of system dependencies, see [docs/installation](https://sec-certs.org/docs/installation.html).

### Requirements

Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ A tool for data scraping and analysis of security certificates from Common Crite

<!-- This project is developed by the [Centre for Research On Cryptography and Security](https://crocs.fi.muni.cz) at Masaryk University, Czech Republic. -->

[![Website](https://img.shields.io/website?down_color=red&down_message=offline&style=flat-square&up_color=SpringGreen&up_message=online&url=https%3A%2F%2Fseccerts.org)](https://seccerts.org)
[![Website](https://img.shields.io/website?label=docs&down_color=red&down_message=offline&style=flat-square&up_color=SpringGreen&up_message=online&url=https%3A%2F%2Fseccerts.org/docs/index.html)](https://seccerts.org/docs/index.html)
[![Website](https://img.shields.io/website?down_color=red&down_message=offline&style=flat-square&up_color=SpringGreen&up_message=online&url=https%3A%2F%2Fsec-certs.org)](https://sec-certs.org)
[![Website](https://img.shields.io/website?label=docs&down_color=red&down_message=offline&style=flat-square&up_color=SpringGreen&up_message=online&url=https%3A%2F%2Fsec-certs.org/docs/index.html)](https://sec-certs.org/docs/index.html)
[![PyPI](https://img.shields.io/pypi/v/sec-certs?style=flat-square)](https://pypi.org/project/sec-certs/)
[![DockerHub](https://img.shields.io/docker/v/seccerts/sec-certs/latest?label=DockerHub&style=flat-square)](https://hub.docker.com/r/seccerts/sec-certs/tags)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/sec-certs?label=Python%20versions&style=flat-square)](https://pypi.org/project/sec-certs/)
Expand All @@ -16,7 +16,7 @@ A tool for data scraping and analysis of security certificates from Common Crite

## Installation

Use Docker with `docker pull seccerts/sec-certs` or just `pip install -U sec-certs && python -m spacy download en_core_web_sm`. For more elaborate description, see [docs](https://seccerts.org/docs/installation.html).
Use Docker with `docker pull seccerts/sec-certs` or just `pip install -U sec-certs && python -m spacy download en_core_web_sm`. For more elaborate description, see [docs](https://sec-certs.org/docs/installation.html).

## Usage

Expand All @@ -27,15 +27,15 @@ There are two main steps in exploring the world of security certificates:

For the first step, we currently provide CLI. For the second step, we provide simple API that can be used directly inside our Jupyter notebook or locally, together with a fully processed datasets that can be downloaded.

More elaborate usage is described in [docs/quickstart](https://seccerts.org/docs/quickstart.html). Also, see [example notebooks](https://github.com/crocs-muni/sec-certs/tree/main/notebooks/examples) either at GitHub or at docs. From docs, you can also run our notebooks in Binder.
More elaborate usage is described in [docs/quickstart](https://sec-certs.org/docs/quickstart.html). Also, see [example notebooks](https://github.com/crocs-muni/sec-certs/tree/main/notebooks/examples) either at GitHub or at docs. From docs, you can also run our notebooks in Binder.

## Data scraping

Run `sec-certs cc all` for Common Criteria processing, `sec-certs fips all` for FIPS 140 processing.

## Data analysis

Most probably, you don't want to fully process the certification artifacts by yourself. Instead, you can use our results and explore them as a data structure. An example snippet follows. For more, see [example notebooks](https://github.com/crocs-muni/sec-certs/tree/main/notebooks/examples). *Tip*: these can be run with Binder from our [docs](https://seccerts.org/docs/index.html).
Most probably, you don't want to fully process the certification artifacts by yourself. Instead, you can use our results and explore them as a data structure. An example snippet follows. For more, see [example notebooks](https://github.com/crocs-muni/sec-certs/tree/main/notebooks/examples). *Tip*: these can be run with Binder from our [docs](https://sec-certs.org/docs/index.html).

```python
from sec_certs.dataset import CCDataset
Expand Down
6 changes: 3 additions & 3 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Sec-certs documentation

Welcome to the technical documentation of *sec-certs* tool for the data analysis of products certified with Common Criteria or FIPS 140 frameworks. If you're looking for general description of the tool, its use cases and capabilites, we refer you to [sec-certs homepage](https://seccerts.org/). If you are looking for more advanced knowledge, e.g. how to mine your own data, how to extend the tool, and so forth, this is the right place.
Welcome to the technical documentation of *sec-certs* tool for the data analysis of products certified with Common Criteria or FIPS 140 frameworks. If you're looking for general description of the tool, its use cases and capabilites, we refer you to [sec-certs homepage](https://sec-certs.org/). If you are looking for more advanced knowledge, e.g. how to mine your own data, how to extend the tool, and so forth, this is the right place.

There are three main parts of this documentation. *User's guide* describes high-level use of our tool. Driven by this knowledge, you can progress to *Notebook examples* that showcase some of the API that we use in the form of Jupyter notebooks. The documentation also contains some of the modules documented with `autodoc`, see *API reference*. Still, some dark corners of our codebase are not documented. To inspect the code directly, see the [sec_certs](https://github.com/crocs-muni/sec-certs/tree/main/src/sec_certs) module. If you want, you can run the notebooks as they are stored in the [project repository](https://github.com/crocs-muni/sec-certs/tree/main/notebooks). If you are interested in contributing to our project or in other aspects of our development, you can consult the relevant *GitHub artifacts*

Expand All @@ -19,8 +19,8 @@ Each of the notebooks can be launched interactively in MyBinder by clicking on
:hidden:
:caption: Navigation
:maxdepth: 1
Seccerts homepage <https://seccerts.org/>
Seccerts docs <https://seccerts.org/docs>
Sec-certs homepage <https://sec-certs.org/>
Sec-certs docs <https://sec-certs.org/docs>
GitHub repo <https://anonymous.4open.science/r/sec-certs-7A92>
```

Expand Down
4 changes: 2 additions & 2 deletions docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ from sec_certs.dataset.cc import CCDataset

dset = CCDataset.from_web_latest()
```
to obtain to obtain freshly processed dataset from [seccerts.org](https://seccerts.org).
to obtain to obtain freshly processed dataset from [sec-certs.org](https://sec-certs.org).

3. Play with the dataset. See [example notebook](./notebooks/examples/cc.ipynb).
:::
Expand All @@ -23,7 +23,7 @@ from sec_certs.dataset.fips import FIPSDataset

dset = FIPSDataset.from_web_latest()
```
to obtain to obtain freshly processed dataset from [seccerts.org](https://seccerts.org).
to obtain to obtain freshly processed dataset from [sec-certs.org](https://sec-certs.org).

3. Play with the dataset. See [example notebook](./notebooks/examples/fips.ipynb).
:::
Expand Down
33 changes: 0 additions & 33 deletions docs/user_guide.md
Original file line number Diff line number Diff line change
@@ -1,34 +1 @@
# User's guide

```{important}
This guide is in the making.
```

## NVD datasets

Our tool matches certificates to their possible CVEs using datasets downloaded from [National Vulnerability Database (NVD)](https://nvd.nist.gov). If you're fully processing the `CCDataset` or `FIPSDataset` by yourself, you must somehow obtain the NVD datasets.

Our tool can seamlessly download the required NVD datasets when needed. We support two download mechanisms:

1. Fetching datasets with the [NVD API](https://nvd.nist.gov/developers/start-here) (preferred way).
1. Fetching snapshots from seccerts.org.

The following two keys control the behaviour:

```yaml
preferred_source_nvd_datasets: "api" # set to "sec-certs" to fetch them from seccerts.org
nvd_api_key: null # or the actual key value
```

If you aim to fetch the sources from NVD, we advise you to get an [NVD API key](https://nvd.nist.gov/developers/request-an-api-key) and set the `nvd_api_key` setting accordingly. The download from NVD will work even without API key, it will just be slow. No API key is needed when `preferred_source_nvd_datasets: "sec-certs"`


## Infering inter-certificate reference context

```{important}
This is an experimental feature.
```

We provide a model that can predict the context of inter-certificate references based on the text embedded in the artifacts. The model output is not incorporated into the `CCCertificate` instances, but can be dumped into a `.csv` file from where it can be correlated with a DataFrame of certificate features.

To train and deploy the model, it should be sufficient to change some paths and run the [prediction notebook](https://github.com/crocs-muni/sec-certs/blob/main/notebooks/cc/reference_annotations/prediction.ipynb). The output of this notebook is a `prediction.csv` file that can be loaded into the [references notebook](https://github.com/crocs-muni/sec-certs/blob/main/notebooks/cc/references.ipynb). This notebook documents the full analysis of references conducted on the Common Criteria certificates. Among others, the notebook generates some further `.csv` files that can subsequently be plotted via [plotting notebook](https://github.com/crocs-muni/sec-certs/blob/main/notebooks/cc/paper2_plots.ipynb).
Loading

0 comments on commit a4aa019

Please sign in to comment.