A simple text classification application, published solely as an artifact for the purpose of demonstrating a software bill of materials (SBOM) in SPDX 3.0 format.
Not recommended for critical text classification tasks.
The main content of this repository is the software bill of materials at bom.spdx3.json. Other files are given to complete the illustration.
SBOM demonstration design goals:
- Comprehensible: Small enough for a human to understand easily.
- Informative: Elaborate enough to showcase the use of various information fields within an SBOM.
- Testable: Designed to facilitate testing and evaluation against specific use case requirements.
Development is in the dev branch.
Will eventually be submitted to
spdx/spdx-examples repo.
├── LICENSE License information
├── README.md This README file
├── bom.spdx3.json Software bill of materials, in SPDX 3 format
├── data Dataset, preprocessed and tokenized
│ ├── test.txt Testing data
│ ├── train.txt Training data
│ └── valid.txt Validation data
├── evaluate.py A script to evaluate prediction performance
├── model.bin A sentiment analysis model
├── predict.py A script to predict a label of a text
├── preprocess.py A script to prepare training data
├── rawdata Raw dataset, before preprocessing
│ ├── test Testing data
│ │ ├── neg.txt Testing samples for label "neg" (negative)
│ │ ├── neu.txt Testing samples for label "neu" (neutral)
│ │ ├── pos.txt Testing samples for label "pos" (positive)
│ │ └── q.txt Testing samples for label "q" (question)
│ ├── train Training data
│ │ └── ...
│ └── valid Validation data
│ └── ...
├── requirements.txt List of required Python libraries
├── techdocs Technical documentation
│ ├── dataprepare.md Data prepration
│ └── instructions.md Instruction for use
└── train.py A script to build a model
The diagram is generated from a PlantUML file: bom.spdx.puml. The PlantUML file is generated by spdx3ToGraph. To save space, spdxIds and long strings are shortened by the shortenid.sh script in tools/, and all but one hyperparameter have been manually removed.
See instruction for use for how to use the application.
See data preparation.
- The energy used by the computer during model training is tracked by energy-tracker. It measures how much energy the computer uses during the training. This means the actual energy used for training the model might be a bit less than the reported amount.
- The SPDX 3.0 SBOM is validated structurally against the JSON Schema at https://spdx.org/schema/3.0.0/spdx-json-schema.json and semantically against the SHACL model at https://spdx.org/rdf/3.0.0/spdx-model.ttl.
- Next steps:
- Add external dependency relationships (e.g.
dependsOn,hasProvidedDependency) - Get tested with an SBOM quality check tool like sbomsq (once it supports SPDX 3.0).
- Using information requirements and obligations in the EU AI Act as a target, labeling all relevant properties and relationships with corresponding difficulty levels and support levels, based on the BOM Maturity Model.
- Add external dependency relationships (e.g.
Apart from the data and components listed in the table below, the code and content in this repository are dedicated to the public domain under the terms of Creative Commons Zero ("CC0") 1.0 Universal, which have no copyright and related or neighboring rights worldwide to the extent allowed by law.
| Component | Name | License | Notes |
|---|---|---|---|
| Training data | Wisesight Sentiment Corpus | Creative Commons Zero v1.0 Universal | Samples from the corpus are in rawdata/. Preprocessed data is in data/. See data preparation for details. |
| Text preprocessor | th-simple-preprocessor | Apache License 2.0 | |
| Word tokenizer | newmm-tokenizer | Apache License 2.0 | Inherited the license from PyThaiNLP. |
| Text classifier | fastText | MIT License | |
| Array package | NumPy | BSD License |
The specific version information can be found in requirements.txt.