Skip to content
This repository has been archived by the owner on Feb 2, 2023. It is now read-only.

Output separate YAML files for cisagov conversion #4

Merged
merged 2 commits into from
Jan 6, 2022

Conversation

mcdonnnj
Copy link
Member

@mcdonnnj mcdonnnj commented Jan 5, 2022

🗣 Description

This pull request changes the convert-cisagov script from printing the YAML for the convert Markdown to stdout to instead output a series of files in a pre-defined format. These files contain entries grouped by the first character (in uppercase) of the product vendor.

💭 Motivation and context

The resultant YAML file from converting the existing Markdown file is too large for the GitHub web UI to work with. There is an API limitation of 1MiB that a file needs to stay under for the web UI to allow in-browser editing/viewing. When the converted YAML is distributed across multiple files we can keep the individual file size to well below the 1MiB limit.

🧪 Testing

Automated tests pass. I confirmed that the processed Markdown is split up in the specified way when run locally.

✅ Pre-approval checklist

  • This PR has an informative and human-readable title.
  • Changes are limited to a single goal - eschew scope creep!
  • All relevant type-of-change labels have been added.
  • I have read the CONTRIBUTING document.
  • These code changes follow cisagov code standards.
  • All new and existing tests pass.

Instead of printing one large YAML file to stdout we instead group the
read software entires by the uppercased first character of the vendor
name and output a file for each of these groups. All non-alphabet
vendor entries are grouped in the "0-9" group manually. When testing
the only entries that were non-alphabet were numeric. This will generate
files that are small enough to be edited using the GitHub web UI.
@mcdonnnj mcdonnnj added the improvement This issue or pull request will add new or improve existing functionality label Jan 5, 2022
@mcdonnnj mcdonnnj self-assigned this Jan 5, 2022
@coveralls
Copy link

coveralls commented Jan 5, 2022

Pull Request Test Coverage Report for Build 1660884981

  • 2 of 21 (9.52%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-1.2%) to 31.579%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/mdyml/convert_cisagov.py 2 21 9.52%
Totals Coverage Status
Change from base Build 1656060700: -1.2%
Covered Lines: 100
Relevant Lines: 315

💛 - Coveralls

Copy link
Member

@jsf9k jsf9k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, but please consider my one suggestion.

src/mdyml/convert_cisagov.py Outdated Show resolved Hide resolved
Copy link
Member

@dav3r dav3r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! 🤩

Use a more general (and accurate) key for the group of vendors that
begin with a non-alphabetic character.

Co-authored-by: Shane Frasier <jeremy.frasier@trio.dhs.gov>
Co-authored-by: dav3r <david.redmin@trio.dhs.gov>
@mcdonnnj mcdonnnj merged commit 24ea411 into v1.1.0 Jan 6, 2022
@mcdonnnj mcdonnnj deleted the improvement/convert_cisagov_group_vendors branch January 6, 2022 15:53
@mcdonnnj mcdonnnj mentioned this pull request Jan 10, 2022
9 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
improvement This issue or pull request will add new or improve existing functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants