Skip to content
This repository has been archived by the owner on Feb 14, 2024. It is now read-only.

DB_Schema update: Data_source table, PSHTT, and combined credential domain alerts tables #126

Merged
merged 6 commits into from
Mar 10, 2022

Conversation

aloftus23
Copy link
Contributor

@aloftus23 aloftus23 commented Jan 19, 2022

🗣 Description

Added data_source table and got rid of any source specific tables. For example, added a data_source column to the "credential_exposures" table to specify the source: either cybersix and hibp (instead of a hibp_exposed_credential table)

Added PSHTT table.
fixes #107
fixes #110

💭 Motivation and context

Normalize the db schema.

🧪 Testing

Successfully built the database using this schema.

✅ Pre-approval checklist

  • This PR has an informative and human-readable title.
  • Changes are limited to a single goal - eschew scope creep!
  • All future TODOs are captured in issues, which are referenced
    in code comments.
  • All relevant type-of-change labels have been added.
  • I have read the CONTRIBUTING document.
  • These code changes follow cisagov code standards.
  • All relevant repo and/or project documentation has been updated
    to reflect the changes in this PR.
  • Tests have been added and/or modified to cover the changes in this PR.
  • All new and existing tests pass.

- Get rid of any source specific tables and add a datasource column to each
@aloftus23 aloftus23 added the improvement This issue or pull request will add or improve functionality, maintainability, or ease of use label Jan 19, 2022
@aloftus23 aloftus23 self-assigned this Jan 19, 2022
@coveralls
Copy link

coveralls commented Jan 19, 2022

Pull Request Test Coverage Report for Build 1964239676

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 17.348%

Totals Coverage Status
Change from base Build 1952778987: 0.0%
Covered Lines: 139
Relevant Lines: 766

💛 - Coveralls

@stewartl97 stewartl97 added this to PR in Sprint 9 (1/16 - 3/1) via automation Jan 20, 2022
@stewartl97 stewartl97 added the Tech Debt Issue or PR from Previous Sprints label Jan 20, 2022
@aloftus23 aloftus23 changed the title Add data_source to db schema DB_Schema update: Data_source table, PSHTT, and more Mar 8, 2022
@aloftus23 aloftus23 changed the title DB_Schema update: Data_source table, PSHTT, and more DB_Schema update: Data_source table, PSHTT, and combined credential domain alerts tables Mar 8, 2022
@aloftus23 aloftus23 marked this pull request as ready for review March 8, 2022 17:52
Copy link
Member

@dav3r dav3r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine to me, though I did have one question about timestamps.

src/pe_reports/data/data_schema.sql Outdated Show resolved Hide resolved
Copy link
Collaborator

@cduhn17 cduhn17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aloftus23
One small adjustment to the column name change at shodan_verified_vulns.

Co-authored-by: cduhn17 <87447403+cduhn17@users.noreply.github.com>
@cduhn17 cduhn17 merged commit 3b3f797 into develop Mar 10, 2022
Sprint 9 (1/16 - 3/1) automation moved this from PR to Done Mar 10, 2022
@cduhn17 cduhn17 deleted the AL-datasource-to-dbschema branch March 10, 2022 17:36
@cduhn17
Copy link
Collaborator

cduhn17 commented Mar 10, 2022

Thank you everyone for your continued diligence and support in moving PR's along!

aloftus23 pushed a commit that referenced this pull request Apr 5, 2022
DB_Schema update:  Data_source table, PSHTT, and combined credential domain alerts tables
aloftus23 pushed a commit that referenced this pull request Apr 15, 2022
DB_Schema update:  Data_source table, PSHTT, and combined credential domain alerts tables
aloftus23 pushed a commit that referenced this pull request Apr 27, 2022
DB_Schema update:  Data_source table, PSHTT, and combined credential domain alerts tables
cisagovbot pushed a commit that referenced this pull request Apr 26, 2023
…nd-wheel-with-pip

Install/upgrade setuptools and wheel when upgrading pip
cisagovbot pushed a commit that referenced this pull request Dec 19, 2023
Add support for Python version 3.12 in build workflow
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
improvement This issue or pull request will add or improve functionality, maintainability, or ease of use Tech Debt Issue or PR from Previous Sprints
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

Modify Database Tables to Use DataSource Table Add Table Called DataSource_Table to P&E Schema
5 participants