Refactor get_training_data for simplicity #115

MarkDavidson · 2021-12-01T12:51:54Z

This change is to simplify how training data is retrieved from the database so that future improvements are easier. I ran this locally and the application behaved as expected. For manual testing, I trained a model and uploaded reports.

What changed:

get_training_data now returns an X, y tuple instead of objects. The X, y tuple has better ergonomics for the SKLearn pipeline
test() and train() both call get_training_data() directly
Removed dead code (y_counts) in SKLearnModel.test
Deleted SKLearnModel._load_and_vectorize_data. It has been replaced by the improved get_training_data().
Replaced _preprocess_text() with lemmatize().
- Input changed from a list of sentences to a single sentence
- Output changed from a list of lemmatized sentences to a single lemmatized sentence

Limitations:

Project test coverage dropped by 0.15% because I deleted covered lines.

codecov · 2021-12-01T12:57:34Z

Codecov Report

Merging #115 (01f0520) into master (cf70164) will decrease coverage by 0.14%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #115      +/-   ##
==========================================
- Coverage   94.29%   94.14%   -0.15%     
==========================================
  Files          17       17              
  Lines         824      803      -21     
==========================================
- Hits          777      756      -21     
  Misses         47       47

Impacted Files	Coverage Δ
src/tram/tram/ml/base.py	`91.83% <100.00%> (-0.82%)`	⬇️
src/tram/tram/models.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf70164...01f0520. Read the comment docs.