Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

0.5.4 #112

Merged
merged 54 commits into from
Jun 15, 2023
Merged

0.5.4 #112

merged 54 commits into from
Jun 15, 2023

Conversation

yiwen-h
Copy link
Member

@yiwen-h yiwen-h commented Jun 15, 2023 •
edited
Loading

New version has:

  • Improved test coverage - up to 47%
  • Attempted 2 layer sklearn model (targeting major categories before the minor categories), performed poorly
  • Improved train_test_splitting process so that all model types use the same split for the same random_seed for better comparability
  • Fixed how distilbert model handles categorical features using CategoryEncoder layer
  • Added analysis to bert pipeline so Excel performance and label files are also outputted
  • Added sentiment async endpoint to API
  • Updated params and dataset to v6 of framework

yiwen-h and others added 30 commits June 1, 2023 15:14
@yiwen-h
fixed xgb model outputting 2d np.ndarray for probabilities unlike oth…
2c6ebb0 
…er sklearn models which are 3d
@yiwen-h
added "labelling not possible" to params
2ca49a9
@yiwen-h
reduced params in gridsearch
acf814a
@yiwen-h
added two layer pipeline
1a1dc1d
@yiwen-h
added fillna to process comments with no FFT question
4550098
@yiwen-h
finalised two layer sklearn model
219eabb
@yiwen-h
Merge pull request #105 from CDU-data-science-team/101_two-model-layers
529cb6c 
101 two model layers
@yiwen-h
removed unnecessary extra models in current_best folder
a9930a9
@yiwen-h
added new tests for predict with sklearn, to prepare for predict with…
035443f 
… bert
@yiwen-h
fixed padding to "max_length" for bert_data_to_dataset
04ca301
@yiwen-h
added new function predict_multilabel_bert
a5cf9d9 
same output as predict_multilabel_sklearn - aim is to be able to use both for API. Also added tests
@yiwen-h
added more tests for predict_unlabelled_text
8853d06
@yiwen-h
added test for write_multilabel_models_and_metrics
e3b0db6
@yiwen-h
added test for write_model_preds
6e60fa7
@yiwen-h
added test for bert_write_model_preds
cc8d5f7
@yiwen-h
amended write_model_preds so it can write bert predictions too
f334cbf
@yiwen-h
added temppath to test_write_results
0c15946
@yiwen-h
fixed unnecessary param in docstring
d8526d9
@yiwen-h
test using same random split with merged categories, 3 different models
72c9ad9
@yiwen-h
coverage up to 44%
07d8e49
@yiwen-h
Merge pull request #106 from CDU-data-science-team/90_bert_analysis
871252a 
90 bert analysis
@yiwen-h
added new minor_cats labels and amended pipeline for v6 of framework
999a0ad
@yiwen-h
fixed wrong minor_cats in params and too many comments getting cleane…
bf05679 
…d in preprocessing due to missing values in q_type
@yiwen-h
train_test_split now exactly the same for all model types regardless …
6537588 
…of whether text is preprocessed
@yiwen-h
final pipeline for v6 of framework
ae4cc5a
@yiwen-h
fixed minor_cats which should still include Impact of treatment/ care
02e05d8
@yiwen-h
updated major_cats_dict with new v6 categories
fccc98c
@yiwen-h
added v5 labels specifically to API, rather than reading from params
81ab2f6
@yiwen-h
rewrote test_raw_data test ficture so it is flexible depending on min…
eb7dcf3 
…or_cats in params
@yiwen-h
Merge pull request #108 from CDU-data-science-team/107_v6_framework
e078e12 
107 v6 framework
yiwen-h and others added 24 commits June 13, 2023 10:14
@yiwen-h
formatted multilabel_pipeline with black, amended param for final v6 …
bd8c71c 
…dataset
@yiwen-h
final models for v6 framework
71d7bd8
@yiwen-h
Merge pull request #109 from CDU-data-science-team/107_v6_framework
60a1f29 
107 v6 framework
@yiwen-h
wrote the test first ;)
04939f3
@yiwen-h
added bert model to gitignore, updated test_api
f988978
@yiwen-h
added test for predict_sentiment_bert
5f08357
@yiwen-h
added test for predict_sentiment_bert
e7746c2
@yiwen-h
added predict_sentiment_bert and its test
46ca964
@yiwen-h
added predict_sentiment_bert and its test (formatted w black)
fbbdfae
@yiwen-h
took out test used for debugging from predict_unlabelled_text
2fe5bc6
@yiwen-h
api partly done. have discovered bug with how q_type is onehotted in …
76e9482 
…bert pipeline
@yiwen-h
fixing bert_additional_feats with CategoricalEncoder layer
c5fcd76
@yiwen-h
converting 0-4 scale of sentiment back to 1-5 as in original data
d157b05
@yiwen-h
Better names for ItemOut type definitions for sentiment vs multilabel…
5dedda1 
… API endpoints
@yiwen-h
api with sentiment complete
ff5b275 
still basic - not using async
@yiwen-h
updated coverage (47%)
186976b
@yiwen-h
fixed bug with predict_sentiment_bert crashing if preprocess_text = F…
d3059bf 
…alse
@yiwen-h
made sentiment API async
ca009cf
@yiwen-h
added updated coverage file
ccab149
@yiwen-h
amended test_api to use Mocked model as distilbert too large for github
9f4713e
@yiwen-h
Merge pull request #111 from CDU-data-science-team/97_sentiment_API
77bb495 
97 sentiment api
@yiwen-h
updated poetry.lock for 0.5.4
5c444ab
@yiwen-h
updated version number in pyproject.toml
1d5582f
@yiwen-h
updated requirements.txt in api folder
842271f
@yiwen-h yiwen-h requested a review from ChrisBeeley June 15, 2023 11:32
@yiwen-h yiwen-h merged commit 0b65e33 into main Jun 15, 2023
@yiwen-h yiwen-h deleted the 0.5.4 branch June 15, 2023 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ChrisBeeley ChrisBeeley Awaiting requested review from ChrisBeeley

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@yiwen-h