Support of multimodal data in DataSourceSplitter + bugfix #1119 #1138

kasyanovse · 2023-08-09T13:34:03Z

Add support of multimodal data to cross validation
Add stratification for classification by default (fix data split #1119)
All data splitter function parameters are unified (shuffle_flag in train_test_data_setup was retained for backward compatibility)
Many checks added in DataSourceSplitter and train_test_data_setup
core.validation.split.py module is moved to core.data.cv_folds.py
New tests are added
DataObjectiveAdvisor is deleted because stratification is enable by default
OneFoldInputDataSplit is deleted because cv_generator get checked parameters from DataSourceSplitter. Cross validation always may be done.
Some old tests rely on unstratified samples. DataSourceSplitter and train_test_data_setup are called
with stratify=False in that cases.

aim-pep8-bot · 2023-08-09T13:34:22Z

Hello @kasyanovse! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file test/unit/data/test_data_split.py:

Line 207:27: E126 continuation line over-indented for hanging indent
Line 223:27: E123 closing bracket does not match indentation of opening bracket's line

Comment last updated at 2023-08-14 13:15:41 UTC

codecov · 2023-08-10T07:04:31Z

Codecov Report

Merging #1138 (d77ecdb) into master (04bceac) will decrease coverage by 0.12%.
The diff coverage is 92.96%.

@@            Coverage Diff             @@
##           master    #1138      +/-   ##
==========================================
- Coverage   78.62%   78.51%   -0.12%     
==========================================
  Files         131      130       -1     
  Lines        9401     9336      -65     
==========================================
- Hits         7392     7330      -62     
+ Misses       2009     2006       -3

Files Changed	Coverage Δ
fedot/api/api_utils/params.py	`89.21% <ø> (-0.11%)`	⬇️
.../core/optimisers/objective/data_source_splitter.py	`92.20% <85.29%> (-3.57%)`	⬇️
fedot/core/data/data_split.py	`94.73% <93.84%> (+3.52%)`	⬆️
fedot/core/data/cv_folds.py	`100.00% <100.00%> (ø)`
...tations/data_operations/sklearn_transformations.py	`91.20% <100.00%> (ø)`

Fix #1119 Add support of MultiModalData in cv_folds Delete some code that solves problems are solved in the new DataSourceSplitter or in new cv_folds

Also some fixes in new code

Fix calls of changed functions, add correct parameters to some test, fix data splitting functions

andreygetmanov · 2023-08-14T09:32:32Z

fedot/core/data/cv_folds.py

+        in the InputData format for cross validation. The function
+        return a generator of tuples, consisting of a pair of train, test.
+
+    :param data: InputData for train and test splitting


Если data может быть и InputData, и MultiModalData, лучше определить обобщённее: data for train and test splitting

Исправлено.

andreygetmanov · 2023-08-14T09:37:15Z

fedot/core/optimisers/objective/data_source_splitter.py

+    Applied only for time series data.
+    If not provided, then value will be calculated.
+    :param random_seed: Random seed for shuffle.
+    :param log: Log for logging.


Порядок переменных в докстринге не совпадает с порядком переменных в классе. Так и должно быть?

Исправлено.

nicl-nno · 2023-08-14T13:01:24Z

Не забудь только pep8-замечания поправить.

fedot/core/optimisers/objective/data_source_splitter.py

kasyanovse · 2023-08-14T13:20:32Z

Не забудь только pep8-замечания поправить.

Поправил. Осталось два замечания, но с кодом в указанных местах все нормально.

kasyanovse linked an issue Aug 9, 2023 that may be closed by this pull request

fix data split #1119

Closed

kasyanovse changed the title ~~Refactor of data splitters~~ Ыupport of multimodal data + bugfix #1119 Aug 10, 2023

kasyanovse changed the title ~~Ыupport of multimodal data + bugfix #1119~~ Support of multimodal data + bugfix #1119 Aug 10, 2023

kasyanovse changed the title ~~Support of multimodal data + bugfix #1119~~ Support of multimodal data in DataSourceSplitter + bugfix #1119 Aug 10, 2023

aPovidlo approved these changes Aug 10, 2023

View reviewed changes

kasyanovse added 10 commits August 10, 2023 17:14

Refactor data splitters logic

5f3fbb2

Fix #1119 Add support of MultiModalData in cv_folds Delete some code that solves problems are solved in the new DataSourceSplitter or in new cv_folds

Rename module split.py to cv_folds.py

b5444f3

Move cv_folds.py to fedot/core/data

2a3a17e

Add tests

b731077

Also some fixes in new code

Add tests and fix some tests

4950dbe

Fixes

c9b2928

Fix calls of changed functions, add correct parameters to some test, fix data splitting functions

Fixes

e57097c

Fixes

6cb04df

Fixes

971617d

Fixes

0e3f875

kasyanovse force-pushed the 1119-bugfix branch from 0888f6d to 0e3f875 Compare August 10, 2023 14:15

nicl-nno requested a review from andreygetmanov August 10, 2023 14:23

Fix problem with lagged window failure on data shortage

b0c854f

andreygetmanov requested changes Aug 14, 2023

View reviewed changes

Fixes

d9002a8

kasyanovse requested a review from andreygetmanov August 14, 2023 10:13

andreygetmanov approved these changes Aug 14, 2023

View reviewed changes

Merge branch 'master' into 1119-bugfix

65e5990

nicl-nno reviewed Aug 14, 2023

View reviewed changes

fedot/core/optimisers/objective/data_source_splitter.py Outdated Show resolved Hide resolved

kasyanovse added 2 commits August 14, 2023 16:05

Fixes

e3da922

Fixes

d77ecdb

kasyanovse merged commit d53dfa1 into master Aug 14, 2023
6 checks passed

kasyanovse deleted the 1119-bugfix branch August 14, 2023 13:35

kasyanovse mentioned this pull request Aug 23, 2023

Balanced / unbalanced data #365

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support of multimodal data in DataSourceSplitter + bugfix #1119 #1138

Support of multimodal data in DataSourceSplitter + bugfix #1119 #1138

kasyanovse commented Aug 9, 2023 •

edited

aim-pep8-bot commented Aug 9, 2023 •

edited

codecov bot commented Aug 10, 2023 •

edited

andreygetmanov Aug 14, 2023

kasyanovse Aug 14, 2023

andreygetmanov Aug 14, 2023

kasyanovse Aug 14, 2023

nicl-nno commented Aug 14, 2023

kasyanovse commented Aug 14, 2023

Support of multimodal data in DataSourceSplitter + bugfix #1119 #1138

Support of multimodal data in DataSourceSplitter + bugfix #1119 #1138

Conversation

kasyanovse commented Aug 9, 2023 • edited

aim-pep8-bot commented Aug 9, 2023 • edited

Comment last updated at 2023-08-14 13:15:41 UTC

codecov bot commented Aug 10, 2023 • edited

Codecov Report

andreygetmanov Aug 14, 2023

Choose a reason for hiding this comment

kasyanovse Aug 14, 2023

Choose a reason for hiding this comment

andreygetmanov Aug 14, 2023

Choose a reason for hiding this comment

kasyanovse Aug 14, 2023

Choose a reason for hiding this comment

nicl-nno commented Aug 14, 2023

kasyanovse commented Aug 14, 2023

kasyanovse commented Aug 9, 2023 •

edited

aim-pep8-bot commented Aug 9, 2023 •

edited

codecov bot commented Aug 10, 2023 •

edited