Industrial Release 0.4.2 #141

v1docq · 2024-05-17T12:10:15Z

Add Set of Valid Pipelines for each task
Add NBEATS for ts forecasting
Add forecasting assumption strategy
Add unit-tests for pipelines
Bug fixes

LoRa impl

…_forecasting_advanced # Conflicts: # fedot_ind/core/architecture/preprocessing/data_convertor.py # fedot_ind/core/models/nn/network_impl/nbeats.py # fedot_ind/core/repository/constanst_repository.py # fedot_ind/core/repository/data/default_operation_params.json # fedot_ind/core/repository/data/industrial_model_repository.json # fedot_ind/core/repository/model_repository.py

LoRa impl

…_forecasting_advanced # Conflicts: # fedot_ind/core/models/nn/network_impl/nbeats.py # fedot_ind/core/models/nn/network_modules/layers/forecasting/nbeats.py # fedot_ind/core/operation/interfaces/industrial_model_strategy.py

LoRa impl

…_forecasting_advanced # Conflicts: # fedot_ind/core/repository/data/default_operation_params.json

pep8speaks · 2024-05-17T12:10:45Z

Thanks for update, @v1docq!

In the file examples/automl_example/api_example/computer_vision/image_classification/mnist_lora_example.py:

Line 1:1: F401 'fedot_ind.core.models.nn.network_modules.layers.lora.LoRAParametrization' imported but unused
Line 12:1: F401 'matplotlib.pyplot as plt' imported but unused
Line 200:121: E501 line too long (200 > 120 characters)
Line 257:121: E501 line too long (132 > 120 characters)

In the file examples/automl_example/api_example/time_series/ts_forecasting/m4_analiysis/ts_forecasting_m4_benchmark.py:

Line 26:37: W503 line break before binary operator

In the file [examples/benchmark_example/LoRA notebooks/lora_nbeats.py](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py):

[Line 22:5](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L22): F841 local variable 'train_loader' is assigned to but never used
[Line 31:5](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L31): F841 local variable 'test_loader' is assigned to but never used
[Line 36:10](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L36): F821 undefined name 'docnn_model'
[Line 36:31](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L36): F821 undefined name 'docnn_model'
[Line 36:52](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L36): F821 undefined name 'docnn_model'
[Line 37:5](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L37): F821 undefined name 'total_parameters_original'
[Line 44:1](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L44): F821 undefined name 'docnn_model_copy'
[Line 44:48](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L44): F821 undefined name 'updated_weight'
[Line 44:70](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L44): F821 undefined name 'device'
[Line 45:1](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L45): F821 undefined name 'docnn_model_copy'
[Line 45:48](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L45): F821 undefined name 'updated_weight'
[Line 45:70](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L45): F821 undefined name 'device'
[Line 46:1](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L46): F821 undefined name 'docnn_model_copy'
[Line 46:48](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L46): F821 undefined name 'updated_weight'
[Line 46:70](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L46): F821 undefined name 'device'
[Line 53:10](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L53): F821 undefined name 'docnn_model_copy'
[Line 53:36](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L53): F821 undefined name 'docnn_model_copy'
[Line 53:62](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L53): F821 undefined name 'docnn_model_copy'
[Line 59:121](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L59): E501 line too long (200 > 120 characters)
[Line 75:15](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L75): F821 undefined name 'transform'
[Line 85:1](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L85): F821 undefined name 'enable_disable_lora'
[Line 90:8](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L90): F821 undefined name 'docnn_model_copy'
[Line 90:43](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L90): F821 undefined name 'docnn_model_copy'
[Line 91:5](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L91): F821 undefined name 'docnn_model_copy'
[Line 92:5](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L92): F821 undefined name 'docnn_model_copy'
[Line 93:5](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L93): F821 undefined name 'docnn_model_copy'
[Line 96:1](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L96): F821 undefined name 'enable_disable_lora'
[Line 99:5](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L99): F821 undefined name 'docnn_model_copy'
[Line 100:5](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L100): F821 undefined name 'original_weights'
[Line 102:1](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L102): F821 undefined name 'enable_disable_lora'
[Line 103:1](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L103): F821 undefined name 'run'
[Line 106:1](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L106): F821 undefined name 'enable_disable_lora'
[Line 107:1](https://github.com/aimclub/Fedot.Industrial/blob/739dc93343ef62adfadde5807a56cfabb65fb14d/examples/benchmark_example/LoRA notebooks/lora_nbeats.py#L107): F821 undefined name 'run'

In the file examples/benchmark_example/time_siries_nbeats_forecast_benchmark.py:

Line 6:1: F401 'torch.nn' imported but unused

In the file examples/real_world_examples/industrial_examples/enviroment_monitoring/ts_forecasting/nbeats_forecasting.py:

Line 1:1: F401 'matplotlib' imported but unused

In the file fedot_ind/api/utils/industrial_strategy.py:

Line 10:1: F401 'golem.core.tuning.optuna_tuner.OptunaTuner' imported but unused
Line 126:30: F541 f-string is missing placeholders
Line 178:32: F841 local variable 'n_classes' is assigned to but never used

In the file fedot_ind/core/architecture/pipelines/abstract_pipeline.py:

Line 4:1: F401 'fedot_ind.core.repository.constanst_repository.VALID_LINEAR_CLF_PIPELINE' imported but unused
Line 8:1: F401 'fedot_ind.api.utils.path_lib.PATH_TO_DEFAULT_PARAMS' imported but unused
Line 9:1: F401 'fedot_ind.core.architecture.settings.pipeline_factory.BasisTransformations' imported but unused
Line 9:1: F401 'fedot_ind.core.architecture.settings.pipeline_factory.FeatureGenerator' imported but unused
Line 9:1: F401 'fedot_ind.core.architecture.settings.pipeline_factory.MlModel' imported but unused
Line 10:1: F401 'fedot_ind.core.metrics.evaluation.PerformanceAnalyzer' imported but unused
Line 167:121: E501 line too long (121 > 120 characters)
Line 168:121: E501 line too long (121 > 120 characters)

In the file fedot_ind/core/architecture/pipelines/classification.py:

Line 73:9: F841 local variable 'classificator_module' is assigned to but never used
Line 154:9: F841 local variable 'classificator_module' is assigned to but never used

In the file fedot_ind/core/architecture/preprocessing/data_convertor.py:

Line 98:13: F841 local variable 'e' is assigned to but never used
Line 296:13: F841 local variable 'e' is assigned to but never used
Line 595:13: F841 local variable 'e' is assigned to but never used

In the file fedot_ind/core/metrics/metrics_implementation.py:

Line 13:1: F811 redefinition of unused 'np' from line 3
Line 84:73: W503 line break before binary operator

In the file fedot_ind/core/models/nn/network_impl/lora_nn.py:

Line 191:9: F841 local variable 'parametrize_by_layer' is assigned to but never used
Line 221:9: F841 local variable 'lora_impl' is assigned to but never used

In the file fedot_ind/core/models/nn/network_impl/nbeats.py:

Line 155:121: E501 line too long (144 > 120 characters)
Line 158:121: E501 line too long (123 > 120 characters)
Line 159:121: E501 line too long (126 > 120 characters)

In the file fedot_ind/core/models/nn/network_modules/layers/forecasting/nbeats.py:

Line 4:1: F401 'typing.Optional' imported but unused

In the file fedot_ind/core/models/nn/network_modules/layers/lora.py:

Line 788:17: W503 line break before binary operator
Line 900:121: E501 line too long (147 > 120 characters)

In the file fedot_ind/core/operation/interfaces/industrial_model_strategy.py:

Line 15:1: F401 'scipy.stats.kurtosis' imported but unused
Line 16:1: F401 'scipy.stats.skew' imported but unused

In the file fedot_ind/core/operation/interfaces/industrial_preprocessing_strategy.py:

Line 46:121: E501 line too long (121 > 120 characters)

In the file fedot_ind/core/repository/constanst_repository.py:

Line 17:121: E501 line too long (121 > 120 characters)
Line 24:1: F403 'from fedot_ind.core.models.topological.topofeatures import ' used; unable to detect undefined names
Line 24:1: F401 'fedot_ind.core.models.topological.topofeatures.' imported but unused
Line 254:121: E501 line too long (127 > 120 characters)

In the file fedot_ind/core/repository/industrial_implementations/abstract.py:

Line 23:1: F401 'typing.Tuple' imported but unused
Line 23:1: F401 'typing.Sequence' imported but unused
Line 23:1: F401 'typing.Dict' imported but unused

In the file fedot_ind/core/repository/initializer_industrial_models.py:

Line 2:1: F401 'types' imported but unused
Line 8:121: E501 line too long (127 > 120 characters)
Line 17:1: F401 'golem.core.optimisers.genetic.operators.crossover.Crossover' imported but unused
Line 24:1: F401 'fedot_ind.core.repository.industrial_implementations.optimisation._crossover_by_type' imported but unused
Line 102:9: F841 local variable 'MutationStrengthEnum' is assigned to but never used

In the file fedot_ind/core/repository/model_repository.py:

Line 14:1: F403 'from fedot.core.operations.evaluation.operation_implementations.data_operations.sklearn_transformations import *' used; unable to detect undefined names
Line 16:121: E501 line too long (127 > 120 characters)
Line 27:1: F401 'fedot.core.operations.evaluation.operation_implementations.models.ts_implementations.statsmodels.GLMImplementation' imported but unused
Line 45:1: F401 'xgboost.XGBClassifier' imported but unused
Line 48:1: F401 'fedot_ind.core.models.nn.network_impl.chronos_tst.ChronosExtractor' imported but unused
Line 74:36: F405 'PCAImplementation' may be undefined, or defined from star imports: fedot.core.operations.evaluation.operation_implementations.data_operations.sklearn_transformations
Line 75:41: F405 'FastICAImplementation' may be undefined, or defined from star imports: fedot.core.operations.evaluation.operation_implementations.data_operations.sklearn_transformations
Line 76:46: F405 'PolyFeaturesImplementation' may be undefined, or defined from star imports: fedot.core.operations.evaluation.operation_implementations.data_operations.sklearn_transformations
Line 139:20: F405 'ScalingImplementation' may be undefined, or defined from star imports: fedot.core.operations.evaluation.operation_implementations.data_operations.sklearn_transformations
Line 140:26: F405 'NormalizationImplementation' may be undefined, or defined from star imports: fedot.core.operations.evaluation.operation_implementations.data_operations.sklearn_transformations
Line 142:30: F405 'ImputationImplementation' may be undefined, or defined from star imports: fedot.core.operations.evaluation.operation_implementations.data_operations.sklearn_transformations
Line 144:23: F405 'KernelPCAImplementation' may be undefined, or defined from star imports: fedot.core.operations.evaluation.operation_implementations.data_operations.sklearn_transformations

In the file fedot_ind/tools/example_utils.py:

Line 119:9: F841 local variable 'probs' is assigned to but never used

In the file tests/unit/api/main/test_api_main.py:

Line 159:5: F841 local variable 'labels' is assigned to but never used
Line 160:5: F841 local variable 'probs' is assigned to but never used
Line 164:5: F841 local variable 'curr_backend' is assigned to but never used

In the file tests/unit/models/forecasting_pipelines.py:

Line 1:1: F401 'numpy as np' imported but unused
Line 2:1: F401 'fedot.core.pipelines.pipeline_builder.PipelineBuilder' imported but unused
Line 5:1: F401 'fedot_ind.core.metrics.metrics_implementation.RMSE' imported but unused
Line 7:1: F401 'fedot_ind.core.repository.initializer_industrial_models.IndustrialModels' imported but unused

Comment last updated at 2024-05-17 15:27:24 UTC

# Conflicts: # examples/automl_example/api_example/time_series/ts_forecasting/ts_forecasting_example.py # fedot_ind/api/utils/checkers_collections.py # fedot_ind/api/utils/industrial_strategy.py # fedot_ind/core/architecture/pipelines/abstract_pipeline.py # fedot_ind/core/repository/constanst_repository.py # fedot_ind/core/repository/industrial_implementations/abstract.py # fedot_ind/tools/example_utils.py # fedot_ind/tools/loader.py

…_forecasting_advanced

…_forecasting_advanced # Conflicts: # fedot_ind/__init__.py

technocreep

добавил ещё удаление неиспользуемых переменных и импортов, на которые ругается pep8speaks

буду наблюдать дальше, возможно чуть позже сформирую конфиг для более тонкой настройки форматирования

codecov · 2024-05-17T15:39:51Z

Codecov Report

Attention: Patch coverage is 16.47577% with 948 lines in your changes are missing coverage. Please review.

Project coverage is 47.51%. Comparing base (4b271c1) to head (6a181dd).
Report is 10 commits behind head on main.

❗ Current head 6a181dd differs from pull request most recent head 9810561

Please upload reports for the commit 9810561 to get more accurate results.

Files	Patch %	Lines
..._ind/core/models/nn/network_modules/layers/lora.py	0.00%	442 Missing ⚠️
...ls/nn/network_modules/layers/forecasting/nbeats.py	20.94%	234 Missing ⚠️
fedot_ind/core/models/nn/network_impl/lora_nn.py	31.57%	78 Missing ⚠️
fedot_ind/core/models/nn/network_impl/nbeats.py	27.71%	60 Missing ⚠️
...d/core/architecture/pipelines/abstract_pipeline.py	0.00%	48 Missing ⚠️
.../repository/industrial_implementations/abstract.py	20.83%	19 Missing ⚠️
fedot_ind/tools/example_utils.py	6.66%	14 Missing ⚠️
fedot_ind/core/models/nn/network_impl/dummy_nn.py	26.66%	11 Missing ⚠️
.../operation/interfaces/industrial_model_strategy.py	33.33%	10 Missing ⚠️
.../core/architecture/preprocessing/data_convertor.py	38.46%	8 Missing ⚠️
... and 12 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #141      +/-   ##
==========================================
- Coverage   56.17%   47.51%   -8.67%     
==========================================
  Files         102      111       +9     
  Lines        9025    10731    +1706     
==========================================
+ Hits         5070     5099      +29     
- Misses       3955     5632    +1677

Flag	Coverage Δ
unittests	`47.51% <16.47%> (-8.67%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

technocreep and others added 30 commits May 6, 2024 12:00

fix versions

807b9e7

NBeats & LoRA implementation (#122)

dfb4206

LoRa impl

add LoRa strategy to industrial

68b36b1

lora fixes

9802c85

m4 analysis

4976633

NBeats & LoRA implementation (#122)

14989ad

LoRa impl

add LoRa strategy to industrial

0150069

lora fixes

28f45f7

m4 analysis

c3e9ad0

add unit_test, add nbeats, add valid linear pipelines

7f0d0be

NBeats & LoRA implementation (#122)

1999c03

LoRa impl

add LoRa strategy to industrial

02f6f58

lora fixes

7ac1ac5

m4 analysis

4bf37f4

add unit_test, add nbeats, add valid linear pipelines

9575a67

NBeats & LoRA implementation (#122)

c2256eb

LoRa impl

add LoRa strategy to industrial

949e2d5

nbeats refactoring

5dfb117

NBeats & LoRA implementation (#122)

5015b86

LoRa impl

add LoRa strategy to industrial

5b1eed2

lora fixes

430eac5

m4 analysis

62344ed

add unit_test, add nbeats, add valid linear pipelines

ee7325a

NBeats & LoRA implementation (#122)

356d468

LoRa impl

add LoRa strategy to industrial

3758b10

nbeats refactoring

884e1a8

NBeats & LoRA implementation (#122)

192ebff

LoRa impl

add LoRa strategy to industrial

6f96225

MisterZurg and others added 3 commits May 17, 2024 15:02

NBeats & LoRA implementation (#122)

0ea37ff

LoRa impl

add LoRa strategy to industrial

35309b1

Merge remote-tracking branch 'origin/ts_forecasting_advanced' into ts…

82173b4

…_forecasting_advanced # Conflicts: # fedot_ind/core/repository/data/default_operation_params.json

v1docq requested a review from technocreep May 17, 2024 12:10

v1docq and others added 15 commits May 17, 2024 15:17

delete abstract pipeline test

647a5b0

Automated autopep8 fixes

46392f4

prerelease fixes

e2cb7a6

Automated autopep8 fixes

3bb3246

fix api test

5001929

Merge remote-tracking branch 'origin/ts_forecasting_advanced' into ts…

30bbe4b

…_forecasting_advanced

fix isinstance

8d092fc

fix loop

a0ff806

update of pep8 action

819f496

updated action

739dc93

Automated autopep8 fixes

6a181dd

prevent unnecessary deletion of useful import

d62eed6

Merge remote-tracking branch 'origin/ts_forecasting_advanced' into ts…

437e8f3

…_forecasting_advanced # Conflicts: # fedot_ind/__init__.py

minor fix

9810561

technocreep approved these changes May 17, 2024

View reviewed changes

v1docq merged commit 474ed75 into main May 17, 2024
5 checks passed

v1docq deleted the ts_forecasting_advanced branch May 29, 2024 09:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Industrial Release 0.4.2 #141

Industrial Release 0.4.2 #141

v1docq commented May 17, 2024

pep8speaks commented May 17, 2024 •

edited

technocreep left a comment

codecov bot commented May 17, 2024

Industrial Release 0.4.2 #141

Industrial Release 0.4.2 #141

Conversation

v1docq commented May 17, 2024

pep8speaks commented May 17, 2024 • edited

Comment last updated at 2024-05-17 15:27:24 UTC

technocreep left a comment

Choose a reason for hiding this comment

codecov bot commented May 17, 2024

Codecov Report

pep8speaks commented May 17, 2024 •

edited