Skip to content

[DOP-5632] Split FileLimit to separate limits#44

Merged
dolfinus merged 1 commit intodevelopfrom
feature/DOP-5632
May 26, 2023
Merged

[DOP-5632] Split FileLimit to separate limits#44
dolfinus merged 1 commit intodevelopfrom
feature/DOP-5632

Conversation

@dolfinus
Copy link
Member

@dolfinus dolfinus commented May 25, 2023

  • Added new onetl.file.limit module
  • Replaced old class FileLimit(count_limit=10) class with MaxFilesCount(10), placed new class to new module, added documentation & unit tests.
  • Moved limits_reached to new module, renamed to limits_stop_at, to show that it checks a specific path against limits, added unit tests & documentation.
  • Added new functions limits_reached and reset_limits which previously were a part of FileConnection.list_dir and .walk. They can help other developers to implement new modules in the future. Documented and tested them as well.
  • Added documentation for BaseFileLimit
  • Changed a bit the behavior of BaseFileLimit.reset() method - now the return value is used as a new limit, so it now depends on developer, should it resett a limit object state in-place or create a copy.

Documentation: https://onetl--44.org.readthedocs.build/en/44/file_limit/index.html

@dolfinus dolfinus force-pushed the feature/DOP-5632 branch 3 times, most recently from 3f571b3 to 0e15d0f Compare May 25, 2023 16:48
@dolfinus dolfinus marked this pull request as ready for review May 25, 2023 16:50
@codecov
Copy link

codecov bot commented May 25, 2023

Codecov Report

Merging #44 (f104a5f) into develop (bafe668) will increase coverage by 0.09%.
The diff coverage is 97.67%.

@@             Coverage Diff             @@
##           develop      #44      +/-   ##
===========================================
+ Coverage    92.94%   93.04%   +0.09%     
===========================================
  Files          120      124       +4     
  Lines         5770     5822      +52     
  Branches      1001     1006       +5     
===========================================
+ Hits          5363     5417      +54     
+ Misses         313      312       -1     
+ Partials        94       93       -1     
Impacted Files Coverage Δ
onetl/base/base_file_filter.py 100.00% <ø> (ø)
onetl/core/file_filter/file_filter.py 97.59% <ø> (ø)
onetl/core/file_limit/__init__.py 100.00% <ø> (ø)
...netl/connection/file_connection/file_connection.py 93.35% <75.00%> (+0.63%) ⬆️
onetl/base/base_file_limit.py 100.00% <100.00%> (ø)
onetl/core/file_limit/file_limit.py 100.00% <100.00%> (ø)
onetl/file/filter/match_all_filters.py 100.00% <100.00%> (ø)
onetl/file/limit/__init__.py 100.00% <100.00%> (ø)
onetl/file/limit/limits_reached.py 100.00% <100.00%> (ø)
onetl/file/limit/limits_stop_at.py 100.00% <100.00%> (ø)
... and 2 more

@dolfinus dolfinus force-pushed the feature/DOP-5632 branch 2 times, most recently from 7dfa680 to ca5439c Compare May 25, 2023 18:01
@dolfinus dolfinus force-pushed the feature/DOP-5632 branch from ca5439c to f104a5f Compare May 26, 2023 07:39
@dolfinus dolfinus requested a review from andy-takker May 26, 2023 08:53
@dolfinus dolfinus merged commit 7ee0878 into develop May 26, 2023
@dolfinus dolfinus deleted the feature/DOP-5632 branch May 26, 2023 09:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants