Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix dev.deprecated and refine its unit tests #647

Merged
merged 15 commits into from
Apr 11, 2024

Conversation

DanielYang59
Copy link
Contributor

@DanielYang59 DanielYang59 commented Mar 31, 2024

Summary:

tests/test_dev.py::TestDecorator::test_install_except_hook
  /Users/runner/work/monty/monty/monty/dev.py:185: UserWarning: Cannot install excepthook, IPyhon.core.ultratb not available
    warnings.warn("Cannot install excepthook, IPyhon.core.ultratb not available")

Summary by CodeRabbit

  • New Features
    • Added ipython==8.22.2 to optional dependencies for enhanced interactive console features.
  • Bug Fixes
    • Improved handling of deadlines and deprecation warnings in development tools, ensuring clearer communication to developers.
  • Refactor
    • Updated test suites for better coverage and clarity in testing deprecation and deadline functionalities.
    • Streamlined code by removing unused imports and unnecessary conditional blocks in test files.

explicitly check for None

fix string format in `test_io`

add missing f for format string

fix date format

fix datetime in string

DEBUG: comment out no warn case 3

DEBUG: comment out no warn case 2

DEBUG: comment out no warn case

try to fix warning

fix time patch

try to fix date patch

revert to monkeypatch
Copy link

coderabbitai bot commented Mar 31, 2024

Walkthrough

The recent modifications focus on enhancing the codebase by refining deadline handling and deprecation warnings, updating dependencies, and improving test coverage and efficiency. These changes streamline development practices, ensuring better maintenance and compliance with modern Python standards, such as the use of f-strings and the removal of unnecessary code and imports.

Changes

Files Summary
monty/dev.py Updated deadline handling and deprecation warnings, including f-string usage and CI checks.
requirements-optional.txt Added ipython==8.22.2 to dependencies.
tests/test_dev.py Enhanced testing for deprecation warnings with additional mocking and setup.
tests/test_tempfile.py, tests/test_termcolor.py Removed unused unittest imports and streamlined tests.

🐇✨
In the realm of code, where changes abound,
A rabbit hopped, making barely a sound.
"Behold," it whispered, with glee in its eyes,
"Updates and fixes, oh, how time flies!"
With each line refined, and tests anew,
It danced in the moonlight, under skies so blue.
🌟📜 "To better days," it cheered, "and nights less drear,
With our code ever evolving, year by year."


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

codecov bot commented Mar 31, 2024

Codecov Report

Attention: Patch coverage is 0% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 81.26%. Comparing base (09d3692) to head (5e58909).

Files Patch % Lines
monty/dev.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #647      +/-   ##
==========================================
+ Coverage   81.12%   81.26%   +0.14%     
==========================================
  Files          27       27              
  Lines        1409     1409              
  Branches      306      306              
==========================================
+ Hits         1143     1145       +2     
+ Misses        213      209       -4     
- Partials       53       55       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@DanielYang59
Copy link
Contributor Author

DanielYang59 commented Mar 31, 2024

My raise_deadline_warning addition to dev.deprecate seems terribly flawed:

  • Accidentally wrong usage of deadline(tuple[int, int, int]) in string format {deadline:%Y-%m-%d}, should be {_deadline:%Y-%m-%d} (_deadline: datetime.datetime)
  • Wrong placement of raise_deadline_warning, should be inside the wrapped function, and therefore the two warnings we noticed previously in CI test.

However even with above two issues fixed, the unit test is still failing in my forked repo, where the first "no warn" case ("date before deadline") still triggers a warning, suggesting the time patch in unit test may be unsuccessful?

The same warning is not triggered here because this is not "the code owner's repo". Also the unit tests pass locally. I don't quite understand this behavior now, can you please give me a hand on this? @janosh Thanks!

@janosh
Copy link
Contributor

janosh commented Mar 31, 2024

  • Wrong placement of raise_deadline_warning, should be inside the wrapped function, and therefore the two warnings we noticed previously in CI test.

are you sure about that? we want the remove-deprecation error to trigger as soon as the code is parsed by the python interpreter, not wait until the function is called, right?

However even with above two issues fixed, the unit test is still failing in my forked repo, where the first "no warn" case ("date before deadline") still triggers a warning, suggesting the time patch in unit test may be unsuccessful?

i think you may have monkeypatched incorrectly. you just want to patch now, not the datetime class, no?

def test_deprecated_deadline_no_warn(self, monkeypatch):
    """Test cases where no warning should be raised."""

    @deprecated(deadline=(2000, 1, 1))
    def func_old():
        pass
    
    monkeypatch.setattr(datetime.datetime, "now", lambda: datetime.datetime(1999, 1, 1)) # changed

    # No warn case 1: date before deadline
    with warnings.catch_warnings(record=True) as warn_msgs:
        func_old()

        for warning in warn_msgs:
            assert "This function should have been removed on" not in str(warning.message)

@DanielYang59
Copy link
Contributor Author

DanielYang59 commented Mar 31, 2024

Hi @janosh, thanks a lot for your quick response and sharing your suggestions.

we want the remove-deprecation error to trigger as soon as the code is parsed by the python interpreter, not wait until the function is called, right?

Yes I think this makes sense, but in both cases, existing unit test was inconsistent and need to be adjusted (done in 49d51de):

    def test_deprecated_deadline(self):
        @deprecated(deadline=(2000, 1, 1))
        def func_old():
            pass

        with warnings.catch_warnings(record=True) as warn_msgs:
            func_old()  # trigger a warning

            assert "will be removed on 2000-01-01" in str(warn_msgs[0].message)

Above unit test assert there would be a warning when func_old() is called, not when the following is defined:

@deprecated(deadline=(2000, 1, 1))
        def func_old():
            pass

i think you may have monkeypatched incorrectly. you just want to patch now, not the datetime class, no?

I have been struggling with patching time for a while and I didn't find much information by searching (some suggestion use freezegun but it'll be an additional dependency). If we do:

monkeypatch.setattr(datetime.datetime, "now", datetime.datetime(1999, 1, 1))

There would be an error suggesting now is immutable:

FAILED tests/test_dev.py::TestDecorator::test_deprecated_deadline_no_warn - TypeError: cannot set 'now' attribute of immutable type 'datetime.datetime'

One last thing I realized from this issue is that if we just issue a DeprecationWarning instead of raise, the CI test will pass with warnings, which could be easily neglected (I doubt much people would check the CI run log when CI run passes). Maybe still use a stronger raise? What do you think?

def func_old():
pass

assert "This function should have been removed" in str(warn_msgs[0].message)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would need to patch this as well. Because currently the code is being pushed from my fork, and as such it is not "in code owner" repo, therefore no warning would be issue. And this is passing in my fork, could confirm my assumption

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that should be easy, no? monkeypatch.setenv("GITHUB_REPOSITORY", "materialsvirtuallab/monty")

Copy link
Contributor Author

@DanielYang59 DanielYang59 Mar 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

monkeypatch.setenv("GITHUB_REPOSITORY", "materialsvirtuallab/monty")

This is a good fix, and I'm thinking if we could make it better:

  • Somehow patch _is_in_owner_repo to always return True (I'm still experimenting on this) such that we don't need to hard code the repo name ("materialsvirtuallab/monty"). Even when we hard code the repo name, unit test would still fail in forks.
  • Set the CI variable as well, such that unit test wouldn't fail locally.

Copy link
Contributor Author

@DanielYang59 DanielYang59 Apr 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I experimented on patching _is_in_owner_repo and did not have much luck, then I decide to patch subprocess instead in bcc5001 (which is closer to the real-world scenario too):

with patch("subprocess.run") as mock_run:
                # Mock "GITHUB_REPOSITORY"
                monkeypatch.setenv("GITHUB_REPOSITORY", "TESTOWNER/TESTREPO")
                mock_run.return_value.stdout.decode.return_value = (
                    "git@github.com:TESTOWNER/TESTREPO.git"
                )

And everything is working as expected so far.

@janosh
Copy link
Contributor

janosh commented Mar 31, 2024

Maybe still use a stronger raise? What do you think?

yes! i was actually going to suggest defining a custom error for this:

class ExpiredDeprecationError(RuntimeError):

maybe some other parent class instead of RuntimeError

There would be an error suggesting now is immutable:

interesting, didn't know that. this SO answer looks promising, have you tried it?

@DanielYang59
Copy link
Contributor Author

yes! i was actually going to suggest defining a custom error for this:

class ExpiredDeprecationError(RuntimeError):

maybe some other parent class instead of RuntimeError

I'm thinking about just raise DeprecationWarning to make things simple? All we need is to "make the CI fail" and provide the developers enough clue such that they know some deprecated code should have been removed, in this case a DeprecationWarning with proper message should suffice. If the developer for any reason do not want CI to fail, they still could choose not to set the deadline arg.

interesting, didn't know that. this SO answer looks promising, have you tried it?

Thanks for providing this, I would have a look and let you know.

@DanielYang59
Copy link
Contributor Author

DanielYang59 commented Apr 1, 2024

There would be an error suggesting now is immutable:

interesting, didn't know that. this SO answer looks promising, have you tried it?

Seems adding a @pytest.fixture() decorator is all we need to resolve this issue (fadaf14). Meanwhile I further refined its unit test to generate more consistent/reliable results, regardless of the environment (CI/GITHUB_REPOSITORY).

@DanielYang59
Copy link
Contributor Author

@coderabbitai review

@DanielYang59 DanielYang59 changed the title Fix dev.deprecated and unit test for io Fix dev.deprecated and refine its unit tests Apr 1, 2024
@DanielYang59 DanielYang59 marked this pull request as ready for review April 1, 2024 03:30
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review Status

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 09d3692 and 5e58909.
Files selected for processing (5)
  • monty/dev.py (3 hunks)
  • requirements-optional.txt (1 hunks)
  • tests/test_dev.py (2 hunks)
  • tests/test_tempfile.py (2 hunks)
  • tests/test_termcolor.py (2 hunks)
Files skipped from review due to trivial changes (1)
  • tests/test_termcolor.py
Additional Context Used
GitHub Check Runs (1)
codecov/patch failure (1)

monty/dev.py: [warning] 94-94: monty/dev.py#L94
Added line #L94 was not covered by tests

Additional comments not posted (5)
requirements-optional.txt (1)

10-10: The addition of ipython==8.22.2 is aligned with the PR objectives to address a specific warning during tests. Ensure compatibility with other dependencies in the project.

tests/test_dev.py (2)

4-4: The addition of patch from unittest.mock is appropriate for mocking in the modified test methods.


92-157: The modifications to the test methods, including additional mocking and environment setup, enhance the unit testing for dev.deprecated. Consider adding comments to explain complex mocking scenarios for future maintainability.

monty/dev.py (2)

77-77: Ensuring os.getenv("CI") is not None is a good practice to confirm the code is running in a CI environment before raising a deprecation warning.


82-82: Using f-strings for the deprecation warning message enhances readability and maintainability.

monty/dev.py Show resolved Hide resolved
@DanielYang59
Copy link
Contributor Author

Please review this as well @shyuep. Thanks!

@shyuep shyuep merged commit 37d1cae into materialsvirtuallab:master Apr 11, 2024
5 checks passed
@DanielYang59 DanielYang59 deleted the fix-deprecation branch April 12, 2024 02:09
@DanielYang59
Copy link
Contributor Author

Thanks for reviewing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants