Fedora packaging

[frenzymadness]

Hello.

I am trying to package Hatch for Fedora and I have some troubles with tests. First (and IMHO the biggest one) is that build of RPM package is done in the restricted environment without access to the internet. But build environment has more restrictions and I don't know Hatch that much that I can fix or skip all of failing tests.

The thing I can do is to check if internet connection is present and if not, skip tests depending on them. What do you think about it?

Could you please help me fix (or find a reason/condition to skip) the rest of failing tests?

Thank you and have a nice day.

Full output: build_tests.txt

[ofek]

@frenzymadness Thanks for working on this! You can ignore any network test e.g. release, (un)install, update, and a few scattered others.

It looks like hatch new/init ... is failing because of a lack of config file. In order to test different config states without accidentally destroying users' config, we temporarily move it if it exists:

https://github.com/ofek/hatch/blob/baeaebec9c5368044c112404315ec57b2d72acaf/hatch/utils.py#L178

This issue is:

_________________________________ test_python __________________________________

    def test_python():
        with temp_chdir() as d:
            runner = CliRunner()
            runner.invoke(hatch, ['init', 'ok', '--basic'])
    
            with temp_move_path(SETTINGS_FILE, d):
                settings = copy_default_settings()
                settings['pypaths']['python'] = sys.executable
                save_settings(settings)
                result = runner.invoke(hatch, ['build', '-py', 'python', '-pp', 'Delphi'])
>               files = os.listdir(os.path.join(d, 'dist'))

tests/commands/test_build.py:265: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib64/python3.6/contextlib.py:88: in __exit__
    next(self.gen)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

path = '/builddir/.local/share/hatch/settings.json', d = '/tmp/tmpy4ekdg0n'

    @contextmanager
    def temp_move_path(path, d):
        if os.path.exists(path):
            dst = shutil.move(path, d)
    
            try:
                yield dst
            finally:
>               os.replace(dst, path)
E               OSError: [Errno 18] Invalid cross-device link: '/tmp/tmpy4ekdg0n/settings.json' -> '/builddir/.local/share/hatch/settings.json'

hatch/utils.py:185: OSError

The config file never gets moved back because it appears the paths are on different file systems

https://docs.python.org/3/library/os.html#os.replace
moby/moby#25409
https://review.openstack.org/#/c/64544/2/keystoneclient/middleware/auth_token.py
pypa/pip#103

Please try again, as I just committed f9f16ff

[frenzymadness]

Thank you very much for your help. I've solved almost all test issues.

Changes I have made:

• Skip tests which depend on internet connection.
• Run tests (I mean hatch test) with -e argument to use pytest/coverage as a module. The reason for this change is the same as with virtualenv because there is no pytest executable in Python 3 only environment.

I'll prepare pull request from them.

I think that the approach in the second change can be somehow automatic or used as a default. I think that is always better to run pytest/virtualenv/coverage etc. as Python modules instead of executables. If you don't want to run this commands as modules then something like if you cannot find pytest executable try to run the same command with python -m pytest could be helpful and fix some future issues. Moreover when you use python -m <module> Python adds CWD to sys.path which could also be useful. But this is up to you, of course.

I am trying to solve one last failing test. test_package in tests/commands/test_test.py is failing because test inside ok package cannot be executed. The reasons are:

• We cannot use pytest for reason described earlier
• We are in the temporary folder and virtualenv where only ok package is installed without pytest.

A possible solution is to install pytest and coverage to the same virtualenv as ok package but then the test will depend on internet connection and will be skipped during a build of RPM. Wha do you think about it?

[ofek]

I'm going to make running pytest/coverage as modules the default and change the option to mean use Hatch's pytest/coverage. Sound ok?

[frenzymadness]

Sorry, but what does Hatch's pytest/coverage means? Running pytest/coverage as modules by default sounds really good to me! Thanks!

And what do you think about mentioned solution for test_package?

[ofek]

As in the pytest/coverage shipped with Hatch. That could be used in the failing test.

[frenzymadness]

Okay, let me know when you commit this change and I'll take a look. I'll also try to build RPM from the latest source and if everything will be ok, I'll create PR with my changes.

[ofek]

@frenzymadness Here it is :) e1bfa6f

[ofek]

@frenzymadness Also, as I'm utterly unfamiliar with Fedora packaging, what would this PR include exactly?

[frenzymadness]

Here is bug for new package review: https://bugzilla.redhat.com/show_bug.cgi?id=1491456

Thank you for your help.

[ofek]

No no, thank you! Added link ba89d02

So how does this work? What is the time frame for a review usually?

[frenzymadness]

And the review is done with a positive outcome. The package is approved!

In bug, there is one suggestion for you:

You should talk with upstream to move tests/ to be a subdirectory of the main module, and keep the tests installed. Users might want to check the installed package too (e.g. to see if everything still works in the installed system).

I'll let you know when the package will be ready in Fedora.

[ofek]

@frenzymadness Wow, that's awesome! You did great work here :)

Is keeping tests outside a deal breaker? I really prefer them outside to reduce download size from PyPI.

[frenzymadness]

No, I don't think that it is a deal breaker. It only means that the tests are not part of RPM so users with hatch from RPM cannot run them. But it is up to you.

[frenzymadness]

Hatch is now available in Fedora rawhide (future Fedora 28) and also requested as an update for Fedora 27 and 26. Packages will be ready to install in approximately one week.

[ofek]

Fantastic, thanks @frenzymadness!!! Now I gotta try Fedora :)

[frenzymadness]

If you do, you can become a tester of Hatch's updates in Fedora and speed up updates of new versions.

[frenzymadness]

Hatch pushed to the Fedora 26 stable package repository.