Making neat-ml poetry-fied (plus tox, black, flake8, and mypy updates)

[hrshdhgd]

• Make poetry the main package management system
• Implement tox within poetry and within it:
  • black
  • flake8
  • mypy

[caufieldjh]

This is pretty strange - I'm having difficulty establishing the path for neat_ml_schema. I can import it without issue, but any attempt to find its path returns None:

>>> import neat_ml_schema
>>> print(neat_ml_schema.__file__)
None
>>> import inspect
>>> inspect.getfile(neat_ml_schema)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/inspect.py", line 659, in getfile
    raise TypeError('{!r} is a built-in module'.format(object))
TypeError: <module 'neat_ml_schema' (namespace)> is a built-in module
>>> import pkg_resources 
>>> pkg_resources.resource_filename('neat_ml_schema','src/schema/neat_ml_schema.yaml')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/harry/neat-env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1144, in resource_filename
    return get_provider(package_or_requirement).get_resource_filename(
  File "/home/harry/neat-env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 364, in get_provider
    return _find_adapter(_provider_factories, loader)(module)
  File "/home/harry/neat-env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1392, in __init__
    self.module_path = os.path.dirname(getattr(module, '__file__', ''))
  File "/usr/lib/python3.8/posixpath.py", line 152, in dirname
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

[hrshdhgd]

Aah ... I think I know what the problem is. I need to fix it in neat-ml-schema

[hrshdhgd]

Alright, releasing a new version of neat-ml-schema (v0.1.11) right now. That should solve the problem. Also, the new path will be pkg_resources.resource_filename('neat_ml_schema','schema/neat_ml_schema.yaml') and this should work. I have made the change in neat-ml and will push the change once the release is done, that way gh-action must pass.

[hrshdhgd]

Alright, this passes! I have implemented tox which in turn implements black. There will be a lot of files committed that will be reformatted. Should we do this in this PR or a separate one?

[caufieldjh]

Let's take care of the reformatting here too

[hrshdhgd]

I will need help with flake8 error correction. Primarily because you both have a better knowledge of what each of the functions do than I. We can go through it together during our hackathons. the command to highlight the areas that need attention : poetry run tox -e flake8

[caufieldjh]

Not sure what's going on with ensmallen or why poetry can't find it to install.

[caufieldjh]

There may be some strange python version interaction between poetry, ensmallen, and OS - the poetry install works perfectly on my machine, running Py3.9.5, but the workflow here is running Py3.9.13 on windows/linux. The windows one fails during install but the linux one works.

Going to try deactivating the windows tests as I don't think grape really supports it anyway.

[caufieldjh]

OK, now it makes it to pytest but just takes far too long. Probably trying to make a real embedding by accident.

[caufieldjh]

Tests are still taking too long - I suspect this test is the culprit:

neat-ml/tests/test_link_prediction.py

Line 95 in 424c365

def test_sklearn_fit(self) -> None:

[caufieldjh]

Hrm, looks like it may just need the extra imports to be test dependencies. Will try that.

[caufieldjh]

See AnacletoLAB/grape#12 - just ran into this

[caufieldjh]

Looks like this is working - tests are still running long, but I think that's because it includes all the tensorflow and sklearn stuff that generally runs quickly (i.e., not 100 minutes) if it isn't on a GH actions instance.

[caufieldjh]

Hrm, the tests install TF and scikit-learn, but it's strange that they still take so long. On my machine it takes <2 min to run all tests.