Adds new CI, Covers Multiple OS, Python Versions and Packaging #226

oke-aditya · 2020-07-26T18:29:15Z

With the current change I added CI for more OS systems. It is good idea to build on multiple OS before publishing.
This technique is taken from PyTorch Lightning.
I have added CI for releasing to both PyPi test server and PyPi official. This is again taken from Lightning
We need a test to build package as well. And test PyPi releasing with a dummy release.

codecov-commenter · 2020-07-26T18:48:14Z

Codecov Report

Merging #226 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #226   +/-   ##
=======================================
  Coverage   80.17%   80.17%           
=======================================
  Files         101      101           
  Lines        1897     1897           
=======================================
  Hits         1521     1521           
  Misses        376      376

Flag	Coverage Δ
#unittests	`80.17% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b2084e...a2b57cf. Read the comment docs.

oke-aditya · 2020-07-26T19:03:33Z

E AttributeError: module 'torch.distributed' has no attribute 'group'

I was too able to reproduce this error on my windows Machine
It arises from conftest.py code in our repo
Rest all, Linux, MacOs build for python 3.6, 3.7 and 3.8
This is a problem if torch.distributed is not initialized properly. Which is strange coz it builds on linux and not on windows.

oke-aditya · 2020-07-26T19:35:15Z

@lgvaz can you fix up the windows build issue of efficient det repo ? Once that is solved and I fix up the packaging We can then add these CI changes.

I will fix up the setup.py issue.
Also we haven't solved how to include efficient det in our settings.ini so that is still a bit of concern.

Helps in #182 #99 #135 #209

lgvaz · 2020-07-27T00:07:46Z

@oke-aditya This is very very hard for me to solve, because, well.. I don't have windows hahahhah

Does windows support multi-gpu after all?

oke-aditya · 2020-07-27T02:37:49Z

Windows does support torch.distributed the issue maybe I am using cpu only which might not ship it. I am raising an issue with PyTorch itself as I tried from torch.distributed import group locally also did not work.

We are installing the same package in Linux and MacOs as It is also CPU only. It should work. Microsoft provides nice support for windows with pytorch now so it should be fixed.

oke-aditya · 2020-07-27T05:13:11Z

I'm facing a very small issue with Packaging install CI.
What it does is checks if the package can be installed properly from the wheel file.

Current scenario: -
I built the wheel, it gets saved in the dist folder. I cannot install that wheel using pip install dist/
More issues are it is version specific wheel name is mantisshrimp.0.0.8.whl

Possible solutions: -
Somehow if I can name the wheel as mantisshrimp-test.whl then I can install it by pip install dist/mantisshrimp-test.whl
Or some way to install wheel like *.whl. I am unable to do that as of now.

Other possible solution: -
Run pip install . Install the setup. This isn't a good thing, if we want to test the.whl.

It's very strange as same workflow runs for PyTorch Lightning. And it doesn't run for us.
Version key error I couldn't understand, it was being paseesd though. When I tried explicit passing, It resulted in error duplicate key found.

oke-aditya · 2020-07-27T05:30:04Z

So Windows CI error is linked to pytorch/pytorch#42095.
I guess torch itself does not support distributed GPU / CPU on windows but maybe soon it will. Let's see if there is a workaround or for now we can drop support for windows altogether.

Another option is mark these tests that fail on windows for now and skip them on windows OS. This is slightly poor technique as its somewhere in between of being supported or not.

Windows users have option of docker container with WSL2, code on windows; train on Linux VMs, This isn't our issue anyways coz if PyTorch doesn't support it. We cannot do it either.

oke-aditya · 2020-07-27T11:46:15Z

AAh the install bug was fixed with MANIFEST.in file. Why did we exclude it anyways ? It was there in nbdev template. We should have kept it.

oke-aditya · 2020-07-27T12:38:34Z

Again the build failed coz we tried to import pandas as pd. We are getting hit by #181 again and again.
I am removing pandas from imports file. I guess we do not need it.

oke-aditya · 2020-07-27T13:17:30Z

Nice, we are getting there. New issue. We import PyTorch Lightning. Note we do not package this with our repo. It is not in our requirements as well, it is in additional requirements. So when we build the wheel, we do not package this dependency or take care that it is installed. The same strategy that of HuggingFace as well. They do not package PyTorch or Tensorflow or take care it is installed. Again this error arises from nasty imports folder, which is causing lot of concern.

Do the tests required Fastai and PyTorch Lightning?
Do we need this in imports ?

Fixes possible.

I can install it while checking our wheel in package.yml file.
We can bundle this in requirements (bad idea limits the minimal installation).

Note that egg or wheel file will always install minimal version of mantisshrimp.

…dds_new_ci

lgvaz · 2020-07-27T15:02:44Z

Tests requires fastai, lightning, efficientdet, albumentations, everything, so for now let's only use with [all]
We don't need it on imports =)

oke-aditya · 2020-07-27T15:40:08Z

Tests run on all.
Wheel build check, checks our code only on our dependencies as imports.

…dds_new_ci

oke-aditya · 2020-07-27T15:53:13Z

Okay. So the final broken things are Tests on Windows. This time it is not due to the distributed issue. It is due to refactor of COCO metrics. Windows we use different COCO Api repository. @lgvaz can you have a look it seems small dtype conversion error.

oke-aditya · 2020-07-27T16:11:37Z

Maybe this is the cause rbgirshick/py-faster-rcnn#481. We are using an upgraded version of numpy but the coco API needs a lower one ? The COCO APi is always painful and a bottleneck.

oke-aditya · 2020-07-27T17:11:23Z

Currently we don't have any test to let us know if it is fixed. So I guess better will be to merge this and try to fix it up. It will be easier to fix as then we will know if Windows CI passes.

oke-aditya · 2020-07-28T12:49:11Z

Following tests fail only on Windows

tests\metrics\test_coco_metric.py FF                            
tests\models\efficient_det\test_coco_metric.py F

@lgvaz @ai-fast-track as we said Windows support is experimental. And reason is PyCoco Tools and PyTorch both have its limitations.

Suggestions to Proceed: -

Merge this PR and try fixing up windows.
Skip these tests for now. Mark them as COCO in PyTest and skip it in windows.
Not support windows for now.

ai-fast-track · 2020-07-28T17:37:55Z

Does this (pip install) fix the issue with Windows when spinning the WIndows test machine?

C.2- Installing cocoapi in Windows:

pycoco cannot be installed using the command above (see issue-185 in the cocoapi repository). We are using this workaround:

pip install "git+https://github.com/philferriere/cocoapi.git#egg=pycocotools&subdirectory=PythonAPI"

oke-aditya · 2020-07-28T17:42:43Z

I am using the same COCO API on workflow file. Still I'm getting error. You can have look at workflow here

Also in a few days. We need to bump to PyTorch 1.6

The reason is COCO API is very old and maybe there is some internal numpy typecast issue from our side or from this github repo itself. This is a liability to be carried forward.

oke-aditya · 2020-07-28T19:03:20Z

Bumping all the tests to PyTorch 1.6 with the new relase today,

oke-aditya · 2020-07-28T19:20:56Z

Wow. Additional trouble now. We can't bump to PyTorch 1.6. @lgvaz this needs multi-headed attention now (pun).

Problem is with Ross's EfficientDet tests. Also, a test of fastai is failing.
Maybe raise an issue with Ross?
Error is

E       RuntimeError: Integer division of tensors using div or / is no longer supported, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead.

from
preds = efficientdet.predict(model=fridge_efficientdet_model, batch=batch)

To avoid this from next time. We should run CI tests on PyTorch current as well as nightly. This was not very expected. But this increases our test count 2x. Which will now be too many.

The Fastai error is

E AttributeError: '_FakeLoader' object has no attribute 'generator'

arising from learner.fine_tune()

A fix for now maybe to ship with PyTorch 1.5 and try to cover 1.6.
Once we reach 1.6 we will start building on PyTorch nightlies.

…dds_new_ci

oke-aditya · 2020-07-30T19:14:58Z

K converting this to draft PR. Will raise 2 new PRs one for CI and one for the PyPI release.

oke-aditya · 2020-07-30T19:15:19Z

Do let me know if we need to delete this branch

oke-aditya added 3 commits July 26, 2020 23:58

Revamps CI

8111572

fixes CI

2e3859c

tries fixing linux and Mac

71aa3c0

oke-aditya added 3 commits July 27, 2020 00:51

adds install package yml

4ff22a9

tries fixing installer

b8c4e99

lowers ubuntu to latest

a04f93d

oke-aditya mentioned this pull request Jul 27, 2020

Import error from torch.distributed pytorch/pytorch#42092

Closed

oke-aditya added 3 commits July 27, 2020 10:06

adds bdist wheel

ae1380f

tries fixing setup

2ec8112

version was passed

d125b8f

tries fixing with manifest

f9b7a9b

oke-aditya added 3 commits July 27, 2020 17:23

added version init, fixes install

2310b07

added venv

c39647e

adds cython

e59a441

oke-aditya added 3 commits July 27, 2020 18:21

Removes pandas

0217f50

passed [all] param to setup for meta dependencies

384a439

all doesn't fix

d5a3ce1

oke-aditya mentioned this pull request Jul 27, 2020

Improper Packaging with fastai requirement #222

Closed

oke-aditya added 2 commits July 27, 2020 19:58

re run CI

a005c5e

Merge branch 'master' of https://github.com/lgvaz/mantisshrimp into a…

8dd5832

…dds_new_ci

tries fixing with requests

0bd0391

oke-aditya added 2 commits July 27, 2020 21:10

Merge branch 'master' of https://github.com/lgvaz/mantisshrimp into a…

ebcdba6

…dds_new_ci

New CI, checks for multiple OS and pypi builds

b271df0

oke-aditya changed the title ~~Revamps CI~~ Adds new CI, Covers Multiple OS, Python Versions and Packaging Jul 27, 2020

shifts to ubuntu latest

46b319f

bumped to PyTorch 1.6

7e1cc5d

oke-aditya added 2 commits July 29, 2020 00:57

Merge branch 'master' of https://github.com/lgvaz/mantisshrimp into a…

6588d63

…dds_new_ci

reverts to pytorch 1.5

a2b57cf

oke-aditya closed this Jul 30, 2020

oke-aditya mentioned this pull request Jul 30, 2020

Adds CI Testing for Linux and Package Intallation #243

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds new CI, Covers Multiple OS, Python Versions and Packaging #226

Adds new CI, Covers Multiple OS, Python Versions and Packaging #226

oke-aditya commented Jul 26, 2020 •

edited

codecov-commenter commented Jul 26, 2020 •

edited

oke-aditya commented Jul 26, 2020

oke-aditya commented Jul 26, 2020 •

edited

lgvaz commented Jul 27, 2020

oke-aditya commented Jul 27, 2020 •

edited

oke-aditya commented Jul 27, 2020 •

edited

oke-aditya commented Jul 27, 2020 •

edited

oke-aditya commented Jul 27, 2020

oke-aditya commented Jul 27, 2020 •

edited

oke-aditya commented Jul 27, 2020 •

edited

lgvaz commented Jul 27, 2020

oke-aditya commented Jul 27, 2020

oke-aditya commented Jul 27, 2020

oke-aditya commented Jul 27, 2020

oke-aditya commented Jul 27, 2020

oke-aditya commented Jul 28, 2020

ai-fast-track commented Jul 28, 2020

oke-aditya commented Jul 28, 2020 •

edited

oke-aditya commented Jul 28, 2020

oke-aditya commented Jul 28, 2020 •

edited

oke-aditya commented Jul 30, 2020

oke-aditya commented Jul 30, 2020

Adds new CI, Covers Multiple OS, Python Versions and Packaging #226

Adds new CI, Covers Multiple OS, Python Versions and Packaging #226

Conversation

oke-aditya commented Jul 26, 2020 • edited

codecov-commenter commented Jul 26, 2020 • edited

Codecov Report

oke-aditya commented Jul 26, 2020

oke-aditya commented Jul 26, 2020 • edited

lgvaz commented Jul 27, 2020

oke-aditya commented Jul 27, 2020 • edited

oke-aditya commented Jul 27, 2020 • edited

oke-aditya commented Jul 27, 2020 • edited

oke-aditya commented Jul 27, 2020

oke-aditya commented Jul 27, 2020 • edited

oke-aditya commented Jul 27, 2020 • edited

lgvaz commented Jul 27, 2020

oke-aditya commented Jul 27, 2020

oke-aditya commented Jul 27, 2020

oke-aditya commented Jul 27, 2020

oke-aditya commented Jul 27, 2020

oke-aditya commented Jul 28, 2020

ai-fast-track commented Jul 28, 2020

C.2- Installing cocoapi in Windows:

oke-aditya commented Jul 28, 2020 • edited

oke-aditya commented Jul 28, 2020

oke-aditya commented Jul 28, 2020 • edited

oke-aditya commented Jul 30, 2020

oke-aditya commented Jul 30, 2020

oke-aditya commented Jul 26, 2020 •

edited

codecov-commenter commented Jul 26, 2020 •

edited

oke-aditya commented Jul 26, 2020 •

edited

oke-aditya commented Jul 27, 2020 •

edited

oke-aditya commented Jul 27, 2020 •

edited

oke-aditya commented Jul 27, 2020 •

edited

oke-aditya commented Jul 27, 2020 •

edited

oke-aditya commented Jul 27, 2020 •

edited

oke-aditya commented Jul 28, 2020 •

edited

oke-aditya commented Jul 28, 2020 •

edited