Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add blake/flake8/isort refactoring + pip installable #69

Merged
merged 17 commits into from
Oct 4, 2022
Merged

feat: add blake/flake8/isort refactoring + pip installable #69

merged 17 commits into from
Oct 4, 2022

Conversation

pommedeterresautee
Copy link
Member

@pommedeterresautee pommedeterresautee commented Oct 2, 2022 •
edited
Loading

  • make package pip installable (setup.py, etc.): move files to src/nucle + add setup.py
  • setup.py check torch is present (exception with info otherwise) + check GPU >= Ampere
  • apply code formatting (blake, flake8, isort): use format source code tool
  • add apache 2 license (LICENSE + headers): insert header with Pycharm

Only unrelated change: remove layernorm 8K len test as not needed.

No other modification have been performed to "ease" as much as possible the review

fix #42
fix #65
fix #70

@pommedeterresautee pommedeterresautee added enhancement New feature or request Python all things related to Python, package, etc. labels Oct 2, 2022
@pommedeterresautee pommedeterresautee self-assigned this Oct 2, 2022
@pommedeterresautee
Copy link
Member Author 

====================================================================================================== warnings summary =======================================================================================================
../../../home/geantvert/.local/share/virtualenvs/triton-xp/lib/python3.9/site-packages/onnxruntime/transformers/float16.py:78: 299 warnings
  /home/geantvert/.local/share/virtualenvs/triton-xp/lib/python3.9/site-packages/onnxruntime/transformers/models/gpt2/../../float16.py:78: DeprecationWarning: The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead
    float32_list = np.fromstring(tensor.raw_data, dtype="float32")

../../../home/geantvert/.local/share/virtualenvs/triton-xp/lib/python3.9/site-packages/onnxruntime/transformers/float16.py:82: 299 warnings
  /home/geantvert/.local/share/virtualenvs/triton-xp/lib/python3.9/site-packages/onnxruntime/transformers/models/gpt2/../../float16.py:82: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    tensor.raw_data = float16_list.tostring()

test/test_debugger.py::test_matmul
  /mnt/workspace/triton-xp/test/test_debugger.py:156: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
    group_id = pid // num_pid_in_group

test/test_debugger.py::test_matmul
  /mnt/workspace/triton-xp/test/test_debugger.py:160: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
    pid_n = (pid % num_pid_in_group) // group_size_m

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================ 2290 passed, 280 skipped, 600 warnings in 4543.08s (1:15:43) =================================================================================

@pommedeterresautee pommedeterresautee marked this pull request as ready for review October 2, 2022 10:56
@pommedeterresautee
Copy link
Member Author

FWIW, pytest has be rerun on #72 this morning, which includes all small modif since last pytest run in this PR.

ayoub-louati
ayoub-louati reviewed Oct 3, 2022
View reviewed changes
README.md

Optimized kernels for `transformer` models.

## Install dependencies

**IMPORTANT**: This package requires `pytorch` being installed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should mention also that we need python3.9 +

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3.9 is 2 years old, but ok to add it if you think it makes sense.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

setup.py Outdated
try:
import torch

assert torch.__version__ >= "1.11.0"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in requirements we ask for : torch>=1.12.0

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

setup.cfg
functorch
torchdynamo
transformers
onnxruntime
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

onnxruntime-gpu (not sure) ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we list namespaces and even for ORT gpu it's still onnxruntime.

test/models/ort_utils.py Outdated
@@ -1,20 +1,36 @@
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should it be after the license ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

test/models/ort_utils.py
import ctypes as C
from ctypes.util import find_library
from typing import Dict, Optional, Tuple

import cupy as cp
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not present in the requirements

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

already present cupy-cuda117

@gaetansnl
Copy link
Contributor

gaetansnl commented Oct 4, 2022 •
edited
Loading

@pommedeterresautee

  • decorator package missing

@pommedeterresautee pommedeterresautee merged commit 5890ec5 into main Oct 4, 2022
@pommedeterresautee pommedeterresautee deleted the feat/pip_install branch October 4, 2022 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gaetansnl gaetansnl gaetansnl approved these changes

@jonathlela jonathlela Awaiting requested review from jonathlela

@ayoub-louati ayoub-louati Awaiting requested review from ayoub-louati

Assignees

@pommedeterresautee pommedeterresautee

Labels
enhancement New feature or request Python all things related to Python, package, etc.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

add OSS licence Check GPU capacity make it pip installable
3 participants
@pommedeterresautee @gaetansnl @ayoub-louati