Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

F/remove sklarn #49

Merged
merged 17 commits into from
May 12, 2022
Merged

F/remove sklarn #49

merged 17 commits into from
May 12, 2022

Conversation

wooohoooo
Copy link
Contributor

  • removed dependency on Sklearn etc, in setup.py and all the places where I could find it
  • removed run() and other pieces of the API that aren't supposed to be used any more
  • made sure the tests cover the use cases
  • added notebook that showcases the use of the package (can be used as secondary testing)

README.md Outdated
@@ -3,8 +3,7 @@
[![PyPI Release](https://github.com/scailable/sclblpy/workflows/PyPI%20Release/badge.svg)](https://pypi.org/project/sclblpy/)


Changelog: the package is currently undergoing a major rework. We try to keep the below up to date, but some features might be stubs.
Most relevant is the removal of support for Scikit learn in favor of focussing fully on ONNX.
Changelog: the package is currently being refactored. As such, it is currently a stub, aas Scailable has changed its focus from Scikit Learn to ONNX, Scikit Learn support has been removed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A minor typo aas


print("Wait, toolchain needs to finish....")
time.sleep(10)
time.sleep(25)
print(" waited for 10 seconds")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10 should be updated to 25

@ayoub-assis
Copy link
Contributor

Everything looks fine to me.
Albeit I couldn't run the notebook and test files since I don't have an account with a device.
But maybe that's something that is irrelevant to this PR.
@robinvanemden

@MKaptein
Copy link
Contributor

MKaptein commented May 10, 2022 via email


## Background
The sclblpy package allows users with a valid scailable account (get one at [https://admin.sclbl.net/signup.html](https://admin.sclbl.net/signup.html))
to upload fitted ML / AI models to the Scailable toolchain server. This will result in:

1. The model being tested for errors on the client side.
2. The model being uploaded to Scailable, tested again, and if all test pass it will be converted to [WebAssembly](https://webassembly.org).
3. The model being made available to deploy to a pre-regsitered (link on how to do that) device
3. The model being made available to deploy to a preregistered (link on how to do that) device
4. (upcoming) The model being testable through test_run(example_input, device_id)

## Getting started
Copy link
Contributor

@ayoub-assis ayoub-assis May 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

upload should be replaced with upload_onnx function:
here
here
and here

Copy link
Contributor

@ayoub-assis ayoub-assis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The run function should be removed

README.md Outdated
@@ -178,7 +177,7 @@ the following packages:
* `sklearn` (legacy, to be removed in future updates)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sklearn is no longer a package dependency



def test_upload():
def test_upload_onnx():
""" Test the upload function (sklearn and onnx"""
Copy link
Contributor

@ayoub-assis ayoub-assis May 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here Test the upload function sklearn and onnx


# ONNX
docs['name'] = "Name of ONNX model"
docs['documentation'] = "A long .md thing...."
check = upload("../test/files/model.onnx", "", docs, model_type="onnx")
check = upload_onnx("../test/files/model.onnx", "", docs)
assert check is True, "ONNX upload test failed."


def test_update():
""" Test the update function (sklearn & onnx"""
Copy link
Contributor

@ayoub-assis ayoub-assis May 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here Test the upload function (sklearn and onnx

@ayoub-assis
Copy link
Contributor

@@ -258,16 +187,31 @@ def test_user_utils():
print("Running simple functional tests of main.py")
print("===============================")

test_upload() # upload sklearn and onnx test
test_upload_onnx() # upload sklearn and onnx test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

upload sklearn and onnx test ---> upload onnx test


print('testing update \n\n')
test_update() # update sklearn and onnx test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update sklearn and onnx test --> update onnx test

@ayoub-assis
Copy link
Contributor

ayoub-assis commented May 10, 2022

Manual Testing

  • test_utils: passed
  • test_bundle: passed
  • test_main: passed
  • Notebook works fine
    .
    .
    .
  • test_jwt: FAILED saying my username and/or password don't exist

@ayoub-assis
Copy link
Contributor

  • test_jwt: Passed after specifying the right usermanager server was correctly specified

@robinvanemden
Copy link
Contributor

Works for me locally. All issues that were found were also resolved by @ayoub-assis .
Merging.

@robinvanemden robinvanemden merged commit 1d7bda8 into develop May 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants