Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

completely factor out numpy dependency from Python runtime #219

Merged
merged 3 commits into from May 22, 2020
Merged

Conversation

StrikerRUS
Copy link
Member

Refer to #115.

Follow the main principle of the library to transpile models into a code with zero dependencies.

numpy can be added back later in a form of another export function/parameter (see discussion in the original issue).

Moreover, native Python list operations faster (not significantly) for small lists compared to numpy. For now we use linear algebra for arrays which dimension equals to number of classes. So in the majority of common cases it is not greater than 100.

from timeit import timeit
import numpy as np

def test_numpy():
    return np.asarray(test1) + np.asarray(test2)

def test_raw_python():
    return [sum(i) for i in zip(test1, test2)]

res = []
for i in range(1, int(1e2), 2):
    N = i
    test1 = np.random.rand(N).tolist()
    test2  = np.random.rand(N).tolist()
    res_numpy = timeit(test_numpy, number=100)
    res_raw = timeit(test_raw_python, number=100)
    res.append((i, res_raw, res_numpy))

print("#elements\traw\tnumpy\tdelta")
for i, raw, numpy in res:
    print("{0}\t{1}\t{2}\t{3}".format(i, raw, numpy, raw - numpy))
#elements	raw	numpy	delta
1	6.250000000029843e-05	0.0004211000000000631	-0.00035859999999976466
3	9.499999999995623e-05	0.0006881000000000803	-0.0005931000000001241
5	0.00012929999999977682	0.00043650000000017286	-0.00030720000000039605
7	0.00016560000000032105	0.00045410000000023487	-0.0002884999999999138
9	0.00019990000000014163	0.00046629999999980853	-0.0002663999999996669
11	0.00022909999999987107	0.0004762000000000377	-0.00024710000000016663
13	0.00026180000000008974	0.0004892000000000785	-0.00022739999999998872
15	0.0002949999999999342	0.0005028999999998618	-0.0002078999999999276
17	0.0003299000000001051	0.0005155000000001131	-0.00018560000000000798
19	0.00036589999999980805	0.0005271000000002246	-0.00016120000000041657
21	0.0003999000000001196	0.0005384999999997753	-0.00013859999999965567
23	0.00043119999999996494	0.0005549000000000248	-0.00012370000000005987
25	0.0004640999999998563	0.0005653999999997161	-0.00010129999999985984
27	0.0012802000000000646	0.0005790000000001072	0.0007011999999999574
29	0.0005319999999997549	0.0005920999999999843	-6.0100000000229414e-05
31	0.0011044999999998417	0.0010659999999997893	3.850000000005238e-05
33	0.0011721999999996235	0.0011746000000001366	-2.400000000513103e-06
35	0.0011851999999996643	0.0012088000000001209	-2.360000000045659e-05
37	0.0012415000000003396	0.0012389999999999901	2.500000000349445e-06
39	0.0013708000000001164	0.0012324999999999697	0.00013830000000014664
41	0.0014009999999999856	0.0012704000000001159	0.0001305999999998697
43	0.0007676999999999268	0.0006812999999996627	8.640000000026404e-05
45	0.0007969999999999366	0.0006905999999999857	0.00010639999999995098
47	0.0010040000000000049	0.000705099999999792	0.0002989000000002129
49	0.0008707999999999494	0.0007174999999999265	0.00015330000000002286
51	0.0009019999999999584	0.000731200000000154	0.00017079999999980444
53	0.0009392999999997542	0.0007426000000001487	0.00019669999999960552
55	0.0010655000000001635	0.0007524000000000974	0.0003131000000000661
57	0.0010200000000000209	0.0008284999999998988	0.00019150000000012213
59	0.0012457999999999636	0.0007822999999995695	0.00046350000000039415
61	0.0010810000000001097	0.0007988999999999358	0.0002821000000001739
63	0.0011092999999999797	0.0008105000000000473	0.00029879999999993245
65	0.0011463999999996588	0.0008184999999998332	0.00032789999999982555
67	0.0011822999999999695	0.0008331000000003641	0.0003491999999996054
69	0.0012094999999998635	0.000846699999999867	0.00036279999999999646
71	0.0012444000000000344	0.0008598000000001882	0.0003845999999998462
73	0.0014647999999999328	0.0008692000000003475	0.0005955999999995853
75	0.0013277000000000427	0.0008892999999998707	0.000438400000000172
77	0.0013636999999997457	0.000897900000000007	0.00046579999999973865
79	0.0013938000000002226	0.000905600000000284	0.0004881999999999387
81	0.0014203999999997663	0.0009268000000002274	0.0004935999999995389
83	0.0014559000000002875	0.0009381999999997781	0.0005177000000005094
85	0.0014894999999999214	0.000943499999999986	0.0005459999999999354
87	0.0015203999999999773	0.001101100000000077	0.0004192999999999003
89	0.001608300000000007	0.0009711000000001135	0.0006371999999998934
91	0.0016391000000002265	0.000981600000000249	0.0006574999999999775
93	0.0016633000000001452	0.0009974999999999845	0.0006658000000001607
95	0.001699500000000409	0.0010091999999999324	0.0006903000000004766
97	0.0019095000000000084	0.0010223000000002536	0.0008871999999997549
99	0.0017678999999999334	0.00103629999999999	0.0007315999999999434

@coveralls
Copy link

coveralls commented May 18, 2020

Coverage Status

Coverage decreased (-0.2%) to 95.027% when pulling 9b6e15f on python into 4869e00 on master.

@StrikerRUS StrikerRUS changed the title completely factor out numpy dependency from runtime completely factor out numpy dependency from Python runtime May 18, 2020
Copy link
Member

@izeigerman izeigerman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks. It's been awhile since we began this conversation. Thank you for bringing this to a closure 👍

@izeigerman izeigerman merged commit 4ecd8b9 into master May 22, 2020
@izeigerman izeigerman deleted the python branch May 22, 2020 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants