[stall]Layer auto detect input size #688

dcslin · 2020-05-02T12:38:00Z

No description provided.

lgtm-com · 2020-05-02T16:46:12Z

This pull request introduces 1 alert when merging dd0dc99 into e4082c6 - view on LGTM.com

new alerts:

1 for Unnecessary pass

dcslin · 2020-05-02T16:46:18Z

changes:

an example of make class Xxx(Operation) to private class _Xxx(Operation). Because class Operation should be only for internal. For the term Operation in the user space, it should be the operation functions def xxx(x):.... Also these operation function should be used by the user
fix bug for set_param when given Tensor as params
Modified Linear constructor to (self,out_features, in_features=None, bias=True):. out_features comes first, leaving in_features as optional. When Linear is constructed with only out_features, it's params, W and b, are not initialized. After set_params or forward/__call__, it's params, W and b, are initialized.

nudles · 2020-05-04T02:34:42Z

changes:

1. an example of make `class Xxx(Operation)` to private `class _Xxx(Operation)`. Because `class Operation` should be only for internal. For the term `Operation` in the user space, it should be the operation functions `def xxx(x):...`. Also these operation function should be used by the user

2. fix bug for `set_param` when given Tensor as params

3. Modified `Linear` constructor to `(self,out_features, in_features=None, bias=True):`. `out_features` comes first, leaving `in_features` as optional. When `Linear` is constructed with only `out_features`, it's params, `W` and `b`, are not initialized. After `set_params` or `forward`/`__call__`,  it's params, `W` and `b`, are initialized.

For the last point, it will break the compatibility..
there are two solutions

use *args and **kwargs
assume the old code passes (in_features, out_features, bias=True) and the new code passes (out_features, bias=True), then we check if in_features is None or not to decide the argument order.

In V4, we can update the API completely.

lgtm-com · 2020-05-04T03:39:28Z

This pull request introduces 1 alert when merging 2b16b37 into e4082c6 - view on LGTM.com

new alerts:

1 for Unnecessary pass

lgtm-com · 2020-05-05T16:49:48Z

This pull request introduces 2 alerts when merging cd289d6 into e4082c6 - view on LGTM.com

new alerts:

2 for Unnecessary pass

dcslin · 2020-05-06T02:00:36Z

changes:
1. an example of make `class Xxx(Operation)` to private `class _Xxx(Operation)`. Because `class Operation` should be only for internal. For the term `Operation` in the user space, it should be the operation functions `def xxx(x):...`. Also these operation function should be used by the user

2. fix bug for `set_param` when given Tensor as params

3. Modified `Linear` constructor to `(self,out_features, in_features=None, bias=True):`. `out_features` comes first, leaving `in_features` as optional. When `Linear` is constructed with only `out_features`, it's params, `W` and `b`, are not initialized. After `set_params` or `forward`/`__call__`,  it's params, `W` and `b`, are initialized.
For the last point, it will break the compatibility..
there are two solutions

use *args and **kwargs

assume the old code passes (in_features, out_features, bias=True) and the new code passes (out_features, bias=True), then we check if in_features is None or not to decide the argument order.

In V4, we can update the API completely.

ok thanks.

Updated:
changes No.4: modified the __ini__ to parse *args and **kwargs for Linear, RNN, LSTM

lgtm-com · 2020-05-06T05:29:38Z

This pull request introduces 2 alerts and fixes 1 when merging cd289d6 into 536f7e4 - view on LGTM.com

new alerts:

2 for Unnecessary pass

fixed alerts:

1 for Unused local variable

lgtm-com · 2020-05-06T05:46:48Z

This pull request introduces 2 alerts when merging cd289d6 into db1846d - view on LGTM.com

new alerts:

2 for Unnecessary pass

lgtm-com · 2020-05-07T04:44:18Z

This pull request introduces 3 alerts when merging e538fef into db1846d - view on LGTM.com

new alerts:

2 for Unnecessary pass
1 for Unused import

lgtm-com · 2020-05-07T11:53:11Z

This pull request introduces 3 alerts when merging 4a98351 into db1846d - view on LGTM.com

new alerts:

2 for Unnecessary pass
1 for Unused import

nudles · 2020-05-12T02:17:02Z

python/singa/autograd.py

            return {"W": self.W, "b": self.b}
        else:
            return {"W": self.W}

+    def set_params_initializer(self, **initializers):


pass initializers as argus of __init__ .

nudles · 2020-05-14T09:14:42Z

The following APIs should be backward compatible. Please test.

class Linear(Layer):
     def __init__(self, num_output, *args, bias=True, **kwargs):
          # the following block is for backward compatibility. 
          # the old code will all Linear(2, 3), or (2, 3, False)
          if len(args) > 0:
             num_input = num_output
             num_output = args[0]
          if len(args) > 1:
             bias = args[1]

       self.num_output = num_output
       self.bias = bias

class Conv2d(Layer):
       def __init__(self,
                 out_channels,
                 kernel_size,
                 *args,
                 stride=1,
                 padding=0,
                 dilation=1,
                 group=1,
                 bias=True,
                 pad_mode="NOTSET",
                 **kwargs):
         # the old code create the layer like: Conv2d(8, 16, 3)， or Conv2d(8, 16, 3, stride=1)
         # the following code block is for backward compatibility
         if len(args) >0:
           in_channel=out_channel
           out_channel = kernel
           kernel = args[0]
         if len(args) > 1:
           stride = args[1]
         if len(args) > 2:
           padding = args[2]

dcslin · 2020-05-15T01:53:28Z

update linear constructor, tested ok

lgtm-com · 2020-05-15T02:00:16Z

This pull request introduces 3 alerts when merging 3d835a0 into db1846d - view on LGTM.com

new alerts:

2 for Unnecessary pass
1 for Unused import

joddiy · 2020-05-18T08:17:33Z

Hi, @dcslin , can I use this PR right now or which operation I can use now? since I need to let the soonx to support the new autograd api.

dcslin · 2020-05-18T08:34:45Z

Hi, @dcslin , can I use this PR right now or which operation I can use now? since I need to let the soonx to support the new autograd api.

Hi @joddiy please refer to #697

joddiy · 2020-05-18T10:36:17Z

Hi, @dcslin , can I use this PR right now or which operation I can use now? since I need to let the soonx to support the new autograd api.

Hi @joddiy please refer to #697

Thanks, shicong, and one thing to confirm, the name of operation should be _ReLU or ReLU?

dcslin · 2020-05-19T15:17:35Z

Hi, @dcslin , can I use this PR right now or which operation I can use now? since I need to let the soonx to support the new autograd api.

Hi @joddiy please refer to #697

Thanks, shicong, and one thing to confirm, the name of operation should be _ReLU or ReLU?

I guess you are building the model? then by convention we use autograd.relu()

nudles · 2020-05-20T01:09:16Z

Hi, @dcslin , can I use this PR right now or which operation I can use now? since I need to let the soonx to support the new autograd api.

Hi @joddiy please refer to #697

Thanks, shicong, and one thing to confirm, the name of operation should be _ReLU or ReLU?

I guess you are building the model? then by convention we use autograd.relu()

I suggest to use layer.ReLU() to avoid mixing operators and layers in constructing the model.
Refer to the first post in #696 (comment)

dcslin · 2020-05-22T03:45:59Z

some of the work is merged into #697
the rest need to be revised as api changed
thus closing this PR

dcslin added 3 commits May 2, 2020 09:12

mark relu operation class as private

0c9bfef

layer set param bug

0541637

linear auto detect in feature size

dd0dc99

modified dropout flatten

2b16b37

dcslin added 2 commits May 5, 2020 06:12

updated linear backward compatibility

7938faa

update LSTM param init

cd289d6

chrishkchris changed the base branch from master to dev May 6, 2020 05:22

chrishkchris changed the base branch from dev to master May 6, 2020 05:23

chrishkchris changed the base branch from master to dev May 6, 2020 05:38

dcslin added 3 commits May 6, 2020 12:09

make operation class deprecated: dropout,flatten,relu

88f6f47

factor out param initializer linear

e538fef

doc for set params initializer

6ecf827

dcslin added 2 commits May 7, 2020 11:26

modified test names

d5481ac

added rnn set initializer

4a98351

dcslin added 2 commits May 7, 2020 17:18

added param initializer for lstm

0517f2b

fix code

e24f710

nudles mentioned this pull request May 12, 2020

Refactor autograd module #696

Closed

nudles reviewed May 12, 2020

View reviewed changes

update linear constructor

3d835a0

dcslin changed the title ~~Autograd refactor~~ [stall]Layer auto detect in features May 18, 2020

dcslin changed the title ~~[stall]Layer auto detect in features~~ [stall]Layer auto detect input size May 18, 2020

dcslin closed this May 22, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[stall]Layer auto detect input size #688

[stall]Layer auto detect input size #688

dcslin commented May 2, 2020

lgtm-com bot commented May 2, 2020

dcslin commented May 2, 2020

nudles commented May 4, 2020

lgtm-com bot commented May 4, 2020

lgtm-com bot commented May 5, 2020

dcslin commented May 6, 2020

lgtm-com bot commented May 6, 2020

lgtm-com bot commented May 6, 2020

lgtm-com bot commented May 7, 2020

lgtm-com bot commented May 7, 2020

nudles May 12, 2020

nudles commented May 14, 2020

dcslin commented May 15, 2020

lgtm-com bot commented May 15, 2020

joddiy commented May 18, 2020 •

edited

Loading

dcslin commented May 18, 2020

joddiy commented May 18, 2020

dcslin commented May 19, 2020

nudles commented May 20, 2020

dcslin commented May 22, 2020

[stall]Layer auto detect input size #688

[stall]Layer auto detect input size #688

Conversation

dcslin commented May 2, 2020

lgtm-com bot commented May 2, 2020

dcslin commented May 2, 2020

nudles commented May 4, 2020

lgtm-com bot commented May 4, 2020

lgtm-com bot commented May 5, 2020

dcslin commented May 6, 2020

lgtm-com bot commented May 6, 2020

lgtm-com bot commented May 6, 2020

lgtm-com bot commented May 7, 2020

lgtm-com bot commented May 7, 2020

nudles May 12, 2020

Choose a reason for hiding this comment

nudles commented May 14, 2020

dcslin commented May 15, 2020

lgtm-com bot commented May 15, 2020

joddiy commented May 18, 2020 • edited Loading

dcslin commented May 18, 2020

joddiy commented May 18, 2020

dcslin commented May 19, 2020

nudles commented May 20, 2020

dcslin commented May 22, 2020

joddiy commented May 18, 2020 •

edited

Loading