Image classification program with fictive new API #68

hedaoyuan · 2017-02-07T13:38:23Z

还有一些问题：

没有考虑evaluator，这里不是必须的；
data_batch的循环中只考虑了最简单的实现，可能还需要增加一些东西；
区分forward/backward主要是为了，可以控制forward只计算到某个特定的layer，方便调试。

wangkuiyi · 2017-02-07T19:27:14Z

image_classification/example.py

+    model.parameter.init()
+
+    # get data from...
+    train_data = paddle.data.create_data_pool(


data_pool怎么知道如何读取图像和label数据呢？这里是不是还是需要定义 data reader (dataprovider)？

是的，这里还没细写。file_reader这是一个用户自己写的读取数据的接口。这部分是参考的api_train.py里面的做法。

wangkuiyi · 2017-02-07T19:27:47Z

image_classification/example.py

+    for i in xrange(10):
+      for batch_id, data_batch in enumerate(train_data):
+        # backward��ʾѵ�ģ�Ͳ���
+        model.calculate(data_batch, backward)


model.calculate(..., backward) ==> model.backward(...) ?

wangkuiyi · 2017-02-07T19:28:17Z

image_classification/example.py

+        model.calculate(data_batch, backward)
+
+        # forward,�����ʾ���㵽output layerΪֹ
+        model.calculate(data_batch, forward, "output"))


model.calculate(..., forward) ==> model.forward(...) ?

wangkuiyi · 2017-02-07T19:30:22Z

image_classification/example.py

+
+    # calculate
+    for i in xrange(10):
+      for batch_id, data_batch in enumerate(train_data):


考虑到有的data provider可能是从Kafka读取infinite data stream，永远不会返回数据结束标记，我理解这里的双重循环应该改成单层：

for mini_batch, end_pass in enumerate(training_data): if end_pass: current_pass++; model.forward(...) model.backward(...)

这样的话，如果training_data是一个streaming data source，那么end_pass永远都是False。

wangkuiyi · 2017-02-07T19:35:53Z

image_classification/example.py

+    cost = paddle.classification_cost(out, label)
+
+    # optimizer
+    optimizer = paddle.optimizer.Optimizer(


考虑到：只有如果有多个optimzier实现的话，我们才会需要一个package叫做optimzier，以便把各个optimizer classes都放在这个package里。那么这里可能用的是某一种optimizer，比如叫做AdamOptimizer，那么应该是下面这样？

optimizer = paddle.optimzier.Adam(...)

嗯，是的这样好。

wangkuiyi · 2017-02-07T19:37:16Z

image_classification/example.py

+    batch_size=128
+
+    # describe the VGG network
+    input = paddle.data_layer(name="image")


考虑到把多种layers放进一个package，这里应该是下面这样？

input = paddle.layer.data(name="image") conv1 = paddle.layer.conv(input, 64, 2, ...) ...

嗯，这样更好。

reyoung · 2017-02-07T13:54:33Z

image_classification/example.py

+    # create paddle model
+    model = paddle.model(...) # some arguments for model, like trainer_count
+    model.network.init(cost)
+    model.optimizer.init(optimizer)


既然如此，为啥不把cost，optimizer都放到paddle.model(...)的参数中呢？

cost, optimizer, flags这些都是model参数是可以放到一起，不过一个函数有太多意义不相关的函数感觉用起来比较麻烦。

reyoung · 2017-02-08T02:27:31Z

image_classification/example.py

+        model.calculate(data_batch, backward)
+
+        # forward,�����ʾ���㵽output layerΪֹ
+        model.calculate(data_batch, forward, "output"))


这里有个问题，如果这个output是可以任意值的话，似乎我们每次调用这个的时候，都会要求重新对Layer做一遍排序，将不需要计算的Layer剃除出去

这个我再想一下，好像不用吧。

helinwang · 2017-02-08T02:38:08Z

image_classification/example.py

+    cost = paddle.classification_cost(out, label)
+
+    # optimizer
+    optimizer = paddle.optimizer.Optimizer(


optimizer需要某一个layer做参数吗？

Default := paddle.optimizer.SGD(...) // derived from paddle.optimizer.Base and overrides `train` def train(...): Default.train(...) train(model, cost, ...) adam = paddle.optimizer.Adam(...) // derived from paddle.optimizer.SGD and overrides `update` adam.train(model, cost, ....) owlqn = paddle.optimizer.OWLQN(...) owlqn.train(model, cost, ...) // derived from paddle.optimizer.Base and overrides `train`

optimizer.train() 中，是不是还有个参数是data，包含了label和vector

reyoung · 2017-02-08T02:52:35Z

image_classification/example.py

+        fc1, act=ReluActivation(), layer_attr=ExtraAttr(drop_rate=0.5))
+    fc2 = paddle.fc_layer(bn, size=512, act=LinearActivation())
+    out = paddle.fc_layer(fc2, size=10, act=SoftmaxActivation())
+    cost = paddle.classification_cost(out, label)


train(model, loss={'prediction', xe_loss(label=data(name="label", type="integer"))})

wangkuiyi · 2017-02-08T02:39:05Z

image_classification/example.py

+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+


import paddle.layer

paddle.layer.data(...)

import 用全路径名

wangkuiyi · 2017-02-08T02:58:24Z

image_classification/example.py

+        fc1, act=ReluActivation(), layer_attr=ExtraAttr(drop_rate=0.5))
+    fc2 = paddle.fc_layer(bn, size=512, act=LinearActivation())
+    out = paddle.fc_layer(fc2, size=10, act=SoftmaxActivation())
+    cost = paddle.classification_cost(out, label)


input = data() label = data() fc = fc(input, parameter_name="Hello") fc2 = fc(fc1, parameter_name="Hello") # 同一个topology里的两个layer共享同一组参数 out = softmax(fc) m1 = paddle.model.create(out) train(m1, paddle.cost.kl_divergence(out, label), ...) print "Label = ", model.eval(read_image, out)) m2 = paddle.model.create(out) train(m2, paddle.cost.cross_entropy(out, label), ...) print "Label = ", model.eval(read_image, out))

out_left = ... out_right = ... m = model(out_left, out_right) train(m, total_cost(left_cost(out_left), right_cost(out_right)), ...)

print test(model, cost, input_image, grouth_truth_label)

两个模型怎么share参数：

input1 = ... fc1 = fc(input1, parameter_name="Hello") out1 = softmax(fc1) m1 = paddle.model.create(out1) input2 = ... fc2 = fc(input, parameter_name="Hello", parameter_model=m1) out2 = softmax(fc2)

wangkuiyi · 2017-02-08T03:15:39Z

image_classification/example.py

+    bn = paddle.batch_norm_layer(
+        fc1, act=ReluActivation(), layer_attr=ExtraAttr(drop_rate=0.5))
+    fc2 = paddle.fc_layer(bn, size=512, act=LinearActivation())
+    out = paddle.fc_layer(fc2, size=10, act=SoftmaxActivation())


讨论结果之一：cost不作为model的一个layer，甚至不是model的一部分。

wangkuiyi · 2017-02-08T03:38:13Z

image_classification/example.py

+    cost = paddle.classification_cost(out, label)
+
+    # optimizer
+    optimizer = paddle.optimizer.Optimizer(


Default := paddle.optimizer.SGD(...) // derived from paddle.optimizer.Base and overrides `train` def train(...): Default.train(...) train(model, cost, ...) adam = paddle.optimizer.Adam(...) // derived from paddle.optimizer.SGD and overrides `update` adam.train(model, cost, ....) owlqn = paddle.optimizer.OWLQN(...) owlqn.train(model, cost, ...) // derived from paddle.optimizer.Base and overrides `train`

wangkuiyi · 2017-02-08T03:49:15Z

image_classification/example.py

+        learning_rate_decay_b=50000 * 100,
+        learning_rate_schedule='discexp',
+        learning_method=MomentumOptimizer(0.9),
+        regularization=L2Regularization(0.0005 * 128))


opt = paddle.optimizer.SGD( learning_rate = ... learining_rate_decay = ... momentum = 0.9)

wangkuiyi · 2017-02-08T04:00:14Z

image_classification/example.py

+        regularization=L2Regularization(0.0005 * 128))
+
+    # create paddle model
+    model = paddle.model(...) # some arguments for model, like trainer_count


L47~L49 是不需要的吧

A example

507ada0

wangkuiyi reviewed Feb 7, 2017

View reviewed changes

wangkuiyi changed the title ~~A example~~ Image classification program with fictive new API Feb 7, 2017

wangkuiyi reviewed Feb 7, 2017

View reviewed changes

wangkuiyi mentioned this pull request Feb 7, 2017

The mnist program with new fictive API #64

Closed

reyoung reviewed Feb 8, 2017

View reviewed changes

helinwang reviewed Feb 8, 2017

View reviewed changes

reyoung reviewed Feb 8, 2017

View reviewed changes

wangkuiyi reviewed Feb 8, 2017

View reviewed changes

follow comments

78b7335

hedaoyuan closed this Mar 3, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Image classification program with fictive new API #68

Image classification program with fictive new API #68

hedaoyuan commented Feb 7, 2017 •

edited

wangkuiyi Feb 7, 2017

hedaoyuan Feb 8, 2017

wangkuiyi Feb 7, 2017

wangkuiyi Feb 7, 2017

wangkuiyi Feb 7, 2017

wangkuiyi Feb 7, 2017

hedaoyuan Feb 8, 2017

wangkuiyi Feb 7, 2017

hedaoyuan Feb 8, 2017

reyoung Feb 7, 2017

hedaoyuan Feb 8, 2017

reyoung Feb 8, 2017

hedaoyuan Feb 8, 2017

helinwang Feb 8, 2017 •

edited

wangkuiyi Feb 8, 2017

jacquesqiao Feb 8, 2017 •

edited

reyoung Feb 8, 2017 •

edited

wangkuiyi Feb 8, 2017

wangkuiyi Feb 8, 2017

wangkuiyi Feb 8, 2017

wangkuiyi Feb 8, 2017

wangkuiyi Feb 8, 2017

wangkuiyi Feb 8, 2017

wangkuiyi Feb 8, 2017

wangkuiyi Feb 8, 2017

wangkuiyi Feb 8, 2017

wangkuiyi Feb 8, 2017

Image classification program with fictive new API #68

Image classification program with fictive new API #68

Conversation

hedaoyuan commented Feb 7, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

helinwang Feb 8, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jacquesqiao Feb 8, 2017 • edited

Choose a reason for hiding this comment

reyoung Feb 8, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hedaoyuan commented Feb 7, 2017 •

edited

helinwang Feb 8, 2017 •

edited

jacquesqiao Feb 8, 2017 •

edited

reyoung Feb 8, 2017 •

edited