Add sequence slice operator #5546

wanghaox · 2017-11-10T06:41:34Z

resolve #5545
Add forward and backward kernel code
Add unitest

chengduoZH · 2017-11-10T08:07:02Z

paddle/operators/sub_sequence_op.cc

+- Case:
+    LoD(x) = {{0, 3, 6, 10}}; Dims(x0) = (10, 3, 2)
+    offset = (0, 1, 1); size = (2, 1, 2)
+    LoD(Out) = {{0, 2, 3, 5}}; Dims(Out) = (5,3,2)


It is better to write the case concretely, like context_project.

chengduoZH · 2017-11-10T08:13:12Z

paddle/operators/sub_sequence_op.cc

+
+    auto dim_0 = 0;
+    for (size_t i = 0; i < sizes.size(); ++i) {
+      dim_0 += sizes[i];


Please ensure sizes[i] is greater than zero.

done, added the PADDLE_ENFORCE_LT check

chengduoZH · 2017-11-10T08:14:29Z

paddle/operators/sub_sequence_op.cc

+                   "Output(Out) of SubSequenceOp should not be null.");
+    auto input_dims = ctx->GetInputDim("X");
+
+    auto offsets = ctx->Attrs().Get<std::vector<int>>("offset");


Please ensure offsets value is valid.

changed from ctx->Attrs() to input tensor

chengduoZH · 2017-11-10T08:18:22Z

paddle/operators/sub_sequence_op.cc

+                        framework::OpAttrChecker* op_checker)
+      : OpProtoAndCheckerMaker(proto, op_checker) {
+    AddInput("X", "(LoDTensor), "
+            "the variable-length input of SubSequenceOp");


Don't forget to put a full stop.

chengduoZH · 2017-11-10T08:21:36Z

paddle/operators/sub_sequence_op.cc

+        "size",
+        "A list<int> to describes size for sub sequence item.");
+    AddOutput("Out",
+              "(Tensor), Variable-length output of "


"Out" should be LoDTensor. And AddOutput("Out",... should be wrote behind AddInput("X", "(LoDTensor), ....

chengduoZH · 2017-11-10T08:39:55Z

paddle/operators/sub_sequence_op.cc

+The operator crop a subsequence from given sequence with given start offset and subsequence size.
+It only supports sequence (LoD Tensor with level number is 1).
+- Case:
+    LoD(x) = {{0, 3, 6, 10}}; Dims(x0) = (10, 3, 2)


Dims(x0) -> Dims(x)

The example case has been rewritten

chengduoZH · 2017-11-10T08:45:47Z

paddle/operators/sub_sequence_op.h

+    PADDLE_ENFORCE_EQ(n, offset_len,
+                      "The length of input and offset should be the same")
+    PADDLE_ENFORCE_EQ(n, size_len,
+                      "The length of input and size should be the same")


Why n, offset_len and n, size_len should be equal.

The three is working on the same sequence.

chengduoZH · 2017-11-10T08:52:39Z

paddle/operators/sub_sequence_op.h

+      auto size = sizes[i];
+      PADDLE_ENFORCE_LT(lod[0][i] + offset + size, lod[0][i + 1],
+                        "The target tensor's length overflow")
+    }


It is better to put lines 63~68 into InferShape. Because InferShape is running in compiling time, if the config (size and offset) is wrong, user can check out mistakes in time.

The offset and length have been changed to input tensor, so they must be checked in the runtime.

chengduoZH · 2017-11-10T08:55:30Z

paddle/operators/sub_sequence_op.h

+      auto size = sizes[i];
+      PADDLE_ENFORCE_LT(lod[0][i] + offset + size, lod[0][i + 1],
+                        "The target tensor's length overflow")
+    }


I think lines 111~123 are unnecessary. Because you have done these in SubSequenceOpKernel.

chengduoZH · 2017-11-10T08:57:32Z

paddle/operators/sub_sequence_op.h

+      auto x_grad_stride = framework::stride(x_grad->dims());
+
+      auto offset = offsets[i];
+      auto size = sizes[i];


The variables of size and offset are unnecessary.

qingqing01 · 2017-11-10T09:25:56Z

paddle/operators/sub_sequence_op.cc

+See the License for the specific language governing permissions and
+limitations under the License. */
+
+#include "paddle/operators/sub_sequence_op.h"


We have:

sequence_conv_op
sequence_pool_op
sequence_softmax_op
...

so, maybe rename to sequence_slice_op ?

sequence_slice_op sounds good.

done, rename to sequence_slice_op

qingqing01 · 2017-11-10T09:29:55Z

paddle/operators/sub_sequence_op.cc

+    auto input_dims = ctx->GetInputDim("X");
+
+    auto offsets = ctx->Attrs().Get<std::vector<int>>("offset");
+    auto sizes = ctx->Attrs().Get<std::vector<int>>("size");


In the raw Paddle Layer, the offset and size are the input, not attributes. Since they may be changed during each mini-batch.

Agree, I think it's better to use 'length' instead of 'size'.

qingqing01 · 2017-11-10T09:33:12Z

paddle/operators/sub_sequence_op.h

+limitations under the License. */
+
+#pragma once
+#include "paddle/framework/eigen.h"


Remove this line, since not use the Eigen function.

qingqing01 · 2017-11-10T09:33:37Z

paddle/operators/sub_sequence_op.cu

+See the License for the specific language governing permissions and
+limitations under the License. */
+
+#define EIGEN_USE_GPU


Remove this line, since there is no Eigen function in the .h file.

qingqing01 · 2017-11-10T09:38:16Z

paddle/operators/sub_sequence_op.h

+    x_grad->set_lod(lod);
+    x_grad->mutable_data<T>(ctx.GetPlace());
+    auto temp = framework::EigenVector<T>::Flatten(*x_grad);
+    temp.device(ctx.GetEigenDevice<Place>()) = temp.constant(static_cast<T>(0));


Use the math::SetConstant or math:: set_constant

https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/math/math_function.h

qingqing01 · 2017-11-10T09:39:26Z

python/paddle/v2/framework/tests/test_sub_sequence_op.py

+            out_lod[0].append(out_lod_offset)
+
+        outs = np.concatenate(outs, axis=0)
+        self.outputs = {'Out': outs}


Also check the output LoD.

self.outputs = {'Out': (outs, out_lod)}

wanghaox

The code has been updated, please review

… sub_sequence_op

chengduoZH · 2017-11-14T06:35:56Z

paddle/operators/sequence_slice_op.cc

+        c1, c2]
+       [d1, d2;
+        e1, e2]]
+    LoD(X) = {{0, 3, 5}}; Dims(X) = (4, 1, 2)


Dims(X) = (4, 1, 2) -> Dims(X) = (5, 2)

chengduoZH · 2017-11-14T06:41:33Z

paddle/operators/sequence_slice_op.cc

+             "(Tensor), "
+             "A vector<int> to describes length for sub sequence item.");
+    AddOutput("Out",
+              "(LoDTensor), output of sequence slice Op.");


the output of SequenceSliceOp.

chengduoZH · 2017-11-14T06:45:56Z

paddle/operators/sequence_slice_op.cc

+             "the input of SequenceSliceOp.");
+    AddInput("Offset",
+             "(Tensor), "
+             "A vector<int> to describes offset for sub sequence item.");


For reference only: a vector<int> to describe the offset of every input sequence for sub sequence item.

chengduoZH · 2017-11-14T06:47:45Z

paddle/operators/sequence_slice_op.cc

+              "(LoDTensor), output of sequence slice Op.");
+    AddComment(R"DOC(
+Sequence slice operator
+The operator crop a subsequence from given sequence with given start offset and subsequence length.


There should be a blank row below "Sequence slice operator"

chengduoZH · 2017-11-14T07:08:42Z

paddle/operators/sequence_slice_op.h

+      offset_cpu.CopyFrom(*offset, platform::CPUPlace(), ctx.device_context());
+      offset_data = offset_cpu.data<int64_t>();
+
+      framework::Tensor length_cpu;


Should these two variables be placed outside of the if_block？

what the difference?

chengduoZH · 2017-11-14T07:14:58Z

paddle/operators/sequence_slice_op.cc

+    Out = [[a1, a2;
+            b1, b2]
+            [e1, e2]]
+    LoD(Out) = {{0, 2, 3}}


This description of case is not consistent with your code.
Your code describes that the input size and output size are equal.

fixed the output dims error

chengduoZH · 2017-11-14T07:19:06Z

paddle/operators/sequence_slice_op.h

+    auto out_lod = SequenceSliceLoD(*in, offset_data, length_data);
+    out->set_lod(out_lod);
+    math::SetConstant<Place, T> set_zero;
+    set_zero(ctx.device_context(), out, static_cast<T>(0));


If your document about the function of this operator is right, you do not need to clean out memory

chengduoZH · 2017-11-14T07:24:44Z

python/paddle/v2/framework/tests/test_sequence_slice_op.py

+        x = np.random.random((100, 3, 2)).astype('float32')
+        lod = [[0, 20, 40, 60, 80, 100]]
+        offset = np.array([1, 2, 3, 4, 5]).flatten().astype("int64")
+        length = np.array([10, 8, 6, 4, 2]).flatten().astype("int64")


You can separate the definitions of data. Like this

chengduoZH · 2017-11-14T07:58:44Z

Please read this document first.

… sub_sequence_op

wanghaox

update sequence_slice_op code

wanghaox · 2017-11-14T10:45:37Z

paddle/operators/sequence_slice_op.cc

+             "the input of SequenceSliceOp.");
+    AddInput("Offset",
+             "(Tensor), "
+             "A vector<int> to describes offset for sub sequence item.");


wanghaox · 2017-11-14T10:45:59Z

paddle/operators/sequence_slice_op.cc

+             "(Tensor), "
+             "A vector<int> to describes length for sub sequence item.");
+    AddOutput("Out",
+              "(LoDTensor), output of sequence slice Op.");


wanghaox · 2017-11-14T10:46:06Z

paddle/operators/sequence_slice_op.cc

+              "(LoDTensor), output of sequence slice Op.");
+    AddComment(R"DOC(
+Sequence slice operator
+The operator crop a subsequence from given sequence with given start offset and subsequence length.


wanghaox · 2017-11-14T10:46:17Z

paddle/operators/sequence_slice_op.cc

+        c1, c2]
+       [d1, d2;
+        e1, e2]]
+    LoD(X) = {{0, 3, 5}}; Dims(X) = (4, 1, 2)


wanghaox · 2017-11-14T10:46:54Z

paddle/operators/sequence_slice_op.cc

+    Out = [[a1, a2;
+            b1, b2]
+            [e1, e2]]
+    LoD(Out) = {{0, 2, 3}}


fixed the output dims error

wanghaox · 2017-11-14T10:48:00Z

paddle/operators/sequence_slice_op.h

+    auto out_lod = SequenceSliceLoD(*in, offset_data, length_data);
+    out->set_lod(out_lod);
+    math::SetConstant<Place, T> set_zero;
+    set_zero(ctx.device_context(), out, static_cast<T>(0));


wanghaox · 2017-11-14T10:48:38Z

python/paddle/v2/framework/tests/test_sequence_slice_op.py

+        x = np.random.random((100, 3, 2)).astype('float32')
+        lod = [[0, 20, 40, 60, 80, 100]]
+        offset = np.array([1, 2, 3, 4, 5]).flatten().astype("int64")
+        length = np.array([10, 8, 6, 4, 2]).flatten().astype("int64")


chengduoZH · 2017-11-15T06:33:03Z

paddle/operators/sequence_slice_op.cc

+                   "Output(Out) of SequenceSliceOp should not be null.");
+    auto input_dims = ctx->GetInputDim("X");
+
+    ctx->SetOutputDim("Out", input_dims);


Why do you set input_dims to Out? In some case, out_dim is not equal to input_dims.

The function InferShape is called at the compiling stage, the offset and length are unknown, so I set the dims to the Maximum, and change them to the real-value at runtime.

chengduoZH · 2017-11-15T06:39:41Z

paddle/operators/sequence_slice_op.h

+    }
+
+    auto lod = in->lod();
+    auto out_lod = SequenceSliceLoD(*in, offset_data, length_data);


You can get out_lod from Out.

chengduoZH · 2017-11-15T06:57:25Z

paddle/operators/sequence_slice_op.h

+    auto lod = in->lod();
+    auto out_lod = SequenceSliceLoD(*in, offset_data, length_data);
+
+    x_grad->set_lod(lod);


x_grad_lod can be assigned in here.

chengduoZH · 2017-11-15T06:59:57Z

paddle/operators/sequence_slice_op.h

+using LoD = framework::LoD;
+
+template <typename T>
+LoD SequenceSliceLoD(const T& in, const int64_t* offset_data,


Please use inline.

chengduoZH · 2017-11-15T07:06:04Z

python/paddle/v2/fluid/tests/test_sequence_slice_op.py

+            out_lod_offset = out_lod_offset + len(sub_x)
+            outs.append(sub_x)
+            out_lod[0].append(out_lod_offset)
+        outs = np.concatenate(outs, axis=0)


I think it is better that putting those code(line 19~25) into a function.

Those codes are short and only use once.

wanghaox

update some code

wanghaox · 2017-11-15T07:27:11Z

paddle/operators/sequence_slice_op.cc

+                   "Output(Out) of SequenceSliceOp should not be null.");
+    auto input_dims = ctx->GetInputDim("X");
+
+    ctx->SetOutputDim("Out", input_dims);


The function InferShape is called at the compiling stage, the offset and length are unknown, so I set the dims to the Maximum, and change them to the real-value at runtime.

wanghaox · 2017-11-15T07:27:49Z

paddle/operators/sequence_slice_op.h

+using LoD = framework::LoD;
+
+template <typename T>
+LoD SequenceSliceLoD(const T& in, const int64_t* offset_data,


wanghaox · 2017-11-15T07:28:41Z

paddle/operators/sequence_slice_op.h

+    }
+
+    auto lod = in->lod();
+    auto out_lod = SequenceSliceLoD(*in, offset_data, length_data);


wanghaox · 2017-11-15T07:30:34Z

paddle/operators/sequence_slice_op.h

+    auto lod = in->lod();
+    auto out_lod = SequenceSliceLoD(*in, offset_data, length_data);
+
+    x_grad->set_lod(lod);


wanghaox · 2017-11-15T07:31:38Z

python/paddle/v2/fluid/tests/test_sequence_slice_op.py

+            out_lod_offset = out_lod_offset + len(sub_x)
+            outs.append(sub_x)
+            out_lod[0].append(out_lod_offset)
+        outs = np.concatenate(outs, axis=0)


Those codes are short and only use once.

qingqing01 · 2017-11-15T11:53:34Z

paddle/operators/sequence_slice_op.h

+    PADDLE_ENFORCE_EQ(offset->dims().size(), 1UL,
+                      "Only support one level sequence now.");
+    PADDLE_ENFORCE_EQ(length->dims().size(), 1UL,
+                      "Only support one level sequence now.");


It seems the offset->dims().size() == 2, length->dims().size() == 2, like the label in the cost operator, the rank is 2. the dims[0] is the batch size, dims[1] = 1.

… sub_sequence_op

…into sub_sequence_op

chengduoZH · 2017-11-16T08:10:14Z

paddle/operators/sequence_slice_op.cc

+    PADDLE_ENFORCE_EQ(length_dim.size(), 2UL,
+                      "Only support one level sequence now.");
+
+    ctx->SetOutputDim("Out", input_dims);


Please add some notes (why do you set input_dims to Out?) .

If not set, it triggers an error
EnforceNotMet: enforce numel() > 0 failed
When calling this method, the Tensor's numel must be larger than zero. Please check Tensor::Resize has been called first. at [/home/work/wanghao/code/Paddle/paddle/framework/tensor_impl.h:118]

so initialize the output dims by set input_dims to the Out

chengduoZH · 2017-11-16T08:12:20Z

paddle/operators/sequence_slice_op.cc

+    PADDLE_ENFORCE_EQ(offset_dim.size(), 2UL,
+                      "Only support one level sequence now.");
+    PADDLE_ENFORCE_EQ(length_dim.size(), 2UL,
+                      "Only support one level sequence now.");


The comment is inconsistent with the expression.

dim 0 is for batch size, dim 1 is for the sequence offset

done,added comment

chengduoZH · 2017-11-16T08:13:58Z

paddle/operators/sequence_slice_op.cc

+             "a vector<int> to describe the length of every input sequence for "
+             "sub sequence item.");
+    AddOutput("Out",
+              "(LoDTensor), The output of SequenceSliceOp.");


The->the

chengduoZH · 2017-11-16T08:17:51Z

paddle/operators/sequence_slice_op.cc

+    AddComment(R"DOC(
+Sequence slice operator
+
+The operator crop a subsequence from given sequence with given start offset and subsequence length.


This is for your reference only.
The operator crops a subsequence from given sequence with given offset and subsequence length.

chengduoZH · 2017-11-16T08:22:47Z

paddle/operators/sequence_slice_op.cc

+            b1, b2]
+            [e1, e2]]
+    LoD(Out) = {{0, 2, 3}}; Dims(Out) = (3, 2)
+NOTE: The length of the input, offset and length should be the same. The offset start from 0.


This is for your reference only.
The first dimension size of input, the size of offset and Length, should be equal

chengduoZH · 2017-11-16T08:28:07Z

paddle/operators/sequence_slice_op.h

+      PADDLE_ENFORCE_LT(
+          lod[0][i] + offset_data[i] + length_data[i],
+          lod[0][i + 1],
+          "The target tensor's length overflow")}


请安装pre-commit，86行代码格式不对

chengduoZH · 2017-11-16T08:42:13Z

paddle/operators/sequence_slice_op.h

+
+    x_grad->mutable_data<T>(ctx.GetPlace());
+    math::SetConstant<Place, T> set_zero;
+    set_zero(ctx.device_context(), x_grad, static_cast<T>(0));


x_grad->mutable_data<T>(ctx.GetPlace()); x_grad->set_lod(in->lod()); ......

…into sub_sequence_op

wanghaox

update code, fix some typos

wanghaox · 2017-11-16T09:27:49Z

paddle/operators/sequence_slice_op.cc

+    PADDLE_ENFORCE_EQ(offset_dim.size(), 2UL,
+                      "Only support one level sequence now.");
+    PADDLE_ENFORCE_EQ(length_dim.size(), 2UL,
+                      "Only support one level sequence now.");


dim 0 is for batch size, dim 1 is for the sequence offset

wanghaox · 2017-11-16T09:30:08Z

paddle/operators/sequence_slice_op.cc

+             "a vector<int> to describe the length of every input sequence for "
+             "sub sequence item.");
+    AddOutput("Out",
+              "(LoDTensor), The output of SequenceSliceOp.");


wanghaox · 2017-11-16T09:31:54Z

paddle/operators/sequence_slice_op.cc

+    AddComment(R"DOC(
+Sequence slice operator
+
+The operator crop a subsequence from given sequence with given start offset and subsequence length.


wanghaox · 2017-11-16T09:37:37Z

paddle/operators/sequence_slice_op.cc

+    PADDLE_ENFORCE_EQ(length_dim.size(), 2UL,
+                      "Only support one level sequence now.");
+
+    ctx->SetOutputDim("Out", input_dims);


If not set, it triggers an error
EnforceNotMet: enforce numel() > 0 failed
When calling this method, the Tensor's numel must be larger than zero. Please check Tensor::Resize has been called first. at [/home/work/wanghao/code/Paddle/paddle/framework/tensor_impl.h:118]

so initialize the output dims by set input_dims to the Out

wanghaox · 2017-11-16T09:42:07Z

paddle/operators/sequence_slice_op.cc

+            b1, b2]
+            [e1, e2]]
+    LoD(Out) = {{0, 2, 3}}; Dims(Out) = (3, 2)
+NOTE: The length of the input, offset and length should be the same. The offset start from 0.


wanghaox · 2017-11-16T09:43:22Z

paddle/operators/sequence_slice_op.h

+      PADDLE_ENFORCE_LT(
+          lod[0][i] + offset_data[i] + length_data[i],
+          lod[0][i + 1],
+          "The target tensor's length overflow")}


wanghaox · 2017-11-16T09:46:40Z

paddle/operators/sequence_slice_op.h

+
+    x_grad->mutable_data<T>(ctx.GetPlace());
+    math::SetConstant<Place, T> set_zero;
+    set_zero(ctx.device_context(), x_grad, static_cast<T>(0));


…into sub_sequence_op

wanghaox

fix comments

wanghaox · 2017-11-16T11:44:17Z

paddle/operators/sequence_slice_op.cc

+    PADDLE_ENFORCE_EQ(offset_dim.size(), 2UL,
+                      "Only support one level sequence now.");
+    PADDLE_ENFORCE_EQ(length_dim.size(), 2UL,
+                      "Only support one level sequence now.");


done,added comment

wanghaox · 2017-11-16T11:44:26Z

paddle/operators/sequence_slice_op.cc

+    PADDLE_ENFORCE_EQ(length_dim.size(), 2UL,
+                      "Only support one level sequence now.");
+
+    ctx->SetOutputDim("Out", input_dims);


… sub_sequence_op

add sub sequence operator code and unittest

23b0388

qingqing01 requested review from pkuyym, luotao1 and qingqing01 November 10, 2017 07:28

qingqing01 added this to Doing in Port Operators Nov 10, 2017

chengduoZH reviewed Nov 10, 2017

View reviewed changes

qingqing01 reviewed Nov 10, 2017

View reviewed changes

chengduoZH added the OpPorting label Nov 10, 2017

wanghaox commented Nov 14, 2017

View reviewed changes

wanghaox added 3 commits November 14, 2017 11:41

update the sub_sequence_op tp sequence_slice_op code.

f23d6cc

update the sub_sequence_op to sequence_slice_op code.

b24afd8

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

8e7c8bb

… sub_sequence_op

chengduoZH reviewed Nov 14, 2017

View reviewed changes

wanghaox added 2 commits November 14, 2017 18:53

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

29c2582

… sub_sequence_op

update sequence slice op, fix some error

9a18e78

wanghaox commented Nov 14, 2017

View reviewed changes

chengduoZH changed the title ~~Add sub_sequence operator~~ Add sequence_sub operator Nov 15, 2017

chengduoZH reviewed Nov 15, 2017

View reviewed changes

update sequence_slice_op.h, change some code

4c426f1

wanghaox commented Nov 15, 2017

View reviewed changes

qingqing01 reviewed Nov 15, 2017

View reviewed changes

qingqing01 changed the title ~~Add sequence_sub operator~~ Add sequence slice operator Nov 15, 2017

wanghaox added 7 commits November 16, 2017 12:11

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

40a6c48

… sub_sequence_op

change offset and length's rank to 2, dim[0] for batch size

3dcf0da

change offset and length's rank to 2, dim[0] for batch size

1d95173

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

7bb2680

… sub_sequence_op

Merge branch 'sub_sequence_op' of https://github.com/wanghaox/Paddle …

4579bd4

…into sub_sequence_op

update some code

352c5a9

Merge branch 'sub_sequence_op' of https://github.com/wanghaox/Paddle …

069dcc2

…into sub_sequence_op

chengduoZH reviewed Nov 16, 2017

View reviewed changes

wanghaox added 2 commits November 16, 2017 18:01

fix some typos

49a5942

Merge branch 'sub_sequence_op' of https://github.com/wanghaox/Paddle …

294570f

…into sub_sequence_op

wanghaox commented Nov 16, 2017

View reviewed changes

wanghaox added 2 commits November 16, 2017 19:29

fix some comments

a0b7a07

Merge branch 'sub_sequence_op' of https://github.com/wanghaox/Paddle …

06dc89e

…into sub_sequence_op

wanghaox commented Nov 16, 2017

View reviewed changes

fix some codes

3603546

chengduoZH previously approved these changes Nov 17, 2017

View reviewed changes

fix some comments

39bce49

wanghaox dismissed chengduoZH’s stale review via 39bce49 November 20, 2017 02:25

wanghaox added 3 commits November 20, 2017 11:24

fix some comments

794117b

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

f7c1562

… sub_sequence_op

fix some comments

d68f861

chengduoZH approved these changes Nov 20, 2017

View reviewed changes

wanghaox merged commit f350be3 into PaddlePaddle:develop Nov 20, 2017

qingqing01 moved this from Doing to Done in Port Operators Nov 20, 2017

wanghaox deleted the sub_sequence_op branch November 24, 2017 13:23

Add sequence slice operator #5546

Add sequence slice operator #5546

Conversation

wanghaox commented Nov 10, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chengduoZH Nov 10, 2017 • edited Loading

Choose a reason for hiding this comment

qingqing01 Nov 10, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qingqing01 Nov 10, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wanghaox left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chengduoZH commented Nov 14, 2017

wanghaox left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wanghaox left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chengduoZH Nov 10, 2017 •

edited

Loading

qingqing01 Nov 10, 2017 •

edited

Loading

qingqing01 Nov 10, 2017 •

edited

Loading