Image processing using OpenCV and multi-threads. #1

qingqing01 · 2016-12-14T10:45:27Z

可供Paddle的PyDataProvider调用的图像处理库，这个库目前支持功能：

提供了python调用接口，目前还采用Boost.Python @reyoung 因为不在Paddle住代码库，目前还想继续采用Boost.Python，以后可以再去掉，你看这样可否？
支持多线程处理
支持resize + random crop + flipping。

后续会继续加其他data argumentation，所以比较希望这部分先merge进去，方便持续开发。

有一个问题是：采用 @reyoung 推荐的ThreadPool，即： https://github.com/progschj/ThreadPool 。
该PR的ThreadPool.h的copyright加的是： https://github.com/progschj/ThreadPool/blob/master/COPYING
不知道这样可否？

reyoung · 2016-12-15T03:32:16Z

transformer/DataTransformer.cpp

+      capacity_(capacity),
+      threadPool_(threadNum),
+      prefetchQueue_(capacity) {
+  fetchCount_ = -1;


放到init list里面

重构后去掉了该参数~

reyoung · 2016-12-15T03:33:05Z

transformer/main.cpp

+
+int main(int argc, char** argv) {
+  float mean[3] = {103.939, 116.779, 123.68};
+  DataTransformer* trans = new DataTransformer(


没有free
不要用new

改成了Python代码单测，这个文件删掉了。

reyoung · 2016-12-15T03:33:27Z

transformer/main.cpp

+int main(int argc, char** argv) {
+  float mean[3] = {103.939, 116.779, 123.68};
+  DataTransformer* trans = new DataTransformer(
+      4, 1024, false, true, 224, 224, 256, false, true, mean);


参数过多了。一般这种情况，我们会写一个DataTransformConfig的POD

reyoung · 2016-12-15T03:33:58Z

transformer/main.cpp

+  float mean[3] = {103.939, 116.779, 123.68};
+  DataTransformer* trans = new DataTransformer(
+      4, 1024, false, true, 224, 224, 256, false, true, mean);
+  std::string src = argv[1];


unittest还是用gtest吧，扩展起来方便。

改成了Python代码单测~

reyoung · 2016-12-15T03:35:05Z

transformer/ThreadPool.h

@@ -0,0 +1,108 @@
+/* Copyright (c) 2012 Jakob Progsch, Václav Zeman


新建一个

third_patry目录，然后git submoudle对方的版本库，不要copy进来。

reyoung · 2016-12-15T03:43:28Z

transformer/DataTransformer.cpp

+
+void DataTransformer::obtain(float* data, int* label) {
+  if (fetchId_ >= fetchCount_) {
+    LOG(FATAL) << "Empty data";


返回值，bool。 true成功，false失败。

reyoung · 2016-12-15T03:43:36Z

transformer/DataTransformer.cpp

+  if (fetchId_ >= fetchCount_) {
+    LOG(FATAL) << "Empty data";
+  }
+  DataTypePtr ret = results_[fetchId_].get();


reyoung · 2016-12-15T03:44:22Z

transformer/DataTransformer.cpp

+  DataTypePtr ret = results_[fetchId_].get();
+  *label = ret->second;
+  memcpy(data, ret->first, sizeof(float) * imgPixels_);
+  prefetchQueue_.enqueue(ret);


其实感觉obtain直接返回float* 效果更好吧？

已重构，使用Boost.Numpy。

reyoung · 2016-12-15T03:44:31Z

transformer/DataTransformer.h

+   * @param filename The file name.
+   * @param im       The image to be saved.
+   */
+  void imsave(std::string filename, cv::Mat& im) { cv::imwrite(filename, im); }


reyoung · 2016-12-15T03:44:39Z

transformer/DataTransformer.h

+   * @param img    The input image Mat to be transformed.
+   * @param target target is used to save the transformed data.
+   */
+  void transform(cv::Mat& img, float* target);


cv::Mat& => const cv::Mat&

如果是复杂类型，要么是 const Blah&(只读)，要么是Blah*(读写)

参考 https://www.zhihu.com/question/20132622/answer/22302151

qingqing01 · 2016-12-18T06:20:57Z

@reyoung Thanks for your review :)

reyoung · 2016-12-18T07:13:50Z

CMakeLists.txt

+
+find_package(PythonLibs 2.7 REQUIRED)
+find_package(PythonInterp 2.7 REQUIRED)
+find_package(Glog)


看起来glog也应该是REQUIRED的吧？

reyoung · 2016-12-18T07:15:53Z

transformer/DataTransformer.cpp

+}
+
+int DataTransformer::Rand(int min, int max) const {
+  std::mt19937 rng(time(0));


random这个搞，每次都调用一个time(0)，似乎会比较慢。

class Random { public: Random() { ... } int randInt(int min, int max) const { std::uniform_int_dist<int> dist(min, max); return dist(eng); } private: std::default_random_engine eng; };

使用了 std::default_random_engine eng，但是没有单独写个class，或许下个PR需要添加其他随机数产生时，再单独提出来。

reyoung · 2016-12-18T07:21:01Z

transformer/DataTransformer.cpp

+  float scale = config_->scale_;
+  float* meanVal = config_->meanValues_;
+  for (int h = 0; h < height; ++h) {
+    const uchar* ptr = cv_cropped_img.ptr<uchar>(h);


uchar => uint8_t ?

uchar也可以

uint8_t在标准库里面，uchar应该不在。

reyoung · 2016-12-18T07:22:50Z

transformer/DataTransformer.cpp

+}
+
+void DataTransformer::transform(cv::Mat& cvImgOri, float* target) const {
+  const int imgChannels = cvImgOri.channels();


看起来这个函数是很多操作的集合，可否把这些操作注释一下或者独立成几个函数。

// crop // sub mean. // etc

下个PR添加更多data argumentation操作时再拆分吧。

reyoung · 2016-12-18T07:24:33Z

transformer/DataTransformer.h

+  T(T const&) = delete; \
+  void operator=(T const& t) = delete
+
+typedef enum { CHANNEL_MEAN = 0, ELEMENT_MEAN = 1, NULL_MEAN = 2 } MeanType;


enum MeanType {...};

reyoung · 2016-12-18T07:30:18Z

transformer/Parallel.h

+  /**
+   * @brief Check whether the PyObject is writable or not.
+   */
+  void pyWritableCheck(PyObject* o) {


这几个check用上了么？

没有用到，已删除。

reyoung · 2016-12-18T07:32:14Z

transformer/tests/CMakeLists.txt

@@ -0,0 +1,4 @@
+add_test(NAME test_dejpeg
+    COMMAND ${PROJ_ROOT}/.set_python_path.sh -d ${CMAKE_CURRENT_BINARY_DIR}/../.


${CMAKE_CURRENT_BINARY_DIR}/../. =>${CMAKE_CURRENT_BINARY_DIR}/..

reyoung · 2016-12-21T03:11:58Z

transformer/DataTransformer.cpp

+  try {
+    cv::Mat im = cv::imread(imgFile, cvFlag);
+    if (!im.data) {
+      LOG(ERROR) << "Could not decode image";


LOG(ERROR) => LOG(FATAL) ?

不是特别理解这里的是不是要程序core掉。如果不core掉，似乎trg里面的数据就是错的了。如果core掉，那似乎应该用LOG(FATAL)。如果想做容忍错误，应该用返回值之类的东西。

原意想容错，更改后去掉了try catch，改成了LOG(FATAL) ，不允许不符合规范的图片，比如gif图片。

reyoung · 2016-12-21T03:13:02Z

transformer/DataTransformer.cpp

+}
+
+int DataTransformer::rand(int min, int max) const {
+  std::default_random_engine eng;


这么搞会慢啊亲。。为啥不改一下啊。。

另外，这么改完了，似乎每次的随机值都一样了？因为random_engine不会初始化随机种子吧？？

将 std::default_random_engine eng 移到class里面~

reyoung · 2016-12-21T03:13:24Z

transformer/DataTransformer.cpp

+  float scale = config_->scale_;
+  float* meanVal = config_->meanValues_;
+  for (int h = 0; h < height; ++h) {
+    const uchar* ptr = cv_cropped_img.ptr<uchar>(h);


uint8_t在标准库里面，uchar应该不在。

reyoung

Basically LGTM, except the argument passing style. Please fix them if you want.

reyoung · 2016-12-21T07:06:12Z

transformer/DataTransformer.h

+   * @param img    The input image Mat to be transformed.
+   * @param target target is used to save the transformed data.
+   */
+  void transform(cv::Mat& img, float* target);


cv::Mat& => const cv::Mat&

如果是复杂类型，要么是 const Blah&(只读)，要么是Blah*(读写)

参考 https://www.zhihu.com/question/20132622/answer/22302151

reyoung · 2016-12-21T07:06:50Z

transformer/DataTransformer.h

+   * @param filename The file name.
+   * @param im       The image to be saved.
+   */
+  void imsave(std::string filename, cv::Mat& im) const {


https://www.zhihu.com/question/20132622/answer/22302151

reyoung · 2016-12-21T07:07:08Z

transformer/Parallel.h

+
+  ~Parallel() {}
+
+  int start(boost::python::list& pysrc, PyObject* pylabel, int mode) {


https://www.zhihu.com/question/20132622/answer/22302151

qingqing01 · 2016-12-22T06:49:56Z

argument passing style 下个PR会一起修改~

Data transformer using OpenCV and multi-threads.

f8a499e

qingqing01 assigned reyoung and luotao1 Dec 14, 2016

reyoung suggested changes Dec 15, 2016

View reviewed changes

Remove BlockQueue and Refine unit tests

16f7c9c

reyoung suggested changes Dec 18, 2016

View reviewed changes

qingqing01 added 2 commits December 19, 2016 22:45

Add git submodule for ThreadPool and follow comments

6c483b2

Rename

9d61f10

reyoung suggested changes Dec 21, 2016

View reviewed changes

change uchar to uint8_t and modify rand function.

016841b

reyoung approved these changes Dec 21, 2016

View reviewed changes

qingqing01 merged commit 397ec9e into PaddlePaddle:master Dec 22, 2016

		@@ -0,0 +1,108 @@
		/* Copyright (c) 2012 Jakob Progsch, Václav Zeman

		@@ -0,0 +1,4 @@
		add_test(NAME test_dejpeg
		COMMAND ${PROJ_ROOT}/.set_python_path.sh -d ${CMAKE_CURRENT_BINARY_DIR}/../.


		~Parallel() {}

		int start(boost::python::list& pysrc, PyObject* pylabel, int mode) {

Image processing using OpenCV and multi-threads. #1

Image processing using OpenCV and multi-threads. #1

Conversation

qingqing01 commented Dec 14, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qingqing01 commented Dec 18, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qingqing01 Dec 19, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qingqing01 Dec 19, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reyoung left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qingqing01 commented Dec 22, 2016

qingqing01 commented Dec 14, 2016 •

edited

Loading

qingqing01 Dec 19, 2016 •

edited

Loading

qingqing01 Dec 19, 2016 •

edited

Loading