Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce DataBufferFromCallback Class #54

Merged
merged 1 commit into from Apr 27, 2020

Conversation

jijoongmoon
Copy link
Collaborator

With this class, it is possible to train with user specific data
generation callback function. This is the default to get training
data unless there is [DataSet] key in configuration file.

  • in Application/Classification/jni/main_func.cpp
    NN.train(getMiniBatch_train, getMiniBatch_val, getMiniBatch_train)

Then, data buffer thread call these functions to get newest data with
size of mini batch.

The format this function should be :
/*

  • @brief Callback function to get user specific data
  • @param[in] X data 3D float vector type
  • @param[in] Y label 3D float vector type
  • @param[out] status status for error handle
  • @RetVal true / false generate all data for this epoch
    */
    bool func(std::vector<std::vector<std::vector>>& X,
    std::vector<std::vector<std::vector>>& Y, int &status)

Self evaluation:

  1. Build test: [X]Passed [ ]Failed [ ]Skipped
  2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon jijoong.moon@samsung.com

@taos-ci
Copy link
Collaborator

taos-ci commented Apr 23, 2020

📝 TAOS-CI Version: 1.4.20191203. Thank you for submitting PR #54. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ec2-54-180-96-14.ap-northeast-2.compute.amazonaws.com/.

@taos-ci
Copy link
Collaborator

taos-ci commented Apr 23, 2020

:octocat: cibot: @jijoongmoon, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://ec2-54-180-96-14.ap-northeast-2.compute.amazonaws.com/nntrainer/ci/repo-workers/pr-checker/54-202004231550260.47635102272034-8329b0b84881c066c4c0592f560cb17981497d85/.

@taos-ci
Copy link
Collaborator

taos-ci commented Apr 23, 2020

:octocat: cibot: @jijoongmoon, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://ec2-54-180-96-14.ap-northeast-2.compute.amazonaws.com/nntrainer/ci/repo-workers/pr-checker/54-202004231601090.43018293380737-09289474f086c915369b0f1ec4b045949257d99c/.

@taos-ci
Copy link
Collaborator

taos-ci commented Apr 23, 2020

:octocat: cibot: @jijoongmoon, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://ec2-54-180-96-14.ap-northeast-2.compute.amazonaws.com/nntrainer/ci/repo-workers/pr-checker/54-202004231610010.526447057724-18aaf743a24717a7d1c597dc1473961f148d3d53/.

With this class, it is possible to train with user specific data
generation callback funciton. This is the default to get training
data unless there is [DataSet] key in configuration file.

- in Application/Classification/jni/main_func.cpp
 NN.train(getMiniBatch_train, getMiniBatch_val, getMiniBatch_train)

Then, data buffer thread call these functions to get newest data with
size of mini batch.

The format this function should be :
/*
 * @brief Callback function to get user specific data
 * @param[in] X data  3D float vector type
 * @param[in] Y label 3D float vector type
 * @param[out] status status for error handle
 * @RetVal true / false generate all data for this epoch
 */
bool  func(std::vector<std::vector<std::vector<float>>>& X,
      std::vector<std::vector<std::vector<float>>>& Y, int &status)

**Self evaluation:**
1. Build test:	 [X]Passed [ ]Failed [ ]Skipped
2. Run test:	 [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>
Copy link
Contributor

@zhoonit zhoonit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jijoongmoon jijoongmoon merged commit 3fd8368 into nnstreamer:master Apr 27, 2020
@jijoongmoon jijoongmoon deleted the data_callback branch April 12, 2022 00:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants