Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

support sgd & dot & embedding with rsp #72

Closed
wants to merge 18 commits into from
Closed

support sgd & dot & embedding with rsp #72

wants to merge 18 commits into from

Conversation

eric-haibin-lin
Copy link
Owner

No description provided.

eric-haibin-lin and others added 16 commits May 27, 2017 05:22
@eric-haibin-lin
merge with 38f7c55
a96efdd 
compiles on GPU

update check alloc:

Checkpoint. Pass elem-sum gpu test

bug fix for copyfromto. sparse sgd test pass on gpu

inefficient implementation for csr copy

update submodule

fix lint

Simple bind with infer storage type (#32)

* Symbol binding for sparse tensor development. (#31)

* Initial checkin

* Add init functions for simple bind in graph_executor

* Add simple_bind c_api

* Add simple bind c-api

* Assign zeros to in_args, arg_grads, and aux_states

* Add simple_bind2 python interface

* Fix python interface bugs

* Interface changes

* Fix

* Fix core dump

* Add bind_ith_exec c_api

* Change simple_bind2

* Fix seg fault

* Finish simple_bind

* Change _bind_ith_exec

* Refactor simple_bind initialization flow for bind

* Consolidate bind and simple_bind graph init flow

* Fix bug

* Clean up

* Add comments

* Clean up

* Clean up

* Minor correction

* Rename APIs in graph executor

* Refactor

* Rebase

* Delete deprecated functions

* Move more front-end work to backend

* Bug fix

* Fix failed tests

* Minor fix

* Fix lint

* Fix lint

* Revert unnecessary changes

* Revert

* Revert

* Clean up

* Fix lint

Conflicts:
	python/mxnet/symbol.py
	src/executor/graph_executor.cc

* Add inferstorage to graph executor

* re-enable tests for sparse embedding with simple_bind

* type switch fix in sparse embedding"
;

change `default` to `default_storage` for cast storage op (#33)

* change default to default_storage

* disable cpp test build temporarily

attempt to fix windows build error, and fix lint (#34)

update nnvm submodule (#37)

Scipy build (#38)

* update nnvm submodule

* add scipy pip install for dockerfile

Python3 unit tests (#39)

* change xrange to range for python3 compatiblity"

* remove more xrange from tests

replace long with int for python3 (#40)

fix the rest of TShape constructor errors (#41)

fix lint (#42)

fix wrong usage of mshadow::Shape1" (#43)

implementation for Csr slice on cpu (#36)

* CPU implementation for CSR

remove seg_len from csr slice

add some docs for slice csr

change indptr, values, etc to be private member

bug fix in sparse embedding

update nnvm submoduel

fix lint

update unit test for sparse nd"

* add const for SliceCsrIndPtr kernel

Fix sparse dot according to the new RSP definition (#35)

* Fix csr dot dns

* Fix sparse dot

* Add fallback and test cases for dot(csr, dns)=dns

* Add int type switch

* Fix

* Fix

* Fix

update mshadow submodule (#44)

Fix dns to rsp (#46)

fix lint (#47)

add runtime storage fallback detection" (#48)

* add runtime storage fallback detection"

* replace cast storage ex with cast storage impl

Fm example (#45)

* update csr slice logic to avoid confusion. add more exmaples.

* add hint to module.update

* more testcases(fallback) for sparse_nd

* add to_csr() and to_rsp() method. More unit test (fallback now)

* add fm test. fix lint

* register sparse sgd under Optim.SGD

* update dmlc-core submoduel

* change indptr to _indptr temporarily. add const ref to fname

fix lint

fix lint; (#51)

Guard gpu cast storage (#50)

* Clean up

* Fix typo

Rearrange unit test files (#52)
@eric-haibin-lin
fix lint. add scipy for python_test. fix scipy.sparse import error. f…
8652043 
…ix truediv for python3
@eric-haibin-lin
fix travis test (#54)
4e2f40f 
* remove pyc files

* add verbose for travis nosetests
@eric-haibin-lin
cleanup some testing code and enums (#57)
be1e63b 
* update Makefile

* refactor test_sparse_operator

* change `default_storage` back to `default`

* remove unused cpp tests
@eric-haibin-lin
port libsvm parser to mxnet as libsvm iter (#55)
0dd6503 
* copied csv iter to libsvm iter

test

libsvm iter draft

handle round batch == false for csr batch loader

code refactoring

add get stype, shape interface to iiter

separate class for sparse iter

add missing file

fix mem corruption'

rename variables

add comments

also read label from libsvm

add test. update docs. update submodule

Conflicts:
	python/mxnet/sparse_ndarray.py

* update submodule

* fix lint

* update test

* revert naming change
@eric-haibin-lin
add benchmark scritp for dot (#59)
965dfd7 
* add benchmark scritp for dot

add gpu option for bench

add get_data funciton for benchmark

print t_sparse, too;

add comment

change nnz to dnesity

add backward

* add comment
@eric-haibin-lin
update fm test (#62)
cbfa792
@eric-haibin-lin
introduce CSRNDarray and rowsparseNDarray to python frontend api (#58)
86d55df 
* introduce CSRNDarray and rowsparseNDarray to python frontend api

* temporarily disable fm_module test
@eric-haibin-lin
fix lint (#64)
3e486d5
@eric-haibin-lin
fix typo. disable libsvm io test (#65)
7c975a5
@reminisce @eric-haibin-lin
Improve dot (#61)
5c0685b 
* Init checkin

* Fix

* Adjust dot parallelization methods

* Set num_omp_threads for benchmark from command line

* Fix omp thread number

* Clean up

* Add scipy as dot baseline

* Fix format
@reminisce @eric-haibin-lin
sparse_retain op (#66)
c109eb6 
* Initial checkin

* Fix bugs

* Add unit test for sparse_retain

* Add example and modify test
@eric-haibin-lin
add storage cast for outputs that have non-default storage (#67)
a7c8fe8
@eric-haibin-lin
fix gpu build (#69)
39d12bc
@reminisce @eric-haibin-lin
Fix test_sparse_retain python3 issue (#68)
d27027a
@eric-haibin-lin
update nnvm submodule (#71)
78a87ad
@eric-haibin-lin eric-haibin-lin force-pushed the walk-around branch 2 times, most recently from fef9f36 to 56afd54 Compare June 8, 2017 00:12
@eric-haibin-lin
Copy link
Owner Author

@reminisce could you review?

reminisce
reminisce reviewed Jun 8, 2017
View reviewed changes
src/operator/tensor/indexing_op.h Outdated
auto req_type = req;
size_t segment_start = i * segment_len;
size_t segment_end = (i + 1) * segment_len;
auto segment_start = i * segment_len;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

segment_start might be greater than num_rows if there are more threads than num_rows. Do we need a check here?

src/operator/tensor/indexing_op.h Outdated
if (req_type == kWriteTo) req_type = kAddTo;
KERNEL_ASSIGN(dst_val[j * width + k], req_type, src[y * width + k]);
for (index_t k = 0; k < num_cols; k++) {
KERNEL_ASSIGN(dst_val[j * num_cols + k], req_type, src[y * num_cols + k]);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since j*num_cols and y*num_cols are not dependent on the loop, is it more efficient to calculate them outside the loop?

src/operator/tensor/indexing_op.h Outdated
(*out_attrs)[0] = kRowSparseStorage;
(*out_attrs)[1] = kRowSparseStorage;
(*out_attrs)[1] = kDefaultStorage;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to use ASSIGN_CHECK?

src/operator/tensor/indexing_op.h Outdated
auto num_cols = out->shape_[1];
// mutate req to avoid extra space allocation
auto out_req = req;
if (out_req == kWriteTo) out_req = kAddTo;
Copy link
Collaborator

@reminisce reminisce Jun 8, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why out_req can be mutated here?

@eric-haibin-lin
draft for sgd rsp rsp
9350855 
support sgd(rsp, rsp)

support dot(csr, rsp) when rsp is full

add ref to const ndarray params

support sparse embedding with rsp weight'

fix lint

modify embedding backward to produce dense grad

remove invalid_rid for rsp->dns

remove previous embedding op changes

pass sparse embedding test

add STORAGE_TYPE_ASSIGN_CHECK

remove backward storage infer
@eric-haibin-lin
revert nnvm
d6833f8
@eric-haibin-lin eric-haibin-lin mentioned this pull request Jun 9, 2017
Merged
@eric-haibin-lin
Copy link
Owner Author

closing this one with #75 for replacement

@eric-haibin-lin eric-haibin-lin deleted the walk-around branch June 15, 2017 20:35
eric-haibin-lin pushed a commit that referenced this pull request Apr 4, 2018
@tqchen
Change license file to be detectable by github (#72)
5b69ffd
eric-haibin-lin pushed a commit that referenced this pull request Apr 4, 2018
@tqchen
fix coreml tools (#72)
fd76749
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@reminisce reminisce reminisce left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@eric-haibin-lin @reminisce