Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BACKEND][CODEGEN] C codegen with tests #2161

Merged
merged 11 commits into from Nov 28, 2018

Conversation

@Mutinifni
Copy link
Contributor

commented Nov 24, 2018

This PR introduces a gcc / g++ compatible C code generator for TVM, tested on x86 host.

@tqchen

This comment has been minimized.

Copy link
Member

commented Nov 24, 2018

@Huyuwei can you help review this?

@Huyuwei

This comment has been minimized.

Copy link
Member

commented Nov 24, 2018

@Mutinifni Can you briefly describe the current compilation flow? Most importantly, is it using DSOModule for the runtime?

@Mutinifni

This comment has been minimized.

Copy link
Contributor Author

commented Nov 24, 2018

@Huyuwei It uses CSourceModule for saving the C source code - I have verified this by checking the type_key. Loading is done via the DSOModule.

I was a bit uncertain about this module design and the export_library functionality as well, so please feel let me know if you think a different flow would be better instead.

python/tvm/module.py Show resolved Hide resolved
python/tvm/module.py Outdated Show resolved Hide resolved
src/codegen/codegen_c_host.cc Outdated Show resolved Hide resolved

@tqchen tqchen changed the title C codegen with tests [BACKEND] C codegen with tests Nov 25, 2018

@tqchen tqchen changed the title [BACKEND] C codegen with tests [BACKEND][CODEGEN] C codegen with tests Nov 25, 2018

src/codegen/codegen_c_host.cc Outdated Show resolved Hide resolved
python/tvm/module.py Outdated Show resolved Hide resolved
src/codegen/codegen_c.cc Outdated Show resolved Hide resolved
src/codegen/codegen_c.h Show resolved Hide resolved
namespace codegen {

CodeGenCHost::CodeGenCHost() {
module_name = GetUniqueName("__tvm_module_ctx");

This comment has been minimized.

Copy link
@Huyuwei

Huyuwei Nov 26, 2018

Member

replace "__tvm_module_ctx" with tvm::runtime::symbol::tvm_module_ctx?

This comment has been minimized.

Copy link
@Mutinifni

Mutinifni Nov 26, 2018

Author Contributor

This would make the generated code necessarily dependent on C++, but we want to keep the option of pure C as well. If there's a reason to use the latter symbol, I can change it.

src/codegen/codegen_c_host.cc Show resolved Hide resolved
src/codegen/codegen_c_host.h Outdated Show resolved Hide resolved
python/tvm/contrib/cc.py Outdated Show resolved Hide resolved
@tqchen
tqchen approved these changes Nov 27, 2018
@tqchen

This comment has been minimized.

Copy link
Member

commented Nov 27, 2018

@Huyuwei please take another look

@Huyuwei Huyuwei merged commit a259b73 into dmlc:master Nov 28, 2018

1 check passed

continuous-integration/jenkins/pr-merge This commit looks good
Details
@srkreddy1238

This comment has been minimized.

Copy link
Member

commented Nov 29, 2018

This PR cause the below issue with llvm target.

-tgt="cuda"
+tgt="llvm"

python3 tutorials/get_started.py

!115 = !{!116, !116, i64 0}
!116 = !{!"float32", !117, i64 0}
!117 = !{!"0x119a190", !4, i64 0}

Traceback (most recent call last):
  File "get_started.py", line 196, in <module>
    cc.create_shared(temp.relpath("myadd.so"), [temp.relpath("myadd.o")])
  File "/home/srk/.local/lib/python3.6/site-packages/tvm-0.5.dev0-py3.6-linux-x86_64.egg/tvm/contrib/cc.py", line 34, in create_shared
    _linux_shared(output, objects, options, cc)
  File "/home/srk/.local/lib/python3.6/site-packages/tvm-0.5.dev0-py3.6-linux-x86_64.egg/tvm/contrib/cc.py", line 53, in _linux_shared
    cmd += ["-I" + path for path in find_include_path()]
  File "/home/srk/.local/lib/python3.6/site-packages/tvm-0.5.dev0-py3.6-linux-x86_64.egg/tvm/_ffi/libinfo.py", line 156, in find_include_path
    raise RuntimeError(message)
RuntimeError: Cannot find the files.
List of candidates:
/home/srk/.local/lib/python3.6/include
/home/srk/.local/lib/python3.6/site-packages/include
/home/srk/.local/lib/python3.6/site-packages/3rdparty/include
/home/srk/.local/lib/python3.6/dlpack/include
/home/srk/.local/lib/python3.6/site-packages/dlpack/include
/home/srk/.local/lib/python3.6/site-packages/3rdparty/dlpack/include
@tqchen

This comment has been minimized.

Copy link
Member

commented Nov 29, 2018

Thanks @srkreddy1238 for reporting this, @Mutinifni can you send in a quick fix? We should not directly raise error when the headers are not found in general create_shared, instead, we should only call find_include_path and add it to the options in created_shared when it is a c module

FrozenGene pushed a commit to FrozenGene/tvm that referenced this pull request Dec 27, 2018
[BACKEND][CODEGEN] C codegen with tests (dmlc#2161)
* Implement C code generation with tests

* Code cleanup

* Implement C code generation with tests

* Code cleanup

* tabs to spaces

* make lint compliant

* update export_library and reserve unique C keywords

* move ReserveKeywordsAsUnique to codegen_c

* some documentation and code cleanup

* use tvm.contrib.util for tempdir in testcases
@ZihengJiang ZihengJiang referenced this pull request Feb 1, 2019
@Mutinifni Mutinifni referenced this pull request Feb 4, 2019
2 of 6 tasks complete
wweic pushed a commit to neo-ai/tvm that referenced this pull request Feb 20, 2019
[BACKEND][CODEGEN] C codegen with tests (dmlc#2161)
* Implement C code generation with tests

* Code cleanup

* Implement C code generation with tests

* Code cleanup

* tabs to spaces

* make lint compliant

* update export_library and reserve unique C keywords

* move ReserveKeywordsAsUnique to codegen_c

* some documentation and code cleanup

* use tvm.contrib.util for tempdir in testcases
wweic pushed a commit to neo-ai/tvm that referenced this pull request Feb 20, 2019
[BACKEND][CODEGEN] C codegen with tests (dmlc#2161)
* Implement C code generation with tests

* Code cleanup

* Implement C code generation with tests

* Code cleanup

* tabs to spaces

* make lint compliant

* update export_library and reserve unique C keywords

* move ReserveKeywordsAsUnique to codegen_c

* some documentation and code cleanup

* use tvm.contrib.util for tempdir in testcases
ricann pushed a commit to ricann/tvm that referenced this pull request Mar 4, 2019
[BACKEND][CODEGEN] C codegen with tests (dmlc#2161)
* Implement C code generation with tests

* Code cleanup

* Implement C code generation with tests

* Code cleanup

* tabs to spaces

* make lint compliant

* update export_library and reserve unique C keywords

* move ReserveKeywordsAsUnique to codegen_c

* some documentation and code cleanup

* use tvm.contrib.util for tempdir in testcases
@dongxiao92

This comment has been minimized.

Copy link

commented Apr 29, 2019

Hello, is there any documents or examples about how to use this c backend?

@Mutinifni

This comment has been minimized.

Copy link
Contributor Author

commented Apr 29, 2019

Hi @dongxiao92, you can look at tests/python/unittest/test_codegen_c_host.py for a vector addition example which runs on the host. You can also print out the generated C code from the function export_library in python/tvm/module.py.

@wujiaqi6630

This comment has been minimized.

Copy link

commented Sep 9, 2019

Hello, I want to generate C code too:

func = tvm.build(s, [A, B, C], target=device, name='myGemm')
temp = util.tempdir()
path_dso = temp.relpath("temp.so")
path = temp.relpath('lib.tar')
func.export_library(path_dso)
m = tvm.module.load(path_dso)
print(m.get_source())

But failed:
TVMError: Module[dso] does not support GetSource
Am I using the wrong method? @Mutinifni

@Mutinifni

This comment has been minimized.

Copy link
Contributor Author

commented Sep 9, 2019

Hi @wujiaqi6630, if you want to see the C source code that is generated, you can use func.get_source() because func is the C source module where as m is the DSO module. Only the C source module supports GetSource.

@wujiaqi6630

This comment has been minimized.

Copy link

commented Sep 10, 2019

Thank you !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.