Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tensorflow support #802

Open
wants to merge 29 commits into
base: master
from

Conversation

@pabloEntropia
Copy link
Collaborator

pabloEntropia commented Nov 26, 2018

Set of algorithms to support Tensorflow C API inside Essentia.

The Algorithm TensorflowPredict accepts Tensorflow graphs in Protobuff format.

Streaming prediction is also supported with the helper functions: VectorRealToTensor and TensorToPool. TensorflowPredict returns a stream of pools that can be converted to a stream of frames with the helper functions PoolToTensor and TensorToVectorReal.
A typical pipelines would be:
AudioLoader >> FrameCutter >> Windowing >> Spectrum >> MelBands >> VectorRealToTensor >> TensorToPool >> TensorFlowPredict >> PoolToTensor >> TensorToVectorReal
This architecture allows retrieving intermediate layers of the graph and saving them in the output pools. Thus, different PoolToTensor instances could be used to extract the ouput of different layers if needed.

This PR introduces Boost::multi_array as the base type for tensors in Essentia.

pabloEntropia and others added 23 commits Oct 10, 2018
ArrayNDReal should be refactored to dinamically supoer N dimentional arrays
Aux function added to handle streams of arraynd and pools
Added support for arraynd in poolstorage
other stuff:
- improved exception messages.
other stuff:
- move from stream of pools to stream of singlePools
- change tensor related algos to receive single token
- improve tensor related algos doc
explanation:
  - this way `Tensor` alias can be used for other types
  - hardcoded dim  is more consistant with boost::multi_array
  - rename Tensor::fromPythonRef to fromPythonCopy
  - remove unuseful `timeAxis` param
  - VectorRealToTensor: add Exceptions
  -  TensorFlowPredict: improve output generation
  - TensorFlowPredict: small style changes
- prevent re-downloading dependencies
- add -lrt flag as needed for absl libs.
-  modify Makefile to prevent compiling example
@pabloEntropia pabloEntropia force-pushed the pabloEntropia:tensorflow_support branch from 6358ec2 to 95cd9a5 Dec 17, 2018
@dbogdanov dbogdanov mentioned this pull request Jan 30, 2019
1 of 2 tasks complete
@ronggong

This comment has been minimized.

Copy link
Contributor

ronggong commented Feb 1, 2019

@pabloEntropia What is the progress of this issue?

generate_pc.py locates the python package for TensorFlow
and creates a pkg-config file to link agaist the .so libs
on it.
TODO: setup_tensorflow.sh should run previous script and
put the .pc file on an appropiate place.
Also make sure that Python is able to locate .so libs on
run time.
remove -Werror flag from nsync

use relative paths when compiling tensorflow to prevent too long
lines

add matmul_op_fused to the list of compiled operations as it is
required by newer models
@pabloEntropia pabloEntropia force-pushed the pabloEntropia:tensorflow_support branch from 5580acc to 13f5d7c Nov 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.