Load from saved model support #68

Enet4 · 2017-03-09T01:28:31Z

This PR exposes TensorFlow's native capability of loading saved model bundles from a directory.

Added TF_LoadSessionFromSavedModel function to tensorflow-sys;
Added function Session::from_saved_model, which provides a safe generic API with the minimum arguments required. In the future, one might consider adding alternative functions that would let users specify run_options and retrieve meta_graph_def.

This feature will hopefully make loading pre-trained models in Rust more accessible. Please let me know if you would like a complete example or an integration test. Some of the examples found in this repository should be easily adjusted to test this feature.

Example of use:

let mut graph = Graph::new();
println!("Loading session...");
let sess = Session::from_saved_model(
    &SessionOptions::new(), &["serve"],
    &mut graph,
    "test-model").unwrap();

println!("Loaded model with {} operations.", graph.operation_iter().count());

- add TF_LoadSessionFromSavedModel function to tensorflow-sys - add function in Session that provides safe loading from a saved model

daschl · 2017-03-09T09:08:46Z

src/session.rs

 use std::ptr;
+use std::result::Result as StdResult;


hm I find this confusing, but if @adamcrume is good with it

Well, I just needed a hint on line 56 to pluck the result out, and this approach sounded elegant to me. We could consider something else though...

In other places we're naming std::result::Result directly without importing it, like so:

pub type Result<T> = std::result::Result<T, Status>;

I'd prefer that, just for consistency. On a side note, I regret adding that type alias. I assumed it was an accepted pattern because of std::fmt::Result, std::io::Result, and std::thread::Result, but having multiple types with the same name (just in different modules) causes no end of headaches.

daschl · 2017-03-09T09:10:20Z

src/session.rs

+        let tags_ptr: Vec<*const c_char> = tags_cstr.iter().map(|t| t.as_ptr()).collect();
+
+        let inner = unsafe {
+        tf::TF_LoadSessionFromSavedModel(


formatting looks a bit off here, can you run it through rustfmt please?

daschl · 2017-03-09T09:12:14Z

@Enet4 I'd love to see at least an example on this, since we are lacking them anyways and it also helps other people onboard more quickly

Enet4 · 2017-03-09T12:40:13Z

@daschl All right, I have formatted the code (my bad!) and added an example based on regression, which was already available. Still, I'm open to file renames or other tweaks.

daschl · 2017-03-09T12:45:28Z

@Enet4 very cool, thanks! Of course @adamcrume has the final say on this ;)

adamcrume

Mostly looks good, with a few tweaks. Please run the code through rustfmt. Also, thanks for the example code; we can always use more examples and more tests.

adamcrume · 2017-03-10T03:26:42Z

src/session.rs

 use std::ptr;
+use std::result::Result as StdResult;


In other places we're naming std::result::Result directly without importing it, like so:

pub type Result<T> = std::result::Result<T, Status>;

I'd prefer that, just for consistency. On a side note, I regret adding that type alias. I assumed it was an accepted pattern because of std::fmt::Result, std::io::Result, and std::thread::Result, but having multiple types with the same name (just in different modules) causes no end of headaches.

adamcrume · 2017-03-10T03:42:02Z

src/session.rs

+                                       .to_str()
+                                       .and_then(|s| CString::new(s.as_bytes()).ok())
+                                       .ok_or_else(|| {
+            Status::new_set(Code::InvalidArgument, "Invalid export directory path").unwrap()


You can use the invalid_arg! macro for this.

adamcrume · 2017-03-10T03:43:13Z

src/session.rs

+                                         .map(|t| CString::new(t.as_ref()))
+                                         .collect::<StdResult<_, _>>()
+                                         .map_err(|_| {
+            Status::new_set(Code::InvalidArgument, "Invalid tag name").unwrap()


invalid_arg! again.

adamcrume · 2017-03-10T04:00:40Z

src/session.rs

+        let inner = unsafe {
+            tf::TF_LoadSessionFromSavedModel(options.inner,
+                                             ptr::null(),
+                                             export_dir_cstr.to_bytes_with_nul().as_ptr() as


The return value of to_bytes_with_nul doesn't live long enough. You can just use export_dir_cstr.as_ptr(), since it already guarantees a null terminator.

- use invalid_arg! - remove std result alias - fix getting char pointer to export_dir

Enet4 · 2017-03-10T10:54:27Z

I still wonder how I got the tag conversion right and then screwed up on the export_dir string... 🤔 Nevertheless, the changes were made. :)

adamcrume · 2017-03-11T05:32:51Z

Thanks!

jhseu · 2017-03-15T03:23:46Z

Note that we're preferring to add a SavedModelBundle wrapper for the return value in other languages. You need the MetaGraphDef to extract out the signatures in the SavedModel.

Enet4 · 2017-03-15T11:32:10Z

@jhseu Admittedly, I knew that at least the Java bindings would be doing it with a bundle (tensorflow/tensorflow#7134), but I had found no reason to replicate that design here, at the time. But given that the meta-graph is still unreachable, I agree that we should still seek to improve this saved model API. Also, a proper MetaGraph abstraction would probably be nicer than just retrieving a byte buffer.

* added new row indexer for parquet data frame * updated all tests and code to use DateTimeOffset * added logical JSON type * added new dataset handling of rows through pivoting * Update PlainValuesReader.cs * built more single responsibility around ParquetReader type to ensure efficient deallocation of resources using IDisposable * updated reader to look at nulls * added branches to set type IList as either nullable or non-nullable and done this against the required attribute on the column header * moved BigDecimal to own file

Load from saved model support

1e11f53

- add TF_LoadSessionFromSavedModel function to tensorflow-sys - add function in Session that provides safe loading from a saved model

daschl reviewed Mar 9, 2017

View reviewed changes

Enet4 added 2 commits March 9, 2017 10:42

Apply rustfmt over previous changes.

4014131

Add example for loading a saved model

bd6c354

adamcrume requested changes Mar 10, 2017

View reviewed changes

Refactor code

91d2a39

- use invalid_arg! - remove std result alias - fix getting char pointer to export_dir

Merge branch 'master' into export-saved-model

e0f4612

adamcrume merged commit 8bd62df into tensorflow:master Mar 11, 2017

Enet4 deleted the export-saved-model branch March 13, 2017 18:18

adamcrume mentioned this pull request Mar 15, 2017

Possibility to Save/Restore Checkpoints #30

Open

Enet4 mentioned this pull request Jul 3, 2017

Add SavedModelBundle API #90

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Load from saved model support #68

Load from saved model support #68

Enet4 commented Mar 9, 2017 •

edited

Loading

daschl Mar 9, 2017

Enet4 Mar 9, 2017

adamcrume Mar 10, 2017

daschl Mar 9, 2017

daschl commented Mar 9, 2017

Enet4 commented Mar 9, 2017

daschl commented Mar 9, 2017

adamcrume left a comment

adamcrume Mar 10, 2017

adamcrume Mar 10, 2017

adamcrume Mar 10, 2017

adamcrume Mar 10, 2017

Enet4 commented Mar 10, 2017

adamcrume commented Mar 11, 2017

jhseu commented Mar 15, 2017

Enet4 commented Mar 15, 2017 •

edited

Loading

Load from saved model support #68

Load from saved model support #68

Conversation

Enet4 commented Mar 9, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daschl commented Mar 9, 2017

Enet4 commented Mar 9, 2017

daschl commented Mar 9, 2017

adamcrume left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Enet4 commented Mar 10, 2017

adamcrume commented Mar 11, 2017

jhseu commented Mar 15, 2017

Enet4 commented Mar 15, 2017 • edited Loading

Enet4 commented Mar 9, 2017 •

edited

Loading

Enet4 commented Mar 15, 2017 •

edited

Loading