New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature] Use Core ML on iOS #10468

Closed
cancan101 opened this Issue Jun 6, 2017 · 18 comments

Comments

Projects
None yet
@cancan101
Contributor

cancan101 commented Jun 6, 2017

Apple announced Core ML which may be useful for abstracting away the complexities of the hardware platform:

image

Related:

@ofirbb

This comment has been minimized.

ofirbb commented Jun 6, 2017

Actually, all that you really need is to write the implementation of the python conversion from a TF graph to a CoreML (also protobuf) model file:
CoreML custom conversion tool

@michaelisard

This comment has been minimized.

Member

michaelisard commented Jun 8, 2017

Thanks for opening this. We we treat this as the tracking issue, assigning to @keveman.

@johndpope

This comment has been minimized.

Contributor

johndpope commented Jun 12, 2017

@songkuixi

This comment has been minimized.

songkuixi commented Jun 12, 2017

Does your tool support transformation from TensorFlow Inception v3 Model to CoreML Model? @johndpope

@johndpope

This comment has been minimized.

Contributor

johndpope commented Jun 12, 2017

The link above is a vanilla implementation of coremltools Python conversion of Kees's models (with tensorflow backend) if you search inception v3 in Keras and swap in /out code above it should work.

@johndpope

This comment has been minimized.

Contributor

johndpope commented Jun 12, 2017

@gsabran

This comment has been minimized.

gsabran commented Jun 15, 2017

FYI if that's helpful I've hosted the package code on github to make conversations easier: https://github.com/gsabran/coremltools

@nojvek

This comment has been minimized.

nojvek commented Jun 29, 2017

Thanks @gsabran.

Is that the official pip package source or just a clone. I see apple mentions it in their docs. https://developer.apple.com/machine-learning/

I do like that Apple just didn't invent their own crazy format but built it on protobuf.

As far as I'm aware keras can load from 'json + h5'.

# serialize model to JSON
model_json = model.to_json()
with open("model.json", "w") as json_file:
    json_file.write(model_json)
# serialize weights to HDF5
model.save_weights("model.h5")
print("Saved model to disk")
 
# later...
 
# load json and create model
json_file = open('model.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
loaded_model = model_from_json(loaded_model_json)
# load weights into new model
loaded_model.load_weights("model.h5")

This should be relatively straightforward to achieve to write a converter for right?

@gsabran

This comment has been minimized.

gsabran commented Jun 30, 2017

@nojvek it's just a clone since there's no official source on github to my knowledge

@andforce

This comment has been minimized.

andforce commented Sep 6, 2017

how convert “ckpt” to "mlmodel"?

@joelteply

This comment has been minimized.

joelteply commented Sep 8, 2017

I'd suggest someone use the caffe to tensorflow project as a starting point, maybe combining both, and output keras since it uses tensorflow as a backend and is compatible with coremltools.Really once you get the weights it's not horrible to do by hand, but still annoying.

@lahayeg

This comment has been minimized.

lahayeg commented Nov 22, 2017

Hi there. I'm novice (2 days)on tensorflow and all deep learning. I managed to train a Mobilenet model (I got a output_graph.pb and a output_labels.txt) in order to test it on iOS. But in fact the coremltools does not provide a converser. How could I manage to do it?
Option 1: tensorflow => keras => core ML
Option 2: caffe (Mobilenet seems to exist but not sure of the official compliance and I tried without to manage to install tools) => core ML

@tensorflowbutler

This comment has been minimized.

Member

tensorflowbutler commented Dec 20, 2017

It has been 14 days with no activity and this issue has an assignee.Please update the label and/or status accordingly.

1 similar comment
@tensorflowbutler

This comment has been minimized.

Member

tensorflowbutler commented Jan 4, 2018

It has been 14 days with no activity and this issue has an assignee.Please update the label and/or status accordingly.

@tensorflowbutler

This comment has been minimized.

Member

tensorflowbutler commented Jan 24, 2018

Nagging Assignee: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

@petewarden

This comment has been minimized.

Member

petewarden commented Jan 29, 2018

There's now an official TensorFlow to CoreML converter at https://github.com/tf-coreml/tf-coreml, so closing this as fixed.

@petewarden petewarden closed this Jan 29, 2018

@igor9silva

This comment has been minimized.

igor9silva commented Jan 31, 2018

@petewarden it doesn't seem official. This seems though -> https://github.com/apple/coremltools.

@oscarbg

This comment has been minimized.

oscarbg commented May 12, 2018

Hi @petewarden and other tensorflow devs,
new to tensoflow world so sorry I'm misinterpreting something but
just seeing Google release MLKit this week with support for Android & IOS I was curious to learn underlying tech..
seems it uses for it's custom model API tensorflow lite underneath which also supports also both Android & IOS..
on Android it uses Android NN API if avaiable to use HW accel. on phone..
problem is from quick search is that altough tensorflow lite supports IOS it doesn't uses CoreML underneath so only using CPU mode..
you say here for using CoreML there is a converter, but then you must use CoreML API and abandon Tensorflow lite API usage.. in other words "only" helps with model conversion issues..
not same inference API for both iOS & Android that is HW accel. on both OSes..
I say that because also at Google I/O also ARCore 1.2 was released and there is IOS support using ARKit underneath..
so just saying same for tensorflow lite..
of course Tensorflow API should support only reading CoreML models in case compiling for IOS..
May I open a separate issue?
thanks..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment