Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Making Activity Classifier updatable #2258

Open
mecid opened this issue Aug 21, 2019 · 14 comments
Open

Making Activity Classifier updatable #2258

mecid opened this issue Aug 21, 2019 · 14 comments

Comments

@mecid
Copy link

@mecid mecid commented Aug 21, 2019

Hello, I try to do my Activity Classifier updatable using this script.

import coremltools

spec = coremltools.utils.load_spec('RawSleepClassifier.mlmodel')

spec.description.trainingInput.extend([spec.description.input[0]])
spec.description.trainingInput.extend([spec.description.output[-1]])

builder = coremltools.models.neural_network.NeuralNetworkBuilder(spec=spec)
builder.make_updatable(['dense_layer1'])
builder.set_categorical_cross_entropy_loss(name='lossLayer', input='activityProbability', target='trueActivityLabel')

from coremltools.models.neural_network import SgdParams
builder.set_sgd_optimizer(SgdParams(lr=0.01, batch=32))

builder.set_epochs(10)

from coremltools.models import MLModel
mlmodel_updatable = MLModel(builder.spec)
mlmodel_updatable.save('SleepClassifier.mlmodel')

But I get the error:

RuntimeWarning: You will not be able to run predict() on this Core ML model. Underlying exception message was: Error compiling model: "Error reading protobuf spec. validator error: Model specification version for an updatable model must be '4' or above.".

I run MacOS 10.15 Beta 6 and Turi Create 5.7 with CoreML tools v3.0b3

@TobyRoseman

This comment has been minimized.

Copy link
Collaborator

@TobyRoseman TobyRoseman commented Aug 21, 2019

It looks like TuriCreate is still exporting to Core ML using specificationVersion: 1. You do indeed need version 4 in order to have an updatable model. There are also lots of other improvements made in Core ML since the first version.

We should update TuriCreate to export models to Core ML using the most recent specification version.

@mecid

This comment has been minimized.

Copy link
Author

@mecid mecid commented Aug 21, 2019

@TobyRoseman so there is no way to generate updatable Activity Classifier with Turi Create 5.7? As I remember it works before.

@TobyRoseman

This comment has been minimized.

Copy link
Collaborator

@TobyRoseman TobyRoseman commented Aug 21, 2019

@mecid - There is no way to generate an updatable Core ML model with Turi Create 5.7. I also don't think this has worked with any previous version of Turi Create.

@mecid

This comment has been minimized.

Copy link
Author

@mecid mecid commented Aug 22, 2019

@TobyRoseman can we expect it anytime soon?

@hollance

This comment has been minimized.

Copy link

@hollance hollance commented Aug 22, 2019

What if you add spec.specificationVersion = 4 to the above script?

@mecid

This comment has been minimized.

Copy link
Author

@mecid mecid commented Aug 22, 2019

@hollance it works! Model works correctly. I will try to run on-device update task.
Is it legal to change version like this? I mean I can set version to 5 or whatever and it also works.

@hollance

This comment has been minimized.

Copy link

@hollance hollance commented Aug 22, 2019

@mecid I'm not sure what version of coremltools you're using, but I think in the latest version, builder.set_categorical_cross_entropy_loss() will automatically fill in that version number. If not, you can fill in anything you like here, but I would use 4 as that is the correct number. ;-)

@mecid

This comment has been minimized.

Copy link
Author

@mecid mecid commented Aug 23, 2019

@hollance so you mean by setting version to 4 and using builder.set_categorical_cross_entropy_loss() my Model packaging will be updated to latest version?
I on the latest coreml tools beta version.

@hollance

This comment has been minimized.

Copy link

@hollance hollance commented Aug 23, 2019

@mecid I was slightly mistaken, builder. make_updatable() will now automatically fill in the specificationVersion for you. Not sure why that didn't happen automatically in your script, as with the current beta version of coremltools it should do that. This means you shouldn't have to fill in specificationVersion yourself anymore.

@mecid

This comment has been minimized.

Copy link
Author

@mecid mecid commented Aug 23, 2019

@hollance I'm on CoreML tools 3.0b3 and Turi Create 5.7.
Is it safe to use that model with manual setting version to 4? I fear that it can be broken.

@hollance

This comment has been minimized.

Copy link

@hollance hollance commented Aug 24, 2019

It is safe. Note that coremltools 3.0b4 is the latest, not b3.

@mecid

This comment has been minimized.

Copy link
Author

@mecid mecid commented Aug 24, 2019

@hollance thanks, just checked it now, we already have b5.

@hollance

This comment has been minimized.

Copy link

@hollance hollance commented Aug 24, 2019

Ah nice!

@TobyRoseman

This comment has been minimized.

Copy link
Collaborator

@TobyRoseman TobyRoseman commented Dec 16, 2019

Created #2860 to track the more general issue (i.e. this issue for all toolkits that support exporting to Core ML).

@mecid mecid closed this Dec 21, 2019
@mecid mecid reopened this Dec 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.