Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception occurs when load SD model in beta5 #8470

Closed
SidneyLann opened this issue Nov 30, 2019 · 6 comments · Fixed by KonduitAI/deeplearning4j#93
Closed

Exception occurs when load SD model in beta5 #8470

SidneyLann opened this issue Nov 30, 2019 · 6 comments · Fixed by KonduitAI/deeplearning4j#93
Assignees

Comments

@SidneyLann
Copy link

@SidneyLann SidneyLann commented Nov 30, 2019

  SameDiff sd;
  File modelFile = new File(NerUtil.DATA_HOME + "/model/ner");

  if (modelFile.list().length > 0) {
    System.out.println("Load model...");
    sd = CheckpointListener.loadLastCheckpoint(modelFile, true);
    // sd=SameDiff.load(modelFile, true);
  } else {
    System.out.println("Build model...");
    sd = SameDiff.create();

    SDVariable input2 = sd.placeHolder("bertOutput", DataType.FLOAT, NerUtil.MINI_BATCH_SIZE, NerUtil.MAX_SENTENCE_LENGTH, NerUtil.MAX_SENTENCE_LENGTH * 6);
    SDVariable label = sd.placeHolder("label", DataType.FLOAT, NerUtil.MINI_BATCH_SIZE, NerUtil.MAX_SENTENCE_LENGTH);

    SDVariable wOut = sd.var("wOut", new XavierInitScheme('c', NerUtil.MAX_SENTENCE_LENGTH * 6, NerUtil.getLabelNum()), DataType.FLOAT, NerUtil.MINI_BATCH_SIZE, NerUtil.MAX_SENTENCE_LENGTH * 6,
        NerUtil.getLabelNum());
    SDVariable bOut = sd.zero("bOut", DataType.FLOAT, NerUtil.getLabelNum());

    SDVariable multiply4Output = sd.mmul("multiply4Output", input2, wOut);
    SDVariable bia4Output = sd.nn.biasAdd("bia4Output", multiply4Output, bOut);
    SDVariable maxprob = sd.math.amax(bia4Output, 2);
    SDVariable output = sd.nn.softmax("nerOutput", maxprob);

    // System.out.println(output.getArr());
    SDVariable loss = sd.loss().softmaxCrossEntropy("loss2", label, output);

    sd.setLossVariables(loss);
    double learningRate = 1e-3;
    TrainingConfig config = new TrainingConfig.Builder().l2(1e-4).updater(new Adam(learningRate)).dataSetFeatureMapping("bertOutput").dataSetLabelMapping("label")
        .trainEvaluation(sd.getVariable("nerOutput"), 0, new Evaluation()).build();

    sd.setTrainingConfig(config);
    // sd.asFlatFile(modelFile, true);
  }

  CheckpointListener checkpointListener = CheckpointListener.builder(modelFile).saveEveryNEpochs(1).deleteExisting(true).build();

  UIListener uiListener = UIListener.builder(new File(NerUtil.DATA_HOME + "/ui/log.bin")).plotLosses(1).trainEvaluationMetrics("nerOutput", 0, Evaluation.Metric.ACCURACY, Evaluation.Metric.F1)
      .updateRatios(1).trainEvalFrequency(10).build();
  
  sd.setListeners(uiListener, new ScoreListener(10), checkpointListener);

  UIServer.getInstance();
  
  BertTrainIterator trainDataIter = new BertTrainIterator();
  trainDataIter.setTokenProvider(new LabeledTokenProvider());

  History history = sd.fit().train(trainDataIter, nEpochs).exec();
  List<Double> acc = history.trainingEval(Metric.ACCURACY);
  System.out.println("Accuracy: " + acc);

///////////////////////////////////////////////////////////////////////////////////////////////////////////
Load model...
java.lang.RuntimeException: org.nd4j.shade.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "empty" (class org.nd4j.linalg.primitives.Counter), not marked as ignorable (0 known properties: ])
at [Source: (String)"{"updater":{"@Class":"org.nd4j.linalg.learning.config.Adam","learningRate":0.001,"beta1":0.9,"beta2":0.999,"epsilon":1.0E-8},"regularization":[{"@Class":"org.nd4j.linalg.learning.regularization.L2Regularization","l2":{"@Class":"org.nd4j.linalg.schedule.FixedSchedule","value":1.0E-4}}],"minimize":true,"dataSetFeatureMapping":["bertOutput"],"dataSetLabelMapping":["label"],"dataSetFeatureMaskMapping":null,"dataSetLabelMaskMapping":null,"lossVariables":null,"iterationCount":5,"epochCount":0,"trainEv"[truncated 569 chars]; line: 1, column: 695] (through reference chain: org.nd4j.autodiff.samediff.TrainingConfig["trainEvaluations"]->java.util.LinkedHashMap["nerOutput"]->java.util.ArrayList[0]->org.nd4j.evaluation.classification.Evaluation["truePositives"]->org.nd4j.linalg.primitives.Counter["empty"])
at org.nd4j.autodiff.samediff.TrainingConfig.fromJson(TrainingConfig.java:556)
at org.nd4j.autodiff.samediff.SameDiff.fromFlatBuffers(SameDiff.java:6060)
at org.nd4j.autodiff.samediff.SameDiff.fromFlatFile(SameDiff.java:5841)
at org.nd4j.autodiff.samediff.SameDiff.load(SameDiff.java:5732)
at org.nd4j.autodiff.listeners.checkpoint.CheckpointListener.loadCheckpoint(CheckpointListener.java:437)
at org.nd4j.autodiff.listeners.checkpoint.CheckpointListener.loadLastCheckpoint(CheckpointListener.java:447)
at com.pcng.ner.bert.NerBertTraining.main(NerBertTraining.java:33)
Caused by: org.nd4j.shade.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "empty" (class org.nd4j.linalg.primitives.Counter), not marked as ignorable (0 known properties: ])
at [Source: (String)"{"updater":{"@Class":"org.nd4j.linalg.learning.config.Adam","learningRate":0.001,"beta1":0.9,"beta2":0.999,"epsilon":1.0E-8},"regularization":[{"@Class":"org.nd4j.linalg.learning.regularization.L2Regularization","l2":{"@Class":"org.nd4j.linalg.schedule.FixedSchedule","value":1.0E-4}}],"minimize":true,"dataSetFeatureMapping":["bertOutput"],"dataSetLabelMapping":["label"],"dataSetFeatureMaskMapping":null,"dataSetLabelMaskMapping":null,"lossVariables":null,"iterationCount":5,"epochCount":0,"trainEv"[truncated 569 chars]; line: 1, column: 695] (through reference chain: org.nd4j.autodiff.samediff.TrainingConfig["trainEvaluations"]->java.util.LinkedHashMap["nerOutput"]->java.util.ArrayList[0]->org.nd4j.evaluation.classification.Evaluation["truePositives"]->org.nd4j.linalg.primitives.Counter["empty"])
at org.nd4j.shade.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61)
at org.nd4j.shade.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:823)
at org.nd4j.shade.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1153)
at org.nd4j.shade.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1589)
at org.nd4j.shade.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1567)
at org.nd4j.shade.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:294)
at org.nd4j.shade.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
at org.nd4j.shade.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
at org.nd4j.shade.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)
at org.nd4j.shade.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:189)
at org.nd4j.shade.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:161)
at org.nd4j.shade.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:130)
at org.nd4j.shade.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:97)
at org.nd4j.shade.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:254)
at org.nd4j.shade.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:288)
at org.nd4j.shade.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:245)
at org.nd4j.shade.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:27)
at org.nd4j.shade.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
at org.nd4j.shade.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
at org.nd4j.shade.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
at org.nd4j.shade.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
at org.nd4j.shade.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)
at org.nd4j.shade.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
at org.nd4j.shade.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4014)
at org.nd4j.shade.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3005)
at org.nd4j.autodiff.samediff.TrainingConfig.fromJson(TrainingConfig.java:554)
... 6 more

@AlexDBlack

This comment has been minimized.

Copy link
Contributor

@AlexDBlack AlexDBlack commented Nov 30, 2019

What version of DL4J/ND4J was used to create the model file?

@SidneyLann

This comment has been minimized.

Copy link
Author

@SidneyLann SidneyLann commented Nov 30, 2019

both are beta5

@AlexDBlack AlexDBlack self-assigned this Nov 30, 2019
AlexDBlack added a commit to KonduitAI/deeplearning4j that referenced this issue Nov 30, 2019
Signed-off-by: AlexDBlack <blacka101@gmail.com>
AlexDBlack added a commit to KonduitAI/deeplearning4j that referenced this issue Nov 30, 2019
Signed-off-by: AlexDBlack <blacka101@gmail.com>
@AlexDBlack

This comment has been minimized.

Copy link
Contributor

@AlexDBlack AlexDBlack commented Nov 30, 2019

Thanks for reporting
Fixed here: KonduitAI#93
It'll be merged back to eclipse repo (and hence will be available on snapshots) some time next week.
It'll also be in the next release of course (which we are working towards now).

@SidneyLann

This comment has been minimized.

Copy link
Author

@SidneyLann SidneyLann commented Nov 30, 2019

Thank you

@AlexDBlack

This comment has been minimized.

Copy link
Contributor

@AlexDBlack AlexDBlack commented Nov 30, 2019

I should also note it's on the evaluation objects in the training config, you should still be able to serialize/deserialize without IEvaluation instances

@SidneyLann

This comment has been minimized.

Copy link
Author

@SidneyLann SidneyLann commented Nov 30, 2019

ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.