PutContexts failed: Given node already exists

[calvinleungyk]

Hi team, I ran a simple pipeline on JupyterLab that consists of InteractiveContext, ExampleGen, StatisticsGen, SchemaGen, ExampleValidator and Trainer but ml-metadata gRPC server keeps outputting this warning:

2020-07-31 05:48:17.165649: W ml_metadata/metadata_store/metadata_store_service_impl.cc:403] PutContexts failed: Given node already exists: type_id: 2
name: "interactive-2020-07-31T05_00_24.026542"
properties {
  key: "pipeline_name"
  value {
    string_value: "interactive-2020-07-31T05_00_24.026542"
  }
}
Internal: mysql_query failed: errno: 1062, error: Duplicate entry '2-interactive-2020-07-31T05_00_24.026542' for key 'type_id'

There are 5 components and this warning appears 4 times in a row and each time the name is identical.

My understanding is pipeline_name is used in both Artifacts and Contexts and it should allow update on a Context with existing name. if name already exists, it would be an update, but I'm not sure why put_contexts is repeatedly called. Has anyone seen this behavior / is this behavior expected or is something wrong?

Thank you.

[hughmiao]

hi @calvinleungyk , this is WAI. in mlmd, the context {name, type} is unique [1]. In this case, this context has been created five times. First time it will succeed, the rest of it, will return an error. In the tfx runtime, this is ignored and log errors to the users, and continue.

[1] https://github.com/tensorflow/tfx/blob/deb618730dc7675c8e9dc75e03b8ca795d49653d/tfx/orchestration/metadata.py#L1021

[calvinleungyk]

Thanks for the response @hughmiao. Is it something I've done in my pipeline that it's created 5 times, or is it something done in a default way in TFX? I don't remember doing any creation other than the one InteractiveContext

[hughmiao]

I believe this is handled by tfx and the interactive context implementation. Let me cc some tfx folks to take a look. @ruoyu90 @charlesccychen

[ruoyu90]

@calvinleungyk this is expected. Basically every node you run will try to register the pipeline context and reuse the one if exists. This is the reason you constantly see four warnings.

[hughmiao]

thanks @ruoyu90 to confirm.

[calvinleungyk]

Thanks @hughmiao and @ruoyu90 !