Wrong field name in processing era dataframe?

[sciaba]

In

CMSSpark/src/python/CMSSpark/schemas.py

Line 32 in 7f72d51

StructField("processing_version", StringType(), True),

we have processing_version, but in

CMSSpark/src/python/CMSSpark/schemas.py

Line 23 in 7f72d51

PROCESSING_ERA_NAME NOT NULL VARCHAR2(120)

we have PROCESSING_ERA_NAME. It seems weird to me that the dataframe for the processing eras does not have the name, in fact. Is this just a typo?

[vkuznet]

Andrea, we should follow to whatever DBS schema is. And, what you found seems a typo, we should have StructField("processing_era_name", StringType(), True) instead of StructField("processing_version", StringType(), True) Feel free to add PR to this issue. V

…

On 0, Andrea Sciaba ***@***.***> wrote: In https://github.com/dmwm/CMSSpark/blob/master/src/python/CMSSpark/schemas.py#L32 we have processing_version, but in https://github.com/dmwm/CMSSpark/blob/7f72d51f1a9c58aa28483483cfbbd412264425e1/src/python/CMSSpark/schemas.py#L23 we have PROCESSING_ERA_NAME. It seems weird to me that the dataframe for the processing eras does not have the name, in fact. Is this just a typo? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #29

[sciaba]

The only thing is that I checked and that field seems to have only numeric values (even if in principle they are strings), so it is more compatible with a version than with a name....

[vkuznet]

I'm adding Yuyi to comment on this.

@yuyiguo , Yuyi could you please let us know what are the attributes of processing era table in DBS. Do we have PROCESSING_ERA_NAME in there or not and where processing version is stored.

[yuyiguo]

@vkuznet @sciaba
Hi Valentin, Andrea,
Here is what in DBS schema:
PROCESSING_ERA_ID(NUMBER)
PROCESSING_VERSION(NUMBER)
CREATION_DATE(NUMBER)
**CREATE_BY(VARCHAR2)
DESCRIPTION(VARCHAR2)

[vkuznet]

This is fixed in master head in this commit 4c7515b
Closing the ticket.