C sharp/ADBC client: Missing information due to misunderstanding of providerType

[qifanzhang-ms]

What feature or improvement would you like to see?

SchemaTableColumn.ProviderType specifies the provider-specific data type of the column, so the following code:
arrow-adbc\csharp\src\Client\SchemaConverter.cs line 64:
row[SchemaTableColumn.ProviderType] = SchemaConverter.GetArrowTypeBasedOnRequestedBehavior(f.DataType, structBehavior);
should be changed to:
row[SchemaTableColumn.ProviderType] = f.DataType;.

Otherwise, the original service data type information (whether it is struct) of connectors such as GBQ will be lost. I noticed that you wrote SchemaConverter.GetArrowTypeBasedOnRequestedBehavior() to handle the conversion from struct to string. I understand that data type conversion is a normal behavior, but it should only modify SchemaTableColumn.DataType, not SchemaTableColumn.ProviderType. I don't know if my understanding is more reasonable, so I raised this issue to discuss with you.

[CurtHagenlocher]

Whether it's this particular column in the schema table or a new (custom) one, I think there ought to be a way to get the original type. What do you think, @davidhcoe?

[davidhcoe]

I don’t necessarily disagree, but I think any changes of this sort would need to be thoroughly tested, especially with the VerifyTypesAndValues calls. Part of the challenge is that even though it’s stored a certain way, it’s not retrieved the same way, even for ODBC. It doesn’t diminish that some caller downstream wants the original information, but I thought that’s what the XDBC_ information was for in the column metadata.

[qifanzhang-ms]

I don't quite understand how we plan to get the original data type in the client level, how do I find the XDBC_ information in the column metadata in the client?

Another point is that the bignumeric data type in GBQ is already lost at the adbc level, let alone the client level.

[CurtHagenlocher]

Another point is that the bignumeric data type in GBQ is already lost at the adbc level, let alone the client level.

Doesn't this come back as a decimal256?

I thought that’s what the XDBC_ information was for in the column metadata.

The column metadata is for when you use the schema function. @qifanzhang-ms, are you interested in table schema or the schema of the result set for a SQL query? I think it's the latter case where we throw the information away and where we could trivially add at least something back by modifying SchemaConverter.cs to add an additional column to the schema table.

[qifanzhang-ms]

Thank you very much, adding an additional column to the schema table can directly solve this problem