Add DECIMAL Type Support to C# Language Extension

[monamaki]

Summary

This PR implements full support for SQL Server DECIMAL type in the C# language extension, enabling seamless conversion between SQL Server's 19-byte SQL_NUMERIC_STRUCT format and .NET's SqlDecimal type. The implementation introduces a new SqlNumericHelper utility class and simplifies existing code by leveraging SqlDecimal's built-in capabilities.

Why These Changes?

Problem: The C# language extension lacked proper support for SQL Server's DECIMAL types, which are critical for financial, scientific, and precision-sensitive applications. Without this support:

• Stored procedures and user-defined functions using DECIMAL parameters couldn't be called from C# extensions
• Result sets containing DECIMAL columns couldn't be properly processed
• OUTPUT parameters of type DECIMAL returned incorrect or corrupted values

Solution: Implement bidirectional conversion between SQL Server's native SQL_NUMERIC_STRUCT (ODBC 19-byte format) and .NET's SqlDecimal type, with proper handling of:

• Precision (1-38 digits) and scale (0-precision)
• NULL values using SqlDecimal.Null
• OUTPUT parameter sentinel detection (uninitialized structs with precision=0 used as a sentinel/marker value to indicate OUTPUT parameters)
• Signed values (positive/negative decimals)

What Changed?

1. SqlNumericHelper.cs

Created a comprehensive utility class for DECIMAL conversions with five core methods:

• ToSqlDecimal(SqlNumericStruct): Converts SQL 19-byte struct → SqlDecimal

  • Handles sign bit, precision, scale validation
  • Reconstructs 128-bit value from four 32-bit integers

• FromSqlDecimal(SqlDecimal, precision, scale): Converts SqlDecimal to SQL struct

  • Adjusts scale using SqlDecimal.AdjustScale() when needed
  • Uses SqlDecimal.Precision property (auto-updated by framework)
  • Validates precision bounds (1-38), throws for overflow
  • Properly handles NULL values (creates zero-initialized struct)

• ToSqlDecimalFromPointer(SqlNumericStruct*): Unsafe pointer version for OUTPUT parameters

  • Detects OUTPUT parameter sentinel (precision=0 → returns SqlDecimal.Null)
  • Centralizes OUTPUT parameter convention handling
  • Prevents duplicate sentinel detection logic across codebase

2. CSharpDecimalTests.cpp

Added comprehensive test coverage with 8 new decimal-specific tests:

Test Name	Purpose	Coverage
GetDecimalOutputParamTest	OUTPUT parameter handling	Sentinel detection, proper value assignment
DecimalPrecisionScaleTest	Precision/scale validation	All valid combinations (1-38 precision, 0-precision scale)
DecimalBoundaryValuesTest	Edge cases	Min/max values, zero, near-overflow scenarios
DecimalStructLayoutTest	Memory layout verification	19-byte ODBC struct correctness
GetDecimalInputColumnsTest	Input column processing	Batch decimal column data handling
GetDecimalResultColumnsTest	Output column generation	Result set decimal columns
DecimalColumnsWithNullsTest	NULL handling	Mixed NULL/non-NULL decimal columns
DecimalHighScaleTest	High precision scenarios	38-digit precision

Test Infrastructure Updates

3. CSharpTestExecutor.cs

Added managed test execution helper for decimal scenarios:

• ExecuteDecimalInputOutput: Tests input columns + OUTPUT parameters
• ExecuteDecimalResultSet: Tests decimal return columns

4. CSharpExtensionApiTests.h/cpp

Extended C++ test framework with decimal test declarations and utility functions.

5. CSharpInitParamTests.cpp

Added InitNumericParamTest for parameter initialization validation.

6. CSharpExecuteTests.cpp

Integrated decimal tests into main test suite execution.

10. Microsoft.SqlServer.CSharpExtension.csproj

Added SqlNumericHelper.cs to build configuration.

11. Microsoft.SqlServer.CSharpExtensionTest.csproj

Added CSharpTestExecutor.cs to test project.

Checklist

• Code builds successfully (Release configuration)
• All unit tests passing (68/68)
• New tests added for decimal support (8 comprehensive tests)
• Code follows existing coding standards
• Comments are concise and explain "why" not "what"

---

[Copilot]

Pull request overview

This PR adds end-to-end SQL Server DECIMAL/NUMERIC support to the .NET Core C# language extension by introducing an ODBC-compatible SQL_NUMERIC_STRUCT representation in managed code and wiring conversions to/from SqlDecimal, plus expanding the native/managed test coverage for decimal parameters and columns.

Changes:

• Introduces SqlNumericHelper with SQL_NUMERIC_STRUCT layout and conversion helpers to/from SqlDecimal.
• Updates parameter and dataset marshalling to support SqlDecimal/SQL_C_NUMERIC for input columns, output columns, and OUTPUT parameters.
• Adds new native and managed tests and updates test harness templates/instantiations for SQL_NUMERIC_STRUCT.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
language-extensions/dotnet-core-CSharp/test/src/native/CSharpInitParamTests.cpp	Adds InitParam template specialization for SQL_NUMERIC_STRUCT to pass precision/scale to InitParam.
language-extensions/dotnet-core-CSharp/test/src/native/CSharpExtensionApiTests.cpp	Adds InitializeColumns specialization for numeric columns to use precision rather than sizeof.
language-extensions/dotnet-core-CSharp/test/src/native/CSharpExecuteTests.cpp	Adds explicit template instantiation for executing numeric column tests.
language-extensions/dotnet-core-CSharp/test/src/native/CSharpDecimalTests.cpp	Adds a new native test suite covering decimal params, output params, and decimal columns.
language-extensions/dotnet-core-CSharp/test/src/managed/Microsoft.SqlServer.CSharpExtensionTest.csproj	Updates build output paths and adds Microsoft.Data.SqlClient dependency for managed tests.
language-extensions/dotnet-core-CSharp/test/src/managed/CSharpTestExecutor.cs	Adds managed executors to drive decimal OUTPUT parameter and precision-overflow scenarios.
language-extensions/dotnet-core-CSharp/test/include/CSharpExtensionApiTests.h	Adds max precision constant and a helper to build SQL_NUMERIC_STRUCT test values.
language-extensions/dotnet-core-CSharp/src/managed/utils/SqlNumericHelper.cs	New utility for SQL_NUMERIC_STRUCT layout plus conversion logic to/from SqlDecimal.
language-extensions/dotnet-core-CSharp/src/managed/utils/Sql.cs	Adds mapping and size metadata for NUMERIC/DECIMAL (SqlDecimal + SQL_NUMERIC_STRUCT).
language-extensions/dotnet-core-CSharp/src/managed/Microsoft.SqlServer.CSharpExtension.csproj	Adjusts output path defaults and adds Microsoft.Data.SqlClient package reference.
language-extensions/dotnet-core-CSharp/src/managed/CSharpParamContainer.cs	Adds NUMERIC param ingestion and output replacement via SqlDecimal + struct conversion.
language-extensions/dotnet-core-CSharp/src/managed/CSharpOutputDataSet.cs	Adds result column extraction path for NUMERIC/DECIMAL into SQL_NUMERIC_STRUCT[].
language-extensions/dotnet-core-CSharp/src/managed/CSharpInputDataSet.cs	Adds input column ingestion path for NUMERIC/DECIMAL into PrimitiveDataFrameColumn<SqlDecimal>.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[SicongLiu2000]

Review Summary

3 bugs found that should be fixed before merge, plus test coverage gaps and minor issues.

Bugs (must fix)

1. FromSqlDecimal(SqlDecimal.Null) crash — value.Precision/value.Scale accessed before IsNull guard (see inline comment on SqlNumericHelper.cs L225-226)
2. Precision overflow check false positives — AdjustScale inflates Precision property causing valid values to be rejected (see inline comment on SqlNumericHelper.cs L266)
3. Precision clamped to 38 but scale not reduced — produces DECIMAL(38,30) that rejects valid integer digits (see inline comment on CSharpOutputDataSet.cs L260-262)

Test coverage gaps

• No value round-trip test: GetDecimalOutputParamTest checks structural properties (precision range, sign ∈ {0,1}) but never checks that val[16] bytes encode the expected number. A byte-ordering or indexing bug in ToSqlDecimal/FromSqlDecimal would pass all existing tests.
• No all-NULL column test: ExtractNumericColumn would produce maxIntDigits=0, and if metadata DecimalDigits=0, precision becomes 0, clamped to 1 → DECIMAL(1,0) regardless of declared type.
• DecimalMixedPrecisionScaleInputTest only checks merged metadata (precision, scale), not actual output data values after scale normalization — an AdjustScale bug (truncating vs zero-padding) would go undetected.
• CSharpTestExecutorDecimalPrecisionOverflow is defined but never called from any C++ test (see inline comment).

Other

• README (language-extensions/dotnet-core-CSharp/README.md line 10) still lists only the original data types — SQL_C_NUMERIC/DECIMAL/NUMERIC should be added to the supported types list.

[monamaki]

Review Summary

3 bugs found that should be fixed before merge, plus test coverage gaps and minor issues.

Bugs (must fix)

1. FromSqlDecimal(SqlDecimal.Null) crash — value.Precision/value.Scale accessed before IsNull guard (see inline comment on SqlNumericHelper.cs L225-226)
2. Precision overflow check false positives — AdjustScale inflates Precision property causing valid values to be rejected (see inline comment on SqlNumericHelper.cs L266)
3. Precision clamped to 38 but scale not reduced — produces DECIMAL(38,30) that rejects valid integer digits (see inline comment on CSharpOutputDataSet.cs L260-262)

Test coverage gaps

• No value round-trip test: GetDecimalOutputParamTest checks structural properties (precision range, sign ∈ {0,1}) but never checks that val[16] bytes encode the expected number. A byte-ordering or indexing bug in ToSqlDecimal/FromSqlDecimal would pass all existing tests.
• No all-NULL column test: ExtractNumericColumn would produce maxIntDigits=0, and if metadata DecimalDigits=0, precision becomes 0, clamped to 1 → DECIMAL(1,0) regardless of declared type.
• DecimalMixedPrecisionScaleInputTest only checks merged metadata (precision, scale), not actual output data values after scale normalization — an AdjustScale bug (truncating vs zero-padding) would go undetected.
• CSharpTestExecutorDecimalPrecisionOverflow is defined but never called from any C++ test (see inline comment).

Other

• README (language-extensions/dotnet-core-CSharp/README.md line 10) still lists only the original data types — SQL_C_NUMERIC/DECIMAL/NUMERIC should be added to the supported types list.

Thanks Sicong for the thorough review of this PR.

Regarding test gaps:

1. Value round-trip:
   GetDecimalOutputParamTest indeed only checks structural properties. BUT our E2E tests in SQL Server engine (PassThroughDecimalColumns, DecimalInputOutputParameter, MaxPrecisionDecimalParameter, etc.) do full value round-trip through SQL Server with exact VerifyEquality comparison. A byte-ordering bug would fail those. I'll have it on the engine's PR internally.

2. All-NULL column: I have DecimalColumnsWithNullsTest which has mixed NULLs but not ALL NULLs. This is a real edge case worth a quick test.

3. Mixed metadata data verification: DecimalMixedPrecisionScaleInputTest checks metadata only. BUT — our E2E MixedPrecisionScaleDecimalColumns test in engine verifies actual data through SQL Server.

Will update the Readme.md too.

[SicongLiu2000]

All 23 review threads addressed. The precision-clamping bug (scale not reduced when precision exceeds 38) is fixed in the latest push. LGTM.