Fix "Constant Folding" info in "Query Processing Architecture Guide" #3698

srutzky · 2019-11-26T19:24:21Z

This PR fixes #3696 .

There are three mistakes in the Constant Folding section:

There is a "note" that states:

If the output type of the folding process is a large object type (text, image, nvarchar(max), varchar(max), or varbinary(max)), then SQL Server does not fold the expression.

That list of datatypes is missing both NTEXT and XML.
Under "Nonfoldable Expressions", it lists:
- User-defined functions (both Transact-SQL and CLR)
That was true until SQL Server 2012. Starting in that version (and still true as of SQL Server 2017), the following two items are foldable if they do not do any data access:
- Deterministic scalar-valued CLR user-defined functions
- Deterministic methods of CLR user-defined types
Please see Behavior Changes in SQL Server 2012: Constant Folding for CLR User-Defined Functions and Methods
The list of items under "Foldable Expressions" is missing the two SQLCLR items noted directly above.

Before executing any of the following tests, be sure to enable "Include Actual Execution Plan" (Control-M). After executing each test, just "View the XML Execution Plan".

The first two queries confirm expected behavior and indicate what it looks like when an expression is not folded.

SELECT TOP 1 CONVERT(NVARCHAR(50), 'bob') AS [Folded] FROM master.dbo.spt_values;
GO
/*
                <DefinedValues>
                  <DefinedValue>
                    <ColumnReference Column="Expr1004" />
                    <ScalarOperator ScalarString="N'bob'">
                      <Const ConstValue="N'bob'" />
                    </ScalarOperator>
                  </DefinedValue>
                </DefinedValues>
*/


SELECT TOP 1 CONVERT(NVARCHAR(MAX), 'bob') AS [NOT Folded-NVCmax] FROM master.dbo.spt_values;
GO
/*
 <DefinedValues>
   <DefinedValue>
     <ColumnReference Column="Expr1004" />
     <ScalarOperator ScalarString="CONVERT(nvarchar(max),'bob',0)">
       <Identifier>
         <ColumnReference Column="ConstExpr1005">
           <ScalarOperator>
             <Convert DataType="nvarchar(max)" Length="2147483647" Style="0" Implicit="false">
               <ScalarOperator>
                 <Const ConstValue="'bob'" />
               </ScalarOperator>
             </Convert>
           </ScalarOperator>
         </ColumnReference>
       </Identifier>
     </ScalarOperator>
   </DefinedValue>
 </DefinedValues>
*/

The next two queries test the other LOB types: NTEXT and XML.

-- Current DB's default collation cannot be Supplementary Character-Aware (collation
-- name has either "_SC", or "_140_" but not "_BIN*") else this will receive an error.
GO
SELECT TOP 1 CONVERT(NTEXT, 'bob') AS [NOT Folded-NTEXT] FROM master.dbo.spt_values;
GO
/*
                <DefinedValues>
                  <DefinedValue>
                    <ColumnReference Column="Expr1004" />
                    <ScalarOperator ScalarString="CONVERT(ntext,'bob',0)">
                      <Identifier>
                        <ColumnReference Column="ConstExpr1005">
                          <ScalarOperator>
                            <Convert DataType="ntext" Style="0" Implicit="false">
                              <ScalarOperator>
                                <Const ConstValue="'bob'" />
                              </ScalarOperator>
                            </Convert>
                          </ScalarOperator>
                        </ColumnReference>
                      </Identifier>
                    </ScalarOperator>
                  </DefinedValue>
                </DefinedValues>
*/


SELECT TOP 1 CONVERT(XML, 'bob') AS [NOT Folded-XML] FROM master.dbo.spt_values;
GO
/*
                <DefinedValues>
                  <DefinedValue>
                    <ColumnReference Column="Expr1004" />
                    <ScalarOperator ScalarString="CONVERT(xml,'bob',0)">
                      <Identifier>
                        <ColumnReference Column="ConstExpr1005">
                          <ScalarOperator>
                            <Convert DataType="xml" Style="0" Implicit="false">
                              <ScalarOperator>
                                <Const ConstValue="'bob'" />
                              </ScalarOperator>
                            </Convert>
                          </ScalarOperator>
                        </ColumnReference>
                      </Identifier>
                    </ScalarOperator>
                  </DefinedValue>
                </DefinedValues>
*/

For the following SQLCLR tests I use the SQL# library (that I wrote) because not only do I have it installed, but it is easy for anyone to download and install in order to duplicate these tests without needing to code or compile anything. Three of the four functions are available in the Free version, and the 4th function is a test function that isn't available in any public version (it's a rare instance of both being deterministic and doing data access).

The next query shows that SQLCLR UDFs can be folded.

SELECT SQL#.String_ToTitleCase4k(N'ŧ', N'') AS [Deterministic_NoDataAccess_NoLobTypes];
/*
            <RelOp ...>
              ...
              <ConstantScan>
                <Values>
                  <Row>
                    <ScalarOperator ScalarString="N'Ŧ'">
                      <Const ConstValue="N'Ŧ'" />
                    </ScalarOperator>
                  </Row>
                </Values>
              </ConstantScan>
            </RelOp>
*/

The next query shows that by changing the string datatypes to NVARCHAR(MAX), the same function (same .NET method, but different T-SQL wrapper object in order to change the datatypes) is no longer foldable.

SELECT SQL#.String_ToTitleCase(N'ŧ', N'') AS [Deterministic_NoDataAccess_LobType];
/*
  <RelOp ...>
    ...
    <ConstantScan>
      <Values>
        <Row>
          <ScalarOperator ScalarString="[SQL#_Full].[SQL#].[String_ToTitleCase](CONVERT_IMPLICIT(nvarchar(max),N'ŧ',0),N'')">
            <UserDefinedFunction FunctionName="[SQL#_Full].[SQL#].[String_ToTitleCase]" IsClrFunction="true">
              <ScalarOperator>
                <Convert DataType="nvarchar(max)" Length="2147483647" Style="0" Implicit="true">
                  <ScalarOperator>
                    <Const ConstValue="N'ŧ'" />
                  </ScalarOperator>
                </Convert>
              </ScalarOperator>
              <ScalarOperator>
                <Const ConstValue="N''" />
              </ScalarOperator>
              <CLRFunction Assembly="SQL#" Class="STRING" Method="ToTitleCase" />
            </UserDefinedFunction>
          </ScalarOperator>
        </Row>
      </Values>
    </ConstantScan>
  </RelOp>
*/

The next query shows that by not being marked as IsDeterministic=true, the UDF is not foldable.

SELECT SQL#.Math_RandomRange(NULL, 11, 200) AS [NonDeterministic_NoDataAccess_NoLobTypes];
/*
  <DefinedValues>
    <DefinedValue>
      <ColumnReference Column="Expr1000" />
      <ScalarOperator ScalarString="[SQL#_Full].[SQL#].[Math_RandomRange](NULL,(11),(200))">
        <UserDefinedFunction FunctionName="[SQL#_Full].[SQL#].[Math_RandomRange]" IsClrFunction="true">
          <ScalarOperator>
            <Const ConstValue="NULL" />
          </ScalarOperator>
          <ScalarOperator>
            <Const ConstValue="(11)" />
          </ScalarOperator>
          <ScalarOperator>
            <Const ConstValue="(200)" />
          </ScalarOperator>
          <CLRFunction Assembly="SQL#" Class="MATH" Method="RandomRange" />
        </UserDefinedFunction>
      </ScalarOperator>
    </DefinedValue>
  </DefinedValues>
*/

The next query shows that even with IsDeterministic=true and no LOB types, doing data access makes the UDF non-foldable.
_{Please note that TEST_GetCurrentServerName is not available in the Free version, or any version, of SQL#.}

SELECT SQL#.TEST_GetCurrentServerName() AS [Deterministic_UserDataAccess_NoLobTypes];
/*
 <DefinedValues>
   <DefinedValue>
     <ColumnReference Column="Expr1000" />
     <ScalarOperator ScalarString="[SQL#_Full].[SQL#].[TEST_GetCurrentServerName]()">
       <UserDefinedFunction FunctionName="[SQL#_Full].[SQL#].[TEST_GetCurrentServerName]" IsClrFunction="true">
         <CLRFunction Assembly="SQL#_2" Class="TEST" Method="GetCurrentServerName" />
       </UserDefinedFunction>
     </ScalarOperator>
   </DefinedValue>
 </DefinedValues>
*/

Take care,
Solomon...
https://SqlQuantumLift.com/
https://SqlQuantumLeap.com/
https://SQLsharp.com/

1. Add datatypes NTEXT and XML to note regarding which output datatypes will prevent constant folding. 2. Clarify that SQLCLR UDFs and UDT methods, as of SQL Server 2012, _can_ be foldable, but only if they are marked as `IsDeterministic=true` and do not do any data access. Full details, including example code to validate these changes, will be in the PR.

PRMerger12 · 2019-11-26T19:24:36Z

@srutzky : Thanks for your contribution! The author(s) have been notified to review your proposed change.

rothja · 2019-12-20T18:55:53Z

@srutzky Thanks for taking the time to fix the article, and sorry for the delayed response. I need some time to get the product team to review and approve these changes. You have done a great job of documenting your findings, so thank you for your patience.

pmasl · 2019-12-21T04:39:59Z

FYI I'll review this during the next few days. Thanks Pedro Lopes - Program Manager - Azure Data SQL Server Sent from Nine<http://www.9folders.com/>. Please excuse my brevity, spelling, and punctuation.

…

________________________________ From: Mike Ray <notifications@github.com> Sent: Friday, December 20, 2019 10:59 AM To: MicrosoftDocs/sql-docs Cc: Pedro Lopes; Review requested Subject: Re: [MicrosoftDocs/sql-docs] Fix "Constant Folding" info in "Query Processing Architecture Guide" (#3698) @MikeRayMSFT<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMikeRayMSFT&data=02%7C01%7Cpedro.lopes%40microsoft.com%7Cc81c0e80e5944d63181208d7857eb758%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637124651600267992&sdata=Cx778hrAq%2Bwop9Sv2cDXBBIHHa%2F7oVCiyWxy9BCpNC8%3D&reserved=0> requested your review on: #3698<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fsql-docs%2Fpull%2F3698&data=02%7C01%7Cpedro.lopes%40microsoft.com%7Cc81c0e80e5944d63181208d7857eb758%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637124651600267992&sdata=QKaDAfbytckZflWvrYN%2B2GukwPMaD2aaWLaml5z79J0%3D&reserved=0> Fix "Constant Folding" info in "Query Processing Architecture Guide". — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fsql-docs%2Fpull%2F3698%3Femail_source%3Dnotifications%26email_token%3DAE2PNQ4SRRNDNGVXFERUAELQZUIYNA5CNFSM4JR4W4UKYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOVT2JYJQ%23event-2901711910&data=02%7C01%7Cpedro.lopes%40microsoft.com%7Cc81c0e80e5944d63181208d7857eb758%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637124651600277995&sdata=%2FdTGZRX9JY7zCk5EP5dwOeVAR3lcxJpk8QG%2FHLn0Muo%3D&reserved=0>, or unsubscribe<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAE2PNQYIVIRFEZWYBG6GVN3QZUIYNANCNFSM4JR4W4UA&data=02%7C01%7Cpedro.lopes%40microsoft.com%7Cc81c0e80e5944d63181208d7857eb758%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637124651600277995&sdata=bY%2BzXxllkXQ%2FjYI6ZQBugWCmaklDSOpaGCKJFEU1jhg%3D&reserved=0>.

ktoliver · 2020-01-23T22:49:33Z

Hi @pmasl

Would you add a comment to show what action to take on this PR?

Merge the PR: #sign-off
Close the PR: #please-close
Wait on the PR: #hold-off

Thanks!

rothja · 2020-02-12T14:56:49Z

@pmasl Let me know if you had a chance to review this change. Thank you!

ktoliver · 2020-02-26T16:18:14Z

I'm seeing the updates in master, so we'll close the PR.

@srutzky Thanks!

srutzky · 2020-02-26T22:03:43Z

@ktoliver (and @rothja , @pmasl , and @MikeRayMSFT ): Yes, it looks like Pedro incorporated my suggested edits into a series of updates he made on this particular page. The commit containing my edits (made earlier today) is here:

909d1d2#diff-c7f5c629fe5c2e5e5339e2998eb14669

I am guessing that my commit for this PR was no longer easily mergable given the number of changes to this page since I submitted the PR.

PRMerger12 added the do-not-merge label Nov 26, 2019

PRMerger12 requested a review from rothja November 26, 2019 19:24

PRMerger12 assigned rothja Nov 26, 2019

PRMerger12 added sql/prod Change sent to author labels Nov 26, 2019

jborsecnik added the aq-pr-triaged tracking label for the PR review team label Nov 26, 2019

srutzky mentioned this pull request Dec 2, 2019

Constant Folding in Q.O.: CLR UDFs 𝑐𝑎𝑛, while NTEXT and XML output types 𝑤𝑜𝑛’𝑡 #3696

Closed

MikeRayMSFT requested review from pmasl and MikeRayMSFT and removed request for rothja December 20, 2019 18:59

MikeRayMSFT assigned MikeRayMSFT and unassigned rothja Dec 20, 2019

ktoliver closed this Feb 26, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix "Constant Folding" info in "Query Processing Architecture Guide" #3698

Fix "Constant Folding" info in "Query Processing Architecture Guide" #3698

srutzky commented Nov 26, 2019 •

edited

PRMerger12 commented Nov 26, 2019

rothja commented Dec 20, 2019

pmasl commented Dec 21, 2019 via email

ktoliver commented Jan 23, 2020

rothja commented Feb 12, 2020

ktoliver commented Feb 26, 2020

srutzky commented Feb 26, 2020

Fix "Constant Folding" info in "Query Processing Architecture Guide" #3698

Fix "Constant Folding" info in "Query Processing Architecture Guide" #3698

Conversation

srutzky commented Nov 26, 2019 • edited

PRMerger12 commented Nov 26, 2019

rothja commented Dec 20, 2019

pmasl commented Dec 21, 2019 via email

ktoliver commented Jan 23, 2020

rothja commented Feb 12, 2020

ktoliver commented Feb 26, 2020

srutzky commented Feb 26, 2020

srutzky commented Nov 26, 2019 •

edited