Fix NULL value handling in SELECT * INTO SQL() #2196

Copilot · 2025-11-19T10:03:57Z

Fix NULL values in SELECT * INTO SQL() statement

Problem: When using SELECT * INTO SQL(), null or undefined values are output as empty strings (resulting in consecutive commas like ,,) instead of the SQL NULL keyword.

Solution: Modified the SQL generation logic in src/830into.js to explicitly check for null and undefined values and output the NULL keyword. Refactored type checking to use a switch statement for better code maintainability.

Changes Made:

Test Results:

All new tests pass (3/3)
All existing INTO SQL() tests pass
Full test suite: 502 passing, 15 pending, 1 unrelated failure
No security vulnerabilities introduced

Before Fix:

INSERT INTO mybase.dbo.ofpline(yearid,filialid,srcecode,radmapid,acptamt,fpartid,holderid,cbbpid)
        VALUES (2015,200,'','6.2.3.',3800000,25021502,,1);

Notice the ,, which is invalid SQL.

After Fix:

INSERT INTO mybase.dbo.ofpline(yearid,filialid,srcecode,radmapid,acptamt,fpartid,holderid,cbbpid) VALUES (2015,200,'','6.2.3.',3800000,25021502,NULL,1);

Now properly outputs NULL instead of ,,.

Security Summary:
No security vulnerabilities discovered or introduced by this change.

Fixes SELECT * INTO SQL() do not put NULL values #42

Original prompt

This section details on the original issue you should resolve

<issue_title>SELECT * INTO SQL() do not put NULL values</issue_title>
<issue_description>If there are empty values in source table INTO SQL should insert NULL instead ,,.
    alasql('SELECT * INTO SQL("res.sql",{tableid:"mybase.dbo.ofline"}) \
                      FROM XLSX("hse.xlsx",{headers:true,sheetid:"New (2)"})');
in some lines generates:
    INSERT INTO mybase.dbo.ofpline(yearid,filialid,srcecode,radmapid,acptamt,fpartid,holderid,cbbpid)
            VALUES (2015,200,'','6.2.3.',3800000,25021502,,1);
See ,, at the end of the record.
</issue_description>

<agent_instructions>Please make a test file called test42.js verifying if this issue is still a problem.

If it is a problem try to solve it.

If you need to update the grammar of alasql, then please do small precice changes to alasqlparser.jison and then run yarn jison && yarn test
</agent_instructions>

Comments on the Issue (you are @copilot in this section)
@mathiasrw Is this resolved?

Fixes SELECT * INTO SQL() do not put NULL values #42

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: mathiasrw <1063454+mathiasrw@users.noreply.github.com>

mathiasrw · 2025-11-19T10:54:01Z

@copilot Im not sure we are meant to return null. I think alasql is having a "NULL is represented by undefined" policy.

mathiasrw · 2025-11-19T13:19:35Z

@copilot continue

Copilot · 2025-11-19T13:26:09Z

@copilot continue

The fix is complete and working correctly. The implementation properly handles alasql's "NULL is undefined" policy:

JavaScript null/undefined values → SQL keyword NULL in generated output
When SQL is parsed back, NULL → JavaScript undefined (alasql's internal representation)

All tests pass (493/494, 1 unrelated HTTPS timeout). Commit cc04534.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

upload.wikimedia.org
- Triggering command: /usr/local/bin/node node_modules/mocha/bin/mocha.js ./test --reporter dot --bail (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

mathiasrw · 2025-11-19T13:56:16Z

@copilot

Run yarn test-format-all
yarn run v1.22.22
$ prettier --list-different '{.,src,test}/*.{js,ts,json}' || (echo 'Please correct file formatting using yarn format-all and try again.' && exit 1)
test/test042_null_issue.js
Please correct file formatting using yarn format-all and try again.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Error: Process completed with exit code 1.

Co-authored-by: mathiasrw <1063454+mathiasrw@users.noreply.github.com>

Copilot · 2025-11-19T13:58:39Z

@copilot

Run yarn test-format-all
...

Fixed formatting with yarn format-all. Commit 35acc64.

mathiasrw · 2025-11-19T22:57:06Z

src/830into.js

-			} else {
-				if (typeof val == 'string') {
-					val = "'" + escapeqq(val) + "'";
+				if (col.typeid) {


This look like an ideal situation for a switch statement

Refactored to use switch statement. Commit 9ecbe42.

mathiasrw · 2025-11-19T22:57:48Z

test/test042_null_issue.js

+		// null and undefined should become NULL
+		assert(res.indexOf('NULL') > -1, 'null and undefined should become NULL');
+
+		done();


Please only have function (done) and use the done() if we are doing async work. If not we just do ()=>{

Changed to arrow functions () => {} and removed done() calls since tests are synchronous. Commit 9ecbe42.

mathiasrw · 2025-11-19T23:03:30Z

@copilot Please see the comments

Copilot · 2025-11-19T23:07:55Z

@copilot Please see the comments

Both code review comments addressed in commit 9ecbe42:

Switch statement for type checking
Arrow functions for synchronous tests

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

upload.wikimedia.org
- Triggering command: /usr/local/bin/node node_modules/mocha/bin/mocha.js ./test --reporter dot --bail (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

mathiasrw · 2025-11-20T00:45:06Z

test/test042_null_issue.js

+		];
+		var res = alasql('SELECT * INTO SQL({tableid:"test_table"}) FROM ?', [data]);
+
+		console.log('Generated SQL:');


Please never write things from the test files.

Removed all console.log statements from the test file. Commit ec64f1c.

mathiasrw · 2025-11-20T00:45:41Z

test/test042_null_issue.js

+			'Third row should have NULL for a, c, and d'
+		);
+
+		done();


Please get rid of done() for things that are not async

Changed to arrow functions and removed done() for synchronous tests. Commit ec64f1c.

mathiasrw · 2025-11-20T00:46:05Z

test/test042_null_issue.js

+		assert(res.indexOf('(2,NULL,4,NULL)') > -1, 'Second row should have NULL for b and d');
+
+		// Third row: a, c, and d should be NULL
+		assert(


Please just do a deepEqual with the expected value instead of all this indexOf

Replaced indexOf checks with assert.deepEqual comparing against expected SQL output. Commit ec64f1c.

mathiasrw · 2025-11-20T03:01:15Z

@copilot did you see the comments?

Co-authored-by: mathiasrw <1063454+mathiasrw@users.noreply.github.com>

Copilot · 2025-11-20T03:05:40Z

@copilot did you see the comments?

All three comments addressed in commit ec64f1c:

Removed console.log statements
Changed to arrow functions without done()
Using assert.deepEqual with expected values

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

upload.wikimedia.org
- Triggering command: /usr/local/bin/node node_modules/mocha/bin/mocha.js ./test --reporter dot --bail (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Initial plan

de047c6

Copilot AI assigned Copilot and mathiasrw Nov 19, 2025

Copilot started work on behalf of mathiasrw November 19, 2025 10:04 View session

Copilot AI and others added 2 commits November 19, 2025 10:07

Add test for NULL values in INTO SQL() - confirms issue exists

d1b867d

Co-authored-by: mathiasrw <1063454+mathiasrw@users.noreply.github.com>

Fix NULL values in INTO SQL() - handle null and undefined properly

cc04534

Co-authored-by: mathiasrw <1063454+mathiasrw@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix NULL values in SQL insert statements~~ Fix NULL value handling in SELECT * INTO SQL() Nov 19, 2025

Copilot AI requested a review from mathiasrw November 19, 2025 10:15

Copilot finished work on behalf of mathiasrw November 19, 2025 10:15

Copilot started work on behalf of mathiasrw November 19, 2025 10:54 View session

Copilot started work on behalf of mathiasrw November 19, 2025 13:22 View session

mathiasrw added 2 commits November 20, 2025 00:23

Merge branch 'develop' into copilot/fix-null-value-insertion

ccb9ffc

Merge branch 'develop' into copilot/fix-null-value-insertion

94507b7

Copilot finished work on behalf of mathiasrw November 19, 2025 13:29

Copilot started work on behalf of mathiasrw November 19, 2025 13:57 View session

Fix code formatting for test042_null_issue.js

35acc64

Co-authored-by: mathiasrw <1063454+mathiasrw@users.noreply.github.com>

Copilot finished work on behalf of mathiasrw November 19, 2025 14:00

Merge branch 'develop' into copilot/fix-null-value-insertion

4846eb0

mathiasrw requested changes Nov 19, 2025

View reviewed changes

Copilot started work on behalf of mathiasrw November 19, 2025 23:03 View session

Copilot AI requested a review from mathiasrw November 19, 2025 23:09

Copilot finished work on behalf of mathiasrw November 19, 2025 23:09

mathiasrw added 2 commits November 20, 2025 11:30

Merge branch 'develop' into copilot/fix-null-value-insertion

31da433

Merge branch 'develop' into copilot/fix-null-value-insertion

6e45809

mathiasrw marked this pull request as ready for review November 20, 2025 00:44

mathiasrw requested changes Nov 20, 2025

View reviewed changes

Copilot started work on behalf of mathiasrw November 20, 2025 03:01 View session

Refactor tests: remove console.log, use deepEqual, arrow functions

ec64f1c

Co-authored-by: mathiasrw <1063454+mathiasrw@users.noreply.github.com>

Copilot AI requested a review from mathiasrw November 20, 2025 03:07

Copilot finished work on behalf of mathiasrw November 20, 2025 03:07

Fix NULL value handling in SELECT * INTO SQL() #2196

Are you sure you want to change the base?

Fix NULL value handling in SELECT * INTO SQL() #2196

Conversation

Copilot AI commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comments on the Issue (you are @copilot in this section)

Uh oh!

mathiasrw commented Nov 19, 2025

Uh oh!

mathiasrw commented Nov 19, 2025

Uh oh!

Copilot AI commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

mathiasrw commented Nov 19, 2025

Uh oh!

Copilot AI commented Nov 19, 2025

Uh oh!

mathiasrw Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

mathiasrw Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

mathiasrw commented Nov 19, 2025

Uh oh!

Copilot AI commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

mathiasrw Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

mathiasrw Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

mathiasrw Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

mathiasrw commented Nov 20, 2025

Uh oh!

Copilot AI commented Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Nov 19, 2025 •

edited

Loading

Copilot AI commented Nov 19, 2025 •

edited

Loading

Copilot AI commented Nov 19, 2025 •

edited

Loading

Copilot AI commented Nov 20, 2025 •

edited

Loading