Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes LISTEN to quote channel name #466

Merged
merged 4 commits into from Mar 26, 2024

Conversation

NeedleInAJayStack
Copy link
Contributor

@NeedleInAJayStack NeedleInAJayStack commented Mar 18, 2024

This adjusts the SQL LISTEN statement called by PostgresConnection.addListener to quote the channel name.

For example, previously addListener("abc") would run LISTEN abc; and it will now run LISTEN "abc";

As far as I can tell, this only matters when the channel name is a reserved Postgres keyword, like default.

@NeedleInAJayStack
Copy link
Contributor Author

NeedleInAJayStack commented Mar 18, 2024

This behavior appears to have been introduced in v1.18.0. Interestingly, by watching SQL logs it appears that version also changed addListener to call LISTEN automatically, so some of the notification tests are calling LISTEN twice.

Copy link
Collaborator

@fabianfett fabianfett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch! See comments!

@@ -594,7 +594,7 @@ final class PostgresChannelHandler: ChannelDuplexHandler {
private func makeStartListeningQuery(channel: String, context: ChannelHandlerContext) -> PSQLTask {
let promise = context.eventLoop.makePromise(of: PSQLRowStream.self)
let query = ExtendedQueryContext(
query: PostgresQuery(unsafeSQL: "LISTEN \(channel);"),
query: PostgresQuery(unsafeSQL: "LISTEN \"\(channel)\";"),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
query: PostgresQuery(unsafeSQL: "LISTEN \"\(channel)\";"),
query: PostgresQuery(unsafeSQL: #"LISTEN "\#(channel)";"),

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks!

@@ -642,7 +642,7 @@ final class PostgresChannelHandler: ChannelDuplexHandler {
private func makeUnlistenQuery(channel: String, context: ChannelHandlerContext) -> PSQLTask {
let promise = context.eventLoop.makePromise(of: PSQLRowStream.self)
let query = ExtendedQueryContext(
query: PostgresQuery(unsafeSQL: "UNLISTEN \(channel);"),
query: PostgresQuery(unsafeSQL: "UNLISTEN \"\(channel)\";"),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
query: PostgresQuery(unsafeSQL: "UNLISTEN \"\(channel)\";"),
query: PostgresQuery(unsafeSQL: #"UNLISTEN "\#(channel)";"#),

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. Done

Comment on lines 231 to 246
func testNotificationWithKeywordChannelName() {
var conn: PostgresConnection?
XCTAssertNoThrow(conn = try PostgresConnection.test(on: eventLoop).wait())
defer { XCTAssertNoThrow( try conn?.close().wait() ) }

var receivedNotifications: [PostgresMessage.NotificationResponse] = []
conn?.addListener(channel: "default") { context, notification in
receivedNotifications.append(notification)
}
XCTAssertNoThrow(_ = try conn?.simpleQuery("LISTEN \"default\"").wait())
XCTAssertNoThrow(_ = try conn?.simpleQuery("NOTIFY \"default\"").wait())
XCTAssertNoThrow(_ = try conn?.simpleQuery("SELECT 1").wait())
XCTAssertEqual(receivedNotifications.count, 1)
XCTAssertEqual(receivedNotifications.first?.channel, "default")
XCTAssertEqual(receivedNotifications.first?.payload, "")
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this test interact with the code that you changed? Maybe we want to change AsyncPostgresConnectionTests.testListenAndNotify to something like this:

    func testListenAndNotify() async throws {
        let channelNames = [
            "foo",
            "default"
        ]

        let eventLoopGroup = MultiThreadedEventLoopGroup(numberOfThreads: 1)
        defer { XCTAssertNoThrow(try eventLoopGroup.syncShutdownGracefully()) }
        let eventLoop = eventLoopGroup.next()

        for channelName in channelNames {
            try await self.withTestConnection(on: eventLoop) { connection in
                let stream = try await connection.listen(channelName)
                var iterator = stream.makeAsyncIterator()

                try await self.withTestConnection(on: eventLoop) { other in
                    try await other.query(#"NOTIFY "\#(unescaped: channelName)", 'bar';"#, logger: .psqlTest)

                    try await other.query(#"NOTIFY "\#(unescaped: channelName)", 'foo';"#, logger: .psqlTest)
                }

                let first = try await iterator.next()
                XCTAssertEqual(first?.payload, "bar")

                let second = try await iterator.next()
                XCTAssertEqual(second?.payload, "foo")
            }
        }
    }

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My change was related to the quoting that occurs in the LISTEN SQL query caused by addListener. So before the changes, line 237 of this new test would fail on the PostgreSQL side when running LISTEN default; with the error:

ERROR: syntax error at or near "default" at character 8
STATEMENT: LISTEN default;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm totally happy to combine it into testListenAndNotify though if you prefer it that way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I've made your recommended changes to testListenAndNotify and removed the testNotificationWithKeywordChannelName test

@fabianfett
Copy link
Collaborator

fabianfett commented Mar 18, 2024

This behavior appears to have been introduced in v1.18.0. Interestingly, by watching SQL logs it appears that version also changed addListener to call LISTEN automatically, so some of the notification tests are calling LISTEN twice.

@NeedleInAJayStack Does this mean you actually used the old LISTEN API?

@NeedleInAJayStack
Copy link
Contributor Author

Does this mean you actually used the old LISTEN API?

Yeah, we were using the old LISTEN API. However, my comment was more related to these tests, and specifically that the lines with conn?.simpleQuery("LISTEN example").wait() are technically no longer needed since addListener now calls LISTEN example;. When watching the SQL logs I could see two LISTEN example queries in a row.

@NeedleInAJayStack
Copy link
Contributor Author

I'm able to recreate the TinyFastSequenceTests.testReserveCapacityIsForwarded CI failures in a swift:5.10-jammy container locally:

/repo/Tests/ConnectionPoolModuleTests/TinyFastSequenceTests.swift:37: error: TinyFastSequenceTests.testReserveCapacityIsForwarded : XCTAssertEqual failed: ("9") is not equal to ("8") - 
/repo/Tests/ConnectionPoolModuleTests/TinyFastSequenceTests.swift:46: error: TinyFastSequenceTests.testReserveCapacityIsForwarded : XCTAssertEqual failed: ("9") is not equal to ("8") -
/repo/Tests/ConnectionPoolModuleTests/TinyFastSequenceTests.swift:53: error: TinyFastSequenceTests.testReserveCapacityIsForwarded : XCTAssertEqual failed: ("9") is not equal to ("8") -

I'm able to recreate these errors on main, and going back to v1.19.0 where these tests were introduced. I'm not sure why the CI passed on previous MRs though. Do you guys have any idea what might be different now compared with those other MRs?

@NeedleInAJayStack NeedleInAJayStack force-pushed the fix/listenWithQuotes branch 2 times, most recently from f37c348 to 30abfd3 Compare March 19, 2024 19:39
Copy link

codecov bot commented Mar 26, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 61.73%. Comparing base (8f8724e) to head (12c5d5d).

Additional details and impacted files
@@            Coverage Diff            @@
##           main     #466       +/-   ##
=========================================
+ Coverage      0   61.73%   +61.73%     
=========================================
  Files         0      125      +125     
  Lines         0    10024    +10024     
=========================================
+ Hits          0     6188     +6188     
- Misses        0     3836     +3836     
Files Coverage Δ
...urces/PostgresNIO/New/PostgresChannelHandler.swift 84.31% <100.00%> (ø)

... and 124 files with indirect coverage changes

@fabianfett fabianfett merged commit 35587e9 into vapor:main Mar 26, 2024
11 of 12 checks passed
@fabianfett fabianfett added the semver-patch No public API change. label Mar 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver-patch No public API change.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants