Close Sftp Session before exception is thrown ?

[corentin-pasquier]

Currently using Spring Integration 5.5.9 but the code looks the same on the main branch.

Describe the bug

Our application is creating sftp connections that stay opened and are never released to the connection pool.
After some time the pool is empty and we get an error "Timed out while waiting to acquire a pool entry".

For instance, this a a view of the sftp connection for the last 2 days :

After some digging I found that ths sftp connection is never closed when an exception is thrown at this moment :
https://github.com/spring-projects/spring-integration/blob/main/spring-integration-file/src/main/java/org/springframework/integration/file/remote/gateway/AbstractRemoteFileOutboundGateway.java#L667

To Reproduce

First you need a sftp server.
Then you need a SessionFactory (this is our configuration) :

	<bean id="sftpSessionFactory"
		class="org.springframework.integration.file.remote.session.CachingSessionFactory">
		<constructor-arg ref="defaultSftpSessionFactory" />
		<property name="poolSize" value="${application.sftp.pool-size}"/>
        	<property name="sessionWaitTimeout" value="${application.sftp.session-timeout}"/>
        	<property name="testSession" value="true"/>
	</bean>

	<bean id="defaultSftpSessionFactory" 
		class="org.springframework.integration.sftp.session.DefaultSftpSessionFactory">
		<property name="host" value="${application.sftp.host}" />
		<property name="port" value="${application.sftp.port}" />
		<property name="user" value="${application.sftp.user}" />
		<property name="password" value="${application.sftp.password}" />
		<property name="allowUnknownKeys" value="true" />
	</bean>

And a sftp-outbound-gateway with a get command and a -stream option :

		<int-sftp:outbound-gateway id="getFileFromSftp"
			session-factory="sftpSessionFactory"
			command="get"
			command-options="-stream"
			reply-timeout="15000"
			expression="headers['pdfFileName']">
		</int-sftp:outbound-gateway>

When you try to get a file that doens't exist, the session stays opened.

Expected behavior

Close the sftp connection before throwing the ErrorMessage so it doesn't stay opened.

I was thinking something like this (maybe adding a non-null verification or other stuff would be better) :

			session = this.remoteFileTemplate.getSessionFactory().getSession();
			try {
				payload = session.readRaw(remoteFilePath);
				return getMessageBuilderFactory()
						.withPayload(payload)
						.setHeader(FileHeaders.REMOTE_DIRECTORY, remoteDir)
						.setHeader(FileHeaders.REMOTE_FILE, remoteFilename)
						.setHeader(FileHeaders.REMOTE_HOST_PORT, session.getHostPort())
						.setHeader(IntegrationMessageHeaderAccessor.CLOSEABLE_RESOURCE, session);
			}
			catch (IOException e) {
				session.close();
				throw new MessageHandlingException(requestMessage,
						"Error handling message in the [" + this
								+ "]. Failed to get the remote file [" + remoteFilePath + "] as a stream", e);
			}
		}

Sample

https://github.com/corentin-pasquier/sample-issue-sftp-connection

You just need a sftp server in local.
You can run this docker command to get one :

docker run -p 22:22 -d atmoz/sftp foo:pass:::upload

Other information

I didn't check if this phenomenom happens with other commands, but it would be interesting to check.

[artembilan]

Confirmed.
That Option.STREAM variant cannot be used for regular this.remoteFileTemplate.execute(session ->) since we must not close the session in the end of the operation, but really defer to that .setHeader(IntegrationMessageHeaderAccessor.CLOSEABLE_RESOURCE, session);.
However, in case error, we really have to close it. Something similar to what you are suggestion.

Feel free to contribute the fix: https://github.com/spring-projects/spring-integration/blob/main/CONTRIBUTING.adoc.

In our tests we use Apache Mina and its SshServer. So, we don't need any external SFTP server for testing.
What you are observing probably can be covered in our SftpServerOutboundTests.

Thank you!