Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [connector-file-sftp] SFTPInputStream.close does not correctly trigger the closing of the file stream #6323

Closed
3 tasks done
panpan2019 opened this issue Feb 1, 2024 · 1 comment
Labels

Comments

@panpan2019
Copy link
Contributor

panpan2019 commented Feb 1, 2024

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

When closing SFTPInputStream after SFTP read/write completion, the SFTPInputStream.close method is not properly closing the file stream. This can lead to resource leaks.

SeaTunnel Version

2.3.2

SeaTunnel Config

env {
  execution.parallelism = 2
  job.mode = "BATCH"

}
source{
   SftpFile {
    path = "/home/dataos/test"
    host = "xxxx"
    port = 22
    user = xxx
    password = "****"
    file_format_type = "text"
    result_table_name = SftpFile_source_4
    schema = {
     fields {
      name = string
      age = int
      }
    }
    delimiter = ","
   }
  } 
  
 transform{
    Sql{
     source_table_name = "SftpFile_source_4"
    result_table_name = "Sql-transform-4"
    query = "select name,age, DES_ENCRYPT('1111111111',name)  des_name from SftpFile_source_4"
    }
  }
  
  sink {
    SftpFile{
      file_format_type="text",
      path = "/home/test1"
      host = "xxx"
      port = 22
      user = xxx
      password = "****"
      file_format_type = "text"
      field_delimiter=",",
      row_delimiter= "/t",
      sink_columns = ["name","age","des_name"]
      source_table_name="Sql-transform-4" 
    }
  }

Running Command

./bin/seatunnel.sh --config ./task/sftp2sftp

Error Exception

1.Terminal shows that the SFTP connection is not released.
2.Debugging reveals that the file stream is not closed when the SFTPInputStream.close method is triggered.

Zeta or Flink or Spark Version

No response

Java or Scala Version

jdk8

Screenshots

1706780364018
1706780493923
1.The initialization of SFTPInputStream.class does not pass the file stream to the parent class FSInputStream.class
2.the SFTPInputStream.close method only triggers the FSInputStream.close method, which is unable to close the file stream actually held by SFTPInputStream.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@panpan2019 panpan2019 added the bug label Feb 1, 2024
@panpan2019
Copy link
Contributor Author

i can fix

panpan2019 added a commit to panpan2019/seatunnel that referenced this issue Feb 2, 2024
hailin0 pushed a commit that referenced this issue Feb 18, 2024
…ectly trigger the closing of the file stream (#6323) (#6329)
@hailin0 hailin0 closed this as completed Feb 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants