Skip to content

Fluss s3/oss filesystem jar shouldn't bundle hadoop-hdfs & hadoop-hdfs-client lib #837

@luoyuxia

Description

@luoyuxia

Search before asking

  • I searched in the issues and found nothing similar.

Description

Currently, module fluss-fs-hadoop will bundle hadoop-hdfs & hadoop-hdfs-client lib, it's needed for accessing hdfs filesystem. But fluss-fs-s3 and fluss-fluss-oss will also include fluss-fs-hadoop which cause to bundle hadoop-hdfs & hadoop-hdfs-client, but for oss and s3, hadoop-hdfs & hadoop-hdfs-client is not needed.

We will need to split the moudle fluss-fluss-hadoop into two moudles then, one moudle maybe called fluss-fs-hadoop-common won't bundle hadoop-hdfs & hadoop-hdfs-client, and
fluss-fs-s3 and fluss-fluss-oss will include fluss-fs-hadoop-common, another moudle, maybe called fluss-fs-hadoop will bundle hadoop-hdfs & hadoop-hdfs-client.

Willingness to contribute

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No fields configured for Task.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions