Skip to content

[AIRFLOW-55] Add support for HDFS remote logging#2119

Closed
dhuang wants to merge 1 commit intoapache:masterfrom
dhuang:AIRFLOW-55
Closed

[AIRFLOW-55] Add support for HDFS remote logging#2119
dhuang wants to merge 1 commit intoapache:masterfrom
dhuang:AIRFLOW-55

Conversation

@dhuang
Copy link
Copy Markdown
Contributor

@dhuang dhuang commented Mar 2, 2017

Implemented using WebHDFS hook because HDFS hook (using snakebite)
does not support a copyFromLocal.

Please accept this PR that addresses the following issues:

Testing Done:

  • Added unit tests to test the path parsing.
  • Was able to view logs in the UI that were pulled from HDFS.

@codecov-io
Copy link
Copy Markdown

codecov-io commented Apr 8, 2017

Codecov Report

❗ No coverage uploaded for pull request base (master@3c450fb). Click here to learn what that means.
The diff coverage is 16.66%.

Impacted file tree graph

@@            Coverage Diff            @@
##             master    #2119   +/-   ##
=========================================
  Coverage          ?   67.98%           
=========================================
  Files             ?      142           
  Lines             ?    11005           
  Branches          ?        0           
=========================================
  Hits              ?     7482           
  Misses            ?     3523           
  Partials          ?        0
Impacted Files Coverage Δ
airflow/configuration.py 83.73% <ø> (ø)
airflow/www/views.py 67.87% <0%> (ø)
airflow/utils/logging.py 23.3% <16.27%> (ø)
airflow/bin/cli.py 51.83% <50%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3c450fb...ab8dc56. Read the comment docs.

Implemented using WebHDFS hook because HDFS hook (using snakebite)
does not support a copyFromLocal.
@Fokko
Copy link
Copy Markdown
Contributor

Fokko commented Jan 22, 2018

Nice one, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants