Skip to content
This repository has been archived by the owner on Nov 22, 2022. It is now read-only.

Support read file from http URL #1317

Closed
wants to merge 2 commits into from

Conversation

hudeven
Copy link
Contributor

@hudeven hudeven commented Apr 14, 2020

Motivation and Context

  1. We want to read pre-train model files from a public http/https URL using PathManager
  2. As fvcore is available in PyPI, we should add it as dependency, such that we can use HTTPURLHandler to read http/https URL

#1220

How Has This Been Tested

I'm able to download model file in Python Console. I'm not adding a unit test as it adds complexity to exclude the test for internal use case.

with PathManager.open("https://dl.fbaipublicfiles.com/pytext/models/roberta/roberta_public.pt1", "rb") as f:
... print(len(f.read()))
...
roberta_public.pt1: 9%|█████▊ | 44.4M/497M [00:07<01:22, 5.47MB/s]
roberta_public.pt1: 497MB [01:19, 6.25MB/s]
497003753

Types of changes

  • Docs change / refactoring / dependency upgrade
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have completed my CLA (see CONTRIBUTING)
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@facebook-github-bot facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Apr 14, 2020
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hudeven has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@hudeven has updated the pull request. Re-import the pull request

Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hudeven has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@hudeven merged this pull request in ba1b16e.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
CLA Signed Do not delete this pull request or issue due to inactivity. Merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants