Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline bugfixes, docstrings, and tests #113

Merged
merged 9 commits into from
Sep 11, 2017

Conversation

nelson-liu
Copy link
Contributor

@nelson-liu nelson-liu commented Sep 8, 2017

This PR fixes a few bugs in data.Pipeline, adds docstrings, and adds tests.

Bugfixes:

  • Pipeline.__call__ was accepting positional args, but wasn't forwarding them to Pipeline.call
  • In Pipeline.call, we were running self(token) for each token in an input list, which has the effect of running the entire pipeline on each token of the list. We were correctly executing only the current pipeline's convert_token if the input was not a list. This becomes a bug when you run a chained pipeline and the same step is executed multiple times.

Enhancements:

  • Pipeline.add_after and Pipeline.add_before now return a reference to the Pipeline (so people don't run into errors doing pipe = pipe.add_*(something))
  • Add a check to ensure that the pipeline's input convert_token is actually callable
  • Replace lambda identity with identity static method, so we can pickle Pipeline objects.

@jekbradbury jekbradbury merged commit be6bbca into pytorch:master Sep 11, 2017
@nelson-liu nelson-liu deleted the pipeline_fixes branch September 13, 2017 23:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants