Finish tests for Field class #119
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I think that would be okay? I can't imagine any situation where someone has data in a string that looks like a float and is asking for a FloatTensor, but doesn't want the data converted from string to float. If it's already of the target datatype, this is a no-op anyway. |
|
If it's already of the target datatype, this is a no-op anyway.
Is this even possible with the standard data flow? I feel like everything
is converted to string.
…On September 14, 2017 at 12:10:51 PM, jekbradbury ***@***.***) wrote:
I think that would be okay? I can't imagine any situation where someone
has data in a string that looks like a float and is asking for a
FloatTensor, but doesn't want the data converted from string to float. If
it's already of the target datatype, this is a no-op anyway.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#119 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AG72X8eFlJ49_fiPLLXKjPOknoitqd8nks5siXo7gaJpZM4PVqTv>
.
|
|
If you write a custom Dataset that e.g. loads from HDF5, maybe? |
nelson-liu
commented
Sep 15, 2017
| "Please raise an issue at " | ||
| "https://github.com/pytorch/text/issues".format(self.tensor_type)) | ||
| numericalization_func = self.tensor_types[self.tensor_type] | ||
| # It doesn't make sense to explictly coerce to a numeric type if |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
Closed
jekbradbury
pushed a commit
that referenced
this pull request
Oct 9, 2017
Also: - Add a dataset for testing numeric features (float and int) - Coerce non-sequential data with use_vocab=False to numeric types
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is a continuation of #47 and finishes testing the various functions for the field class (
build_vocabandnumericalize).I also wanted to fix #78 and add a test for it, but I'm unsure how to properly do this. One way would be to simply convert the numerical features to python ints dictated by the input
tensor_typemember (e.g.intforLongTensor,floatforFloatTensor, etc). This feels pretty brittle, does anyone else have any other suggestions?