Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update README.md with a working example on data.TabularDataset for json files #133

Closed
czhang99 opened this issue Oct 2, 2017 · 0 comments

Comments

@czhang99
Copy link
Contributor

czhang99 commented Oct 2, 2017

In README.md, the current example for data.TabularDataset with json file input looks like this:

sentiment = data.TabularDataset(
path='data/sentiment/train.json', format='json',
fields=[{'sentence_tokenized': ('text', data.Field(sequential=True)),
'sentiment_gold': ('labels', data.Field(sequential=False))}])

where "fields" attribute is a list that contains a dictionary. This will lead to an AttributeError: 'list' object has no attribute 'items' because in torchtext/data/example.py we have:

@classmethod
def fromJSON(cls, data, **fields**):
    return cls.fromdict(json.loads(data), **fields**)

@classmethod
def fromdict(cls, data, **fields**):
    ex = cls()
    for key, vals in **fields.items()**:

where the field value (currently a list) is processed by function fromdict with a ".item()" operator.

So the example in the README.md file should be changed to:

sentiment = data.TabularDataset(
path='data/sentiment/train.json', format='json',
fields={'sentence_tokenized': ('text', data.Field(sequential=True)),
'sentiment_gold': ('labels', data.Field(sequential=False))})

where "fields" attribute is fed with a dictionary directly.

czhang99 added a commit to czhang99/text that referenced this issue Oct 2, 2017
czhang99 added a commit to czhang99/text that referenced this issue Oct 2, 2017
jekbradbury pushed a commit that referenced this issue Oct 3, 2017
jekbradbury pushed a commit that referenced this issue Oct 9, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant