Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Expose tf.lookup.TextFileIndex in 2.0 #26644

Merged
merged 4 commits into from
Mar 19, 2019
Merged

Expose tf.lookup.TextFileIndex in 2.0 #26644

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
97ac448
Expose tf.lookup.TextFileIndex in 2.0
yongtang Mar 13, 2019
a887a0c
Update API golden for tf.lookup.TextFileIndex
yongtang Mar 13, 2019
5312931
Added documentation for tf.lookup.TextFileIndex
yongtang Mar 15, 2019
65f25df
Add the reference of lookup.TextFileIndexer to lookup.TextFileInitial…
yongtang Mar 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
15 changes: 15 additions & 0 deletions tensorflow/python/ops/lookup_ops.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -419,7 +419,22 @@ def initialize(self, table):
return init_op


@tf_export("lookup.TextFileIndex")
class TextFileIndex(object):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets add some documentation here describing what these constants mean and how can one use them. I think they are described in the file elsewhere.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation helps a lot, but I still don't know which endpoints in the TF API accept a TextFileIndex. Mentioning it here is helpful

"""The key and value content to get from each line.

This class defines the key and value used for tf.lookup.TextFileInitializer.

The key and value content to get from each line is specified either
by the following, or a value `>=0`.
* `TextFileIndex.LINE_NUMBER` means use the line number starting from zero,
expects data type int64.
* `TextFileIndex.WHOLE_LINE` means use the whole line content, expects data
type string.

A value `>=0` means use the index (starting at zero) of the split line based
on `delimiter`.
"""
WHOLE_LINE = -2
LINE_NUMBER = -1

Expand Down
16 changes: 16 additions & 0 deletions tensorflow/tools/api/golden/v1/tensorflow.lookup.-text-file-index.pbtxt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
path: "tensorflow.lookup.TextFileIndex"
tf_class {
is_instance: "<class \'tensorflow.python.ops.lookup_ops.TextFileIndex\'>"
is_instance: "<type \'object\'>"
member {
name: "LINE_NUMBER"
mtype: "<type \'int\'>"
}
member {
name: "WHOLE_LINE"
mtype: "<type \'int\'>"
}
member_method {
name: "__init__"
}
}
4 changes: 4 additions & 0 deletions tensorflow/tools/api/golden/v1/tensorflow.lookup.pbtxt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ tf_module {
name: "StaticVocabularyTable"
mtype: "<type \'type\'>"
}
member {
name: "TextFileIndex"
mtype: "<type \'type\'>"
}
member {
name: "TextFileInitializer"
mtype: "<type \'type\'>"
Expand Down
16 changes: 16 additions & 0 deletions tensorflow/tools/api/golden/v2/tensorflow.lookup.-text-file-index.pbtxt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
path: "tensorflow.lookup.TextFileIndex"
tf_class {
is_instance: "<class \'tensorflow.python.ops.lookup_ops.TextFileIndex\'>"
is_instance: "<type \'object\'>"
member {
name: "LINE_NUMBER"
mtype: "<type \'int\'>"
}
member {
name: "WHOLE_LINE"
mtype: "<type \'int\'>"
}
member_method {
name: "__init__"
}
}
4 changes: 4 additions & 0 deletions tensorflow/tools/api/golden/v2/tensorflow.lookup.pbtxt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ tf_module {
name: "StaticVocabularyTable"
mtype: "<type \'type\'>"
}
member {
name: "TextFileIndex"
mtype: "<type \'type\'>"
}
member {
name: "TextFileInitializer"
mtype: "<type \'type\'>"
Expand Down