Skip to content

Init _backbone, _tokenizer and _preprocessor in Task #899

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 22, 2023
Merged

Init _backbone, _tokenizer and _preprocessor in Task #899

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8ab38f8
Init _backbone in Task
jbischof Mar 21, 2023
4b99e2a
Fix `preprocessor` as well
jbischof Mar 21, 2023
9ebe8b5
format
jbischof Mar 21, 2023
2b1848c
Fix `_tokenizer`
jbischof Mar 21, 2023
4bcad55
Chain to super() first
jbischof Mar 22, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions keras_nlp/models/preprocessor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,10 @@
class Preprocessor(keras.layers.Layer):
"""Base class for model preprocessors."""

def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._tokenizer = None

@property
def tokenizer(self):
"""The tokenizer used to tokenize strings."""
Expand Down
5 changes: 5 additions & 0 deletions keras_nlp/models/task.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,11 @@
class Task(PipelineModel):
"""Base class for Task models."""

def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._backbone = None
self._preprocessor = None

def preprocess_samples(self, x, y=None, sample_weight=None):
return self.preprocessor(x, y=y, sample_weight=sample_weight)

Expand Down