Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Torchscript] Add Date feature preprocessing #2178

Merged
merged 5 commits into from
Jun 22, 2022
Merged

Conversation

geoffreyangus
Copy link
Collaborator

This PR implements Date feature preprocessing in Torchscript.

Date feature preprocessing in Torchscript deviates from vanilla preprocessing in one key way: instead of being able to pass in the raw date-formatted string into the torchscript model, the user must first featurize it into a vector with the values found in vector (below).

yearday = datetime_obj.toordinal() - date(datetime_obj.year, 1, 1).toordinal() + 1

midnight = datetime_obj.replace(hour=0, minute=0, second=0, microsecond=0)
second_of_day = (datetime_obj - midnight).seconds

vector = [
    datetime_obj.year,
    datetime_obj.month,
    datetime_obj.day,
    datetime_obj.weekday(),
    yearday,
    datetime_obj.hour,
    datetime_obj.minute,
    datetime_obj.second,
    second_of_day,
]

A utility function for converting datetime objects into this exact type of vector can be found at ludwig.features.date_feature.create_vector_from_datetime_obj.

This deviation occurs because the datetime and dateutil libraries are not currently scriptable (https://pytorch.org/docs/stable/jit.html#python-functions-and-modules).

In general, if the user plans on deploying with Torchscript with a Python backend, they can use the utility function ludwig.models.inference.to_inference_module_input_from_dataframe to ensure that their input features are being passed into the torchscript model correctly.

@github-actions
Copy link

Unit Test Results

       6 files  ±0         6 suites  ±0   2h 20m 1s ⏱️ + 20m 22s
2 831 tests +1  2 797 ✔️ +1    34 💤 ±0  0 ±0 
8 493 runs  +3  8 387 ✔️ +3  106 💤 ±0  0 ±0 

Results for commit 9abc608. ± Comparison against base commit fdcbcc7.

@geoffreyangus geoffreyangus merged commit 0dd7d56 into master Jun 22, 2022
@geoffreyangus geoffreyangus deleted the ts-add-date-preproc branch June 22, 2022 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants