fix: Request data api update #2488

kevjumba · 2022-04-04T23:36:55Z

What this PR does / why we need it:
Update RequestDataSource and change the name to RequestSource
Which issue(s) this PR fixes:

Fixes #

codecov-commenter · 2022-04-04T23:42:30Z

Codecov Report

Merging #2488 (340b98a) into master (712653e) will increase coverage by 0.13%.
The diff coverage is 95.45%.

@@            Coverage Diff             @@
##           master    #2488      +/-   ##
==========================================
+ Coverage   84.50%   84.64%   +0.13%     
==========================================
  Files         130      130              
  Lines       11015    11043      +28     
==========================================
+ Hits         9308     9347      +39     
+ Misses       1707     1696      -11

Flag	Coverage Δ
integrationtests	`74.57% <70.45%> (-0.26%)`	⬇️
unittests	`58.97% <84.09%> (+0.09%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...ts/integration/feature_repos/repo_configuration.py	`96.07% <ø> (ø)`
sdk/python/feast/request_feature_view.py	`67.56% <50.00%> (ø)`
sdk/python/feast/data_source.py	`68.85% <100.00%> (+0.55%)`	⬆️
sdk/python/feast/feature_store.py	`89.84% <100.00%> (ø)`
sdk/python/feast/inference.py	`85.29% <100.00%> (ø)`
sdk/python/feast/on_demand_feature_view.py	`95.12% <100.00%> (ø)`
...tegration/feature_repos/universal/feature_views.py	`83.87% <100.00%> (ø)`
...n/tests/integration/registration/test_inference.py	`100.00% <100.00%> (ø)`
...on/tests/integration/registration/test_registry.py	`95.81% <100.00%> (ø)`
...istration/test_universal_odfv_feature_inference.py	`100.00% <100.00%> (ø)`
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 712653e...340b98a. Read the comment docs.

sdk/python/feast/data_source.py

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

sdk/python/feast/data_source.py

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

Signed-off-by: Author <kzhang@tecton.ai> Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

Signed-off-by: Kevin Zhang <kzhang@tecton.ai> Signed-off-by: Author <kzhang@tecton.ai> Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

achals · 2022-04-05T18:52:40Z

sdk/python/feast/data_source.py

@@ -496,6 +495,15 @@ def source_datatype_to_feast_value_type() -> Callable[[str], ValueType]:
        raise NotImplementedError


+class RequestDataSource(RequestSource):


Can you also keep a test for RequestDataSource to make sure that it continues to work, and prints the deprecation warning? I think there's examples of this in the unit tests already.

achals · 2022-04-05T18:53:10Z

sdk/python/feast/request_feature_view.py

@@ -30,7 +30,7 @@ class RequestFeatureView(BaseFeatureView):
    """

    name: str
-    request_data_source: RequestDataSource
+    request_data_source: RequestSource


change field name?

achals · 2022-04-05T18:53:44Z

sdk/python/feast/request_feature_view.py

@@ -40,7 +40,7 @@ class RequestFeatureView(BaseFeatureView):
    def __init__(
        self,
        name: str,
-        request_data_source: RequestDataSource,
+        request_data_source: RequestSource,


change field name? also, change to use kwargs? Or is that a different PR?

This is a different PR, I am not deprecating feature view params here.

achals · 2022-04-05T18:54:15Z

sdk/python/tests/integration/feature_repos/universal/feature_views.py

 def create_conv_rate_request_data_source():
-    return RequestDataSource(
-        name="conv_rate_input", schema={"val_to_add": ValueType.INT32}
-    )
+    return RequestSource(name="conv_rate_input", schema={"val_to_add": ValueType.INT32})


 def create_similarity_request_data_source():
-    return RequestDataSource(
+    return RequestSource(


create_*_source?

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

achals

/lgtm

feast-ci-bot · 2022-04-05T20:29:33Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: achals, kevjumba

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [achals,kevjumba]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

achals · 2022-04-05T21:07:08Z

hmmm @kevjumba there seem to be a bunch of test failures.

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

achals · 2022-04-05T23:29:14Z

/lgtm

# [0.20.0](v0.19.0...v0.20.0) (2022-04-14) ### Bug Fixes * Add inlined data sources to the top level registry ([#2456](#2456)) ([356788a](356788a)) * Add new value types to types.ts for web ui ([#2463](#2463)) ([ad5694e](ad5694e)) * Add PushSource proto and Python class ([#2428](#2428)) ([9a4bd63](9a4bd63)) * Add spark to lambda dockerfile ([#2480](#2480)) ([514666f](514666f)) * Added private_key auth for Snowflake ([#2508](#2508)) ([c42c9b0](c42c9b0)) * Added Redshift and Spark typecheck to data_source event_timestamp_col inference ([#2389](#2389)) ([04dea73](04dea73)) * Building of go extension fails ([#2448](#2448)) ([7d1efd5](7d1efd5)) * Bump the number of versions bumps expected to 27 ([#2549](#2549)) ([ecc9938](ecc9938)) * Create __init__ files for the proto-generated python dirs ([#2410](#2410)) ([e17028d](e17028d)) * Don't prevent apply from running given duplicate empty names in data sources. Also fix repeated apply of Spark data source. ([#2415](#2415)) ([b95f441](b95f441)) * Dynamodb deduplicate batch write request by partition keys ([#2515](#2515)) ([70d4a13](70d4a13)) * Ensure that __init__ files exist in proto dirs ([#2433](#2433)) ([9b94f7b](9b94f7b)) * Fix DataSource constructor to unbreak custom data sources ([#2492](#2492)) ([712653e](712653e)) * Fix default feast apply path without any extras ([#2373](#2373)) ([6ba7fc7](6ba7fc7)) * Fix definitions.py with new definition ([#2541](#2541)) ([eefc34a](eefc34a)) * Fix entity row to use join key instead of name ([#2521](#2521)) ([c22fa2c](c22fa2c)) * Fix Java Master ([#2499](#2499)) ([e083458](e083458)) * Fix registry proto ([#2435](#2435)) ([ea6a9b2](ea6a9b2)) * Fix some inconsistencies in the docs and comments in the code ([#2444](#2444)) ([ad008bf](ad008bf)) * Fix spark docs ([#2382](#2382)) ([d4a606a](d4a606a)) * Fix Spark template to work correctly on feast init -t spark ([#2393](#2393)) ([ae133fd](ae133fd)) * Fix the feature repo fixture used by java tests ([#2469](#2469)) ([32e925e](32e925e)) * Fix unhashable Snowflake and Redshift sources ([cd8f1c9](cd8f1c9)) * Fixed bug in passing config file params to snowflake python connector ([#2503](#2503)) ([34f2b59](34f2b59)) * Fixing Spark template to include source name ([#2381](#2381)) ([a985f1d](a985f1d)) * Make name a keyword arg for the Entity class ([#2467](#2467)) ([43847de](43847de)) * Making a name for data sources not a breaking change ([#2379](#2379)) ([71d7ae2](71d7ae2)) * Minor link fix in `CONTRIBUTING.md` ([#2481](#2481)) ([2917e27](2917e27)) * Preserve ordering of features in _get_column_names ([#2457](#2457)) ([495b435](495b435)) * Relax click python requirement to >=7 ([#2450](#2450)) ([f202f92](f202f92)) * Remove date partition column field from datasources that don't s… ([#2478](#2478)) ([ce35835](ce35835)) * Remove docker step from unit test workflow ([#2535](#2535)) ([6f22f22](6f22f22)) * Remove spark from the AWS Lambda dockerfile ([#2498](#2498)) ([6abae16](6abae16)) * Request data api update ([#2488](#2488)) ([0c9e5b7](0c9e5b7)) * Schema update ([#2509](#2509)) ([cf7bbc2](cf7bbc2)) * Simplify DataSource.from_proto logic ([#2424](#2424)) ([6bda4d2](6bda4d2)) * Snowflake api update ([#2487](#2487)) ([1181a9e](1181a9e)) * Support passing batch source to streaming sources for backfills ([#2523](#2523)) ([90db1d1](90db1d1)) * Timestamp update ([#2486](#2486)) ([bf23111](bf23111)) * Typos in Feast UI error message ([#2432](#2432)) ([e14369d](e14369d)) * Update feature view APIs to prefer keyword args ([#2472](#2472)) ([7c19cf7](7c19cf7)) * Update file api ([#2470](#2470)) ([83a11c6](83a11c6)) * Update Makefile to cd into python dir before running commands ([#2437](#2437)) ([ca32155](ca32155)) * Update redshift api ([#2479](#2479)) ([4fa73a9](4fa73a9)) * Update some fields optional in UI parser ([#2380](#2380)) ([cff7ac3](cff7ac3)) * Use a single version of jackson libraries and upgrade to 2.12.6.1 ([#2473](#2473)) ([5be1cc6](5be1cc6)) * Use dateutil parser to parse materialization times ([#2464](#2464)) ([6c55e49](6c55e49)) * Use the correct dockerhub image tag when building feature servers ([#2372](#2372)) ([0d62c1d](0d62c1d)) ### Features * Add `/write-to-online-store` method to the python feature server ([#2423](#2423)) ([d2fb048](d2fb048)) * Add description, tags, owner fields to all feature view classes ([#2440](#2440)) ([ed5e928](ed5e928)) * Add DQM Logging on GRPC Server with FileLogStorage for Testing ([#2403](#2403)) ([57a97d8](57a97d8)) * Add Feast types in preparation for changing type system ([#2475](#2475)) ([4864252](4864252)) * Add Field class ([#2500](#2500)) ([1279612](1279612)) * Add support for DynamoDB online_read in batches ([#2371](#2371)) ([702ec49](702ec49)) * Add Support for DynamodbOnlineStoreConfig endpoint_url parameter ([#2485](#2485)) ([7b863d1](7b863d1)) * Add templating for dynamodb table name ([#2394](#2394)) ([f591088](f591088)) * Allow local feature server to use Go feature server if enabled ([#2538](#2538)) ([a2ef375](a2ef375)) * Allow using entity's join_key in get_online_features ([#2420](#2420)) ([068c765](068c765)) * Data Source Api Update ([#2468](#2468)) ([6b96b21](6b96b21)) * Go server ([#2339](#2339)) ([d12e7ef](d12e7ef)), closes [#2354](#2354) [#2361](#2361) [#2332](#2332) [#2356](#2356) [#2363](#2363) [#2349](#2349) [#2355](#2355) [#2336](#2336) [#2361](#2361) [#2363](#2363) [#2344](#2344) [#2354](#2354) [#2347](#2347) [#2350](#2350) [#2356](#2356) [#2355](#2355) [#2349](#2349) [#2352](#2352) [#2341](#2341) [#2336](#2336) [#2373](#2373) [#2315](#2315) [#2372](#2372) [#2332](#2332) [#2349](#2349) [#2336](#2336) [#2361](#2361) [#2363](#2363) [#2344](#2344) [#2354](#2354) [#2347](#2347) [#2350](#2350) [#2356](#2356) [#2355](#2355) [#2349](#2349) [#2352](#2352) [#2341](#2341) [#2336](#2336) [#2373](#2373) [#2379](#2379) [#2380](#2380) [#2382](#2382) [#2364](#2364) [#2366](#2366) [#2386](#2386) * Graduate write_to_online_store out of experimental status ([#2426](#2426)) ([e7dd4b7](e7dd4b7)) * Make feast PEP 561 compliant ([#2405](#2405)) ([3c41f94](3c41f94)), closes [#2420](#2420) [#2418](#2418) [#2425](#2425) [#2426](#2426) [#2427](#2427) [#2431](#2431) [#2433](#2433) [#2420](#2420) [#2418](#2418) [#2425](#2425) [#2426](#2426) [#2427](#2427) [#2431](#2431) [#2433](#2433) * Makefile for contrib for Issue [#2364](#2364) ([#2366](#2366)) ([a02325b](a02325b)) * Support on demand feature views in go feature server ([#2494](#2494)) ([6edd274](6edd274)) * Switch from Feature to Field ([#2514](#2514)) ([6a03bed](6a03bed)) * Use a daemon thread to monitor the go feature server exclusively ([#2391](#2391)) ([0bb5e8c](0bb5e8c))

kevjumba requested a review from a team as a code owner April 4, 2022 23:36

kevjumba requested review from MattDelac and removed request for a team April 4, 2022 23:36

feast-ci-bot added the approved label Apr 4, 2022

github-actions bot assigned woop Apr 4, 2022

feast-ci-bot added the size/M label Apr 4, 2022

kevjumba added ok-to-test kind/housekeeping labels Apr 4, 2022

kevjumba added ok-to-test and removed ok-to-test labels Apr 4, 2022

achals requested changes Apr 5, 2022

View reviewed changes

sdk/python/feast/data_source.py Show resolved Hide resolved

kevjumba requested a review from achals April 5, 2022 17:51

kevjumba added 2 commits April 5, 2022 10:58

Update request data source

7cea469

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

Fix

7b7395c

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

kevjumba force-pushed the requestdata_api_update branch from 285d96d to fd82da1 Compare April 5, 2022 18:00

achals reviewed Apr 5, 2022

View reviewed changes

sdk/python/feast/data_source.py Outdated Show resolved Hide resolved

kevjumba added 5 commits April 5, 2022 11:38

Fix

945e0f8

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

Fix

19158f8

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

Data source update

b284d5f

Signed-off-by: Author <kzhang@tecton.ai> Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

Data source

c90a224

Signed-off-by: Kevin Zhang <kzhang@tecton.ai> Signed-off-by: Author <kzhang@tecton.ai> Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

Temp fix

a362e9f

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

kevjumba force-pushed the requestdata_api_update branch from 0bdc471 to a362e9f Compare April 5, 2022 18:38

Fix

f3c9d7a

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

kevjumba requested a review from achals April 5, 2022 18:49

achals reviewed Apr 5, 2022

View reviewed changes

kevjumba added 4 commits April 5, 2022 12:31

Update internal var names

b442599

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

Add test

af8e98f

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

Fix

b28932d

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

Fix

9ecf9c8

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

Fix

61230c8

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

feast-ci-bot added size/L and removed size/M labels Apr 5, 2022

kevjumba added ok-to-test and removed ok-to-test labels Apr 5, 2022

achals approved these changes Apr 5, 2022

View reviewed changes

feast-ci-bot assigned achals Apr 5, 2022

feast-ci-bot added the lgtm label Apr 5, 2022

Fix

340b98a

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

feast-ci-bot removed the lgtm label Apr 5, 2022

feast-ci-bot added the lgtm label Apr 5, 2022

feast-ci-bot merged commit 0c9e5b7 into feast-dev:master Apr 5, 2022

felixwang9817 mentioned this pull request Apr 11, 2022

Feature repo setup for Java integration tests is broken #2522

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Request data api update #2488

fix: Request data api update #2488

kevjumba commented Apr 4, 2022

codecov-commenter commented Apr 4, 2022 •

edited

achals Apr 5, 2022

kevjumba Apr 5, 2022

achals Apr 5, 2022

kevjumba Apr 5, 2022

achals Apr 5, 2022

kevjumba Apr 5, 2022

achals Apr 5, 2022

kevjumba Apr 5, 2022

achals left a comment

feast-ci-bot commented Apr 5, 2022

achals commented Apr 5, 2022

achals commented Apr 5, 2022

		@@ -496,6 +495,15 @@ def source_datatype_to_feast_value_type() -> Callable[[str], ValueType]:
		raise NotImplementedError


		class RequestDataSource(RequestSource):

fix: Request data api update #2488

fix: Request data api update #2488

Conversation

kevjumba commented Apr 4, 2022

codecov-commenter commented Apr 4, 2022 • edited

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

achals left a comment

Choose a reason for hiding this comment

feast-ci-bot commented Apr 5, 2022

achals commented Apr 5, 2022

achals commented Apr 5, 2022

codecov-commenter commented Apr 4, 2022 •

edited