How to store assets per user id? #20413
-
Hey. I guess it's is very common scenario when assets should be placed in a folder of uniq id, let it be user id. But a built-in IOManager doesn't have an option to setup such the way of contextual asset placing. Let's consider the following basic example @asset
def user_profile(context, http_resource, user_id: str, access_token: str) -> dict:
profile_data = http_resource.get('/any/profile/url', user_id, access_token)
extra_handling(profile_data)
return profile_data # heree i would like an IOManager to write data into filesystem as `/parent_folder/{user_id}/profile_data`
@asset
def profile_score(user_profile: dict) -> int:
s = score(user_profile) # here i would like an IOManager to read input data from a folder `/parent_folder/{user_id}/profile_data`
return s Just in case, currently i'm using |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
@4l1fe That feature is not currently supported by a built-in IOManager. To support the use case, the unique folder name would need to be persisted somewhere, so that the IOManager could get the correct path when loading an asset for downstream usage. This being said, you could subclass S3PickleIOManager and implement a different |
Beta Was this translation helpful? Give feedback.
@4l1fe That feature is not currently supported by a built-in IOManager. To support the use case, the unique folder name would need to be persisted somewhere, so that the IOManager could get the correct path when loading an asset for downstream usage.
This being said, you could subclass S3PickleIOManager and implement a different
get_asset_relative_path
method that creates the correct path for your use case.