You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Background
When pipeline code is upgraded, it often requires a more recent dlt version that the one that user has. We should at least warn the user before updating the code.
Tasks
all pipelines have a requirements.txt. use it to get the dlt dependency version
there are helper functions to get the local dlt version. use standard python means (distutils??) to check if local version is matching the version string in requirements
perform that check during dlt init. if version does not match ask user if to continue
when listing pipelines check for the version and show it in the listing. warn about incompatibilities.
when adding/updating pipeline we generate requirements.txt, now it assumes that dlt dependency is never there. it must be smarter and actually use the dlt provided one. just remember to add extra to it with destination name
keep the required dlt version in TVerifiedSourceFileIndex which is dumped to .sources folder after successful dlt init
Background
When pipeline code is upgraded, it often requires a more recent
dlt
version that the one that user has. We should at least warn the user before updating the code.Tasks
requirements.txt
. use it to get the dlt dependency versionrequirements.txt
, now it assumes thatdlt
dependency is never there. it must be smarter and actually use thedlt
provided one. just remember to add extra to it with destination nameTVerifiedSourceFileIndex
which is dumped to.sources
folder after successfuldlt init
You may prepare pipelines for this upgrade first here: dlt-hub/verified-sources#97
The text was updated successfully, but these errors were encountered: