Skip to content

Corrupted dlt pipelines metadata #2394

@dexters1

Description

@dexters1

dlt version

1.7.0

Describe the problem

I've somehow managed to corrupt my dlt metadata at: "/Users/{username}/.dlt/pipelines" ( I don't know how )

Due to this dlt stopped creating it's tables in databases and I was getting the following error:

"(sqlite3.OperationalError) no such table: main._dlt_loads"

I had a hard time debugging what was the problem with my dlt afterwards as reinstalling dlt with pip, making fresh environments and etc. didn't help. When I manually deleted the dlt pipelines data dlt managed to work again.

Maybe consider also removing this pipeline data when dlt in uninstalled? Or maybe have a suggestion to clean this dlt pipelines metadata in the traceback message.

It's not an easy issue to find, debug and resolve for users currently.

Expected behavior

No response

Steps to reproduce

Corrupt the dlt pipelines metadata at: "/Users/username/.dlt/pipelines"

Operating system

macOS

Runtime environment

Local

Python version

3.12

dlt data source

No response

dlt destination

No response

Other deployment details

No response

Additional information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    QoLQuality of Life: improve the developer experience

    Type

    No type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions