Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-26683: Make exporting dimension data friendlier #375

Merged
merged 10 commits into from
Sep 17, 2020
Merged

Commits on Sep 16, 2020

  1. Improve type annotations on immutable decorator.

    Without generics, decoration hid all type information about the
    wrapped type.
    TallJimbo committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    0c3ddb0 View commit details
    Browse the repository at this point in the history
  2. Fix inf recursion DimensionUniverse.__repr__.

    This was a relic from when DimensionUniverse inherited from
    DimensionGraph and DimensionGraph defined __str__.
    TallJimbo committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    5578084 View commit details
    Browse the repository at this point in the history
  3. Move repository import/export code into separate package.

    This should not change functionality at all; it's just moving code
    around.  A separate package gives the code a bit more room for
    expansion and more clearly separates between interfaces and
    implementations.
    
    Moving it out of 'core' fixes the existing circular dependency issues:
    the import/export code needs to depend on both Registry and Datastore,
    and hence shouldn't go in core.  But the registry tests depend on it,
    so there is one new circular dependency in registry.tests, but because
    tests conceptually depend on everything I'm not bothered by this
    (maybe we should have named that 'registry_tests' instead of making it
    a registry subpackage, but I'm not going to).
    TallJimbo committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    d062e3e View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    2031534 View commit details
    Browse the repository at this point in the history
  5. Defer export writes in order to deduplicate dimension records.

    This should address the core complaint of DM-26683.
    TallJimbo committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    0c3008f View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    b3ce605 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    35d518c View commit details
    Browse the repository at this point in the history
  8. Make element argument in RepoExportContext more user-friendly.

    Strings are now accepted as well as DimensionElement instances, and
    passing elements without tables (like 'htmN') is now correctly ignored
    (because the user shouldn't care which elements have tables).
    TallJimbo committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    7baced3 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2020

  1. Remove/adjust confusing comments in butler tests.

    I wrote these a year ago and don't understand them anymore.
    TallJimbo committed Sep 17, 2020
    Configuration menu
    Copy the full SHA
    b10c432 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7eef36c View commit details
    Browse the repository at this point in the history