Skip to content
Permalink
Branch: master
Commits on May 17, 2019
  1. Excise PipelineContextDefinition

    mgasner committed May 16, 2019
    Summary: #1343
    
    Test Plan: Unit tests
    
    Reviewers: alangenfeld, natekupp
    
    Reviewed By: natekupp
    
    Subscribers: schrockn
    
    Differential Revision: https://dagster.phacility.com/D149
Commits on May 16, 2019
  1. Move solid_has_* onto ISolidDefinition

    natekupp committed May 16, 2019
    Test Plan: tests pass
    
    Reviewers: max, alangenfeld
    
    Reviewed By: alangenfeld
    
    Differential Revision: https://dagster.phacility.com/D150
  2. fix preset sorting

    alangenfeld committed May 16, 2019
    Test Plan: buildkite
    
    Reviewers: natekupp
    
    Reviewed By: natekupp
    
    Differential Revision: https://dagster.phacility.com/D152
  3. remove need to yield None for Nothing output

    alangenfeld committed May 16, 2019
    Test Plan: updated unit tests, run many_events toy in dagit
    
    Reviewers: max, natekupp
    
    Reviewed By: max
    
    Differential Revision: https://dagster.phacility.com/D148
  4. [nightly] dont prune in phab tag cleanup

    alangenfeld committed May 16, 2019
    Summary: -P still isnt supported in v2.11 which is what the default circle image uses. I don't think we need it since we are doing a fresh checkout - so just do the clean up part.
    
    Test Plan: eyes
    
    Reviewers: max
    
    Reviewed By: max
    
    Differential Revision: https://dagster.phacility.com/D147
  5. move presets from repo yml to pipeline

    alangenfeld committed May 16, 2019
    Summary: Move preset definition from repository.yml to `PresetDefinition`s being passed to `PipelineDefinition`.
    
    Test Plan: unit, load error_monster in dagit and select the preset
    
    Reviewers: schrockn, natekupp, max
    
    Reviewed By: natekupp
    
    Differential Revision: https://dagster.phacility.com/D144
  6. [dagit] change Nothing type display

    alangenfeld committed May 16, 2019
    Summary: In the initial pass, I choose to hide details about `Nothing` type IO. This was probably not the right choice, so add it back everywhere except the type signature section.
    
    Test Plan: screenshots
    
    Reviewers: schrockn, natekupp, bengotow, max
    
    Reviewed By: natekupp
    
    Differential Revision: https://dagster.phacility.com/D145
  7. Add a default ModeDefinition

    mgasner committed May 14, 2019
    Summary: Depends on D135
    
    Test Plan: Unit tests
    
    Reviewers: alangenfeld
    
    Reviewed By: alangenfeld
    
    Differential Revision: https://dagster.phacility.com/D143
  8. [nightly] add --autoclean to publish.py

    alangenfeld committed May 16, 2019
    Summary:
    hitting 1 pyc file in the nightly job, so lets add an autoclean flag for automation
    https://circleci.com/gh/dagster-io/dagster/166468
    
    Test Plan: not sure how to test this without doing a publish
    
    Reviewers: max
    
    Reviewed By: max
    
    Differential Revision: https://dagster.phacility.com/D146
  9. Get airline demo running using ModeDefinition

    mgasner committed May 14, 2019
    Test Plan: Unit tests
    
    Reviewers: alangenfeld, natekupp
    
    Reviewed By: alangenfeld, natekupp
    
    Differential Revision: https://dagster.phacility.com/D135
  10. enable arc lint

    alangenfeld committed May 3, 2019
    Summary:
    Turn on local linting to reduce round trips from lint failure.
    
    Note: this is blocked on a bug with arc: https://discourse.phabricator-community.org/t/arcanistpylintlinter-version-regex-issue/2688
    
    Test Plan: arc lint
    
    Reviewers: schrockn, natekupp, max
    
    Reviewed By: natekupp, max
    
    Differential Revision: https://dagster.phacility.com/D36
  11. fixing nightly

    alangenfeld committed May 16, 2019
Commits on May 15, 2019
  1. Use os.sep instead of '/'

    mgasner committed May 15, 2019
    Summary: Should fix our Azure pipelines builds
    
    Test Plan: Azure pipelines
    
    Reviewers: alangenfeld
    
    Reviewed By: alangenfeld
    
    Differential Revision: https://dagster.phacility.com/D142
  2. guard against npm install

    alangenfeld committed May 15, 2019
    Test Plan: {P7}
    
    Reviewers: max
    
    Reviewed By: max
    
    Differential Revision: https://dagster.phacility.com/D139
  3. [nightly] do phab tags last

    alangenfeld committed May 15, 2019
    Summary: We check out the nightly branch (from N days ago since this has been broken) so we should run this step after we've updated to master and pushed the new nightly.
    
    Test Plan: ???
    
    Reviewers: max
    
    Reviewed By: max
    
    Differential Revision: https://dagster.phacility.com/D140
  4. Relax Flask requirement

    mgasner committed May 15, 2019
    Summary: #1360
    
    Test Plan: Unit tests & fresh install
    
    Reviewers: natekupp
    
    Reviewed By: natekupp
    
    Differential Revision: https://dagster.phacility.com/D141
  5. Fix bug in python file handle function

    natekupp committed May 15, 2019
    Test Plan: TBD
    
    Reviewers: alangenfeld
    
    Reviewed By: alangenfeld
    
    Differential Revision: https://dagster.phacility.com/D138
  6. Update js packages

    mgasner committed May 14, 2019
    Summary: Fixes issues identified by npm audit
    
    Test Plan: Unit tests
    
    Reviewers: alangenfeld
    
    Reviewed By: alangenfeld
    
    Differential Revision: https://dagster.phacility.com/D133
  7. Frontend display for mode-configured loggers

    mgasner committed May 14, 2019
    Test Plan: Jest
    
    Reviewers: alangenfeld
    
    Reviewed By: alangenfeld
    
    Differential Revision: https://dagster.phacility.com/D127
  8. Add hammer toy

    natekupp committed May 14, 2019
    Summary: This changeset adds a pointer-chase "hammer" pipeline, for use with multiproc testing. We also rename BaseEngine to IEngine for consistency with the rest of the codebase, and pass around the new ExecutionTargetHandle instead of a pipeline_fn.
    
    Test Plan: Tested hammer pipeline
    
    Reviewers: alangenfeld
    
    Reviewed By: alangenfeld
    
    Differential Revision: https://dagster.phacility.com/D132
  9. [dagster] Fan In Dependency

    alangenfeld committed May 10, 2019
    Summary:
    Add `FanInDependencyDefinition` which is constructed with a list of `DependencyDefinition`s. Currently only supports `Nothing` type inputs.
    
    Updates dagit and dagster-graphql to support an input depending on multiple outputs.
    
    Test Plan: added unit tests
    
    Reviewers: schrockn, max, natekupp
    
    Reviewed By: schrockn
    
    Differential Revision: https://dagster.phacility.com/D109
Commits on May 14, 2019
  1. Expose CompositeSolid in GraphQL

    alangenfeld committed May 10, 2019
    Summary: See `schema.graphql` changes
    
    Test Plan:
    Query
    {P5}
    
    Response
    {P4}
    
    Reviewers: bengotow, schrockn
    
    Reviewed By: schrockn
    
    Differential Revision: https://dagster.phacility.com/D79
  2. fix nightly

    alangenfeld committed May 14, 2019
    Summary: Circle uses an older version of git that doesnt have --prune-tags / -P so use -p -t to attempt to prune tags
    
    Test Plan: ???
    
    Reviewers: max
    
    Reviewed By: max
    
    Differential Revision: https://dagster.phacility.com/D130
  3. Expose mode loggers in graphql

    mgasner committed May 14, 2019
    Test Plan: Unit tests
    
    Reviewers: alangenfeld
    
    Reviewed By: alangenfeld
    
    Differential Revision: https://dagster.phacility.com/D126
  4. Make doc test as python3 only

    schrockn committed May 14, 2019
    Summary: I was cleaning up some virtual environments and ran into this
    
    Test Plan: Run unit tests in python 2
    
    Reviewers: max, alangenfeld
    
    Reviewed By: max
    
    Differential Revision: https://dagster.phacility.com/D131
  5. Consolidate PipelineTargetInfo and RepositoryTargetInfo into a single…

    natekupp committed May 13, 2019
    … serializable ExecutionTargetHandle object
    
    Summary:
    This changeset removes `*TargetInfo` objects in favor of a single ExecutionTargetHandle object we can thread everywhere in dagster.
    
    The new object is serializable and enables newing up a repo/pipeline wherever needed across the codebase.
    
    Test Plan: Changes are under test; manually tested dagit to confirm no regression there
    
    Reviewers: max, alangenfeld, schrockn
    
    Reviewed By: schrockn
    
    Differential Revision: https://dagster.phacility.com/D123
  6. Fix crash in dagit when steps are skipped

    schrockn committed May 13, 2019
    Summary:
    Dagit was crashing anytime that it encountered a skip event. It appears to happen because immer's
    produce doesn't play totally nicely with typescript correctness. In this case "skipped" needs to be
    treated like "start" and create a metadata structure from scratch. Immer allowed an IRunMetaDict struct
    to be created without a displayEvents property, and that caused a crash in ExecutionPlanBox
    
    Test Plan: Run pipeline with failure and skip event
    
    Reviewers: max, alangenfeld, bengotow
    
    Reviewed By: alangenfeld
    
    Differential Revision: https://dagster.phacility.com/D119
  7. hide *.generated.json in phab

    alangenfeld committed May 13, 2019
    Test Plan: shows up as generated in phabricator
    
    Reviewers: schrockn, max
    
    Reviewed By: schrockn
    
    Differential Revision: https://dagster.phacility.com/D116
  8. Top level logging implemented through ModeDefinition

    mgasner committed May 14, 2019
    Summary: Opening this for review as it is code complete, will write changelog and update docs before landing.
    
    Test Plan: Unit tests
    
    Reviewers: schrockn, alangenfeld
    
    Reviewed By: schrockn
    
    Differential Revision: https://dagster.phacility.com/D78
  9. Move event pipeline demo to use ModeDefinition

    schrockn committed May 13, 2019
    Summary: See above
    
    Test Plan: CI/CD
    
    Reviewers: natekupp, max
    
    Reviewed By: max
    
    Differential Revision: https://dagster.phacility.com/D124
  10. Rip out dependency on dagster-pandas

    mgasner committed May 13, 2019
    Summary: Replaces dagster-pandas with an ad hoc PoorMansDataFrame that serializes to and from .csv
    
    Test Plan: Unit tests
    
    Reviewers: schrockn
    
    Reviewed By: schrockn
    
    Differential Revision: https://dagster.phacility.com/D121
  11. Move intro tutorial code to ModeDefinition

    schrockn committed May 13, 2019
    Summary: Still need to port actual docs. Tracking here: #1354
    
    Test Plan: Unit tests
    
    Reviewers: max, natekupp
    
    Reviewed By: max
    
    Differential Revision: https://dagster.phacility.com/D122
Commits on May 13, 2019
  1. Eliminate PipelineContextDefinition from graphql

    schrockn committed May 13, 2019
    Summary:
    Can do this now that dagit has been used to use Mode instead of Context.
    
    Depends on D118
    
    Test Plan: Unit tests. Run pipelines from dagit.
    
    Reviewers: max
    
    Reviewed By: max
    
    Differential Revision: https://dagster.phacility.com/D120
  2. Add mode selection to dagit

    schrockn committed May 10, 2019
    Summary:
    This adds a mode selector to dagit. One can take a pipeline with modes, change them, and the config editor, execution plan rendering, and execution itself gets a new mode.
    
    Depends on D117
    
    Test Plan:
    Test within UI.
    
    1) Select new mode that requires new config
    2) See config schema change show syntax editor
    3) See execution plan no longer render
    4) Fix config
    5) Instigate execution; see things happen
    
    Reviewers: bengotow, alangenfeld, max
    
    Reviewed By: max
    
    Differential Revision: https://dagster.phacility.com/D118
  3. Render modes instead of contexts in explore tab

    schrockn committed May 13, 2019
    Summary: See above. Very straightforward
    
    Test Plan: View pipelines both with and without modes
    
    Reviewers: max, alangenfeld, bengotow
    
    Reviewed By: max
    
    Differential Revision: https://dagster.phacility.com/D117
Older
You can’t perform that action at this time.