Skip to content

DeepSharp 0.3.0

Choose a tag to compare

@github-actions github-actions released this 28 Sep 17:53
· 190 commits to main since this release

A pipeline you can look at: every step written as a block of a notebook, and the data at any block shown on
request. Underneath it, the pipeline became strict where it was only polite. Every way of writing one keeps
the same rules, a row is known by what it says rather than where it stands, and a file says everything that is
wrong with it at once, each at its line and column. A model can be asked for more than one answer — the shares
of a whole, labels, a value some rows on — and its predictions come back in their own units; and the columns a
pipeline decides about can be changed from the notebook, saved beside it and taken over again. An application of
your own can host the notebook, too, and DeepSharp's own server shows it in a browser with nothing else installed.

Upgrading from 0.2

  • A file is read with a catalog. PipelineDeclaration.FromJson(json) and PreparedData.FromJson(json)
    are gone; write FromJson(json, StepCatalog.BuiltIn()), and add .WithIndicators() when the file holds
    indicators. Without a catalog a file could only ever hold this package's own verbs, and it was read as if
    nothing else existed.

  • The file names its version, and what was fitted names the steps it was fitted behind. A pipeline is
    written as {"version": 2, "declaration": [...], "fitted": [...]}. Every fitted entry carries a key made
    from its own step and every step above it, so a fit is never used under steps that changed after it was
    learned: a fit spliced under another declaration used to serve a price of 135.7 where 0.9048 was meant. A
    file from 0.2 names no version and is read as the first one. Its declaration still loads, except where a
    verb's meaning has changed since — the three splits and drop.warmup — and those are refused by name
    rather than run the new way. Write them again. The fitted half of a 0.2 file is refused: fit again. A verb
    that is new in this release needs "version": 2 in the file that names it.

  • The columns are declared directly after the source. Declare comes straight after ReadCsv or Read,
    and every other step stands below it. A step written above the schema used to fail a whole run later, as a
    column nobody could find.

  • DeclareStep.Columns is every column the schema names; DeclareStep.Taking is the ones that take part. A
    schema can now name a column and exclude it, so the two can differ. Code that reads Columns to learn which
    columns a pipeline carries should read Taking. A schema written with 0.2 excludes nothing, and there the two
    are the same.

  • The rules hold wherever a step comes from. A pipeline has one source, one schema, one split, one order
    and one output. Every step does something the run acts on. Rows are dropped and put in order before the
    split, never after it. A column is read only where it exists, and only by a step that can work on its kind.
    The chain, the extension point, a hand-written file and a notebook all meet these rules in the same place,
    and a refusal names every fault at once: DeclarationException.Faults, each with its step.

  • A split divides the rows by what they say. Each row is ranked by a digest of its own contents and the
    seed rather than by its place in the file. So the same rows land in the same parts whatever order they
    arrive in, and every copy of a repeated row lands where its first copy does. The parts hold as many rows as
    before, but not the same ones, so what is learned from the training rows moves a little: on the Titanic
    data the median age a fill learns goes from 28 to 29. A split in time keeps every row of one moment on the
    same side of the line.

  • The order of the rows is declared before anything reads it. An indicator, DropWarmUp and a fill that
    carries the previous value forward all read the rows in their order, and nothing said that order was time.
    OrderBy("Date") above them says it; without it they are refused. The same prices listed newest first
    gave a five-day average of 98.352 where the right one is 99.74.

  • A list of columns names each column once. NormaliseRow refuses a column written twice, in C# and in
    a file, a 0.2 file included; 0.2 accepted it and counted the column twice in the row's size. The same holds
    for every new verb that takes a list. An indicator is the exception: its roles may name one price in more
    than one place.

  • A step reads what it is given and nothing else. A key a step does not take is refused, naming the keys
    it does take; it used to be read past and then vanish when the step was written back. A whole number has
    to be whole: a seed of 3.5 was truncated, and is now refused.

  • What learns refuses what is not a number. A fit that meets a not-a-number or an infinity among the
    training values refuses it and names fill.nan, instead of learning a centre from it. fill.nan now
    catches infinities as well, and Batch refuses both. A cell that overflows, such as 1e999, is refused
    where it is read.

  • EncodeCategories() is one step, encode.categories, with one list of categories per column, rather
    than one encode step per column.

  • What a fill counts is counted on the training rows. The number of gaps a fill records is taken from
    the rows it learns from, like everything else it learns: 133 in the Titanic ages, not the 177 in the file.

  • For somebody writing a step of their own. IAssignsParts and ILearnsFromData are merged into
    ISplitStep and IFittedStep: saying a step splits or learns is now the same as doing it. A step
    implements exactly one of the capabilities the run acts on — opening rows, declaring columns, ordering,
    adding, dropping rows or columns, splitting, learning, producing evidence. IPipelineStep<TSelf> asks for
    a Purpose and its Parameters, and the step is written, checked and described from those, so a key is
    typed in one place. The StepCatalog.Register that took a name and a function is gone: a verb arrives with
    its description or not at all. IOpensRows.Open is handed the SourceFolder a relative path is read from.
    What a model is asked to predict is a step that implements INamesTheAnswer, and one that makes its answer
    from the rows implements IMakesTheAnswer. A step that drops rows is shown a served row without the answers
    it awaits: those are gaps in every served row, and no reason to drop one.

Added

  • DeepSharp.Verso.Notebooks — a pipeline written as a notebook. An extension for
    Verso in which every block is one step, written as the step's own
    JSON, and the blocks in the order they stand are the pipeline. A block is edited as text, with the verbs,
    keys and columns offered as you type, or field by field in Verso's properties panel; both write the same
    text. In the schema's form a column set "not taken" stays named, excluded with its kind. Each block shows
    what its step is and does, or every fault at its line. "Show the data here" runs the
    pipeline down to that block and shows the rows there: fifty at a time, each column coloured over the
    training rows, with every row knowing the part the split below will put it in. On the grid every column has
    a box for whether it is in and, when the schema takes it, one for whether it is a category: unticking and
    ticking again write the steps that do it and take them back, a column's kind kept. A column that is not in is
    black, its values left off the page, and a category is one colour, darker for a value the training rows never
    held. "Choose the columns", on the
    schema's block, lists every column of the source with its first values and a box that takes it in or leaves it
    out, marking new the columns the saved file never showed; each row's kind select commits the kind a keyboard walk
    ends on, as one pick from the state the list was drawn in, so a category walked away from and back to remembers
    the kind it was. Above the rows a select picks the kind of output, and each row's output box puts its column into
    that output or takes it out, with a box of its own that takes the output away; the output's own values — rows
    ahead, a return, how many ones, what the shares are shares of — are selects offering only the values the rules keep.
    The list ticks one column or a range: two ticks take in seventy columns, or make them the answer, in one change
    with one kind picked for the range. What the blocks decide about
    their columns is saved beside the notebook, as <notebook>.columns.json, after every change they accept and
    every run of the whole pipeline. An output block is a stage of
    its own, and its form swaps it for any other kind of output. A profile block names, for
    every problem it finds, the step that answers it, and a correlation block is drawn as a heatmap over the
    complete training rows. The toolbar runs the whole pipeline, fitting every step on the training rows, and
    exports it as the same pipeline file the chain writes. Its "Take over the saved columns" lists, at the schema's
    block, every column whose decision taking the saved ones over would change — whether the source may lack it too, and
    what becomes of the columns the schema does not name — with every saved drop the blocks cannot make and why, and
    changes nothing until the list's own box is ticked. Every change to a notebook takes its turn — a gesture on the
    grid or a list, a change in the form, the toolbar's run, export and take-over — so a host that hands them over side
    by side still changes the blocks one at a time, and a change still on its way to a block the change before it
    rewrote is not made. A block added, taken away or moved, a cell turned into another kind, or text typed into a
    block — none of which Verso's editors tell a part of — is caught up with at the next gesture: a grid the blocks no
    longer make is cleared — before it is forgotten, so one a stop leaves on the screen is cleared by the next catch-up —
    and the pipeline handed to C# cells is taken back unless they still make it, while a cell turned into another kind
    keeps what it shows. An application that changes cells itself tells the notebook at once,
    with StepCellType.BlocksChangedAsync, and tells it of a stop with StepCellType.StoppedAsync, which gives the
    notebook back at once and ends once every write let through before the stop has landed; from then on what the
    stopped run asks for writes nothing, and a change whose run was stopped before it began does nothing at all. What a
    block's run writes — its card, a grid and what it measured, the list of the columns, the columns saved beside the
    notebook, what is handed to C# cells — is written whole or not at all, so a stop, Verso's own editors' Stop among
    them, never splits one. C# cells in the same notebook are handed the pipeline as text, under deepsharp.pipeline,
    with the notebook's folder under deepsharp.folder. Saving the
    notebook keeps the steps and leaves out what the blocks show, the data included. It is saved as .verso, the one
    format that keeps a block a block: saving it as Jupyter or Markdown, which keep a cell's text and lose its kind, is
    refused, and so is opening a file of another format whose code cells are steps. It is
    installed from Verso's Extensions panel and runs in Verso's VS Code extension, in the browser editor
    verso serve opens, and inside an application that takes Verso's engine as a dependency, DeepSharp's own server
    among them.

  • DeepSharp.Verso.Api — the notebook in an application of your own. It opens a notebook the way Verso's own
    editors open one — through the serializer for its format, past the guards that run after reading, with the cells
    that are only ever shown rendered drawn — and registers the notebook's parts itself, so a program published as a
    single file, with nothing beside it for Verso to find, still has them. One file is one notebook, however many views
    show it: the same path twice, or two spellings of one path, hand back the same host, and a notebook that cannot be
    opened leaves nothing open behind it. An extension a notebook asks for is refused rather than fetched. Nothing the
    engine falls back on — a kernel, a layout, a theme — is written into a notebook, as Verso's browser editor writes
    none: a file that names none is saved naming none, and a layout the engine lacks is shown in the notebook's own and
    kept by its name. What the layouts arranged, such as the dashboard's tiles, and what the parts keep as settings are
    handed back at open and taken back at every save. Its cells, and what they show, are handed over as plain values.
    Typing a cell's text, running a cell and a click on a control a block drew take their turn one at a time, in the
    order they came; typing or running a cell a change before it rewrote is refused, while a click from its card still
    reaches the part, which knows the block it became, and what a part answers a click with is what the cell shows next.
    A cell is added after another or at the end, of any kind the engine has — code in a language it runs, Markdown, a
    pipeline block and the rest, each saying whether a person writes its text and whether it is shown rendered once it
    has run — and starts empty; a kind named with no language takes the one Verso's editors give it. It is taken away,
    moved past the neighbour it passes, or turned into another kind in one step, keeping its text. Each goes through the
    port the notebook's layout guards, so a layout that does not allow it refuses it, typing included, and every version
    says which layout the notebook is shown in, what it allows and whether it has a properties panel: the dashboard and
    the presentation have none, and there a cell's panel and its fields are refused. As a cell's text is typed, its
    kernel offers what may come next and says what a word means, as Verso's editors ask it: a block's verbs, keys and
    columns, C#'s members. A new notebook is made as one block that reads a CSV file, naming C# as its kernel and when it
    was made, as a .verso file written whole before it appears, and never over a file that is there already. Typing
    into a block tells the notebook at once, so what was worked out from the block as it was is taken back. A run exists
    from the moment it is asked, numbered, and a stop names the run it means. A run that never ends is stopped by a
    fresh kernel — of what ran at the stop, found as the engine finds it: the kernel of the cell's type, else of its
    language, else the notebook's default, and none for a cell that only draws, which takes no C# turn either; for code a
    button runs in no cell, its language, else the default. A stop while nothing runs, between two cells, starts none. A
    stop is one step: unless the run already ended by itself, it takes the run's end, marks the run, tells the notebook —
    what the run left behind writes nothing more, no grid and no pipeline handed to C# cells, and the notebook takes its
    next change at once — and only then takes what runs as the kernel to start afresh, which happens once everything let
    through before the stop has landed; so whether a kernel starts afresh never depends on when a thread happens to run.
    C# runs take their turn across the whole process, so none prints
    into another; a C# run waiting for its turn is told on every version, and a stop ends the wait, so it never runs.
    Every version carries the toolbar: every button the engine has, each asked at that version whether it can be
    pressed — a button on a cell's toolbar for each cell, as Verso's editors ask it, so a page draws a cell's buttons
    where they can be pressed — and a button whose part cannot say is not pressable, and says why. A press is one run,
    and stopped, the cell under way is left behind and no cell the button would still run begins; a file a button hands
    over goes to whoever pressed it, unless the press was stopped, and nothing is written beside the notebook. A cell's
    properties panel is a section
    from every part that has one for it, and a changed field is made by that part. The panel, what a kernel offers and
    what a word means are read beside the notebook's turn, so they answer while a run is under way and through a stop's
    fresh kernel; one that a start afresh overlapped says nothing, even when the kernel failed as it was put away, since the
    kernel it asked is gone. A look — a button asked whether it can be pressed, a panel asked to draw its section — is refused every
    verb, so what is only looked at does nothing to the notebook. A change — a click on a control a cell drew, a changed
    field — runs DeepSharp's blocks as its own; anything else it asks to run is a run of its own, told and stopped as
    any run is, and once that run is stopped nothing else the change asks is done, and what it would answer or hand over
    is dropped. The layouts and the themes the engine
    has are listed, and each is switched by its id in the notebook's turn and saved as the notebook's choice
    (SwitchLayoutAsync, SwitchThemeAsync). What a person does to what a layout drew — a tile moved, resized or run —
    goes to the layout's own part as a change (InteractAsync): a move runs nothing and waits for no C# run, a tile's
    run is a run told and stopped as any, and a file the part hands over goes to whoever acted. The title is changed as
    Verso's editors change it (RetitleAsync): a change every view is told, unsaved until saved, and the name of what
    the notebook is exported as. A notebook is saved the way Verso's editors save it — what a block shows left out, when
    it was last saved stamped in, written whole before it takes the file's place — and saved under another name it is
    that file from then on, what DeepSharp names after it following; a name another open notebook holds is refused.
    Every view of a notebook is told what changed: the notebook as it stands when the view begins, then each change after
    it, numbered — the cells that came or changed as they now stand, and every id in order when cells came, went or
    moved, whatever made the change, a clear and a form included, of which the engine says nothing. A turn tells every
    view what it does while it does it, and one version is made at a time, so no view loses a change. A cell carries
    what Verso's editors draw it with: how it is shown, how many times it ran, how its last run ended and how long it
    took; what it shows carries a failure's name and where it happened, and whether a text came on standard output or
    standard error. What a C# cell displays while it runs reaches a view before the run ends, and every version says
    which run is under way, from its start — the cell that runs or waits, none between a button's cells, since when, and
    the number a stop names. Every version says, too, what the engine runs that no run owns — a block a change runs, and
    what a stop left behind — each until the engine says it ended; whether the notebook differs from its file, worked
    out at open too, so a save is told as a version; what became of its kernels — how many times one was started afresh,
    by a stop or by Verso's Restart Kernel, whether one is being started now, and why the last start failed; the theme
    the notebook chose; what the layout draws of its own — the dashboard's tiles, the presentation's column — as HTML
    with a slot for each cell it shows, or why it could not; and what the notebook says of itself: its title, its
    default kernel, when it was made and last saved, and the version of its format. A view that reads slowly holds one
    change, the latest look at each cell, and never holds the notebook up; it takes the change waiting, or waits for the
    next, whenever it is ready for one. A request about a cell that is gone says from which version on. A file that
    repeats a cell's id gives each repeat an id of its own, as Jupyter's own reader does. Given a grace, a notebook
    closes by itself once no view has shown it for that long, nothing runs or waits in it — nothing a stop left behind
    either — and nothing in it differs from the file it was last saved to — what a block shows never counts, as it is
    never saved; one with changes not yet saved stays open until it is saved. An application can close one notebook at
    once: its run under way is stopped, never waited for, a change under way finishes, and whatever still waits its turn
    is refused; a closed notebook tells its views nothing more, and closing them all stops every notebook's run before
    any of them closes.

  • DeepSharp.Verso.Serve — DeepSharp's own server. The tool deepsharp-serve, installed with
    dotnet tool install --global DeepSharp.Verso.Serve and run beside a notebook or a folder of them, built on
    DeepSharp.Verso.Api and needing nothing else installed. The notebook it serves runs code, so it listens on this
    computer alone, answers only a request carrying the token it said when it started — in the address, or in the cookie
    its first page sets — opens a notebook's connection and makes a change only for its own page, since a browser carries
    that cookie for a page from any port of this computer, answers only under a name of this computer, and serves its own
    page and nothing from the folder it runs in. It says where it is once, then nothing; --port names the port,
    --no-browser opens none, a port already taken stops it before it says anything, and Ctrl+C ends it within moments
    whatever a notebook runs: every run is stopped as a close stops it, and a thread a cell left going does not keep the
    tool alive. It carries a build for .NET 8 and one for .NET 10, and keeps starting on the newer runtime once .NET 8
    is gone.

    A page holds one connection to a notebook, a WebSocket, as Verso's editor holds one for each tab: the notebook as it
    stands first, then each change, numbered. On it the page asks for what a person does — typing, sent as it is typed;
    a run, and a stop naming the run it means, answered at once, past everything asked before it; a click on a block's
    controls; a cell added, taken away, moved or turned into another kind; what a cell's kernel offers and what a word
    means; a toolbar button; a cell's properties, and a field changed, its value handed on in the form Verso's browser
    editor hands it; the layout, the theme and the title; an act on what a layout drew; a save and a close — and each is
    answered to that page alone, never before the change it made. A dropped connection opens again by itself, and the
    notebook is drawn onto what the page shows, so the cell being written keeps its text and its cursor; text typed while
    it is down is kept, marked as not sent yet, and sent once it is back. A notebook the server does not serve is refused
    in words, and a cell another change took away is said to be gone in plain words. A file a button hands over arrives
    as a download in the page that pressed it, under its own name, and nothing is written beside the notebook; in a
    folder, the page makes a new notebook under the bare name of a .verso file, never over a file that is there.

    The page draws the notebook as Verso's editor draws it, and carries everything it draws with, so it fetches nothing
    from anywhere: a block's output; a failure with its name and where it happened, and standard error labelled; JSON as
    a tree, CSV as a table, progress as a bar; Mermaid diagrams, and formulas in Markdown, from Mermaid 11.17.2 and KaTeX
    0.18.9 carried inside it, their notices in the package; a widget in a sandboxed frame of its own, shown as the state
    it saved. A browser that has the page is told it is unchanged rather than sent it again. The dashboard and the
    presentation are drawn as the engine arranges them, the dashboard's tiles moved, resized and run where they stand.
    The panels are Verso's: Metadata, where the title is changed; the chosen cell's Properties, where the layout has
    them, each field described under its label; and View, which switches the layout and the theme, the page drawn in the
    notebook's theme once it chose one. A cell is chosen as in Verso's editor, and a cell shown rendered shows its text
    while it is chosen; cells are added, taken away once you say yes, moved and turned into another kind, each only where
    the notebook's layout allows it, and a cell's bar and the export menu hold only the engine's buttons that can be
    pressed there. A cell's text takes Verso's keys — Shift+Enter, Ctrl+Enter, Alt+Enter, Ctrl+Alt+Enter, Tab and Escape
    — lists what its kernel offers as it is typed, on Ctrl+Space and after a dot or a quote, says what the word at the
    cursor means, and, in a code cell, folds to its first lines. Outputs set hidden say so, and outputs cut short keep to
    the cell's lines. The parameters form draws itself, and its controls reach Verso's parameters part as Verso's own
    script sends them. Beside the notebook's name stand its kernel and how many cells it has, a status of what its
    kernels do, and a dot on Save while anything is unsaved. A run is offered only while none is under way, and its Stop
    stands where Run All stood; a run that waits for another notebook's C# run says so, and a cell a change runs, or a run
    a stop left behind, shows running with nothing to stop. The page says things as Verso's editor does: a notice that
    goes after three seconds — "Saved to titanic.verso", "Kernel restarted" — a sentence that stands while what it says
    holds, and an error in a banner with Dismiss; when the connection goes while something is unsaved, it says that is
    lost if the server has stopped. Close goes at once, even while a run is under way, and asks first only when something
    would be lost; a folder's list, shown even when it holds one notebook, makes a new one.

  • What a model is asked to predict comes in kinds. Target(column) names one column, as before.
    Distribution(columns, scaleBy), target.distribution, names the columns a whole is divided among — a flock
    weighed in seventy bands of fifty grams — whose shares are at least nought and sum to one on every row; named
    with the column saying how many there were, which is none of the shares, the shares come back as how many fell in
    each band. Labels(columns, ones), target.labels, names columns that are each nought or one, with how many ones
    a row holds when that is said, never more than there are labels. Ahead(column, ahead, as), target.ahead,
    makes its answer from a column rows later in the declared order: the price five days on, or the return on today's
    price by then. Reading ahead is held to rules of its own: only an output may, below a split in time whose gap is
    at least as wide, the rows ordered by that split's column alone; and a return stands above every step that
    changes the column it is made from.

  • Every answer is handed over. Batch.AnswerNames and Batch.Answers hold as many numbers a row as the
    output names, in its order; Labels stays the one number a row for an output of one answer. Each answer is
    refused for a gap or a value that is not finite, as a feature is, and each kind of output refuses a row its
    answers could not be: shares that do not sum to one, a label that is neither nought nor one.

  • A split in time can keep a gap. SplitByTime(column, train, validation, gap) and the gap of
    split.byTime set apart the last moments of every part, before each line and at the end, fitted on by nothing
    and handed to nothing: Part.Gap. Without it, the last training rows of an answer read five days ahead learn
    their answers from the rows a model is measured on. The fit writes down how many rows the gap held, and a file
    without a gap is written exactly as before.

  • The way back in your own units, for a part and for served rows. BackToOriginal(predictions, part) puts
    the predictions for a part back, and BackToOriginal(predictions, served, rows) the predictions for served
    rows, each row found again by where it was handed in and checked by its key, ServedBatch.Keys, so rows
    handed in again in another order are refused rather than answered with another row's numbers. A way back
    that needs the row a number belongs to reads it as it was read: a share of a flock comes back as birds by the
    flock's own size, a return as a price by the day's own price. For somebody writing a step of their own,
    IUndoesItself can say which columns it undoes, what a column was made from, and read that row.

  • A column can be left out and brought back as it was. A declared column can say it is excluded: the schema
    still names it, with its kind, but nothing reads it and the source is not asked for it, and a step that reads it
    is refused at that step, in the words "which the schema excludes". Taking it in again is clearing the word, and
    the column comes back with the kind it had. A category can say which kind it was before it became one, so it
    can go back to it. Both are written into the file only where they say something, so a schema that uses neither
    is written exactly as before and keeps its key. A schema that excludes every column it names and keeps none of
    the rest is refused, since no column would take part.

  • What can be done to a column is said once. Including, Excluding and WithKind on a declaration hand back
    the steps a change to one column makes — Including takes several in at once too, with one kind: a column taken
    back in as it was, or out of the drop that left it out; a column no step reads excluded in the schema with its
    kind, and any other dropped below the last step that reads it and the step that makes it; a category that
    remembers the kind it came from. Asked for what already is, each hands back the steps it was given.
    ChoicesFor(columns) says how each column stands — taking part, excluded, dropped, made by a step and which one,
    kept with the rest, or not declared, and whether the source may lack it — and what can be done to it without
    breaking a rule, and KindsFor(column, header) the kinds it can be given, so the answer is never offered to be
    left out and a column a later step scales as a number is never offered to become a category. The schema's own
    WithColumn, WithColumnExcluded and WithColumnKind change one block alone. WithOutput places an output, in
    the place of the one standing or at the end, and a return directly after the split; WithoutOutput takes it
    away; each row says what its column is to the output — an answer comes back to it, or the way back reads it — and
    ChoicesFor names the output it asked about.

  • An output is made under its verb in one place. StepCatalog.Make(verb, stated, carrying) makes a step from
    the verb's template, the values the step it replaces holds under the verb's keys, and the values said, which win;
    it is read as a file's step is, so the verb's own rules hold. The notebook's form swaps one kind of output for
    another through it.

  • What a pipeline decided about its columns can be saved on its own. PipelinePreset holds the schema, the
    columns dropped after the steps that read them, the output, and the source's columns as they were last shown
    — nothing fitted and no rows, so a pipeline written again next month over new rows can take it over.
    PipelinePreset.Of(declaration, header) takes it from a pipeline, ToJson() writes it, and
    PipelinePreset.FromJson(text, catalog) reads it back through the same door as a pipeline file: every fault at
    once at its line and column, and a file from a newer version refused whole. A part it does not hold is not
    written. preset.TakeOver(pipeline, header) makes the steps the preset's decisions make of a pipeline and lists,
    before anything is applied, every column whose decision changes — its standing, its kind, the kind a category was,
    whether the source may lack it — the output before and after, the schema's order and what it does with the columns
    it does not name when those change, the source's columns the preset never showed, and every saved drop it cannot
    make because its column no longer reaches the end, with how that column stands; steps that would break a rule are
    refused with every fault. preset.NewColumns(header) names the source's columns the decisions never showed.
    CsvRowSource.HeaderOf(path) reads a file's header alone. In a chain,
    .Declare(preset, out declared) takes the saved schema at the source and .Output(preset, out taken) the drops and
    the output where the chain names its answer, each saying what it decides before anything runs.

  • Every package runs on .NET 8 as well as .NET 10. Verso's browser editor runs on .NET 8 for as long as
    .NET 8 is installed, and a package built for .NET 10 alone does not load there. Each package now carries a
    build for both, a host takes the one for the runtime it is on, and every test runs on both.

  • OrderBy, order.by — the rows in the order of one or more columns, smallest first, so a time column
    puts the oldest row first. It refuses a gap in a key, and two rows whose keys are equal: nothing says which
    of them came first.

  • Drop, drop.columns — a column taken away, above the split or below it.

  • DropGaps, drop.gaps — the rows with a gap in the named columns, dropped before the split. It
    replaces the With.DropRow that was once planned as a fill strategy: dropping rows after the split would
    quietly change the shares the split promised.

  • FillMissing(column, strategy, refuseAbove: 0.5) — a point beyond which filling is invention. Above
    that share of gaps in the training rows the column is not filled, and the column that marks where the gaps
    were speaks for it.

  • Profile and Correlation, evidence.profile and evidence.correlation — proof declared with the
    pipeline and produced by every run, measured on the rows the split trains on. A profile gives, per column,
    the rows, gaps and values that are not numbers, the distinct values and, for numbers, the smallest, largest,
    mean and median, and names the step that answers each thing it finds. A correlation says which training
    rows it is drawn from and how many were left out. The results are in PreparedData.Evidence, and never in
    the pipeline file: evidence is output, not the pipeline.

  • Pipeline.ViewAt(steps) — the data after any number of steps, with where every row stands: the part
    the split puts it in, dropped before the split reaches it, or undivided when nothing divides it. A range or
    a profile drawn above the split is drawn over the rows the split below will train on, not over all of them.
    MeasuredCategories(column) names the categories those rows hold, by the rule an encoder learns them by.

  • Faults at their line and column. A file that cannot be read throws PipelineFileException, whose
    Faults hold every problem at once, each with its line and column. A verb nobody has heard of is told the
    nearest one there is; a verb another DeepSharp package brings is told which package to reference. A value
    a step refuses is said in the file's words, without the name of a C# parameter the file never had.

  • A description every door is derived from. The JSON Schema of the pipeline file
    (pipeline.schema.json), the
    reference of every verb (VERBS.md), the
    template a new step starts from and the notebook's form are all generated from the steps' own parameters,
    and tests fail the moment one of them falls behind.

  • PreparedData.Served(rows) — rows that arrived after training, replayed and handed over without an
    answer, each saying which of the handed-in rows it is, since a replay can drop rows and put them in order.

  • SourceFolder — one rule for where a relative path is read from: the folder of the file or notebook
    the pipeline came from, or the working directory for one written in code.

  • Table.Duplicates(parts) — the rows that are there more than once, and how many of them landed in
    different parts.

Fixed

  • A run and a replay walk the same steps in the same order. The run lifted every feature above every
    dropped row and the replay dropped nothing, so the same sixty rows came out as fifty-six from one and sixty
    from the other. There is one walk now, in the order the steps were written, and a replay drops and orders
    rows exactly as the run did.

  • The way back is checked on the right rows. With DropWarmUp in the pipeline, the check that a
    prediction leads back to the value that was read compared each row with the one four places further on,
    and refused a pipeline that was fine: 127.83 came back as 133.

  • A row can be served without the answer. Serving refused a row that lacked the column being predicted,
    so a host had to invent an answer to ask the question.

  • An answer made from another column comes back in the units that column was read in. A logarithm taken
    into a new column from a scaled fare came back in the scaled units — 0.014 where the fare was 7.25 — and the
    run's check could not tell, because it compared with the answer as read and an answer a step made was never
    read. The way back now goes on through whatever was done to the column the answer was made from, and the
    check compares with that column as it was read.

  • An answer that is a moment in time no longer stops the run. The check of the way back read it as numbers
    and refused the whole pipeline.

  • Two things 0.2.0 said now hold. A file naming a verb that is not registered is told whether the verb is
    unknown or belongs to a package that is not installed; 0.2.0 gave one message for both. And a declaration
    holds one split: the builder refused a second one, but the extension point and a file could still carry
    it.

  • DropWarmUp refuses a warm-up that covers every row, instead of leaving none.