Allow configuring any notebook to have transient outputs

[amunger]

Transient outputs

Current state

Definition

In the VS Code notebook model, transientOutputs is a property in the TransientOptions interface. It's a boolean flag that determines whether notebook cell outputs should be treated as transient or persisted.

Interface

The TransientOptions interface is defined in notebookCommon.ts and includes:

export interface TransientOptions {
  readonly transientOutputs: boolean;
  readonly transientCellMetadata: TransientCellMetadata;
  readonly transientDocumentMetadata: TransientDocumentMetadata;
  readonly cellContentMetadata: CellContentMetadata;
}

Usage in the Code

When creating a snapshot of a notebook in the NotebookTextModel.createSnapshot method, the code checks transientOptions.transientOutputs:

cellData.outputs = !transientOptions.transientOutputs ? cell.outputs : [];

If transientOutputs is true, the cell outputs are not included in the snapshot (represented as an empty array).
If transientOutputs is false, the cell outputs are included in the snapshot.

Configuration

Notebook serializers set these options when they register with VS Code.
In the VS Code API, this is exposed through the NotebookDocumentContentOptions interface:

export interface NotebookDocumentContentOptions {
  transientOutputs?: boolean;
  transientCellMetadata?: { [key: string]: boolean | undefined };
  transientDocumentMetadata?: { [key: string]: boolean | undefined };
}

Control Mechanism

Extension authors control this behavior when registering a notebook serializer:

vscode.workspace.registerNotebookSerializer(notebookType, serializer, {
  transientOutputs: true, // or false
  // other options
});

When transientOutputs is set to true, notebook outputs are not saved in the file, and changes to outputs don't mark the document as dirty.
When false, outputs are persisted in the file and changes to outputs mark the document as dirty.

Default Value

The default value for transientOutputs is false, meaning outputs are persisted by default.
This can be seen in the NotebookTextModel class where it initializes:

Impact

This setting affects the notebook's dirty state: when outputs are transient, changes to outputs don't mark the notebook as dirty.
It affects whether outputs are included when saving the notebook file.
It determines whether output changes are considered in the diff editor.
So, in summary, the transientOutputs configuration for notebooks is controlled by extensions that register notebook serializers through the NotebookDocumentContentOptions interface. This determines whether cell outputs should be saved with the notebook file or treated as transient (not saved). The default behavior is to persist outputs (transientOutputs: false).

Making Transient outputs configurable

Requirements

A user should be able to disable saving or backing up cell outputs in a notebook to reduce the amount of data that needs to be serialized while saving the document. This should be a user setting that will disable saving outputs for all notebooks, and also a button in the notebook toolbar that will run a new command to disable saving outputs for just that notebook by setting a value in the notebook's metadata that is specific to VS code.

For notebooks that already have transient outputs due to the setting in the serializer, the setting should have no effect, nor should the button/command be enabled.

Required code changes

1. Add new user setting for notebook transient outputs

   Add a new setting to notebookCommon.ts in the NotebookSetting object:

   export const NotebookSetting = {
     // ...existing settings...
     transientOutputs: 'notebook.transientOutputs',
   };

   Define this setting in the VS Code configuration schema (in notebookConfigurationDefaults):

   [NotebookSetting.transientOutputs]: {
     description: nls.localize('notebook.transientOutputs', "When enabled, notebook cell outputs won't be saved with the notebook document. This can reduce file size and improve performance for large notebooks."),
     type: 'boolean',
     default: false,
     scope: ConfigurationScope.RESOURCE
   }

2. Create new notebook command for toggling transient outputs

   Define a new command ID in the appropriate constants file:

   export const TOGGLE_NOTEBOOK_TRANSIENT_OUTPUTS = 'notebook.toggleTransientOutputs';

   Register the command handler in the notebook contribution file:

   registerAction2(class ToggleTransientOutputsAction extends Action2 {
     constructor() {
       super({
         id: TOGGLE_NOTEBOOK_TRANSIENT_OUTPUTS,
         title: { value: nls.localize('notebook.toggleTransientOutputs', "Toggle Transient Outputs"), original: 'Toggle Transient Outputs' },
         f1: true,
         menu: {
           id: MenuId.NotebookToolbar,
           when: ContextKeyExpr.and(
             NOTEBOOK_EDITOR_FOCUSED,
             ContextKeyExpr.not('notebookOutputsTransient')
           ),
           group: 'notebook/cell/execute'
         }
       });
     }
   
     async run(accessor: ServicesAccessor, context?: INotebookActionContext): Promise<void> {
       const notebookEditor = getActiveNotebookEditor(accessor);
       if (!notebookEditor) {
         return;
       }
       
       const model = notebookEditor.textModel;
       if (!model) {
         return;
       }
       
       const newMetadata = {
         ...model.metadata,
         transientOutputs: !model.metadata.transientOutputs
       };
       
       const edit: ICellEditOperation = {
         editType: CellEditType.DocumentMetadata,
         metadata: newMetadata
       };
       
       await model.applyEdits([edit], true, undefined, () => undefined, undefined, true);
     }
   });

3. Update NotebookTextModel to handle the setting and metadata

   Modify the createSnapshot method in notebookTextModel.ts to check both serializer options and user settings/metadata:

   createSnapshot(options: INotebookSnapshotOptions): NotebookData {
     const transientOptions = options.transientOptions ?? this.transientOptions;
     const data: NotebookData = {
       metadata: filter(this.metadata, key => !transientOptions.transientDocumentMetadata[key]),
       cells: [],
     };
   
     // Check if outputs should be transient based on multiple sources
     const shouldTreatOutputsAsTransient = 
       transientOptions.transientOutputs || // From serializer
       !!this.metadata.transientOutputs || // From notebook metadata
       this._configurationService.getValue<boolean>(NotebookSetting.transientOutputs, { resource: this.uri }); // From user setting
   
     let outputSize = 0;
     for (const cell of this.cells) {
       const cellData: ICellDto2 = {
         cellKind: cell.cellKind,
         language: cell.language,
         mime: cell.mime,
         source: cell.getValue(),
         outputs: [],
         internalMetadata: cell.internalMetadata
       };
   
       if (options.context === SnapshotContext.Backup && options.outputSizeLimit > 0) {
         cell.outputs.forEach(output => {
           output.outputs.forEach(item => {
             outputSize += item.data.byteLength;
           });
         });
         if (outputSize > options.outputSizeLimit) {
           throw new Error('Notebook too large to backup');
         }
       }
   
       cellData.outputs = !shouldTreatOutputsAsTransient ? cell.outputs : [];
       cellData.metadata = filter(cell.metadata, key => !transientOptions.transientCellMetadata[key]);
   
       data.cells.push(cellData);
     }
   
     return data;
   }

4. Add context key for UI state

   Register a new context key for notebook outputs transient state:

   export const NOTEBOOK_OUTPUTS_TRANSIENT = new RawContextKey<boolean>('notebookOutputsTransient', false);

   Update this context key when notebook editor is created or metadata changes:

   private _updateTransientOutputsState(): void {
     if (!this.notebookDocument) {
       return;
     }
   
     const isTransient = 
       !!this.notebookDocument.metadata.transientOutputs ||
       this._configurationService.getValue<boolean>(NotebookSetting.transientOutputs, { resource: this.notebookDocument.uri }) ||
       this._notebookService.getNotebookSerializer(this.viewType)?.options.transientOutputs === true;
   
     this._notebookOutputsTransientContext.set(isTransient);
   }

5. Update toolbar with visual indicator

   Add a toolbar item with icon that indicates current transient outputs state:

   {
     id: 'workbench.notebook.toolbar.transientOutputs',
     name: nls.localize('notebook.transientOutputsIndicator', "Transient Outputs"),
     tooltip: nls.localize('notebook.transientOutputsIndicator.tooltip', "When enabled, notebook cell outputs are not saved with the document"),
     icon: Codicon.saveAll.with({ strikethrough: true }),
     when: ContextKeyExpr.equals('notebookOutputsTransient', true),
     group: 'status'
   }

6. Update dirty state tracking

   Modify the cell output change handling to consider the transient state:

   private _handleCellOutputChange(cellHandle: number): void {
     // Check if we should mark as dirty
     const isOutputTransient = 
       this.transientOptions.transientOutputs ||
       !!this.metadata.transientOutputs ||
       this._configurationService.getValue<boolean>(NotebookSetting.transientOutputs, { resource: this.uri });
   
     if (!isOutputTransient) {
       this._increaseVersionId(true);
       // Trigger dirty state change
     }
   }