Output cells should not auto-collapse more than once per execution

[burnpanck]

Description

I just upgraded to jupyter notebook 7.0, thus migrating from the "classic" experience to the "jupyterlab" based version. With that migration, my user-experience has degraded significantly. I have a notebook that hosts a number of weakly related interactive tasks that I run sporadically while exploring a problem. The output from some of these cells can sometimes be 100s of lines long. In general, that output being collapsed is great, but while a specific task is running, I want to be able to use the full screen real-estate to monitor the (long-running) task as it goes. Previously, I would un-collapse the output after it started, and then be able to just follow the output, as it is generated. Now, if I attempt to do that, after a several seconds and potentially hundreds of lines of output later, it auto-collapses again. All of a sudden, this leaves me (who followed the output in the uncollapsed state) in the middle of nowhere far away in that document. It appears, previously (Notebook v 6.x), it would only auto-collapse once per execution, but now it does it repeatedly.

(I have been directed here from the JupyterLab repo, where I reported the same issue: jupyterlab/jupyterlab#11760 )

Reproduce

1. Execute a cell containing long-running code that continuously produces output, such as the following:

import time

for k in range(10):
   print("\n".join(f"- {k}.{v}" for v in range(100 if not k else 20)))
   time.sleep(5)

2. Try keep the output area expanded and follow the output at it generates. It will constantly collapse away under your nose. If there is a lot of cells/output following the currently executing cell, you'll have a hard time getting back to that output.

Expected behavior

The output area should stay uncollapsed while I'm following the output.

Context

• Operating System and version: macOS 13.4
• Browser and version: Chome Version 115.0.5790.114 (Official Build) (arm64)
• Jupyter Notebook version: 7.0.2

Screencast

Both examples show the same notebook executed in two different versions of jupyter notebook.
I have added one cell above and below each containing dummy output ((pre-fill) and (post-fill)) to highlight the user experience if that happens in the middle of a long notebook.

Version 7.0.2

Bad.Autocollapse.mov

Version 6.5.5

Good.Autocollapse.mov

[jtpio]

Thanks @burnpanck for reporting 👍

(I have been directed here from the JupyterLab repo, where I reported the same issue: jupyterlab/jupyterlab#11760 )

Thanks! The implementation in Notebook 7 is a bit clunky at the moment and was added here only to mimic the behavior of the classic notebook. For reference the code is available here:

notebook/packages/notebook-extension/src/index.ts

Lines 290 to 375 in b5133c6

	/**
	* A plugin to enable scrolling for outputs by default.
	* Mimic the logic from the classic notebook, as found here:
	* https://github.com/jupyter/notebook/blob/a9a31c096eeffe1bff4e9164c6a0442e0e13cdb3/notebook/static/notebook/js/outputarea.js#L96-L120
	*/
	const scrollOutput: JupyterFrontEndPlugin<void> = {
	id: '@jupyter-notebook/notebook-extension:scroll-output',
	autoStart: true,
	requires: [INotebookTracker],
	optional: [ISettingRegistry],
	activate: async (
	app: JupyterFrontEnd,
	tracker: INotebookTracker,
	settingRegistry: ISettingRegistry | null
	) => {
	const autoScrollThreshold = 100;
	let autoScrollOutputs = true;
	// decide whether to scroll the output of the cell based on some heuristics
	const autoScroll = (cell: CodeCell) => {
	if (!autoScrollOutputs) {
	// bail if disabled via the settings
	return;
	}
	const { outputArea } = cell;
	// respect cells with an explicit scrolled state
	const scrolled = cell.model.getMetadata('scrolled');
	if (scrolled !== undefined) {
	return;
	}
	const { node } = outputArea;
	const height = node.scrollHeight;
	const fontSize = parseFloat(node.style.fontSize.replace('px', ''));
	const lineHeight = (fontSize || 14) * 1.3;
	// do not set via cell.outputScrolled = true, as this would
	// otherwise synchronize the scrolled state to the notebook metadata
	const scroll = height > lineHeight * autoScrollThreshold;
	cell.toggleClass(SCROLLED_OUTPUTS_CLASS, scroll);
	};
	const handlers: { [id: string]: () => void } = {};
	const setAutoScroll = (cell: Cell) => {
	if (cell.model.type === 'code') {
	const codeCell = cell as CodeCell;
	const id = codeCell.model.id;
	autoScroll(codeCell);
	if (handlers[id]) {
	codeCell.outputArea.model.changed.disconnect(handlers[id]);
	}
	handlers[id] = () => autoScroll(codeCell);
	codeCell.outputArea.model.changed.connect(handlers[id]);
	}
	};
	tracker.widgetAdded.connect((sender, notebook) => {
	// when the notebook widget is created, process all the cells
	notebook.sessionContext.ready.then(() => {
	notebook.content.widgets.forEach(setAutoScroll);
	});
	notebook.model?.cells.changed.connect((sender, args) => {
	notebook.content.widgets.forEach(setAutoScroll);
	});
	});
	if (settingRegistry) {
	const loadSettings = settingRegistry.load(scrollOutput.id);
	const updateSettings = (settings: ISettingRegistry.ISettings): void => {
	autoScrollOutputs = settings.get('autoScrollOutputs')
	.composite as boolean;
	};
	Promise.all([loadSettings, app.restored])
	.then(([settings]) => {
	updateSettings(settings);
	settings.changed.connect((settings) => {
	updateSettings(settings);
	});
	})
	.catch((reason: Error) => {
	console.error(reason.message);
	});
	}
	},
	};

Ideally a more robust (and configurable) implementation will be implemented in JupyterLab first (as part of jupyterlab/jupyterlab#11760) so it can be reused in Notebook 7 directly.

[burnpanck]

I'm not sure how this worked in the notebook versions < 7, but I actually had trouble reproducing the auto-collapsing behaviour for the video above after I had tested it before screen-recoding. Maybe it has nothing to do with "once per execution", but rather that with notebook < 7, expanding the collapsed output would set the scroll-mode explicitly, which would then prevent any auto-scroll:

notebook/packages/notebook-extension/src/index.ts

Lines 315 to 319 in b5133c6

	// respect cells with an explicit scrolled state
	const scrolled = cell.model.getMetadata('scrolled');
	if (scrolled !== undefined) {
	return;
	}

So then, the difference in the user-experience really would come from the fact that the new notebook doesn't set that scrolled metadata when the user undoes the auto-scroll, or at least not in a way for the above code to notice.

[burnpanck]

Ideally a more robust (and configurable) implementation will be implemented in JupyterLab first (as part of jupyterlab/jupyterlab#11760) so it can be reused in Notebook 7 directly.

I fear this is going to take some time. Given that the current auto-collapse behaviour is extremely disruptive to some workflows, is there a workaround available, e.g. just completely disabling that behaviour?

[jtpio]

I'm not sure how this worked in the notebook versions < 7

The classic notebook also had some arbitrary thresholds:

notebook/notebook/static/notebook/js/outputarea.js

Lines 96 to 120 in a9a31c0

	/**
	* Should the OutputArea scroll?
	* Returns whether the height (in lines) exceeds the current threshold.
	* Threshold will be OutputArea.minimum_scroll_threshold if scroll_state=true (manually requested)
	* or OutputArea.auto_scroll_threshold if scroll_state='auto'.
	* This will always return false if scroll_state=false (scroll disabled).
	*
	*/
	OutputArea.prototype._should_scroll = function () {
	var threshold;
	if (this.scroll_state === false) {
	return false;
	} else if (this.scroll_state === true) {
	threshold = OutputArea.minimum_scroll_threshold;
	} else {
	threshold = OutputArea.auto_scroll_threshold;
	}
	if (threshold <=0) {
	return false;
	}
	// line-height from https://stackoverflow.com/questions/1185151
	var fontSize = this.element.css('font-size') || '14px';
	var lineHeight = Math.floor((parseFloat(fontSize.replace('px','')) || 14) * 1.3);
	return (this.element.height() > threshold * lineHeight);
	};

just completely disabling that behaviour

Sure. You can disable the corresponding plugin:

jupyter labextension disable @jupyter-notebook/notebook-extension:scroll-output

[mvoelk]

For those who end up here because they want to completely deactivate the auto-scroll behavior. Go to "Setting" > "Settings Editor" > "Jupyter Notebook Notebook" and uncheck "Auto Scroll Outputs".

Note, the menu item "Settings Editor" is not available in the notebook view, only in the file browser.