Seems like the behavior between --no-download and --to-remote is inconsistent. We can fix in this PR or follow up with another one. For --to-remote, the outs metadata is updated with the new info but the workspace remains untouched, while --no-download drop the outs metadata and deletes anything in the workspace.
Originally posted by @dberenbaum in #8752 (comment)