In inspector, nav arrows work even though inspector is not turned on

[lukaszpolowczyk]

Describe the bug

In inspector, nav arrows work even though inspector is not turned on.

Reproduction

1. enable the page with svelte.config.js.

import adapter from '@sveltejs/adapter-auto';

/** @type {import('@sveltejs/kit').Config} */
const config = {
  kit: {
    adapter: adapter()
  },
  vitePlugin: {
    experimental: {
      inspector: {
        toggleKeyCombo: "os-alt",
        holdMode: true,
        showToggleButton: "never"
      }
    }
  }
};

export default config;

2. enter the page
3. press the arrow keys

Effect: a red frame will appear on random elements.

Pressing toggleKeyCombo twice, clears this red frame.

Evidently, inpector does not distinguish whether inspector is in selection mode or not.
It always behaves as if it is in selection mode. Because of this, the Nav arrows work even though the inspector is off.

Logs

No response

System Info

System:
    OS: Linux 5.10 Arch Linux
    CPU: (4) x64 AMD Athlon(tm) X4 950 Quad Core Processor
    Memory: 8.79 GB / 15.07 GB
    Container: Yes
    Shell: 5.1.8 - /bin/bash
  Binaries:
    Node: 18.4.0 - ~/.nvm/versions/node/v18.4.0/bin/node
    Yarn: 1.22.10 - /usr/bin/yarn
    npm: 8.12.1 - ~/.nvm/versions/node/v18.4.0/bin/npm
  Browsers:
    Brave Browser: 100.1.37.109
    Firefox: 102.3.0esr
  npmPackages:
    @sveltejs/adapter-auto: next => 1.0.0-next.76 
    @sveltejs/kit: next => 1.0.0-next.490 
    svelte: ^3.44.0 => 3.50.1 
    vite: ^3.1.0-beta.1 => 3.1.3

Severity

blocking all usage of vite-plugin-svelte