startDevWorker - Milestone 1 (#4497)

* Revert "Revert "startDevWorker - Milestone 1" (#4171)"

This reverts commit 88f15f6.

* fix: don't show logs to ProxyWorker(s)
unless log level is debug

* fix: show console.log's in remote mode
remote inspector websocket upgrade request required auth headers
so use `fetch` with `Upgrade: websocket` header instead of `new WebSocket`

* use miniflare verbose mode only if debug log level

* use single Miniflare instance
for (Inspector)ProxyWorker

* port: clear remote runtime logs
upon UserWorker restarts

* default unstable_dev inspectorPort to 0

* parallelise cleanup to minimise chance of hanging
previously, sequential cleanups fail to fully cleanup if earlier steps in the sequence fail

* ensure InspectorProxyWorker unsafeDirectPort is set

* don't use file-system for (Inspector)ProxyWorker DOs

* prevent eviction of the Durable Objects
with (Inspector)ProxyWorker

* remove miniflare workaround for parallel requests

* considerations for race between control messages and user fetches

* use port: undefined vs 0 for UserWorker
to force different port across reloads to workaround workerd bug on Windows

* Don't try to parse `node-internal:* import specifiers

* improve InspectorProxyWorker debug logs

* only proxy consoleAPICalled events in remote mode

* enable consoleAPICalled events proxying if local mode AND service-worker format

* fix userWorkerInnerUrlOverrides host/hostname/port
mainly, base innerUrl off of request.url not userWorkerUrl

* use ProxyWorker ip/port for DEV_SERVER_READY event
instead of UserWorker ip/port

* always disable the UserWorker miniflare pretty error
using the MF-Disable-Pretty-Error header on the UserWorker request

the ProxyWorker will still interpret the json error response
depending on its own MF-Disable-Pretty-Error header

* recover from 'address in use' errors
by trying to start on a random port

* run unit tests in parallel again
by removing --runInBand flag for jest

* add handleRuntimeStdio option to ProxyWorker miniflare instance

* expand containsHexStack check for windows

* logger.debug runtime websocket errors from InspectorProxyWorker
+ remove miniflare log.error overrides no longer needed

* log workerd warnings with logger.warn
not logger.error/info

* enable Cloudflare Access auth for remote previews

* only send Runtime.discardConsoleEntries if currently connected to runtime

---------

Co-authored-by: Samuel Macleod <smacleod@cloudflare.com>
Co-authored-by: MrBBot <bcoll@cloudflare.com>

.changeset/serious-toys-repeat.md

@@ -0,0 +1,12 @@
+---
+"wrangler": minor
+---
+
+Reintroduces some internal refactorings of wrangler dev servers (including `wrangler dev`, `wrangler dev --remote`, and `unstable_dev()`).
+
+These changes were released in 3.13.0 and reverted in 3.13.1 -- we believe the changes are now more stable and ready for release again.
+
+There are no changes required for developers to opt-in. Improvements include:
+
+- fewer 'address in use' errors upon reloads
+- upon config/source file changes, requests are buffered to guarantee the response is from the new version of the Worker

fixtures/dev-env/.gitignore

@@ -0,0 +1 @@
+dist

fixtures/dev-env/package.json

@@ -0,0 +1,25 @@
+{
+	"name": "dev-env",
+	"version": "1.0.1",
+	"private": true,
+	"description": "",
+	"license": "ISC",
+	"author": "",
+	"main": "src/index.js",
+	"scripts": {
+		"test": "npx vitest run",
+		"test:ci": "npx vitest run",
+		"test:watch": "npx vitest",
+		"type:tests": "tsc -p ./tests/tsconfig.json"
+	},
+	"devDependencies": {
+		"@types/ws": "^8.5.7",
+		"@cloudflare/workers-tsconfig": "workspace:^",
+		"get-port": "^7.0.0",
+		"miniflare": "3.20231025.1",
+		"undici": "^5.23.0",
+		"wrangler": "workspace:*",
+		"ws": "^8.14.2"
+	},
+	"dependencies": {}
+}