internal: remove secret hidden internals and introduce new way to call native code from js

[paperdave]

What does this PR do?

ooooh magic javascript preprocessor shenanigans.

This PR removes $lazy, and replaces it with two alternatives: $cpp and $zig. These both take a filename and a function name, and they will call the function defined with no arguments.

• Preprocessor Transform
• Codegen
• New impl of @lazy

Examples:

With this done, I will also be removing the following definitions

• globalThis[Symbol.for("Bun.lazy")]
• Bun.fs
• Bun._Path
• Bun._Os()

The API Bun.DO_NOT_USE_OR_YOU_WILL_BE_FIRED_mimalloc_dump has been renamed to Bun.unsafe.dumpMimalloc(). While left undocumented, this will hopefully reduce the number of people fired from bun.

If internals are required for unit testing, two mechanisms will be provided. In order for either to work, the environment variables BUN_GARBAGE_COLLECTOR_LEVEL and BUN_FEATURE_FLAG_INTERNAL_FOR_TESTING must be set, or you must be running in a debug build.

From there, runtime versions of $zig and $cpp will exist on the global object. These work the same as the bundled ones, but only map to symbols that have a used binding in the src/js. Perhaps we can add a way to add additional bindings that aren't used in src/js but able to be exposed for testing, such as the syntax highlighter.

From there, you can use bun:internal-for-testing, which currently contains three declarations:

Extra Tasks

• Internal Typescript Definitions and Documentation
• Merge bundle-functions and bundle-modules once and for all
• Finish porting old code to new API

[Jarred-Sumner]

Exciting

We can keep DO_NOT_USE_OR_YOU_WILL_BE_FIRED_mimalloc_dump imo

[Jarred-Sumner]

v fancy

[github-actions]

❌ @paperdave 4 files with test failures on bun-darwin-x64:

• test/cli/install/registry/bun-install-registry.test.ts 2 failing
• test/integration/next/default-pages-dir/test/next-build.test.ts SIGTERM
• test/js/bun/test/test-test.test.ts 1 failing
• test/js/third_party/prisma/prisma.test.ts SIGABRT

View test output

^{#d8aca187b79deed863adb8a0f51348e04d6a8824}

[github-actions]

❌🪟 @paperdave, there are 9 test regressions on Windows x86_64

• test\cli\install\bunx.test.ts
• test\cli\install\migration\migrate.test.ts
• test\cli\install\registry\bun-install-registry.test.ts
• test\js\node\dns\node-dns.test.js
• test\js\third_party\esbuild\esbuild-child_process.test.ts
• test\js\web\fetch\body-stream.test.ts
• test\regression\issue\09041.test.ts
• test\js\third_party\postgres\postgres.test.ts
• test\js\web\workers\worker.test.ts

Full Test Output

[Jarred-Sumner]

merge conflicts

[paperdave]

this PR is marked breaking as it removes some hidden apis. we will merge this before 1.1 but before the last 1.0.x

[github-actions]

❌ @paperdave 3 files with test failures on linux-x64-baseline:

• test/integration/next/default-pages-dir/test/next-build.test.ts SIGTERM
• test/js/node/watch/fs.watchFile.test.ts 1 failing
• test/napi/napi.test.ts 6 failing

View test output

^{#d8aca187b79deed863adb8a0f51348e04d6a8824}

[paperdave]

test/js/bun/wasm/wasi.test.js test i think is real

[github-actions]

❌ @paperdave 3 files with test failures on linux-x64:

• test/integration/next/default-pages-dir/test/next-build.test.ts SIGTERM
• test/js/node/http2/node-http2.test.js 1 failing
• test/napi/napi.test.ts 6 failing

View test output

^{#d8aca187b79deed863adb8a0f51348e04d6a8824}