From e4de6235b3a7f44b7e346b065166579e973b1d4e Mon Sep 17 00:00:00 2001 From: Timo Stamm Date: Wed, 6 Dec 2023 23:04:59 +0100 Subject: [PATCH 01/11] Add CommonJS support as a plugin option --- docs/writing_plugins.md | 45 +++-- packages/protoc-gen-es/README.md | 21 +- packages/protoc-gen-es/src/javascript.ts | 4 +- packages/protoc-gen-es/src/typescript.ts | 4 +- .../src/protoc-gen-twirp-es.ts | 11 +- .../src/byo-transpile.test.ts | 95 +++++++++ .../src/file-export-decl.test.ts | 101 +++++++++ .../protoplugin-test/src/import_style.test.ts | 125 ++++++++++++ packages/protoplugin/README.md | 4 +- packages/protoplugin/src/create-es-plugin.ts | 191 ++---------------- .../src/ecmascript/export-declaration.ts | 32 +++ .../src/ecmascript/generated-file.ts | 131 ++++++++++-- packages/protoplugin/src/ecmascript/index.ts | 10 +- .../protoplugin/src/ecmascript/parameter.ts | 182 +++++++++++++++++ packages/protoplugin/src/ecmascript/schema.ts | 65 +++--- .../protoplugin/src/ecmascript/transpile.ts | 5 + 16 files changed, 761 insertions(+), 265 deletions(-) create mode 100644 packages/protoplugin-test/src/byo-transpile.test.ts create mode 100644 packages/protoplugin-test/src/file-export-decl.test.ts create mode 100644 packages/protoplugin-test/src/import_style.test.ts create mode 100644 packages/protoplugin/src/ecmascript/export-declaration.ts create mode 100644 packages/protoplugin/src/ecmascript/parameter.ts diff --git a/docs/writing_plugins.md b/docs/writing_plugins.md index 86e53701b..7b0e23b95 100644 --- a/docs/writing_plugins.md +++ b/docs/writing_plugins.md @@ -97,7 +97,12 @@ In most cases, implementing the `generateTs` function only and letting the plugi As mentioned, if you do not provide a `generateJs` and/or a `generateDts` function and either `js` or `dts` is specified as a target out, the plugin framework will use its own TypeScript compiler to generate these files for you. This process uses a stable version of TypeScript with lenient compiler options so that files are generated under most conditions. However, if this is not sufficient, you also have the option of providing your own `transpile` function, which can be used to override the plugin framework's transpilation process. ```ts -transpile(fileInfo: FileInfo[], transpileJs: boolean, transpileDts: boolean): FileInfo[] +transpile( + fileInfo: FileInfo[], + transpileJs: boolean, + transpileDts: boolean, + jsImportStyle: "module" | "legacy_commonjs", +): FileInfo[] ``` The function will be invoked with an array of `FileInfo` objects representing the TypeScript file content @@ -334,36 +339,34 @@ The natural instinct would be to simply print your own import statements as `f.p ### Exporting -Working with exports is accomplished via the `export` function on the generated file object. Let's walk through an example: +To export a declaration from you code, use `exportDecl`: -Suppose you generate a validation function for every message. If you have a nested message, such as: - -```proto -message Bar { - Foo foo = 1; -} +```typescript +const name = "foo"; +f.exportDecl("const", name); ``` -You may want to import and use the validation function generated for message `Foo` when generating the code for message `Bar`. To generate the validation function, you would use `export` as follows: +This method takes two arguments: +1. The declaration, for example `const`, `enum`, `abstract class`, or anything + you might need. +2. The name of the declaration, which is also used for the export. -```ts -const fn = f.export("validateFoo"); -f.print("function ", fn, "() {"); -f.print(" return true;"); -f.print("}"); +The return value of the method can be passed to `print`: + +```typescript +const name = "foo"; +f.print(f.exportDecl("const", name), " = 123;"); ``` -Note that `export` returns an `ImportSymbol` that can then be used by another dependency. The trick is to store this `ImportSymbol` and use it when you generate the validation function for `Bar`. Storing the symbol is as simple as putting it in a global map: +The example above will generate the following code: -```ts -const exportMap = new Map() +```typescript +export const foo = 123; ``` -That way, when you need to use it for `Bar`, you can simply access the map: +If the plugin option `js_import_style=legacy_commonjs` is set, the example will +automatically generate the correct export for CommonJS. -```ts -const fooValidationFn = exportMap.get(bar); // bar is of type DescMessage -``` ### Parsing plugin options diff --git a/packages/protoc-gen-es/README.md b/packages/protoc-gen-es/README.md index 0235cc8cd..2cdf57c87 100644 --- a/packages/protoc-gen-es/README.md +++ b/packages/protoc-gen-es/README.md @@ -98,12 +98,25 @@ By default, [protoc-gen-es](https://www.npmjs.com/package/@bufbuild/protoc-gen-e uses a `.js` file extensions in import paths, even in TypeScript files. This is unintuitive, but necessary for [ECMAScript modules in Node.js](https://www.typescriptlang.org/docs/handbook/esm-node.html). -Unfortunately, not all bundlers and tools have caught up yet, and Deno -requires `.ts`. With this plugin option, you can replace `.js` extensions +Unfortunately, not all bundlers and tools have caught up yet, and Deno +requires `.ts`. With this plugin option, you can replace `.js` extensions in import paths with the given value. For example, set -- `import_extension=none` to remove the `.js` extension -- `import_extension=.ts` to replace the `.js` extension with `.ts` +- `import_extension=none` to remove the `.js` extension. +- `import_extension=.ts` to replace the `.js` extension with `.ts`. + +### `js_import_style` + +By default, [protoc-gen-es](https://www.npmjs.com/package/@bufbuild/protoc-gen-es) +(and all other plugins based on [@bufbuild/protoplugin](https://www.npmjs.com/package/@bufbuild/protoplugin)) +generate ECMAScript `import` and `export` statements. For use cases where +CommonJS is difficult to avoid, this option can be used to generate CommonJS +`require()` calls. + +Possible values: +- `js_import_style=module` generate ECMAScript `import` / `export` statements - + the default behavior. +- `js_import_style=legacy_commonjs` generate CommonJS `require()` calls. ### `keep_empty_files=true` diff --git a/packages/protoc-gen-es/src/javascript.ts b/packages/protoc-gen-es/src/javascript.ts index 1a9bdcb60..660adf744 100644 --- a/packages/protoc-gen-es/src/javascript.ts +++ b/packages/protoc-gen-es/src/javascript.ts @@ -52,7 +52,7 @@ export function generateJs(schema: Schema) { function generateEnum(schema: Schema, f: GeneratedFile, enumeration: DescEnum) { const protoN = getNonEditionRuntime(schema, enumeration.file); f.print(makeJsDoc(enumeration)); - f.print("export const ", enumeration, " = ", protoN, ".makeEnum(") + f.print(f.exportDecl("const", enumeration), " = ", protoN, ".makeEnum(") f.print(` "`, enumeration.typeName, `",`) f.print(` [`) if (enumeration.sharedPrefix === undefined) { @@ -74,7 +74,7 @@ function generateEnum(schema: Schema, f: GeneratedFile, enumeration: DescEnum) { function generateMessage(schema: Schema, f: GeneratedFile, message: DescMessage) { const protoN = getNonEditionRuntime(schema, message.file); f.print(makeJsDoc(message)); - f.print("export const ", message, " = ", protoN, ".makeMessageType(") + f.print(f.exportDecl("const", message), " = ", protoN, ".makeMessageType(") f.print(` `, literalString(message.typeName), `,`) if (message.fields.length == 0) { f.print(" [],") diff --git a/packages/protoc-gen-es/src/typescript.ts b/packages/protoc-gen-es/src/typescript.ts index 0fe9b19db..dd56b9897 100644 --- a/packages/protoc-gen-es/src/typescript.ts +++ b/packages/protoc-gen-es/src/typescript.ts @@ -53,7 +53,7 @@ export function generateTs(schema: Schema) { function generateEnum(schema: Schema, f: GeneratedFile, enumeration: DescEnum) { const protoN = getNonEditionRuntime(schema, enumeration.file); f.print(makeJsDoc(enumeration)); - f.print("export enum ", enumeration, " {"); + f.print(f.exportDecl("enum", enumeration), " {"); for (const value of enumeration.values) { if (enumeration.values.indexOf(value) > 0) { f.print(); @@ -84,7 +84,7 @@ function generateMessage(schema: Schema, f: GeneratedFile, message: DescMessage) JsonValue } = schema.runtime; f.print(makeJsDoc(message)); - f.print("export class ", message, " extends ", Message, "<", message, "> {"); + f.print(f.exportDecl("class", message), " extends ", Message, "<", message, "> {"); for (const member of message.members) { switch (member.kind) { case "oneof": diff --git a/packages/protoplugin-example/src/protoc-gen-twirp-es.ts b/packages/protoplugin-example/src/protoc-gen-twirp-es.ts index ba6497d10..983f6db2d 100755 --- a/packages/protoplugin-example/src/protoc-gen-twirp-es.ts +++ b/packages/protoplugin-example/src/protoc-gen-twirp-es.ts @@ -16,13 +16,13 @@ import { createEcmaScriptPlugin, runNodeJs } from "@bufbuild/protoplugin"; import { version } from "../package.json"; +import type { Schema } from "@bufbuild/protoplugin/ecmascript"; import { literalString, - makeJsDoc, localName, + makeJsDoc, } from "@bufbuild/protoplugin/ecmascript"; import { MethodKind } from "@bufbuild/protobuf"; -import type { Schema } from "@bufbuild/protoplugin/ecmascript"; const protocGenTwirpEs = createEcmaScriptPlugin({ name: "protoc-gen-twirp-es", @@ -39,19 +39,16 @@ function generateTs(schema: Schema) { Message, JsonValue } = schema.runtime; - // Convert the Message ImportSymbol to a type-only ImportSymbol - const MessageAsType = Message.toTypeOnly(); for (const service of file.services) { - const localServiceName = localName(service); f.print(makeJsDoc(service)); - f.print("export class ", localServiceName, "Client {"); + f.print(f.exportDecl("class", localName(service) + "Client"), " {"); f.print(" private baseUrl: string = '';"); f.print(); f.print(" constructor(url: string) {"); f.print(" this.baseUrl = url;"); f.print(" }"); f.print(); - f.print(" async request>("); + f.print(" async request>("); f.print(" service: string,"); f.print(" method: string,"); f.print(" contentType: string,"); diff --git a/packages/protoplugin-test/src/byo-transpile.test.ts b/packages/protoplugin-test/src/byo-transpile.test.ts new file mode 100644 index 000000000..5518f3b6d --- /dev/null +++ b/packages/protoplugin-test/src/byo-transpile.test.ts @@ -0,0 +1,95 @@ +// Copyright 2021-2023 Buf Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +import { describe, expect, test } from "@jest/globals"; +import { CodeGeneratorRequest } from "@bufbuild/protobuf"; +import { createEcmaScriptPlugin } from "@bufbuild/protoplugin"; +import type { FileInfo } from "@bufbuild/protoplugin/ecmascript"; + +describe("bring your own transpile", () => { + test("does not transpile target=ts", () => { + const lines = testGenerate("target=ts"); + expect(lines).toStrictEqual(["fake typescript source"]); + }); + + test("transpiles to target js", () => { + const lines = testGenerate("target=js"); + expect(lines).toStrictEqual(["fake transpiled to js"]); + }); + + test("transpiles to target dts", () => { + const lines = testGenerate("target=dts"); + expect(lines).toStrictEqual(["fake transpiled to dts"]); + }); + + function testGenerate(parameter: string): string[] { + const plugin = createEcmaScriptPlugin({ + name: "test", + version: "v1", + generateTs: (schema) => { + const f = schema.generateFile("test.ts"); + f.print("fake typescript source"); + }, + transpile: (files, transpileJs, transpileDts) => { + const out: FileInfo[] = []; + for (const file of files) { + if (transpileJs) { + switch (file.content) { + case "fake typescript source\n": + out.push({ + name: "test.js", + preamble: file.preamble, + content: "fake transpiled to js\n", + }); + break; + default: + out.push({ + name: "test.js", + preamble: file.preamble, + content: "failed to transpile to js\n", + }); + break; + } + } + if (transpileDts) { + switch (file.content) { + case "fake typescript source\n": + out.push({ + name: "test.d.ts", + preamble: file.preamble, + content: "fake transpiled to dts\n", + }); + break; + default: + out.push({ + name: "test.js", + preamble: file.preamble, + content: "failed to transpile to js\n", + }); + break; + } + } + } + return out; + }, + }); + const req = new CodeGeneratorRequest({ + parameter, + }); + const res = plugin.run(req); + expect(res.file.length).toBeGreaterThanOrEqual(1); + const content = res.file[0]?.content ?? ""; + return content.trim().split("\n"); + } +}); diff --git a/packages/protoplugin-test/src/file-export-decl.test.ts b/packages/protoplugin-test/src/file-export-decl.test.ts new file mode 100644 index 000000000..f0b03cd12 --- /dev/null +++ b/packages/protoplugin-test/src/file-export-decl.test.ts @@ -0,0 +1,101 @@ +// Copyright 2021-2023 Buf Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +import { describe, expect, test } from "@jest/globals"; +import { UpstreamProtobuf } from "upstream-protobuf"; +import type { DescEnum, DescMessage } from "@bufbuild/protobuf"; +import { CodeGeneratorRequest } from "@bufbuild/protobuf"; +import type { Schema } from "@bufbuild/protoplugin"; +import { createEcmaScriptPlugin } from "@bufbuild/protoplugin"; +import type { GeneratedFile } from "@bufbuild/protoplugin/ecmascript"; + +describe("file exportDecl", () => { + test("works as documented", async () => { + const lines = await testGenerate((f) => { + const name = "foo"; + f.print(f.exportDecl("const", name), " = 123;"); + }); + expect(lines).toStrictEqual(["export const foo = 123;"]); + }); + + test("declaration can be empty string", async () => { + const lines = await testGenerate((f) => { + f.print("const foo = 123;"); + f.print(f.exportDecl("", "foo"), ";"); + }); + expect(lines).toStrictEqual(["const foo = 123;", "export foo;"]); + }); + + test("accepts DescMessage as name", async () => { + const lines = await testGenerate((f, descMessage) => { + f.print(f.exportDecl("const", descMessage), " = 123;"); + }); + expect(lines).toStrictEqual(["export const SomeMessage = 123;"]); + }); + + test("accepts DescEnum as name", async () => { + const lines = await testGenerate((f, _, descEnum) => { + f.print(f.exportDecl("const", descEnum), " = 123;"); + }); + expect(lines).toStrictEqual(["export const SomeEnum = 123;"]); + }); + + async function testGenerate( + gen: ( + f: GeneratedFile, + descMessage: DescMessage, + descEnum: DescEnum, + ) => void, + ) { + const plugin = createEcmaScriptPlugin({ + name: "test", + version: "v1", + generateTs: generateAny, + generateJs: generateAny, + generateDts: generateAny, + }); + + function generateAny(schema: Schema) { + gen( + schema.generateFile("test.ts"), + schema.files[0].messages[0], + schema.files[0].enums[0], + ); + } + + const upstream = new UpstreamProtobuf(); + const protoFiles = { + "x.proto": ` + syntax="proto3"; + message SomeMessage {} + enum SomeEnum { + SOME_ENUM_UNRECOGNIZED = 0; + SOME_ENUM_A = 1; + } + `, + }; + const req = CodeGeneratorRequest.fromBinary( + await upstream.createCodeGeneratorRequest(protoFiles, { + parameter: "target=ts", + }), + ); + expect(req.protoFile.length).toBe(1); + expect(req.protoFile[0]?.messageType.length).toBe(1); + expect(req.protoFile[0]?.enumType.length).toBe(1); + const res = plugin.run(req); + expect(res.file.length).toBeGreaterThanOrEqual(1); + const content = res.file[0]?.content ?? ""; + return content.trim().split("\n"); + } +}); diff --git a/packages/protoplugin-test/src/import_style.test.ts b/packages/protoplugin-test/src/import_style.test.ts new file mode 100644 index 000000000..80d94dc58 --- /dev/null +++ b/packages/protoplugin-test/src/import_style.test.ts @@ -0,0 +1,125 @@ +// Copyright 2021-2023 Buf Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +import { describe, expect, test } from "@jest/globals"; +import { CodeGeneratorRequest } from "@bufbuild/protobuf"; +import type { Schema } from "@bufbuild/protoplugin"; +import { createEcmaScriptPlugin } from "@bufbuild/protoplugin"; + +describe("js_import_style", () => { + const linesEsm = [ + `import { third } from "party";`, + ``, + `const thirdParty = third;`, + `export class MyClass {}`, + `import { hand } from "written";`, + `hand();`, + ]; + describe("unset", () => { + test.each(["js", "ts", "dts"])("uses module with target %p", (target) => { + const lines = testGenerate(`target=${target}`); + expect(lines).toStrictEqual(linesEsm); + }); + }); + + describe("module", () => { + test.each(["js", "ts", "dts"])("uses module with target %p", (target) => { + const lines = testGenerate(`js_import_style=module,target=${target}`); + expect(lines).toStrictEqual(linesEsm); + }); + }); + + describe("legacy_commonjs", () => { + test.each(["ts", "dts"])("uses CommonJs with target %p", (target) => { + const lines = testGenerate( + `js_import_style=legacy_commonjs,target=${target}`, + ); + expect(lines).toStrictEqual(linesEsm); + }); + test(`uses CommonJs with target "js"`, () => { + const lines = testGenerate(`js_import_style=legacy_commonjs,target=js`); + expect(lines).toStrictEqual([ + `use strict;`, + `Object.defineProperty(exports, "__esModule", { value: true });`, + ``, + `const { third } = require("party");`, + ``, + "const thirdParty = third;", + `class MyClass {}`, + `const { hand } = require("written");`, + `hand();`, + ``, + `exports.MyClass = MyClass;`, + ]); + }); + test(`uses CommonJs with built-in transpile`, () => { + const lines = testGenerate( + `js_import_style=legacy_commonjs,target=js`, + true, + ); + expect(lines).toStrictEqual([ + `"use strict";`, + `Object.defineProperty(exports, "__esModule", { value: true });`, + `exports.MyClass = void 0;`, + `const party_1 = require("party");`, + `const thirdParty = party_1.third;`, + `class MyClass {`, + `}`, + `exports.MyClass = MyClass;`, + `const written_1 = require("written");`, + `(0, written_1.hand)();`, + ]); + }); + }); + + function testGenerate( + parameter: string, + useBuiltInTranspileFromTsToJs = false, + ) { + const plugin = createEcmaScriptPlugin({ + name: "test", + version: "v1", + generateTs: generateImportAndExportExamples, + generateJs: useBuiltInTranspileFromTsToJs + ? undefined + : generateImportAndExportExamples, + generateDts: generateImportAndExportExamples, + }); + + function generateImportAndExportExamples( + schema: Schema, + target: "js" | "ts" | "dts", + ) { + const f = schema.generateFile(`test.${target}`); + f.print("const thirdParty = ", f.import("third", "party"), ";"); + f.print(f.exportDecl("class", "MyClass"), " {}"); + switch (f.jsImportStyle) { + case "module": + f.print(`import { hand } from "written";`); + break; + case "legacy_commonjs": + f.print(`const { hand } = require("written");`); + break; + } + f.print("hand();"); + } + const req = new CodeGeneratorRequest({ + parameter, + }); + const res = plugin.run(req); + expect(res.file.length).toBeGreaterThanOrEqual(1); + const content = res.file[0]?.content ?? ""; + return content.trim().split("\n"); + } +}); diff --git a/packages/protoplugin/README.md b/packages/protoplugin/README.md index eedb31025..8ae452e8b 100644 --- a/packages/protoplugin/README.md +++ b/packages/protoplugin/README.md @@ -26,8 +26,8 @@ declaration files automatically using our internal TypeScript compiler. it to generate JavaScript and declaration files with your own version of TypeScript and your own compiler options. -With `protoplugin`, you have all the tools at your disposal to produce ECMAScript-compliant -code. +With `@bufbuild/protoplugin`, you have all the tools at your disposal to produce +ECMAScript-compliant code. ## Usage diff --git a/packages/protoplugin/src/create-es-plugin.ts b/packages/protoplugin/src/create-es-plugin.ts index a1d0de3cb..399a402fc 100644 --- a/packages/protoplugin/src/create-es-plugin.ts +++ b/packages/protoplugin/src/create-es-plugin.ts @@ -12,13 +12,11 @@ // See the License for the specific language governing permissions and // limitations under the License. -import type { Target } from "./ecmascript"; import { createSchema, Schema, toResponse } from "./ecmascript/schema.js"; import type { FileInfo } from "./ecmascript/generated-file.js"; import type { Plugin } from "./plugin.js"; import { transpile } from "./ecmascript/transpile.js"; -import { PluginOptionError } from "./error.js"; -import type { RewriteImports } from "./ecmascript/import-path.js"; +import { parseParameter } from "./ecmascript/parameter.js"; interface PluginInit { /** @@ -70,17 +68,23 @@ interface PluginInit { generateDts?: (schema: Schema, target: "dts") => void; /** - * A optional function which will transpile a given set of files. + * An optional function which will transpile a given set of files. * - * This funcion is meant to be used in place of either generateJs, + * This function is meant to be used in place of either generateJs, * generateDts, or both. However, those functions will take precedence. * This means that if generateJs, generateDts, and this transpile function * are all provided, this transpile function will be ignored. + * + * If jsImportStyle is "module" (the standard behavior), the function is + * expected to use ECMAScript module import and export statements when + * transpiling to JS. If jsImportStyle is "legacy_commonjs", the function is + * expected to use CommonJs require() and exports when transpiling to JS. */ transpile?: ( files: FileInfo[], transpileJs: boolean, transpileDts: boolean, + jsImportStyle: "module" | "legacy_commonjs", ) => FileInfo[]; } @@ -96,27 +100,8 @@ export function createEcmaScriptPlugin(init: PluginInit): Plugin { name: init.name, version: init.version, run(req) { - const { - targets, - tsNocheck, - bootstrapWkt, - rewriteImports, - importExtension, - keepEmptyFiles, - pluginParameter, - } = parseParameter(req.parameter, init.parseOption); - const { schema, getFileInfo } = createSchema( - req, - targets, - tsNocheck, - bootstrapWkt, - rewriteImports, - importExtension, - keepEmptyFiles, - init.name, - init.version, - pluginParameter, - ); + const parameter = parseParameter(req.parameter, init.parseOption); + const schema = createSchema(req, parameter, init.name, init.version); const targetTs = schema.targets.includes("ts"); const targetJs = schema.targets.includes("js"); @@ -134,6 +119,7 @@ export function createEcmaScriptPlugin(init: PluginInit): Plugin { (targetJs && !init.generateJs) || (targetDts && !init.generateDts) ) { + schema.prepareGenerate("module"); init.generateTs(schema, "ts"); // Save off the generated TypeScript files so that we can pass these @@ -146,11 +132,12 @@ export function createEcmaScriptPlugin(init: PluginInit): Plugin { // a generateJs function and expect to transpile declarations. // 3. Transpiling is somewhat expensive and situations with an // extremely large amount of files could have performance impacts. - tsFiles = getFileInfo(); + tsFiles = schema.getFileInfo(); } if (targetJs) { if (init.generateJs) { + schema.prepareGenerate(parameter.jsImportStyle); init.generateJs(schema, "js"); } else { transpileJs = true; @@ -159,6 +146,7 @@ export function createEcmaScriptPlugin(init: PluginInit): Plugin { if (targetDts) { if (init.generateDts) { + schema.prepareGenerate("module"); init.generateDts(schema, "dts"); } else { transpileDts = true; @@ -170,7 +158,7 @@ export function createEcmaScriptPlugin(init: PluginInit): Plugin { // generated TypeScript files to assist in transpilation. If they were // generated but not specified in the target out, we shouldn't produce // these files in the CodeGeneratorResponse. - let files = getFileInfo(); + let files = schema.getFileInfo(); if (!targetTs && tsFiles.length > 0) { files = files.filter( (file) => !tsFiles.some((tsFile) => tsFile.name === file.name), @@ -183,7 +171,12 @@ export function createEcmaScriptPlugin(init: PluginInit): Plugin { if (transpileJs || transpileDts) { const transpileFn = init.transpile ?? transpile; // Transpile the TypeScript files and add to the master list of files - const transpiledFiles = transpileFn(tsFiles, transpileJs, transpileDts); + const transpiledFiles = transpileFn( + tsFiles, + transpileJs, + transpileDts, + parameter.jsImportStyle, + ); files.push(...transpiledFiles); } @@ -191,143 +184,3 @@ export function createEcmaScriptPlugin(init: PluginInit): Plugin { }, }; } - -function parseParameter( - parameter: string | undefined, - parseOption: PluginInit["parseOption"], -) { - let targets: Target[] = ["js", "dts"]; - let tsNocheck = true; - let bootstrapWkt = false; - let keepEmptyFiles = false; - const rewriteImports: RewriteImports = []; - let importExtension = ".js"; - const rawParameters: string[] = []; - for (const { key, value, raw } of splitParameter(parameter)) { - // Whether this key/value plugin parameter pair should be - // printed to the generated file preamble - let printToFile = true; - switch (key) { - case "target": - targets = []; - for (const rawTarget of value.split("+")) { - switch (rawTarget) { - case "js": - case "ts": - case "dts": - if (targets.indexOf(rawTarget) < 0) { - targets.push(rawTarget); - } - break; - default: - throw new PluginOptionError(raw); - } - } - value.split("+"); - break; - case "ts_nocheck": - switch (value) { - case "true": - case "1": - tsNocheck = true; - break; - case "false": - case "0": - tsNocheck = false; - break; - default: - throw new PluginOptionError(raw); - } - break; - case "bootstrap_wkt": - switch (value) { - case "true": - case "1": - bootstrapWkt = true; - break; - case "false": - case "0": - bootstrapWkt = false; - break; - default: - throw new PluginOptionError(raw); - } - break; - case "rewrite_imports": { - const parts = value.split(":"); - if (parts.length !== 2) { - throw new PluginOptionError( - raw, - "must be in the form of :", - ); - } - const [pattern, target] = parts; - rewriteImports.push({ pattern, target }); - // rewrite_imports can be noisy and is more of an implementation detail - // so we strip it out of the preamble - printToFile = false; - break; - } - case "import_extension": { - importExtension = value === "none" ? "" : value; - break; - } - case "keep_empty_files": { - switch (value) { - case "true": - case "1": - keepEmptyFiles = true; - break; - case "false": - case "0": - keepEmptyFiles = false; - break; - default: - throw new PluginOptionError(raw); - } - break; - } - default: - if (parseOption === undefined) { - throw new PluginOptionError(raw); - } - try { - parseOption(key, value); - } catch (e) { - throw new PluginOptionError(raw, e); - } - break; - } - if (printToFile) { - rawParameters.push(raw); - } - } - - const pluginParameter = rawParameters.join(","); - - return { - targets, - tsNocheck, - bootstrapWkt, - rewriteImports, - importExtension, - keepEmptyFiles, - pluginParameter, - }; -} - -function splitParameter( - parameter: string | undefined, -): { key: string; value: string; raw: string }[] { - if (parameter == undefined) { - return []; - } - return parameter.split(",").map((raw) => { - const i = raw.indexOf("="); - return { - key: i === -1 ? raw : raw.substring(0, i), - value: i === -1 ? "" : raw.substring(i + 1), - raw, - }; - }); -} diff --git a/packages/protoplugin/src/ecmascript/export-declaration.ts b/packages/protoplugin/src/ecmascript/export-declaration.ts new file mode 100644 index 000000000..ed4451372 --- /dev/null +++ b/packages/protoplugin/src/ecmascript/export-declaration.ts @@ -0,0 +1,32 @@ +// Copyright 2021-2023 Buf Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +import { DescEnum, DescMessage } from "@bufbuild/protobuf"; + +export type ExportDeclaration = { + readonly kind: "es_export_decl"; + declaration: string; + name: string | DescMessage | DescEnum; +}; + +export function createExportDeclaration( + declaration: string, + name: ExportDeclaration["name"], +): ExportDeclaration { + return { + kind: "es_export_decl", + declaration, + name, + }; +} diff --git a/packages/protoplugin/src/ecmascript/generated-file.ts b/packages/protoplugin/src/ecmascript/generated-file.ts index 1ba1ff9eb..0e45fe4f4 100644 --- a/packages/protoplugin/src/ecmascript/generated-file.ts +++ b/packages/protoplugin/src/ecmascript/generated-file.ts @@ -18,6 +18,8 @@ import { createImportSymbol } from "./import-symbol.js"; import { literalString, makeFilePreamble } from "./gencommon.js"; import type { RuntimeImports } from "./runtime-imports.js"; import { makeImportPathRelative } from "./import-path.js"; +import type { ExportDeclaration } from "./export-declaration.js"; +import { createExportDeclaration } from "./export-declaration.js"; /** * All types that can be passed to GeneratedFile.print() @@ -29,6 +31,7 @@ export type Printable = | bigint | Uint8Array | ImportSymbol + | ExportDeclaration | DescMessage | DescEnum | Printable[]; @@ -84,6 +87,27 @@ export interface GeneratedFile { */ export(name: string): ImportSymbol; + /** + * Create a printable export statement. For example: + * + * ```ts + * f.print(f.exportDecl("abstract class", "MyClass"), " {}") + * ``` + * + * Will generate as: + * ```ts + * export abstract class MyClass {} + * ``` + * + * Using this method is preferred over a calling print() with a literal export + * statement. If the plugin option `js_import_style=legacy_commonjs` is set, + * exports will automatically be generated for CommonJS. + */ + exportDecl( + declaration: string, + name: string | DescMessage | DescEnum, + ): Printable; + /** * Import a message or enumeration generated by protoc-gen-es. */ @@ -100,10 +124,16 @@ export interface GeneratedFile { * relative to the current file. */ import(name: string, from: string): ImportSymbol; + + /** + * In case you need full control over exports and imports, use print() and + * formulate your own imports and exports based on this property. + */ + readonly jsImportStyle: "module" | "legacy_commonjs"; } -export interface GenerateFileToFileInfo { - getFileInfo(): FileInfo | undefined; +export interface GeneratedFileController extends GeneratedFile { + getFileInfo(): FileInfo; } type CreateTypeImportFn = (desc: DescMessage | DescEnum) => ImportSymbol; @@ -113,6 +143,7 @@ type RewriteImportPathFn = (path: string) => string; export function createGeneratedFile( name: string, importPath: string, + jsImportStyle: "module" | "legacy_commonjs", rewriteImportPath: RewriteImportPathFn, createTypeImport: CreateTypeImportFn, runtimeImports: RuntimeImports, @@ -122,8 +153,7 @@ export function createGeneratedFile( pluginParameter: string; tsNocheck: boolean; }, - keepEmpty: boolean, -): GeneratedFile & GenerateFileToFileInfo { +): GeneratedFileController { let preamble: string | undefined; const el: El[] = []; return { @@ -165,6 +195,9 @@ export function createGeneratedFile( export(name) { return createImportSymbol(name, importPath); }, + exportDecl(declaration, name) { + return createExportDeclaration(declaration, name); + }, import(typeOrName: DescMessage | DescEnum | string, from?: string) { if (typeof typeOrName == "string") { // eslint-disable-next-line @typescript-eslint/no-non-null-assertion @@ -172,55 +205,99 @@ export function createGeneratedFile( } return createTypeImport(typeOrName); }, + jsImportStyle, getFileInfo() { - const content = elToContent(el, importPath, rewriteImportPath); - if (!keepEmpty && content.length === 0) { - return; - } return { name, - content, + content: elToContent( + el, + importPath, + rewriteImportPath, + jsImportStyle == "legacy_commonjs", + ), preamble, }; }, }; } -type El = ImportSymbol | string; +type El = + | string + | ImportSymbol + | { kind: "es_export_stmt"; declaration?: string; name: string }; function elToContent( el: El[], importerPath: string, rewriteImportPath: RewriteImportPathFn, + legacyCommonJs: boolean, ): string { const c: string[] = []; + if (legacyCommonJs) { + c.push(`use strict;\n`); + c.push(`Object.defineProperty(exports, "__esModule", { value: true });\n`); + c.push(`\n`); + } const symbolToIdentifier = processImports( el, importerPath, rewriteImportPath, (typeOnly, from, names) => { - const p = names.map(({ name, alias }) => - alias == undefined ? name : `${name} as ${alias}`, - ); - const what = `{ ${p.join(", ")} }`; - if (typeOnly) { - c.push(`import type ${what} from ${literalString(from)};\n`); + if (legacyCommonJs) { + const p = names.map(({ name, alias }) => + alias == undefined ? name : `${name}: ${alias}`, + ); + const what = `{ ${p.join(", ")} }`; + c.push(`const ${what} = require(${literalString(from)});\n`); } else { - c.push(`import ${what} from ${literalString(from)};\n`); + const p = names.map(({ name, alias }) => + alias == undefined ? name : `${name} as ${alias}`, + ); + const what = `{ ${p.join(", ")} }`; + if (typeOnly) { + c.push(`import type ${what} from ${literalString(from)};\n`); + } else { + c.push(`import ${what} from ${literalString(from)};\n`); + } } }, ); if (c.length > 0) { c.push("\n"); } + const legacyCommonJsExportNames: string[] = []; for (const e of el) { if (typeof e == "string") { c.push(e); - continue; + } else { + switch (e.kind) { + case "es_symbol": { + const ident = symbolToIdentifier.get(e.id); + if (ident != undefined) { + c.push(ident); + } + break; + } + case "es_export_stmt": + if (legacyCommonJs) { + legacyCommonJsExportNames.push(e.name); + } else { + c.push("export "); + } + if (e.declaration !== undefined && e.declaration.length > 0) { + c.push(e.declaration, " "); + } + c.push(e.name); + break; + } } - const ident = symbolToIdentifier.get(e.id); - if (ident != undefined) { - c.push(ident); + } + if (legacyCommonJs) { + if (legacyCommonJsExportNames.length > 0) { + c.push(`\n`); + } + for (const name of legacyCommonJsExportNames) { + c.push(`exports.`, name, " = ", name, ";\n"); } } return c.join(""); @@ -258,6 +335,16 @@ function printableToEl( case "es_symbol": el.push(p); break; + case "es_export_decl": + el.push({ + kind: "es_export_stmt", + declaration: p.declaration, + name: + typeof p.name == "string" + ? p.name + : createTypeImport(p.name).name, + }); + break; case "message": case "enum": el.push(createTypeImport(p)); @@ -309,7 +396,7 @@ function processImports( // Walk through all symbols used and populate the collections above. for (const s of el) { - if (typeof s == "string") { + if (typeof s != "object" || s.kind !== "es_symbol") { continue; } symbolToIdentifier.set(s.id, s.name); diff --git a/packages/protoplugin/src/ecmascript/index.ts b/packages/protoplugin/src/ecmascript/index.ts index c16f2df6b..3879f12fc 100644 --- a/packages/protoplugin/src/ecmascript/index.ts +++ b/packages/protoplugin/src/ecmascript/index.ts @@ -14,11 +14,11 @@ import { codegenInfo } from "@bufbuild/protobuf"; export { reifyWkt } from "./reify-wkt.js"; -export { Target } from "./target.js"; -export { Schema } from "./schema.js"; -export { RuntimeImports } from "./runtime-imports.js"; -export { GeneratedFile, FileInfo, Printable } from "./generated-file.js"; -export { ImportSymbol } from "./import-symbol.js"; +export type { Target } from "./target.js"; +export type { Schema } from "./schema.js"; +export type { RuntimeImports } from "./runtime-imports.js"; +export type { GeneratedFile, FileInfo, Printable } from "./generated-file.js"; +export type { ImportSymbol } from "./import-symbol.js"; export const { localName } = codegenInfo; diff --git a/packages/protoplugin/src/ecmascript/parameter.ts b/packages/protoplugin/src/ecmascript/parameter.ts new file mode 100644 index 000000000..c130d68b9 --- /dev/null +++ b/packages/protoplugin/src/ecmascript/parameter.ts @@ -0,0 +1,182 @@ +// Copyright 2021-2023 Buf Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +import { Target } from "./target.js"; +import { RewriteImports } from "./import-path.js"; +import { PluginOptionError } from "../error.js"; + +export interface ParsedParameter { + targets: Target[]; + tsNocheck: boolean; + bootstrapWkt: boolean; + keepEmptyFiles: boolean; + rewriteImports: RewriteImports; + importExtension: string; + jsImportStyle: "module" | "legacy_commonjs"; + sanitizedParameter: string; +} + +export function parseParameter( + parameter: string | undefined, + parseExtraOption: ((key: string, value: string) => void) | undefined, +): ParsedParameter { + let targets: Target[] = ["js", "dts"]; + let tsNocheck = true; + let bootstrapWkt = false; + let keepEmptyFiles = false; + const rewriteImports: RewriteImports = []; + let importExtension = ".js"; + let jsImportStyle: "module" | "legacy_commonjs" = "module"; + const rawParameters: string[] = []; + for (const { key, value, raw } of splitParameter(parameter)) { + // Whether this key/value plugin parameter pair should be + // printed to the generated file preamble + let sanitize = false; + switch (key) { + case "target": + targets = []; + for (const rawTarget of value.split("+")) { + switch (rawTarget) { + case "js": + case "ts": + case "dts": + if (targets.indexOf(rawTarget) < 0) { + targets.push(rawTarget); + } + break; + default: + throw new PluginOptionError(raw); + } + } + value.split("+"); + break; + case "ts_nocheck": + switch (value) { + case "true": + case "1": + tsNocheck = true; + break; + case "false": + case "0": + tsNocheck = false; + break; + default: + throw new PluginOptionError(raw); + } + break; + case "bootstrap_wkt": + switch (value) { + case "true": + case "1": + bootstrapWkt = true; + break; + case "false": + case "0": + bootstrapWkt = false; + break; + default: + throw new PluginOptionError(raw); + } + break; + case "rewrite_imports": { + const parts = value.split(":"); + if (parts.length !== 2) { + throw new PluginOptionError( + raw, + "must be in the form of :", + ); + } + const [pattern, target] = parts; + rewriteImports.push({ pattern, target }); + // rewrite_imports can be noisy and is more of an implementation detail + // so we strip it out of the preamble + sanitize = true; + break; + } + case "import_extension": { + importExtension = value === "none" ? "" : value; + break; + } + case "js_import_style": + switch (value) { + case "module": + jsImportStyle = value; + break; + case "legacy_commonjs": + jsImportStyle = value; + break; + default: + throw new PluginOptionError(raw); + } + break; + case "keep_empty_files": { + switch (value) { + case "true": + case "1": + keepEmptyFiles = true; + break; + case "false": + case "0": + keepEmptyFiles = false; + break; + default: + throw new PluginOptionError(raw); + } + break; + } + default: + if (parseExtraOption === undefined) { + throw new PluginOptionError(raw); + } + try { + parseExtraOption(key, value); + } catch (e) { + throw new PluginOptionError(raw, e); + } + break; + } + if (!sanitize) { + rawParameters.push(raw); + } + } + + const sanitizedParameter = rawParameters.join(","); + + return { + targets, + tsNocheck, + bootstrapWkt, + rewriteImports, + importExtension, + jsImportStyle, + keepEmptyFiles, + sanitizedParameter, + }; +} + +function splitParameter( + parameter: string | undefined, +): { key: string; value: string; raw: string }[] { + if (parameter == undefined) { + return []; + } + return parameter.split(",").map((raw) => { + const i = raw.indexOf("="); + return { + key: i === -1 ? raw : raw.substring(0, i), + value: i === -1 ? "" : raw.substring(i + 1), + raw, + }; + }); +} diff --git a/packages/protoplugin/src/ecmascript/schema.ts b/packages/protoplugin/src/ecmascript/schema.ts index 4024d1330..be1794b97 100644 --- a/packages/protoplugin/src/ecmascript/schema.ts +++ b/packages/protoplugin/src/ecmascript/schema.ts @@ -29,7 +29,7 @@ import { import type { FileInfo, GeneratedFile, - GenerateFileToFileInfo, + GeneratedFileController, } from "./generated-file.js"; import { createGeneratedFile } from "./generated-file.js"; import { createRuntimeImports, RuntimeImports } from "./runtime-imports.js"; @@ -39,8 +39,8 @@ import { deriveImportPath, makeImportPath, rewriteImportPath, - RewriteImports, } from "./import-path.js"; +import { ParsedParameter } from "./parameter"; /** * Schema describes the files and types that the plugin is requested to @@ -78,69 +78,72 @@ export interface Schema { readonly proto: CodeGeneratorRequest; } -interface SchemaController { - schema: Schema; +interface SchemaController extends Schema { getFileInfo: () => FileInfo[]; + prepareGenerate(jsImportStyle: "module" | "legacy_commonjs"): void; } export function createSchema( request: CodeGeneratorRequest, - targets: Target[], - tsNocheck: boolean, - bootstrapWkt: boolean, - rewriteImports: RewriteImports, - importExtension: string, - keepEmptyFiles: boolean, + parameter: Omit, pluginName: string, pluginVersion: string, - pluginParameter: string, ): SchemaController { const descriptorSet = createDescriptorSet(request.protoFile); const filesToGenerate = findFilesToGenerate(descriptorSet, request); - const runtime = createRuntimeImports(bootstrapWkt); + const runtime = createRuntimeImports(parameter.bootstrapWkt); const createTypeImport = (desc: DescMessage | DescEnum): ImportSymbol => { const name = codegenInfo.localName(desc); - const from = makeImportPath(desc.file, bootstrapWkt, filesToGenerate); + const from = makeImportPath( + desc.file, + parameter.bootstrapWkt, + filesToGenerate, + ); return createImportSymbol(name, from); }; - const generatedFiles: GenerateFileToFileInfo[] = []; - const schema: Schema = { - targets, + let jsImportStyle: "module" | "legacy_commonjs" | undefined; + const generatedFiles: GeneratedFileController[] = []; + return { + targets: parameter.targets, runtime, proto: request, files: filesToGenerate, allFiles: descriptorSet.files, generateFile(name) { + if (jsImportStyle === undefined) { + throw new Error( + "prepareGenerate() must be called before generateFile()", + ); + } const genFile = createGeneratedFile( name, deriveImportPath(name), + jsImportStyle, (importPath: string) => - rewriteImportPath(importPath, rewriteImports, importExtension), + rewriteImportPath( + importPath, + parameter.rewriteImports, + parameter.importExtension, + ), createTypeImport, runtime, { pluginName, pluginVersion, - pluginParameter, - tsNocheck, + pluginParameter: parameter.sanitizedParameter, + tsNocheck: parameter.tsNocheck, }, - keepEmptyFiles, ); generatedFiles.push(genFile); return genFile; }, - }; - return { - schema, getFileInfo() { - return generatedFiles.flatMap((file) => { - const fileInfo = file.getFileInfo(); - // undefined is returned if the file has no content - if (!fileInfo) { - return []; - } - return [fileInfo]; - }); + return generatedFiles + .map((f) => f.getFileInfo()) + .filter((fi) => parameter.keepEmptyFiles || fi.content.length > 0); + }, + prepareGenerate(newJsImportStyle) { + jsImportStyle = newJsImportStyle; }, }; } diff --git a/packages/protoplugin/src/ecmascript/transpile.ts b/packages/protoplugin/src/ecmascript/transpile.ts index d6890a789..f6cddd562 100644 --- a/packages/protoplugin/src/ecmascript/transpile.ts +++ b/packages/protoplugin/src/ecmascript/transpile.ts @@ -98,6 +98,7 @@ export function transpile( files: FileInfo[], transpileJs: boolean, transpileDts: boolean, + jsImportStyle: "module" | "legacy_commonjs", ): FileInfo[] { const options: ts.CompilerOptions = { ...defaultOptions, @@ -105,6 +106,10 @@ export function transpile( emitDeclarationOnly: transpileDts && !transpileJs, }; + if (jsImportStyle == "legacy_commonjs") { + options.module = ts.ModuleKind.CommonJS; + } + // Create the transpiler (a ts.Program object) const program = createTranspiler(options, files); From 07e3cd1e0e25869259ecfd8e93971b7d2de4f0ff Mon Sep 17 00:00:00 2001 From: Timo Stamm Date: Thu, 7 Dec 2023 12:13:02 +0100 Subject: [PATCH 02/11] Update importing docs in writing_plugins.md --- docs/writing_plugins.md | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/docs/writing_plugins.md b/docs/writing_plugins.md index 7b0e23b95..c8654aced 100644 --- a/docs/writing_plugins.md +++ b/docs/writing_plugins.md @@ -252,13 +252,13 @@ function generateTs(schema: Schema) { ### Importing -Generating import statements is accomplished via a combination of the `print` function and another function on the generated file object: `import`. The approach varies depending on the type of import you would like to generate. +Generating import statements is accomplished via a combination of the methods `import` and `print` on the generated file +object. #### Importing from an NPM package -To generate an import statement from an NPM package dependency, you first invoke the `import` function, passing the name of the import and the package in which it is located. - -For example, to import the `useEffect` hook from React: +To import from an NPM package, you first invoke the `import` function, passing the name of the symbol to import, and the +package in which it is located. For example, to import the `useEffect` hook from React: ```ts const useEffect = f.import("useEffect", "react"); @@ -278,7 +278,7 @@ When the `ImportSymbol` is printed (and only when it is printed), an import stat #### Importing from `protoc-gen-es` generated code -Imports in this way work similarly. Again, the `print` statement will automatically generate the import statement for you when invoked. +To import a message or enumeration from `protoc-gen-es` generated code, you can simply pass the descriptor to `import()`: ```ts declare var someMessageDescriptor: DescMessage; @@ -327,14 +327,12 @@ Note that some of the `ImportSymbol` types in the schema runtime (such as `JsonV The natural instinct would be to simply print your own import statements as `f.print("import { Foo } from 'bar'")`, but this is not the recommended approach. Using `f.import()` has many advantages such as: -- **Conditional imports** - - Import statements belong at the top of a file, but you usually only find out later whether you need the import, such as further in your code in a nested if statement. Conditionally printing the import symbol will only generate the import statement when it is actually used. +- **Conditional imports**: Import statements belong at the top of a file, but you usually only find out later whether you need the import, such as further in your code in a nested if statement. Conditionally printing the import symbol will only generate the import statement when it is actually used. -- **Preventing name collisions** - - For example if you `import { Foo } from "bar"` and `import { Foo } from "baz"` , `f.import()` will automatically rename one of them `Foo$1`, preventing name collisions in your import statements and code. +- **Preventing name collisions**: For example if you `import { Foo } from "bar"` and `import { Foo } from "baz"` , `f.import()` will automatically rename one of them `Foo$1`, preventing name collisions in your import statements and code. -- **Extensibility of import generation** - - Abstracting the generation of imports allows the library to potentially offer other import styles in the future without affecting current users. +- **Import styles**: If the plugin option `js_import_style=legacy_commonjs` is set, code is automatically generated + with `require()` calls instead of `import` statements. ### Exporting From e1cb9d84c6c2d02c61fda626bc18a97443ae31d2 Mon Sep 17 00:00:00 2001 From: Timo Stamm Date: Thu, 7 Dec 2023 12:35:46 +0100 Subject: [PATCH 03/11] "use strict"; --- packages/protoplugin/src/ecmascript/generated-file.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/protoplugin/src/ecmascript/generated-file.ts b/packages/protoplugin/src/ecmascript/generated-file.ts index 0e45fe4f4..ba5082cf0 100644 --- a/packages/protoplugin/src/ecmascript/generated-file.ts +++ b/packages/protoplugin/src/ecmascript/generated-file.ts @@ -234,7 +234,7 @@ function elToContent( ): string { const c: string[] = []; if (legacyCommonJs) { - c.push(`use strict;\n`); + c.push(`"use strict";\n`); c.push(`Object.defineProperty(exports, "__esModule", { value: true });\n`); c.push(`\n`); } From b4dcfb9a7edd6df42d9db9d215793a8040369679 Mon Sep 17 00:00:00 2001 From: Timo Stamm Date: Thu, 7 Dec 2023 12:40:49 +0100 Subject: [PATCH 04/11] Rename import_style.test.ts --- .../src/{import_style.test.ts => js_import_style.test.ts} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename packages/protoplugin-test/src/{import_style.test.ts => js_import_style.test.ts} (100%) diff --git a/packages/protoplugin-test/src/import_style.test.ts b/packages/protoplugin-test/src/js_import_style.test.ts similarity index 100% rename from packages/protoplugin-test/src/import_style.test.ts rename to packages/protoplugin-test/src/js_import_style.test.ts From ca7464d9c63a2072002f399fd27e8ba7d5b1c6d9 Mon Sep 17 00:00:00 2001 From: Timo Stamm Date: Thu, 7 Dec 2023 13:12:28 +0100 Subject: [PATCH 05/11] "use strict"; --- packages/protoplugin-test/src/js_import_style.test.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/protoplugin-test/src/js_import_style.test.ts b/packages/protoplugin-test/src/js_import_style.test.ts index 80d94dc58..046a5c422 100644 --- a/packages/protoplugin-test/src/js_import_style.test.ts +++ b/packages/protoplugin-test/src/js_import_style.test.ts @@ -50,7 +50,7 @@ describe("js_import_style", () => { test(`uses CommonJs with target "js"`, () => { const lines = testGenerate(`js_import_style=legacy_commonjs,target=js`); expect(lines).toStrictEqual([ - `use strict;`, + `"use strict";`, `Object.defineProperty(exports, "__esModule", { value: true });`, ``, `const { third } = require("party");`, From 8ca585e1ebe798796e962ba61696250e0ac0f4f7 Mon Sep 17 00:00:00 2001 From: Timo Stamm Date: Thu, 7 Dec 2023 13:40:13 +0100 Subject: [PATCH 06/11] Fix omission of empty CJS files --- packages/protoplugin/src/ecmascript/generated-file.ts | 3 +++ 1 file changed, 3 insertions(+) diff --git a/packages/protoplugin/src/ecmascript/generated-file.ts b/packages/protoplugin/src/ecmascript/generated-file.ts index ba5082cf0..100b86654 100644 --- a/packages/protoplugin/src/ecmascript/generated-file.ts +++ b/packages/protoplugin/src/ecmascript/generated-file.ts @@ -232,6 +232,9 @@ function elToContent( rewriteImportPath: RewriteImportPathFn, legacyCommonJs: boolean, ): string { + if (el.length == 0) { + return ""; + } const c: string[] = []; if (legacyCommonJs) { c.push(`"use strict";\n`); From 9c1ecf6e1eb75f1444faebb6c284cd5c5e16a4ab Mon Sep 17 00:00:00 2001 From: Timo Stamm Date: Thu, 7 Dec 2023 13:40:23 +0100 Subject: [PATCH 07/11] Add tests for plugin option keep_empty_files --- .../src/keep_empty_files.test.ts | 121 ++++++++++++++++++ 1 file changed, 121 insertions(+) create mode 100644 packages/protoplugin-test/src/keep_empty_files.test.ts diff --git a/packages/protoplugin-test/src/keep_empty_files.test.ts b/packages/protoplugin-test/src/keep_empty_files.test.ts new file mode 100644 index 000000000..cf7fd02c8 --- /dev/null +++ b/packages/protoplugin-test/src/keep_empty_files.test.ts @@ -0,0 +1,121 @@ +// Copyright 2021-2023 Buf Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +import { describe, expect, test } from "@jest/globals"; +import type { DescFile } from "@bufbuild/protobuf"; +import { CodeGeneratorRequest } from "@bufbuild/protobuf"; +import type { Schema } from "@bufbuild/protoplugin"; +import { createEcmaScriptPlugin } from "@bufbuild/protoplugin"; +import type { GeneratedFile } from "@bufbuild/protoplugin/ecmascript"; +import { UpstreamProtobuf } from "upstream-protobuf"; + +describe("keep_empty_files", () => { + describe("unset", () => { + test.each(["js", "ts", "dts"])( + "does not generate empty file with target %p", + async (target) => { + const { fileCount, lines } = await testGenerate( + ` + // detached syntax comment + + // syntax comment + syntax="proto3"; + + // detached package comment + + // package comment + package test; + + message M {} + `, + `target=${target}`, + (f, descFile) => { + // A preamble does not count as non-empty + f.preamble(descFile); + // An unused import does not count as non-empty + f.import("foo", "bar"); + // An unused export declaration does not count as non-empty + f.exportDecl("foo", "bar"); + }, + ); + expect(lines).toBeUndefined(); + expect(fileCount).toBe(0); + }, + ); + test.each(["js", "ts", "dts"])( + "printing empty line generates a file with target %p", + async (target) => { + const { lines } = await testGenerate( + `syntax="proto3"; message M {}`, + `target=${target}`, + (f) => { + f.print(); + }, + ); + expect(lines).toStrictEqual([""]); + }, + ); + }); + + describe("set", () => { + test.each(["js", "ts", "dts"])( + "generates empty file with target %p", + async (target) => { + const { lines } = await testGenerate( + `syntax="proto3"; message M {}`, + `target=${target},keep_empty_files=true`, + (f) => { + f.print(); + }, + ); + expect(lines).toStrictEqual([""]); + }, + ); + }); + + async function testGenerate( + protoSource: string, + parameter: string, + gen: (f: GeneratedFile, descFile: DescFile) => void, + ) { + const plugin = createEcmaScriptPlugin({ + name: "test", + version: "v1", + generateTs: generateAny, + generateJs: generateAny, + generateDts: generateAny, + }); + + function generateAny(schema: Schema, target: "js" | "ts" | "dts") { + const f = schema.generateFile(`test.${target}`); + gen(f, schema.files[0]); + } + + const upstream = new UpstreamProtobuf(); + const protoFiles = { + "x.proto": protoSource, + }; + const req = CodeGeneratorRequest.fromBinary( + await upstream.createCodeGeneratorRequest(protoFiles, { + parameter, + }), + ); + expect(req.fileToGenerate.length).toBe(1); + const res = plugin.run(req); + return { + fileCount: res.file.length, + lines: res.file[0]?.content?.trim().split("\n"), + }; + } +}); From d56bd4efa20a8fbfb30c279b3f1e3a33c27e00ba Mon Sep 17 00:00:00 2001 From: Timo Stamm Date: Thu, 7 Dec 2023 13:48:07 +0100 Subject: [PATCH 08/11] make format --- packages/protoplugin-test/src/keep_empty_files.test.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/protoplugin-test/src/keep_empty_files.test.ts b/packages/protoplugin-test/src/keep_empty_files.test.ts index cf7fd02c8..ba44608c8 100644 --- a/packages/protoplugin-test/src/keep_empty_files.test.ts +++ b/packages/protoplugin-test/src/keep_empty_files.test.ts @@ -49,8 +49,8 @@ describe("keep_empty_files", () => { f.exportDecl("foo", "bar"); }, ); - expect(lines).toBeUndefined(); - expect(fileCount).toBe(0); + expect(lines).toBeUndefined(); + expect(fileCount).toBe(0); }, ); test.each(["js", "ts", "dts"])( From 526fe50f26eecbd9f59cb2148eddce3cd6c8729c Mon Sep 17 00:00:00 2001 From: Timo Stamm Date: Thu, 7 Dec 2023 16:22:27 +0100 Subject: [PATCH 09/11] Update docs for generated_code.md --- docs/generated_code.md | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/docs/generated_code.md b/docs/generated_code.md index dd176fce8..52247edcb 100644 --- a/docs/generated_code.md +++ b/docs/generated_code.md @@ -78,17 +78,12 @@ By default, we generate JavaScript _and_ TypeScript declaration files, so the ge code can be used in JavaScript or TypeScript projects without transpilation. If you prefer to generate TypeScript, use the plugin option `target=ts`. -Note that we generate ECMAScript modules, which means we use `import` and `export` statements. -All import paths include a `.js` extension, so you can use the generated code in Node.js -with `"type": "module"` in your project's `package.json` without transpilation. -If you do require support for the legacy CommonJS format, you can generate TypeScript and -transpile it, for example with the extremely fast [esbuild](https://github.com/evanw/esbuild) -bundler. - -It is also possible to modify the extension used in the import paths via the -[`import_extension`](https://github.com/bufbuild/protobuf-es/tree/main/packages/protoc-gen-es#import_extensionjs) plugin option. -This option allows you to choose which extension will used in the imports, -providing flexibility for different environments. +By default, we generate ECMAScript modules, which means we use `import` and `export` statements. +If you need CommonJS, set the plugin option [`js_import_style=legacy_commonjs`](https://github.com/bufbuild/protobuf-es/tree/main/packages/protoc-gen-es#js_import_style). + +All import paths include a `.js` extension by default. You can remove or change the +extension via the [`import_extension`](https://github.com/bufbuild/protobuf-es/tree/main/packages/protoc-gen-es#import_extensionjs) +plugin option. ### Messages From 39c11644ab98b4c43837ebfefc204c4136372cf1 Mon Sep 17 00:00:00 2001 From: Timo Stamm Date: Thu, 7 Dec 2023 16:22:54 +0100 Subject: [PATCH 10/11] r --- docs/writing_plugins.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/writing_plugins.md b/docs/writing_plugins.md index c8654aced..969e2d76c 100644 --- a/docs/writing_plugins.md +++ b/docs/writing_plugins.md @@ -337,7 +337,7 @@ The natural instinct would be to simply print your own import statements as `f.p ### Exporting -To export a declaration from you code, use `exportDecl`: +To export a declaration from your code, use `exportDecl`: ```typescript const name = "foo"; From e110d9ebf74682168bbc8c3454c9dcf3e77353cd Mon Sep 17 00:00:00 2001 From: Timo Stamm Date: Thu, 7 Dec 2023 16:25:35 +0100 Subject: [PATCH 11/11] Link target=ts to the plugin readme --- docs/generated_code.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/generated_code.md b/docs/generated_code.md index 52247edcb..c5d6f7241 100644 --- a/docs/generated_code.md +++ b/docs/generated_code.md @@ -76,7 +76,7 @@ we generate `foo/bar_pb.js`. By default, we generate JavaScript _and_ TypeScript declaration files, so the generated code can be used in JavaScript or TypeScript projects without transpilation. If you -prefer to generate TypeScript, use the plugin option `target=ts`. +prefer to generate TypeScript, use the plugin option `[target=ts`](https://github.com/bufbuild/protobuf-es/tree/main/packages/protoc-gen-es#target). By default, we generate ECMAScript modules, which means we use `import` and `export` statements. If you need CommonJS, set the plugin option [`js_import_style=legacy_commonjs`](https://github.com/bufbuild/protobuf-es/tree/main/packages/protoc-gen-es#js_import_style).