Updating to 4.0.0 breaks integration with grpc-web: reader.readStringRequireUtf8 is not a function

[kostyay]

Hi
After upgrading the proto compiler to version 4.0.0 our application started failing to deserialize messages and failing with this error:
reader.readStringRequireUtf8 is not a function, response: undefined
Downgrading to v3.21.4 solves the problem.
I can see in the change log notes referring to some breaking change related to utf decoding, so I suppose its related, however its not clear how to resolve this.

[dibenede]

Can you tell me a bit more about your setup? I'm assuming you're using grpc-web 2.0.0 and the google-protobuf 4.0.0 npm package rather than building from source, is that correct?

Is there any relevant stack track within grpc-web/google-protobuf code?

[kostyay]

Hey
This is our buf.gen.yaml.
grpc-web is not available via buf at the moment so we npm installed it.

version: v1
plugins:
  - name: grpc-web
    out: gen/typescript
    opt:
      - mode=grpcwebtext
      - import_style=commonjs
   # dependencies
  - plugin: buf.build/protocolbuffers/js:v3.21.4
    out: gen/typescript
    opt:
      - import_style=commonjs

[dibenede]

I'm having trouble reproducing the issue.

The function readStringRequireUtf8 still exists in 4.0.0. However, we dropped closure compiler @export annotations for it along with some other closure compiler changes to the BinaryReader. Normally, reader.readStringRequireUtf8 is not a function is the kind of error I would expect to see from trying to use readStringRequireUtf8 with no @export.

However, I made a trivial test project (npm install'ing google-protobuf @4.0.0) that round trips a string binary proto serialization. Everything works though.

I'm not familiar with using buf, but tweaking your config to:

version: v1
plugins:
  - name: grpc-web
    out: gen/typescript
    opt:
      - mode=grpcwebtext
      - import_style=commonjs
   # dependencies
  - plugin: buf.build/protocolbuffers/js
    out: gen/typescript
    opt:
      - import_style=commonjs

and swapping to using the produced version of my test proto also still works.

I'm wondering if this could be an issue with conflicting versions of google-protobuf deps within your project. Alternatively, maybe a proto that has not be generated under 4.0.0.

My testing setup is just:

// string_test.js

const {TestAllTypesEdition2023} = require('./gen/typescript/protos/test_messages_edition2023_pb.js');
const test = require('node:test');
const assert = require('node:assert');

test('strings', () => {
  const request = new TestAllTypesEdition2023().setOptionalString('hello').setRepeatedStringList(['a','b','c']);
  const wire = request.serializeBinary(request);

  const response =
	TestAllTypesEdition2023.deserializeBinary(wire);

  assert.strictEqual(response.getOptionalString(), 'hello');
  assert.deepStrictEqual(response.getRepeatedStringList(), ['a', 'b', 'c']);
});

Using the node --test string_test.js to run.

We ship TestAllTypesEdition2023 here: conformance/protos/test_messages_edition2023.proto.