Enable the batch tsp & chat api tsp #2090

MaryGao · 2023-10-30T02:40:56Z

Enable the batch_modular & Chat API modular in smoke testing
Small bugfixs
- undefined and null are not allowed as header value which would happen for optional headers so adding conditions to omit these values
- Generate token credential signiture for empty scopes and set the default scopes with ${baseurl}/.default

packages/typespec-ts/test/unit/clientFactoryGenerator.spec.ts

qiaozha · 2023-10-30T09:35:22Z

packages/typespec-ts/test/modularUnit/operations.spec.ts

+          return context.path("/", ).post({...operationOptionsToRequestParameters(options), 
+          headers: {
+            "required-header": requiredHeader,
+            ...(options?.optionalHeader !== undefined ? {"optional-header": options?.optionalHeader} : {})


what kind of error will we get if it's undefined in the hearders?

If it's from core client rest, should we improve the core ?

The header definition is a dictionary of string | number | boolean items. If the value is optional which could be undefined, it would cause compile error.

https://github.com/azure/azure-sdk-for-js/blob/main/sdk/core/core-rest-pipeline/src/interfaces.ts#L15

headers?: RawHttpHeadersInput; ... export declare type RawHttpHeadersInput = Record<string, string | number | boolean>;

I think the core interface is accurate which only allows valid value as record items. I think it's the codegen's role to prepare headers object correctly.

Have you considered null in header ?

In my opinion, even if codegen has a way to prepare headers object correctly, we can not stop customer from pass undefined/null with as any, are you sure it can work as expected in such case ? After all, core is the one handle the ultimate request we send, I think it should be robust.

I am fine to temporary fix this in the codegen to unblock our customer, but I think we need some discussion here.

@joheredi @xirzec Could you help share your thoughts here ? Thanks

In general, it's recommended to work with strings in HTTP headers because the HTTP specification (RFC 7230) defines header fields as comprising a field name and a string value. While you can represent numbers, booleans, or other data as strings in headers, the native format for header values is string-based.

If we set undefined or null it's a string containing "undefined" or "null" that goes out to the wire. We support numbers and boolean as they can be serialized as strings easily, but on the wire it is strings.

I would prefer to not support null and if the value is set to undefined not included it. If customers want to pass those values they can do it as strings and would have the same effect if we supported them.

I found in openapi3, it has specified that headers could be primitives, arrays, objects here. https://swagger.io/docs/specification/describing-parameters/

per https://swagger.io/docs/specification/serialization/ it seems that Header parameters always use the simple style, that is, comma-separated values. so arrays/objects are always flattened into a csv of some sort.

that's is the serialization, but in the generated code, I don't think we should just generate string at least in Modular ?

To clarify something after having some offline discussion with @MaryGao,

For complex arrays and objects, codegen should generate the serialize helper functions, for modular, we should leverage the rlc helper functions to build the object/array as string.

For primitive types, boolean/number/string, core can handle this now.

We need to generate compilable code if it's undefined or null in typespec, which means we need to have a serializer helper function for undefined or null as there's type mismatch. And because we currently don't have model based overall serialier yet, we will have to put them inline the send request function body.

@xirzec @joheredi let me know if you have any concerns about it. Thanks

Also, sorry about the previous confusions.

Thanks for Qiaoqiao's summary and I supported the omit logic for both null and undefined in latest pr and feel free to review it again.

MaryGao · 2023-10-30T12:03:32Z

packages/rlc-common/src/interfaces.ts

+   * - undefined, no credentialScopes and relevant settings would be generated
+   * - [], which means we would generate TokenCredential but no credentialScopes and relevant settings
+   * - ["..."], which means we would generate credentialScopes and relevant settings with the given values
+   */


Now we handle the undefined and empty array as different cases.

undefined means we don't have OAuth2 defined;

[] means we have OAuth2 defined but with empty scopes.

Previously we treat the above two cases the same so all of them will not generate TokenCredential in constructor. But now they are different. The first would not but the latter would generate.

Please notice that we will not set the credentialScopes default value as []. If the value is absent the core would populate with {endpoint}/.default automatically.

This is another place where we should consider if we are generating an Azure SDK or not since the {endpoint}/.default value only makes sense for Azure services.

Haha, there may be another question - how to handle the empty scopes for non-Azure? How about explicitly set this as an empty array? Do we need to change the default logic for in @typespec/ts-http-runtime?

credentialScopes: []

But I think this topic would not block current pr considering we have a on-going one for integration #2083.

packages/rlc-common/src/buildClient.ts

packages/typespec-ts/test/unit/clientFactoryGenerator.spec.ts

MaryGao · 2023-10-31T12:06:10Z

packages/typespec-ts/src/modular/helpers/operationHelpers.ts

+  param: Parameter
+): string {
+  if (!param.optional && param.type.nullable === true) {
+    reportDiagnostic(program, {


Report warning if null value is allowed in required header item.

qiaozha

LGTM, left some minor comments

packages/typespec-ts/test/unit/clientFactoryGenerator.spec.ts

packages/typespec-ts/test/modularUnit/operations.spec.ts

MaryGao added 3 commits October 30, 2023 10:40

enable the batch tsp

aa2b4c8

Fix batch header issues

c81f020

Fix the empty credential scopes case in RLC

be7bba1

MaryGao changed the title ~~enable the batch tsp~~ Enable the batch tsp & chat api tsp Oct 30, 2023

MaryGao added 4 commits October 30, 2023 15:28

Set the proper credentialScopes values

375d30d

Refactor code little

1091ccc

Update the api view for api chat sdk

447162e

Remove useless files

c92bfbc

MaryGao marked this pull request as ready for review October 30, 2023 08:27

MaryGao requested review from sarangan12, joheredi and qiaozha as code owners October 30, 2023 08:27

MaryGao added 6 commits October 30, 2023 16:53

Add UTs to cover empty scopes

5ce99da

Update the comments

4f3817d

Update the comments

54a4c80

Debug git dirty

d18c217

Merge remote-tracking branch 'origin/main' into enable-batch-test

5566793

Update the batch testing

19beec6

qiaozha reviewed Oct 30, 2023

View reviewed changes

MaryGao commented Oct 30, 2023

View reviewed changes

packages/rlc-common/src/buildClient.ts Show resolved Hide resolved

MaryGao commented Oct 30, 2023

View reviewed changes

packages/typespec-ts/test/unit/clientFactoryGenerator.spec.ts Show resolved Hide resolved

MaryGao added 6 commits October 30, 2023 20:21

Revert changes

4697bc2

Merge branch 'main' into enable-batch-test

f8df8c1

Update the default value for empty scopes

223ebea

Support the nullable header in modular

96b30e2

Fix lint issue

18ad380

Remove useless files

b0ec9fe

MaryGao commented Oct 31, 2023

View reviewed changes

MaryGao requested a review from qiaozha October 31, 2023 13:45

MaryGao requested a review from xirzec October 31, 2023 13:45

joheredi approved these changes Nov 1, 2023

View reviewed changes

qiaozha approved these changes Nov 1, 2023

View reviewed changes

packages/typespec-ts/test/unit/clientFactoryGenerator.spec.ts Outdated Show resolved Hide resolved

packages/typespec-ts/test/modularUnit/operations.spec.ts Show resolved Hide resolved

MaryGao added 4 commits November 1, 2023 16:36

Refactor the credential option positions

4b00247

Update the ut after refactoring code

f318b2e

Regenerate smoke and integration files

b00c128

Regenerate in autorest package

b1dec56

MaryGao merged commit ba18c89 into Azure:main Nov 1, 2023
28 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable the batch tsp & chat api tsp #2090

Enable the batch tsp & chat api tsp #2090

MaryGao commented Oct 30, 2023 •

edited

qiaozha Oct 30, 2023

qiaozha Oct 30, 2023

MaryGao Oct 30, 2023 •

edited

qiaozha Oct 30, 2023

joheredi Oct 30, 2023

qiaozha Oct 31, 2023

xirzec Oct 31, 2023

qiaozha Oct 31, 2023

qiaozha Oct 31, 2023 •

edited

MaryGao Oct 31, 2023

MaryGao Oct 30, 2023 •

edited

xirzec Oct 30, 2023

MaryGao Oct 31, 2023 •

edited

MaryGao Oct 31, 2023 •

edited

qiaozha left a comment

Enable the batch tsp & chat api tsp #2090

Enable the batch tsp & chat api tsp #2090

Conversation

MaryGao commented Oct 30, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MaryGao Oct 30, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qiaozha Oct 31, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MaryGao Oct 30, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MaryGao Oct 31, 2023 • edited

Choose a reason for hiding this comment

MaryGao Oct 31, 2023 • edited

Choose a reason for hiding this comment

qiaozha left a comment

Choose a reason for hiding this comment

MaryGao commented Oct 30, 2023 •

edited

MaryGao Oct 30, 2023 •

edited

qiaozha Oct 31, 2023 •

edited

MaryGao Oct 30, 2023 •

edited

MaryGao Oct 31, 2023 •

edited

MaryGao Oct 31, 2023 •

edited