[Bug]: Toml quoted keys are not parsed properly

[sameerajayasoma]

Description

For each toml quoted key, I can see additional double quotes around it.

[{"basePath":"/api/entitlement-service","url":["https://abc.com/api/entitlement-service"],"config":{""foo.bar.baz"":"value"}}]

Config.toml

[[endpoints]]
basePath = "/api/entitlement-service"
url = ["https://abc.com/api/entitlement-service"]
config = {"foo.bar.baz" = "value"}

Steps to Reproduce

Use the below code snippet

type Endpoint record {|
    string basePath;
    string[] url;
    map<string> config;
|};

configurable Endpoint[] endpoints = [];

Config.toml

[[endpoints]]
basePath = "/api/entitlement-service"
url = ["https://abc.com/api/entitlement-service"]
config = {"foo.bar.baz" = "value"}

Affected Version(s)

2201.5.0

OS, DB, other environment details and versions

No response

Related area

-> Compilation

Related issue(s) (optional)

No response

Suggested label(s) (optional)

No response

Suggested assignee(s) (optional)

No response

[sameerajayasoma]

This will be a blocker pretty soon. We need this to implement service registration and discovery proposal.

[xlight05]

Had a look at this today.
From toml parsers side, As I remember, adding the quotes into quoted keys was intentional. This is due to complexities when retrieving via dotted keys.

Consider the following sample -

"foo.bar" = "value"

[foo]
bar = "value1"

Both the entries are different right? First one is a key value par in root level, and 2nd one is inside a table. But when we want to retrieve via dotted keys. we always have to use "foo.bar" for both cases. I believe that's why we made surrounding quotes required when we wanna retrieve quoted strings. This is the same behavior in parsers like Toml4j as well.

However, in Config.toml case this doesn't really matter. Had a chat with @HindujaB today. We'll introduce an API without quotes for the Configurables to use in the AST level.

[HindujaB]

I have a minor concern.
Since we support identifiers with special characters in Ballerina and if we have a configurable variable with quotes as follows,
(This is not yet supported)

configurable string \"foo\.bar\" = ?;

What should be the Config.toml representation?

"foo.bar" = "hello-foo.bar"

or

"\"foo.bar\"" = "hello-foo.bar"

As " in Toml spec is used to enclose toml reserved characters inside a key, I suppose the user has to enclose the entire variable name as option 2. In that case, we can remove the inconsistency from the above example in the issue and separately handle the identifier with special characters.

WDYT?
@sameerajayasoma @warunalakshitha

[warunalakshitha]

@HindujaB Option 1 would be more user friendly. But if there are any ambiguities we can support both

[HindujaB]

I agree.
With approach 1, we will need to handle the ambiguities.
For example,

type Rec record {
    string \"foo\.bar\";
    string foo\.bar; 
};

configurable Rec rec = ?;

This can create ambiguity as we can't provide values separately for both fields.

rec = {"foo.bar" = "hello"}