Skip to content

feat: add support for maps #36

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed

feat: add support for maps #36

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ae626de
map properties
mhmd-azeez Oct 22, 2024
fb224b9
map schemas
mhmd-azeez Oct 22, 2024
02fd6fd
aaaaa
mhmd-azeez Oct 23, 2024
0f0038e
handle imports and exports
mhmd-azeez Oct 23, 2024
65b3f29
check for null
mhmd-azeez Oct 23, 2024
492b697
address PR comments
mhmd-azeez Oct 24, 2024
b09ed5c
refactor
mhmd-azeez Oct 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 141 additions & 2 deletions src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
let tp
if (property.$ref) {
tp = property.$ref.name
} else if (property.additionalProperties) {

Check failure on line 8 in src/index.ts

View workflow job for this annotation

GitHub Actions / test 

Property 'additionalProperties' does not exist on type 'Property | Parameter'.
tp = `Record<string, ${toTypeScriptType(property.additionalProperties)}>`

Check failure on line 9 in src/index.ts

View workflow job for this annotation

GitHub Actions / test 

Property 'additionalProperties' does not exist on type 'Property | Parameter'.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use Record over Map? my understanding is Record is pretty limited to the key and value types it can hold.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think i see the logic here. looks like keys can only really be strings anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a strong opinion here, but it seems like Record has a more straightforward path for json serialization: https://stackoverflow.com/questions/50153172/how-to-serialize-a-map-in-javascript

} else {
switch (property.type) {
case "string":
Expand Down Expand Up @@ -44,11 +46,143 @@
}
}

if (!tp) throw new Error("Cant convert property to typescript type: " + property.type)
if (!tp) throw new Error("Cant convert property to typescript type: " + JSON.stringify(property))
if (!property.nullable) return tp
return `${tp} | null`
}

function getBaseRef(property: Property): string {
if (property.additionalProperties) {

Check failure on line 55 in src/index.ts

View workflow job for this annotation

GitHub Actions / test 

Property 'additionalProperties' does not exist on type 'Property'.
if (property.additionalProperties.items?.$ref) {

Check failure on line 56 in src/index.ts

View workflow job for this annotation

GitHub Actions / test 

Property 'additionalProperties' does not exist on type 'Property'.
return property.additionalProperties.items.$ref.name

Check failure on line 57 in src/index.ts

View workflow job for this annotation

GitHub Actions / test 

Property 'additionalProperties' does not exist on type 'Property'.
}

return property.additionalProperties.$ref?.name || ''

Check failure on line 60 in src/index.ts

View workflow job for this annotation

GitHub Actions / test 

Property 'additionalProperties' does not exist on type 'Property'.
}

if (property.items) {
return property.items.$ref?.name || ''
}

return property.$ref?.name || ''
}

function propertyParsingSnippet(prop: Property): string | null {
const baseP = (prop.items ? prop.items : prop) as Property;
const baseRef = getBaseRef(prop);

if (helpers.isDateTime(baseP)) {
return `cast(dateFromJson, obj.${prop.name})`;
} else if (isBuffer(baseP)) {
return `cast(bufferFromJson, obj.${prop.name})`;
} else if (helpers.isMap(prop)) {

Check failure on line 78 in src/index.ts

View workflow job for this annotation

GitHub Actions / test 

Property 'isMap' does not exist on type '{ hasComment: (p: Property | Parameter | Export | null | undefined) => boolean; formatCommentLine: (s: string | null) => string; formatCommentBlock: (s: string | null, prefix?: string | undefined) => string; ... 8 more ...; snakeToPascalCase: (s: string) => string; }'.
return mapParsingSnippet(`obj.${prop.name}`, prop);
} else if (!helpers.isPrimitive(baseP)) {
return `cast(${baseRef}.fromJson, obj.${prop.name})`;
}
return null;
}
function mapParsingSnippet(source: string, prop: Property): string {
// Extract map's value type, handling both direct and array cases
const valueType = prop.additionalProperties?.items || prop.additionalProperties;

Check failure on line 87 in src/index.ts

View workflow job for this annotation

GitHub Actions / test 

Property 'additionalProperties' does not exist on type 'Property'.

Check failure on line 87 in src/index.ts

View workflow job for this annotation

GitHub Actions / test 

Property 'additionalProperties' does not exist on type 'Property'.
if (!valueType) return `mapFromJson(${source}, (v) => v)`;

// Get type annotation for the resulting map values
const typeAnnotation = getTypeAnnotation(valueType as Property);

// Generate converter based on value type
const valueConverter = buildParsingConverter(valueType as Property);

// Handle arrays
if (prop.additionalProperties?.items) {

Check failure on line 97 in src/index.ts

View workflow job for this annotation

GitHub Actions / test 

Property 'additionalProperties' does not exist on type 'Property'.
return `mapFromJson(${source}, (arr: any[]): ${typeAnnotation}[] => arr.map((v: any) => ${valueConverter}(v)))`;
}

// Handle simple maps
return `mapFromJson(${source}, ${valueConverter})`;
}

function buildParsingConverter(prop: Property): string {
// Handle nested maps
if (helpers.isMap(prop)) {
const innerValueType = prop.additionalProperties as Property;
const valueRef = getBaseRef(innerValueType);

if (!helpers.isPrimitive(innerValueType)) {
return `(v: any) => mapFromJson(v, ${valueRef}.fromJson)`;
}
return `(v: any) => mapFromJson(v, (vv) => vv as ${toTypeScriptType(innerValueType)})`;
}

// Handle non-map types
const ref = getBaseRef(prop);
if (!helpers.isPrimitive(prop)) {
return `${ref}.fromJson`;
}
return `(v: any) => v as ${toTypeScriptType(prop)}`;
}

function getTypeAnnotation(prop: Property): string {
if (helpers.isMap(prop)) {
const valueType = prop.additionalProperties as Property;
return getBaseRef(valueType) || toTypeScriptType(valueType);
}
return getBaseRef(prop) || toTypeScriptType(prop);
}

function propertyToJsonSnippet(prop: Property): string | null {
const baseP = (prop.items ? prop.items : prop) as Property;
const baseRef = getBaseRef(prop);

if (helpers.isDateTime(baseP)) {
return `cast(dateToJson, obj.${prop.name})`;
} else if (isBuffer(baseP)) {
return `cast(bufferToJson, obj.${prop.name})`;
} else if (helpers.isMap(prop)) {
return mapToJsonSnippet(`obj.${prop.name}`, prop);
} else if (!helpers.isPrimitive(baseP)) {
return `cast(${baseRef}.toJson, obj.${prop.name})`;
}
return null;
}

function mapToJsonSnippet(source: string, prop: Property): string {
// Extract map's value type, handling both direct and array cases
const valueType = prop.additionalProperties?.items || prop.additionalProperties;
if (!valueType) return `mapToJson(${source}, (v) => v)`;

// Generate converter based on value type
const valueConverter = buildValueConverter(valueType as Property);

// Handle arrays
if (prop.additionalProperties?.items) {
return `mapToJson(${source}, (arr) => arr.map((v) => ${valueConverter}(v)))`;
}

// Handle simple maps
return `mapToJson(${source}, ${valueConverter})`;
}

function buildValueConverter(prop: Property): string {
// Handle nested maps
if (helpers.isMap(prop)) {
const innerValueType = prop.additionalProperties as Property;
const valueRef = getBaseRef(innerValueType);

if (!helpers.isPrimitive(innerValueType)) {
return `(v) => mapToJson(v, ${valueRef}.toJson)`;
}
return `(v) => mapToJson(v, (vv) => vv)`;
}

// Handle non-map types
const ref = getBaseRef(prop);
if (!helpers.isPrimitive(prop)) {
return `${ref}.toJson`;
}
return `(v) => v`;
}

// TODO: can move this helper up to shared library?
function isBuffer(property: Property | Parameter): boolean {
return property.type === 'buffer'
Expand All @@ -61,7 +195,12 @@
...helpers,
isBuffer,
toTypeScriptType,
getBaseRef,
propertyToJsonSnippet,
mapToJsonSnippet,
propertyParsingSnippet,
mapParsingSnippet
}
const output = ejs.render(tmpl, ctx)
Host.outputString(output)
}
}
11 changes: 10 additions & 1 deletion template/src/index.ts.ejs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ import {
<% Object.values(schema.schemas).forEach(schema => { -%>
<%- schema.name %>,
<% }) -%>

mapFromJson,
mapToJson,
} from './pdk'
<% } %>

Expand All @@ -14,7 +17,10 @@ export function <%- ex.name %>(): number {
<% if (isBuffer(ex.input)) { -%>
const input: <%- toTypeScriptType(ex.input) %> = Host.base64ToArrayBuffer(JSON.parse(Host.inputString()))
<% } else if (isPrimitive(ex.input)) { -%>
const input: <%- toTypeScriptType(ex.input) %> = JSON.parse(Host.inputString())
const input: <%- toTypeScriptType(ex.input) %> = JSON.parse(Host.inputString())
<% } else if (isMap(ex.input)) { -%>
const untypedInput = JSON.parse(Host.inputString())
const input = <%- mapParsingSnippet('untypedInput', ex.input) %>
<% } else { -%>
const untypedInput = JSON.parse(Host.inputString())
const input = <%- toTypeScriptType(ex.input) %>.fromJson(untypedInput)
Expand Down Expand Up @@ -44,6 +50,9 @@ export function <%- ex.name %>(): number {
Host.outputString(JSON.stringify(Host.arrayBufferToBase64(output)))
<% } else if (isPrimitive(ex.output)) { -%>
Host.outputString(JSON.stringify(output))
<% } else if (isMap(ex.output)) { -%>
const untypedOutput = <%- mapToJsonSnippet('output', ex.output) %>
Host.outputString(JSON.stringify(untypedOutput))
<% } else { -%>
const untypedOutput = <%- toTypeScriptType(ex.output) %>.toJson(output)
Host.outputString(JSON.stringify(untypedOutput))
Expand Down
54 changes: 37 additions & 17 deletions template/src/pdk.ts.ejs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,30 @@ function cast(caster: (v: any) => any, v: any): any {
return caster(v)
}

export function mapFromJson<T>(obj: any, cast: (value: any) => T): Record<string, T> {
const result: Record<string, T> = {};

if (!obj) return result

for (const [key, value] of Object.entries(obj)) {
result[key] = cast(value);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure it's safe to mutate this and we may need to copy. we'll be changing the type of a property that a user my still be holding right?

Copy link
Contributor Author

@mhmd-azeez mhmd-azeez Oct 24, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are mutating the local copy result, so i think we should be fine

}

return result;
}

export function mapToJson<T>(obj: Record<string, T>, cast: (value: T) => any): any {
const result: Record<string, any> = {};

if (!obj) return result

for (const [key, value] of Object.entries(obj)) {
result[key] = cast(value);
}

return result;
}

function dateToJson(v: Date): string {
return v.toISOString()
}
Expand Down Expand Up @@ -48,31 +72,21 @@ export class <%- schema.name %> {
return {
...obj,
<% schema.properties.forEach(p => { -%>
<% let baseP = p.items ? p.items : p -%>
<% let baseRef = p.$ref ? p.$ref.name : (p.items && p.items.$ref ? p.items.$ref.name : null) -%>
<% if (isDateTime(baseP)) { -%>
<%- p.name -%>: cast(dateFromJson, obj.<%- p.name -%>),
<% } else if (isBuffer(baseP)) {-%>
<%- p.name -%>: cast(bufferFromJson, obj.<%- p.name -%>),
<% } else if (!isPrimitive(baseP)) {-%>
<%- p.name -%>: cast(<%- baseRef -%>.fromJson, obj.<%- p.name -%>),
<% let prop = propertyParsingSnippet(p) -%>
<% if (prop) { -%>
<%- p.name -%>: <%- prop %>,
<% } -%>
<% }) -%>
}
}

static toJson(obj: <%- schema.name %>): any{
static toJson(obj: <%- schema.name %>): any {
return {
...obj,
<% schema.properties.forEach(p => { -%>
<% let baseP = p.items ? p.items : p -%>
<% let baseRef = p.$ref ? p.$ref.name : (p.items && p.items.$ref ? p.items.$ref.name : null) -%>
<% if (isDateTime(baseP)) { -%>
<%- p.name -%>: cast(dateToJson, obj.<%- p.name -%>),
<% } else if (isBuffer(baseP)) {-%>
<%- p.name -%>: cast(bufferToJson, obj.<%- p.name -%>),
<% } else if (!isPrimitive(baseP)) {-%>
<%- p.name -%>: cast(<%- baseRef -%>.toJson, obj.<%- p.name -%>),
<% let prop = propertyToJsonSnippet(p) -%>
<% if (prop) { -%>
<%- p.name %>: <%- prop %>,
<% } -%>
<% }) -%>
}
Expand Down Expand Up @@ -117,6 +131,9 @@ export function <%- imp.name %>(<%- imp.input ? `input: ${toTypeScriptType(imp.i
<% if (isJsonEncoded(imp.input)) { -%>
<% if (isPrimitive(imp.input)) { %>
const mem = Memory.fromJsonObject(input as any)
<% } else if (isMap(imp.input)) { %>
const serialized = <%- mapToJsonSnippet('input', imp.input) %>
const mem = Memory.fromJsonObject(serialized)
<% } else { %>
const casted = <%- toTypeScriptType(imp.input) %>.toJson(input)
const mem = Memory.fromJsonObject(casted)
Expand All @@ -138,6 +155,9 @@ export function <%- imp.name %>(<%- imp.input ? `input: ${toTypeScriptType(imp.i
<% if (isJsonEncoded(imp.output)) { -%>
<% if (isPrimitive(imp.output)) { -%>
return Memory.find(ptr).readJsonObject();
<% } else if (isMap(imp.output)) { -%>
const output = Memory.find(ptr).readJsonObject();
return <%- mapParsingSnippet('output', imp.output) %>
<% } else { -%>
const output = Memory.find(ptr).readJsonObject();
return <%- toTypeScriptType(imp.output) %>.fromJson(output)
Expand Down
47 changes: 46 additions & 1 deletion tests/schemas/fruit.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,23 @@
exports:
mapExport:
description: This demonstrates how you can create an export that takes a map of
strings and returns a map of strings.
input:
type: object
additionalProperties:
type: array
items:
$ref: "#/components/schemas/WriteParams"
description: A map of strings
contentType: application/json
output:
type: object
additionalProperties:
type: array
items:
type: string
description: A map of strings
contentType: application/json
topLevelBuffJSON:
description: Top level json buffers
input:
Expand Down Expand Up @@ -55,6 +74,23 @@ exports:
contentType: application/json
$ref: "#/components/schemas/ComplexObject"
imports:
mapImport:
input:
type: object
additionalProperties:
type: array
items:
$ref: "#/components/schemas/WriteParams"
description: A map of strings
contentType: application/json
output:
type: object
additionalProperties:
type: array
items:
type: string
description: A map of strings
contentType: application/json
eatAFruit:
input:
contentType: text/plain; charset=utf-8
Expand All @@ -78,7 +114,6 @@ imports:
description: the raw byte values at key
kv_write:
description: kvwrite
contentType: application/json
input:
contentType: application/json
"$ref": "#/components/schemas/WriteParams"
Expand Down Expand Up @@ -138,4 +173,14 @@ components:
writeParams:
"$ref": "#/components/schemas/WriteParams"
nullable: true
anIntMap:
type: object
additionalProperties:
type: integer
format: int32
description: A map of integers
anObjectMap:
type: object
additionalProperties:
$ref: "#/components/schemas/WriteParams"
description: A complex json object
Loading