Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FAI-6093: Handle different timestamp types in adapter #137

Merged
merged 1 commit into from
May 2, 2023
Merged

FAI-6093: Handle different timestamp types in adapter #137

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions src/adapter/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,11 @@ export function getFieldPaths(
} else if (isLeafType(type)) {
const fieldPath = fieldStack.join('.');
let newFieldPath = newFieldStack.join('.');
let fieldType = asLeafValueType(type);
// Timestamps in v1 are always stringified epoch millis, except when
// they're inside embedded object lists. In that case, they're stored
// as epoch millis.
const stringifyTimestamps = !mirrorPaths;
let fieldType = asLeafValueType(type, stringifyTimestamps);
if (mirrorPaths) {
fieldPaths[fieldPath] = {path: fieldPath, type: fieldType};
return undefined;
Expand All @@ -172,7 +176,7 @@ export function getFieldPaths(
// In V2, it's stored and typed like every other timestamp.
// We force conversion from ISO 8601 string => epoch millis
// string by overriding the type from string to timestamp.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment is out-of-date

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's accurate? This is pointing out an inconsistency in the v1 schema where refreshedAt is serialized as a timestamp, but is typed as a string. If we left the type as a string instead of overriding it, then we wouldn't convert the ISO timestamp to an epoch millis string.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I was being too literal. The override type is now epoch_millis_string.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. True.

fieldType = 'timestamp';
fieldType = 'epoch_millis_string';
}
} else {
// Prefix the last field name with the embedded field name
Expand Down Expand Up @@ -346,16 +350,15 @@ export class QueryAdapter {
return `${v2Value}`;
}
throw new VError('invalid long: %s', v2Value);
} else if (type === 'timestamp') {
// A timestamp will be an ISO string in v2, unless it's inside a list of
// embedded objects, in which case it will be an epoch millis number
} else if (type === 'epoch_millis' || type === 'epoch_millis_string') {
const stringify = type === 'epoch_millis_string';
if (_.isString(v2Value)) {
const millis = DateTime.fromISO(v2Value).toMillis();
if (!isNaN(millis)) {
return `${millis}`;
return stringify ? `${millis}` : millis;
}
} else if (_.isNumber(v2Value)) {
return v2Value;
return stringify ? `${v2Value}` : v2Value;
}
throw new VError('invalid timestamp: %s', v2Value);
}
Expand Down
13 changes: 10 additions & 3 deletions src/adapter/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ const primitiveTypes = [
'int',
'long',
'string',
'timestamp',
'epoch_millis',
'epoch_millis_string',
] as const;
type PrimitiveType = typeof primitiveTypes[number];

Expand Down Expand Up @@ -68,7 +69,10 @@ export interface FieldPaths {
readonly [path: string | symbol]: PathValue;
}

export function asLeafValueType(type: gql.GraphQLType): LeafValueType {
export function asLeafValueType(
type: gql.GraphQLType,
stringifyTimestamps = true
): LeafValueType {
function asPrimitiveType(type: gql.GraphQLLeafType): PrimitiveType {
if (gql.isEnumType(type) || type.name === 'String' || type.name === 'ID') {
return 'string';
Expand All @@ -83,7 +87,10 @@ export function asLeafValueType(type: gql.GraphQLType): LeafValueType {
} else if (type.name === 'Long') {
return 'long';
} else if (type.name === 'Timestamp') {
return 'timestamp';
if (stringifyTimestamps) {
return 'epoch_millis_string';
}
return 'epoch_millis';
}
throw new VError('unknown GraphQL leaf type: %s', type);
}
Expand Down
69 changes: 36 additions & 33 deletions test/adapter.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@ describe('AST utilities', () => {
},
'deployment.refreshedAt': {
path: 'deployment.metadata.refreshedAt',
type: 'timestamp'
type: 'epoch_millis_string'
},
'commit.sha': {
path: 'commit.sha',
Expand All @@ -204,7 +204,7 @@ describe('AST utilities', () => {
},
'commit.refreshedAt': {
path: 'commit.metadata.refreshedAt',
type: 'timestamp'
type: 'epoch_millis_string'
}
}
}
Expand Down Expand Up @@ -374,7 +374,7 @@ describe('AST utilities', () => {
nestedPaths: {
changedAt: {
path: 'changedAt',
type: 'timestamp'
type: 'epoch_millis'
},
'status.category': {
path: 'status.category',
Expand Down Expand Up @@ -548,90 +548,93 @@ describe('AST utilities', () => {
nestedPaths: {
uid: {path: 'uid', type: 'string'},
description: {path: 'description', type: 'string'},
createdAt: {path: 'createdAt', type: 'timestamp'},
createdAt: {path: 'createdAt', type: 'epoch_millis_string'},
origin: {path: 'metadata.origin', type: 'string'},
refreshedAt: {path: 'metadata.refreshedAt', type: 'timestamp'},
refreshedAt: {
path: 'metadata.refreshedAt',
type: 'epoch_millis_string',
},
releases: {
path: 'releases.nodes',
nestedPaths: {
'release.uid': {
path: 'release.uid',
type: 'string'
type: 'string',
},
'release.releasedAt': {
path: 'release.releasedAt',
type: 'timestamp'
type: 'epoch_millis_string',
},
'release.origin': {
path: 'release.metadata.origin',
type: 'string'
type: 'string',
},
'release.refreshedAt': {
path: 'release.metadata.refreshedAt',
type: 'timestamp'
}
}
type: 'epoch_millis_string',
},
},
},
tasks: {
path: 'tasks.nodes',
nestedPaths: {
'task.uid': {
path: 'task.uid',
type: 'string'
type: 'string',
},
'task.typeCategory': {
path: 'task.type.category',
type: 'string'
type: 'string',
},
'task.typeDetail': {
path: 'task.type.detail',
type: 'string'
type: 'string',
},
'task.origin': {
path: 'task.metadata.origin',
type: 'string'
type: 'string',
},
'task.refreshedAt': {
path: 'task.metadata.refreshedAt',
type: 'timestamp'
type: 'epoch_millis_string',
},
'task.parent.uid': {
path: 'task.parent.uid',
type: 'string'
type: 'string',
},
'task.parent.createdAt': {
path: 'task.parent.createdAt',
type: 'timestamp'
type: 'epoch_millis_string',
},
'task.additionalFields': {
path: 'task.additionalFields',
nestedPaths: {
name: {path: 'name', type: 'string'},
value: {path: 'value', type: 'string'}
}
value: {path: 'value', type: 'string'},
},
},
'task.statusChangelog': {
path: 'task.statusChangelog',
nestedPaths: {
changedAt: {
path: 'changedAt',
type: 'timestamp'
type: 'epoch_millis',
},
'status.category': {
path: 'status.category',
type: 'string'
type: 'string',
},
'status.detail': {
path: 'status.detail',
type: 'string'
}
}
}
}
}
}
}
}
type: 'string',
},
},
},
},
},
},
},
},
});
});
});
Expand Down Expand Up @@ -757,7 +760,7 @@ describe('query adapter', () => {
status: {category: 'c1a', detail: 'd1a'}
},
{
changedAt: 1667871145261,
changedAt: '2022-11-08T01:32:25.261Z',
Copy link
Contributor Author

@eskrm eskrm May 2, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test a mix of epoch millis and ISO timestamps that are inside embedded object lists are converted to epoch millis.

status: {category: 'c1b', detail: 'd1b'}
}
]
Expand All @@ -774,7 +777,7 @@ describe('query adapter', () => {
status: {category: 'c2a', detail: 'd2a'}
},
{
changedAt: 1667871145261,
changedAt: '2022-11-08T01:32:25.261Z',
status: {category: 'c2b', detail: 'd2b'}
}
]
Expand Down
Loading