Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(smithy-client): support strict timestamp parsing #2737

Merged
merged 1 commit into from
Sep 7, 2021

Conversation

adamthom-amzn
Copy link
Contributor

Description

From context, we know what the format of a timestamp should be, and we can
restrict the input to match that format rather than relying on the unspecified
behavior of new Date(string).

Testing

Unit tests, plus regeneration of clients, client protocol tests, server protocol tests. (those PRs to follow)

Additional context

  • We generally try to avoid adding dependencies, which is why I wrote these functions here, but there's potential for even more verification (like day of week) that I'm avoiding altogether that we might not have to with a real date/time library.
  • We could write a more performant parser without using regex, but getting the behavior and abstraction right first is more important to me.
  • I was split between putting these in date-utils or parse-utils, I'm open to suggestions.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Copy link
Contributor

@AllanZhengYP AllanZhengYP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't look very deep into all the date RFCs, only a few minor points to the syntax. You can leave the function in date-util. Otherwise, most other function in the smithy-client are parser-util.

packages/smithy-client/src/date-utils.ts Outdated Show resolved Hide resolved
packages/smithy-client/src/date-utils.ts Show resolved Hide resolved
packages/smithy-client/src/date-utils.ts Outdated Show resolved Hide resolved
packages/smithy-client/src/date-utils.ts Outdated Show resolved Hide resolved
packages/smithy-client/src/date-utils.ts Outdated Show resolved Hide resolved
packages/smithy-client/src/date-utils.ts Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented Sep 2, 2021

Codecov Report

❗ No coverage uploaded for pull request base (main@afeccd7). Click here to learn what that means.
The diff coverage is n/a.

❗ Current head 6aae1ae differs from pull request most recent head 7cf40f7. Consider uploading reports for the commit 7cf40f7 to get more accurate results
Impacted file tree graph

@@           Coverage Diff           @@
##             main    #2737   +/-   ##
=======================================
  Coverage        ?   61.48%           
=======================================
  Files           ?      540           
  Lines           ?    27617           
  Branches        ?     6745           
=======================================
  Hits            ?    16981           
  Misses          ?    10636           
  Partials        ?        0           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update afeccd7...7cf40f7. Read the comment docs.

Copy link
Member

@JordonPhillips JordonPhillips left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I put this in a few spots, but you're gonna have to account for leading zeroes being valid all over the place, where the strictParseX methods explicitly reject them. It'd be great if you could just use the built in parseInt but that'll probably take a single leading zero as an octal.

packages/smithy-client/src/date-utils.spec.ts Show resolved Hide resolved
packages/smithy-client/src/date-utils.ts Outdated Show resolved Hide resolved
packages/smithy-client/src/date-utils.ts Outdated Show resolved Hide resolved
packages/smithy-client/src/date-utils.ts Outdated Show resolved Hide resolved
packages/smithy-client/src/date-utils.ts Outdated Show resolved Hide resolved
Copy link
Member

@JordonPhillips JordonPhillips left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoops, meant to do request changes

From context, we know what the format of a timestamp should be, and we can
restrict the input to match that format rather than relying on the unspecified
behavior of new Date(string).
@adamthom-amzn adamthom-amzn merged commit c244678 into aws:main Sep 7, 2021
@adamthom-amzn adamthom-amzn deleted the strict-date-parsing branch September 7, 2021 16:51
kr8n3r added a commit to alphagov/paas-admin that referenced this pull request Sep 14, 2021
In release 3.30.0 (aws/aws-sdk-js-v3#2737)
AWS-SDK put stricter timestamp parsing.

Our stub data for cloudwatch metrics was outputing unix timestamp
strings, so aws-sdk parser was throwing `TypeError: Invalid RFC-3339
date-time`

In 7b1edc2
we updated the `getGappyRandomData` function to return unix timestamp
to address Prometheus stub data formats.

We don't want to change the output of the `getGappyRandomData` function,
so for cloudwatch stub data, we'll:

- parse the unixtimestamp string to an integer
- convert it to a valid date
- format date to RFC-3339 (ISO 8601-ish) format

Now AWS-SDK parser is happy.
kr8n3r added a commit to alphagov/paas-admin that referenced this pull request Sep 15, 2021
In release 3.30.0 (aws/aws-sdk-js-v3#2737)
AWS-SDK put stricter timestamp parsing.

Our stub data for cloudwatch metrics was outputing unix timestamp
strings, so aws-sdk parser was throwing `TypeError: Invalid RFC-3339
date-time`

In 7b1edc2
we updated the `getGappyRandomData` function to return unix timestamp
to address Prometheus stub data formats.

We don't want to change the output of the `getGappyRandomData` function,
so for cloudwatch stub data, we'll:

- parse the unixtimestamp string to an integer
- convert it to a valid date
- format date to RFC-3339 (ISO 8601-ish) format

Now AWS-SDK parser is happy.
@github-actions
Copy link

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 22, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants