Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store the Mongo timestamps as Date objects #372

Closed
frbattid opened this issue Apr 27, 2015 · 3 comments
Closed

Store the Mongo timestamps as Date objects #372

frbattid opened this issue Apr 27, 2015 · 3 comments
Assignees
Milestone

Comments

@frbattid
Copy link
Member

Instead of "recvTimeTs": "1402409899391", use a Date object.

Nevertheless, an interesting behaviour to take into account: Date objects store miliseconds-based timestamps, but the Mongo console shows things like ISODate("2012-12-19T06:01:17.171Z"). This human-redable version of the timestamp may make unnecessary our "recvTime" field within the Json documents.

We should check if MySQL and CKAN have objects working in the same direction; if they have such objects, their usage should be study in order to probably remove the "recvTime" field in those backends. The problem here is HDFS, which does not have such a functionality and will be the unique backend keeping the "recvTime" field. This would break the homegeneity regarding the information all the backends are storing.

Effort: 1 man day

@fgalan
Copy link
Member

fgalan commented Apr 27, 2015

Depending on "implementation timing" migration scripts may be required. For example, if Cygnus 0.8.0 is released before implementing this (either in FIWARE or IoP) and people starts using the Mongo sink, a migration script would be needed to transform from string-based recvTimeTs to Date() based recvTimeTs.

Similar situation with MySQL (in this case, Cygnus with the support to MySQL has been already released, so the need for the migration script/procedure is confirmed in this case).

@frbattid
Copy link
Member Author

frbattid commented May 5, 2015

At first stage, OrionMongoSink will add this ISODate objects for the recvTime field. In addition, recvTimeTs will be discarded. The same way current STH implementation works.

@gtorodelvalle
Copy link
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants