Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decode "timestamp" from oplog.timestamp #324

Closed
howardkhl opened this issue Sep 10, 2015 · 8 comments

Comments

@howardkhl
Copy link

commented Sep 10, 2015

On the wiki it says:
The exact format of this file depends on MongoDB's toplogy. For a single replica set, the format is:

["oplog name", timestamp]

An example would be:

["Collection(Database(MongoClient('localhost', 27017), u'local'), u'oplog.rs')", 6185588281674039941]

What format is this timestamp in, and how do you convert say, 6185588281674039941, to a normal date?

Thanks.

@howardkhl

This comment has been minimized.

Copy link
Author

commented Sep 10, 2015

In local.oplog.rs, oplog entry has timestamp value like this:

"ts" : Timestamp(1440194492, 391)

How is it relate to the timestamp format in oplog.timestamp? Perhaps this in oplog_manager.py?

timestamp = util.bson_ts_to_long(entry['ts'])

If so, is there a way to convert this format back to a human readable date in mongo shell?

@howardkhl

This comment has been minimized.

Copy link
Author

commented Sep 11, 2015

Okay, found a solution for this.

util.py has a method long_to_bson_ts, use Python to run the method directly:

python -c 'import util; print util.long_to_bson_ts(<val>)'

For example:

python -c 'import util; print util.long_to_bson_ts(6185145118358503425)'

And you will get a Timestamp object back.

Timestamp(1440091319, 1)

In python, use datetime to convert Timestamp to human readable form:

python
>>> from datetime import datetime
>>> str(datetime.fromtimestamp(1440091319))

It will return:

'2015-08-20 10:21:59'

For comparison purpose, this command will return current system time:

>>> str(datetime.now())

@howardkhl howardkhl closed this Sep 11, 2015

@behackett

This comment has been minimized.

Copy link
Contributor

commented Sep 11, 2015

Please note that the BSON Timestamp type is seconds since the Unix epoch plus an increment to represent multiple operations within the same second. See this for more information:

http://docs.mongodb.org/master/reference/bson-types/#timestamps

@upenblog

This comment has been minimized.

Copy link

commented Sep 15, 2015

Thanks@ behackett

followed the same steps but I get current time stamp no matter what value you I use
rs0:SECONDARY> new ISODate(new Timestamp(1442245369, 356) * 1000)
ISODate("2015-09-15T20:28:11.099Z")

rs0:SECONDARY> new ISODate(new Timestamp(1442244777, 386) * 1000)
ISODate("2015-09-15T20:28:30.838Z")

@howardkhl

This comment has been minimized.

Copy link
Author

commented Sep 15, 2015

@upenblog Please follow updated step. Go to python and use datetime.

@howardkhl

This comment has been minimized.

Copy link
Author

commented Mar 31, 2017

troubleshooting notes, if you get:

ImportError: No module named util
Make sure you git pull mongo-connector project, and you run the python command from mongo-connector/mongo_connector directory where util.py is.

ImportError: No module named mongo_connector.compat
Edit util.py, change line 26: from mongo_connector.compat import reraise to from compat import reraise

@howardkhl

This comment has been minimized.

Copy link
Author

commented Mar 31, 2017

Extra credit: if you need to convert from bson timestamp to long (integer format used in oplog.timestamp):
python -c 'import util; from bson.timestamp import Timestamp; print util.bson_ts_to_long(Timestamp(<val>, <inc>))'

E.g.
python -c 'import util; from bson.timestamp import Timestamp; print util.bson_ts_to_long(Timestamp(1490951863, 8))'
6403589491495272456

@leroichou

This comment has been minimized.

Copy link

commented Apr 4, 2018

Traceback (most recent call last):
File "<string>", line 1, in <module>
File "util.py", line 22, in <module>
from bson import Timestamp
ImportError: cannot import name Timestamp

It failed when I imported the util module. How do I resolve the problem like this?

All right, I know what question is. I need to install the pymongo module.
#pip install pymonngo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.