Timestamp of each frame

[crearo]

Is calling cameraTexture.getTimestamp() after cameraTexture.updateTexImage() the closest time in nanoseconds we can get to when the camera sensor exposed first row of image sensor?

I see a comment by @fadden on this SO post that says you can get the timestamp from the camera if you send the frames to a SurfaceTexture, and do your work in GLES instead of Java. Is the timestamp from SurfaceTexture the time when frame was captured, or received by the SurfaceTexture? Is the time between these two significantly low (<1ms) to be ignorable?

[crearo]

Alright, I'm almost certain the timestamp isn't accurate. Though the surfacetexture doc states
timestamps cannot generally be compared across SurfaceTexture instances, or across multiple program invocations., I am assuming these are System.nanoTime() - now System.nanoTime() in a single JVM instance uses one reference as timebase and so I should be able to compare them.

public void onFrameAvailable(SurfaceTexture cameraTexture) {
     long nanoTimeDiff = System.nanoTime() - cameraTexture.getTimestamp();
     // nanoTimeDiff is not a constant whole number (isn't zero, nor is it constant); 
     // nanoTimeDiff is different (almost) every time I open the app, 
     // but settles at a constant average after 100 frames
}

Any pointers would help!

[crearo]

Alriiiiight. I am probably wrong with my above comment. I checked the internals of Camera API 2 that claims CaptureResult.SENSOR_TIMESTAMP gives the Time at start of exposure of first row of the image sensor active array, in nanoseconds.. I dug into the AOSP and found the internal camera v2 CaptureCollector class uses a SurfaceTexture inside, just as in Grafika.

This probably makes sense because surfacetexture acts as a consumer for the sensor to start writing into. I'm just not sure if this is really the exact time at start of exposure of first row, or timestamp of writing onto surfacetexture when updateTexImage is called; but the difference if any is probably ignorable.

Another strange thing, I could not find SENSOR_INFO_TIMESTAMP_SOURCE_REALTIME being used anywhere (no getter, not setter), meaning all CaptureResult.SENSOR_TIMESTAMP will return SOURCE_UNKNOWN; which in my opinion is System.nanoTime().

Right before I stop trying to get a more accurate timestamp, the real reason I am digging into this is I'm trying to sync Android's sensor timestamps with camera's frame timestamps. This has turned out insanely challenging.

[paleozogt]

@crearo I've had the same problem in trying to sync the timestamps from the camera and the microphone. It's still unclear to me how that's supposed to work.

[fadden]

You may have better luck getting answers on stackoverflow, especially if you can get Eddy Talvala's attention. For example:

• https://stackoverflow.com/questions/38585761/what-is-the-timebase-of-the-timestamp-of-cameradevice/

Note in particular the last sentence, which seems to say that you generally can't do what you're trying to do... but the third comment suggests a possible workaround.

[crearo]

Thanks @fadden. I've tried his suggestions, and though logically it makes sense, the non-deterministic difference in gyro and camera frames timestamps persists. I'll put up a question on SO though.

[paleozogt]

@crearo if you post on SO, can you link to it here?