Training usingDSEC-Flow dataset

[Harin99]

I'm trying to train the model using desc-flow dataset.
While trying, I got some questions about it.

In this page (https://dsec.ifi.uzh.ch/dsec-datasets/download/), I could check that gt flow is consisted of png.

• png : ( 480, 640, 3 , uint8) _ like below
• Is it correct gt flow? Or Should i transform that png file for using in training?
  

And In 'dsec_loader.py', there is an function "load_flow" and I thought that function makes the flow & valid2d from png.
But When i apply the training dataset(.png flow) to the function('load_flow'), I get the tensor filled with only 0. And also that function require uint16 data type, but the png file that i downloaded in above link is uint8.

So I want to ask :
How can i generate the correct GT Flow & Mask(valid2d) from training dataset(.png flow)?
And If i want to train the E-Raft model using dsec dataset, should i train the model by comparing the flow generated from png and predictions?

Also When i debugged the 'dsec_loader.py', flow_timestamp variable is entered the image timestamps in "image_timestamp.txt".
Why should we put the image timestamp value in flow timestamp variable?
Is it because the test dataset generate the gt flow as a image and measures the metric?
If then, When i want to train the model using training dataset(dsec) in above link, should flow timestamp.txt value be entered as the flow timestamp variable?

Your reply will help me a lot. Thank you so much:)

[magehrig]

load_flow should be fixed now. Can you try again with the new version of load_flow?

Basically I changed

flow_16bit = imageio.imread(str(flowfile))

to

flow_16bit = imageio.imread(str(flowfile), format='PNG-FI')

[Harin99]

Oh! Thank you so much.. It was so helpful for me!!

1. Then, To train the model, should i compare the results of "load_flow" with the predictions?

Also, When i debugged the 'dsec_loader.py', flow_timestamp variable is entered the image timestamps in "image_timestamp.txt". But In the training dataset of DSEC, there are the flow_timestamp file.

2. Why did you put the image timestamp value in flow timestamp variable?

Thank you :)

[magehrig]

1. Yes
2. I don't understand. Could you point me to the line(s) of code you are referring to?

The optical flow reference time coincides with frames at 10 Hz, as described here:
https://dsec.ifi.uzh.ch/data-format/ -> see Optical Flow section
Maybe that answers your question?

[Harin99]

I should have asked the question in more detail, sorry.
My question is :

In "loader_dsec.py" ,
"image_timestamp.txt" is loaded, and then information of that file are input into the variable ( self.timestamp_flow ) __ [ like below figure ]

But we already have the file (test_forward_flow_timestamps.csv or train_forward_flow_timestamps.csv).

1. So Is there any reason to put the information in the image timestamp file into self.timestamp_flow variable?

2. Will it be a problem if i load the flow timestamp data right away, get the appropriate event, and put it in the model?

[magehrig]

The reason, if I remember correctly was that it was convenient to use the same pattern [::2][1:-1] to also extract the image indices which are useful (but also not required) for the benchmark submission: https://dsec.ifi.uzh.ch/optical-flow-submission-format/

In other words, you are right in that you don't need the image timestamps but can directly use the flow timestamps (which any way is more elegant). The code you see was just a quick hacky way to extract both timestamps and indices with the same pattern.

[magehrig]

I assume that I answered your question. Let me know if you still have questions regarding this topic.

[Harin99]

Thank you so much for your help. It will be great help to me.
Thank you!