Describe the bug
Brandon Davis 11:51 AM
Good morning, David.
Bryon and I have just run headlong into a weird video codec / rendering issue. Here's some ffprobe output.
ffprobe version 3.4.8-0ubuntu0.2 Copyright (c) 2007-2020 the FFmpeg developers
built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
configuration: --prefix=/usr --extra-version=0ubuntu0.2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
libavutil 55. 78.100 / 55. 78.100
libavcodec 57.107.100 / 57.107.100
libavformat 57. 83.100 / 57. 83.100
libavdevice 57. 10.100 / 57. 10.100
libavfilter 6.107.100 / 6.107.100
libavresample 3. 7. 0 / 3. 7. 0
libswscale 4. 8.100 / 4. 8.100
libswresample 2. 9.100 / 2. 9.100
libpostproc 54. 7.100 / 54. 7.100
Input #0, mpeg, from 'wrasse.mpg/wrasse.mpg':
Duration: 00:00:07.16, start: 0.224400, bitrate: 4824 kb/s
Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv, top first), 720x480 [SAR 32:27 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
Stream #0:1[0x80]: Audio: ac3, 48000 Hz, stereo, fltp, 256 kb/s
When transcode to mp4, the width, height, DAR, PAR are the same. When we play it in a browser, the width is reported by a video player as 853. Indeed, the displayed aspect ratio (as checked with a measuring tape) is 16:9 (not 3:2, as 720x480 actually is)
Naturally, when we store annotations from this stretched coordinate space, they don't match up with the video pixels. Have you had to deal with this problem before? What's a good way to manage these? Does GeoJS have tools that will put drawn shapes into the video's coordinate space given the PAR? Should we transform them at export time?
@bryon Lewis please add any detail I may have missed. (edited)
manthey 1:33 PM
If you know you have non-square pixels, there are several ways to apply a transform in GeoJS. ...
1:35
You can construct an appropriate proj4 string to do the transform. Or you could add a position() accessor to the shape feature layers to transform the coordinates for rendering (e.g., feature.position((d, i) => {x: d.x * 1.2, y: d.y}))
1:36
I think if the video is already being stretched to the correct shape in the quad, then you'd want to use the position accessor.
Brandon Davis 1:48 PM
I'd like to try both and see which makes sense for us. the feature position thing seems relatively straightforward.
Do you have any references to an example where geo.transform is used? If I construct an affine transform and apply it to the whole map through geo.map.spec.gcs, will that do what I epxect?
manthey 1:49 PM
The feature position will work for rendering, but if you are drawing annotations it will probably be more work than the transform. I know I have an example of applying the transform, but will take me a short while to dig it up.
1:50
The transform is applies via gcs, yes.
Brandon Davis 1:53 PM
Thank you.
manthey 2:09 PM
Try a gcs string like +proj=longlat +axis=enu +s11=1.2 +s12=0 +s21=0 +s22=1 +xoff=0 +yoff=0.
2:10
You would do something like layer.gcs('+proj=longlat +axis=enu +s11=1.2 +s12=0 +s21=0 +s22=1 +xoff=0 +yoff=0') for the annotation layer. (edited)
Brandon Davis 2:11 PM
I think this will get us most of the way there. Thank you!
Related data
https://viame.kitware.com/girder#user/60243dea4d85a3cb4e2250c5/folder/602441854d85a3cb4e2250d2
Describe the bug
Brandon Davis 11:51 AM
Good morning, David.
Bryon and I have just run headlong into a weird video codec / rendering issue. Here's some ffprobe output.
When transcode to mp4, the width, height, DAR, PAR are the same. When we play it in a browser, the width is reported by a video player as 853. Indeed, the displayed aspect ratio (as checked with a measuring tape) is 16:9 (not 3:2, as 720x480 actually is)
Naturally, when we store annotations from this stretched coordinate space, they don't match up with the video pixels. Have you had to deal with this problem before? What's a good way to manage these? Does GeoJS have tools that will put drawn shapes into the video's coordinate space given the PAR? Should we transform them at export time?
@bryon Lewis please add any detail I may have missed. (edited)
manthey 1:33 PM
If you know you have non-square pixels, there are several ways to apply a transform in GeoJS. ...
1:35
You can construct an appropriate proj4 string to do the transform. Or you could add a position() accessor to the shape feature layers to transform the coordinates for rendering (e.g., feature.position((d, i) => {x: d.x * 1.2, y: d.y}))
1:36
I think if the video is already being stretched to the correct shape in the quad, then you'd want to use the position accessor.
Brandon Davis 1:48 PM
I'd like to try both and see which makes sense for us. the feature position thing seems relatively straightforward.
Do you have any references to an example where geo.transform is used? If I construct an affine transform and apply it to the whole map through geo.map.spec.gcs, will that do what I epxect?
manthey 1:49 PM
The feature position will work for rendering, but if you are drawing annotations it will probably be more work than the transform. I know I have an example of applying the transform, but will take me a short while to dig it up.
1:50
The transform is applies via gcs, yes.
Brandon Davis 1:53 PM
Thank you.
manthey 2:09 PM
Try a gcs string like +proj=longlat +axis=enu +s11=1.2 +s12=0 +s21=0 +s22=1 +xoff=0 +yoff=0.
2:10
You would do something like layer.gcs('+proj=longlat +axis=enu +s11=1.2 +s12=0 +s21=0 +s22=1 +xoff=0 +yoff=0') for the annotation layer. (edited)
Brandon Davis 2:11 PM
I think this will get us most of the way there. Thank you!
Related data
https://viame.kitware.com/girder#user/60243dea4d85a3cb4e2250c5/folder/602441854d85a3cb4e2250d2