Rotation problem when appending movies from different camera on the phone

[kkho]

Hi.

Maybe this has been reported before.
I have this method below that merges my videos into a final mp4 file.
However. When I record with the front camera first and then the back. The orientation changes.
Suddenly, the movie recorded from the back is now showing upside down.
The same happens when I record the backcamera and then the frontcamera.

What is the simple rotate mechanism for a video track?
It doesn't help to set orientation on video recording or the camera. It will give me the same issue.

my code

private void mergeMovies(File outFile) {
    try {
        int totalFilesToMerge = mMoviesToMergeList.get().size();
        Movie finalMovie = new Movie();
        Track[] movieTracks = new Track[totalFilesToMerge];
        Track[] audioTracks = new Track[totalFilesToMerge];

        for (int i = 0; i < totalFilesToMerge; i++) {
            Movie movie = MovieCreator.build(mMoviesToMergeList.get().get(i).getPath());
            if(movie.getTracks().get(0).getHandler().equals("vide")) {
                movieTracks[i] = movie.getTracks().get(0);
            }

            if(movie.getTracks().get(1).getHandler().equals("soun")) {
                audioTracks[i] = movie.getTracks().get(1);
            }

        }
        finalMovie.addTrack(new AppendTrack(movieTracks));
        finalMovie.addTrack(new AppendTrack(audioTracks));

        FileOutputStream fos = new FileOutputStream(outFile);
        BasicContainer container = (BasicContainer) new DefaultMp4Builder().build(finalMovie);
        container.writeContainer(fos.getChannel());
        fos.close();
    } catch (IOException ie) {
        Log.e("ERROR MOVIE", "Something is wrong");
    }
}

[sannies]

Oh my, front and back camera in this are physically oriented differently
and the orientation is correced in the file format. Unfortunately the
rotation is applied globally and there is no way to rotate just 'half of
the movie'. You will need to make sure that the video is fed to the
MediaCodec already in the correct orientation to make the append
operatation work as you expect.

2015-11-04 8:47 GMT+01:00 Khiem-Kim Ho Xuan notifications@github.com:

Hi.

Maybe this has been reported before.
I have this method below that merges my videos into a final mp4 file.
However. When I record with the front camera first and then the back. The
orientation changes.
Suddenly, the movie recorded from the back is now showing upside down.
The same happens when I record the backcamera and then the frontcamera.

What is the simple rotate mechanism for a video track?
It doesn't help to set orientation on video recording or the camera. It
will give me the same issue.

my code

private void mergeMovies(File outFile) {
try {
int totalFilesToMerge = mMoviesToMergeList.get().size();
Movie finalMovie = new Movie();
Track[] movieTracks = new Track[totalFilesToMerge];
Track[] audioTracks = new Track[totalFilesToMerge];

    for (int i = 0; i < totalFilesToMerge; i++) {
        Movie movie = MovieCreator.build(mMoviesToMergeList.get().get(i).getPath());
        if(movie.getTracks().get(0).getHandler().equals("vide")) {
            movieTracks[i] = movie.getTracks().get(0);
        }

        if(movie.getTracks().get(1).getHandler().equals("soun")) {
            audioTracks[i] = movie.getTracks().get(1);
        }

    }
    finalMovie.addTrack(new AppendTrack(movieTracks));
    finalMovie.addTrack(new AppendTrack(audioTracks));

    FileOutputStream fos = new FileOutputStream(outFile);
    BasicContainer container = (BasicContainer) new DefaultMp4Builder().build(finalMovie);
    container.writeContainer(fos.getChannel());
    fos.close();
} catch (IOException ie) {
    Log.e("ERROR MOVIE", "Something is wrong");
}

}

—
Reply to this email directly or view it on GitHub
#130.

[kkho]

hmmm yeah I thought that doing what I did now would solve the problem.
But so as far as I have understood it. I have set the mediarecorder orientation to what I want. But merging the files. It gives med the wrong orientation. I might have misunderstood what you mean. But if it is setting the orientation before recording a movie. Then I have done that. It just doesn't seem to be in effect when merging the files.

[sannies]

There is an 'encoding' orientation. That's what the encoder thinks where
the top is. The mp4 container has an orientation as well which is basically
a matrix transforming (this case: rotating) the decoded picture. If your
two parts have different orientation on encoder level you cannot correct
that on mp4 level as this is applied globally to the full track.
Does that help you?

Khiem-Kim Ho Xuan notifications@github.com schrieb am Mi., 4. Nov. 2015
9:33 AM:

hmmm yeah I thought that doing what I did now would solve the problem.
But so as far as I have understood it. I have set the mediarecorder
orientation to what I want. But merging the files. It gives med the wrong
orientation. I might have misunderstood what you mean. But if it is setting
the orientation before recording a movie. Then I have done that. It just
doesn't seem to be in effect when merging the files.

—
Reply to this email directly or view it on GitHub
#130 (comment).

[PunitD]

@kkho did you find any alternative to overcome this problem back then?

[PunitD]

@sannies is it possible to merge 2 video recorded in different orientation by changing some lib logic or is this impossible to achieve?

[ghost]

Has anyone managed to solve this issue? And if yes, could you be so kind to put some code snippet, to push other devs.

[BigNateBombBomb]

I'm guessing that no one has solved this issue using MP4Parser. I love the ease of use of the library but it seems like this one sticking point is going to force me to choose another way of joining the video files from opposite cameras.

[ShagunParikh]

Facing the same issue. Found any solution yet? Please help.
I have 2 video files. 1st is recorded from the front camera and 2nd is recorded from the back camera. When i play them individual, they seems correctly oriented. But after merging them, the orientation of 2nd video changes to upside down.

[kkho]

Hmmm any news? This should be solved somehow since there are many developers that are using the library

[ShagunParikh]

@kkho Have you been able to solve it somehow with some workaround?

[kkho]

No not yet.. My app had been on hold ever since. But a workaround should be possible but we will see

[kkho]

Explain how you did it?

[ShagunParikh]

@wandering7man Yes please explain. If possible please post a sample code showing how you did it?

[ShagunParikh]

I tried it with FFMEG library as well. But it has the same issue. @sannies Hope to see the issue solved soon as this is a great library except for this issue.

[kkm]

I need this, too. I believe, we need that a own playback.

[brianlee0113]

@wandering7man How did you solved it with ffmpeg? I also tried, but still ended with the same result.

[mendax92]

I hope to help you.
I am trying to encode video from the camera and audio from the microphone using MediaCodec and MediaMuxer. Rendering with OpenGL.
https://github.com/lumyus/FlexCam

[buntupana]

I've been stuck with this problem for a while. I tried with ffmpeg and besides that it's really big size library it took ages to append 90 seconds videos. I found a solution using the library CameraView. When I use this library to record videos from both cameras and after use Mp4Parser to append them, they are with the right rotation. I think the library is rotating the videos as it records or something like that. I hope some people find this useful.

EXTRA: I found CameraView library very useful and simple I would recommend people to use it

[HBiSoft]

That is how android works and there is a way around it.

When recording a video with the rear camera, in portrait mode, the rotation will be set to 90° because landscape (with the camera on the left) is 0°.

When recording a video with the front camera, in portrait mode, the rotation will be set to 270°.

Here is an image demonstrating this more clearly:

So, when you record a video in landscape (with the camera on the left hand side), you will not face this issue because the rotation is set to 0° (To be more precise - no rotate tag will be added).

---

So what you can do is, get the rotation of the video file - You can do so using Matrix as shown below:

public class GetVideoRotationInDegrees {
    private final String MP4_VIDEO_TAG = "vide";
    private final MatrixReader sMatrixReader = new MatrixReader();
    private final Object EXTRACT_LOCKER = new Object();
    // disable public construction.
    GetVideoRotationInDegrees(){}

    @SuppressLint("Assert")
    Matrix getRotation(String filePath) throws IOException, IllegalArgumentException {
        Movie video;
        synchronized (EXTRACT_LOCKER){
            video = MovieCreator.build(filePath);
        }

        assert(video.getTracks().size() > 0);
        for(int i=0; i<video.getTracks().size();i++){
            Track videoTrack = video.getTracks().get(i);
            if(videoTrack.getHandler().equals(MP4_VIDEO_TAG)){
                TrackMetaData metaData = videoTrack.getTrackMetaData();
                return getMatrixFromReader(metaData.getMatrix(), metaData.getWidth(), metaData.getHeight());
            }
        }
        throw new IllegalArgumentException("Cannot find video track in target file.");
    }
    
    private Matrix getMatrixFromReader(Matrix matrix, double width, double height) throws IllegalArgumentException{
        if(rotationEquals(matrix, Matrix.ROTATE_0)){
            Log.e("Rotation Was =", ""+Matrix.ROTATE_0);
            // using with and heigt to fix incorrect orientation on android
            if (width <= height){
                return Matrix.ROTATE_90;
            }
            return Matrix.ROTATE_0;
        }else if(rotationEquals(matrix, Matrix.ROTATE_90)){
            Log.e("Rotation Was =", ""+Matrix.ROTATE_90);
            return Matrix.ROTATE_90;
        }else if(rotationEquals(matrix, Matrix.ROTATE_180)){
            Log.e("Rotation Was =", ""+Matrix.ROTATE_180);
            return Matrix.ROTATE_180;
        }else if(rotationEquals(matrix, Matrix.ROTATE_270)){
            Log.e("Rotation Was =", ""+Matrix.ROTATE_270);
            return Matrix.ROTATE_270;
        }else{
            // The file did not have a rotate tag
            // On android, there will be no rotate tag added if recorded in landscape AFAICT..
            Log.e("Matrix -", "File did not have a rotate tag");
            return Matrix.ROTATE_0;
        }
    }

    private boolean rotationEquals(Matrix source, Matrix target){
        try {
            return Double.compare(sMatrixReader.getA(source), sMatrixReader.getA(target)) == 0 && Double.compare(sMatrixReader.getB(source), sMatrixReader.getB(target)) == 0 && Double.compare(sMatrixReader.getC(source), sMatrixReader.getC(target)) == 0 && Double.compare(sMatrixReader.getD(source), sMatrixReader.getD(target)) == 0;
        }   catch(Exception ex){return false;}
    }

    private class MatrixReader{
        final String FIELD_NAME_A = "a";
        final String FIELD_NAME_B = "b";
        final String FIELD_NAME_C = "c";
        final String FIELD_NAME_D = "d";

        private Field mField_A;
        private Field mField_B;
        private Field mField_C;
        private Field mField_D;

        MatrixReader(){
            try{
                mField_A = Matrix.class.getDeclaredField(FIELD_NAME_A);
                mField_A.setAccessible(true);
                mField_B = Matrix.class.getDeclaredField(FIELD_NAME_B);
                mField_B.setAccessible(true);
                mField_C = Matrix.class.getDeclaredField(FIELD_NAME_C);
                mField_C.setAccessible(true);
                mField_D = Matrix.class.getDeclaredField(FIELD_NAME_D);
                mField_D.setAccessible(true);
            }catch (NoSuchFieldException ex){
                // safe ignore
            }
        }
        double getA(Matrix m) throws IllegalAccessException{
            return (Double) mField_A.get(m);
        }
        double getB(Matrix m) throws IllegalAccessException{
            return (Double) mField_B.get(m);
        }
        double getC(Matrix m) throws IllegalAccessException{
            return (Double) mField_C.get(m);
        }
        double getD(Matrix m) throws IllegalAccessException{
            return (Double) mField_D.get(m);
        }
    }
}

You can call the above, from wherever, like this:

GetVideoRotationInDegrees getVideoRotationInDegrees = new GetVideoRotationInDegrees();
Matrix mRotation = getVideoRotationInDegrees.getRotation(path);

---

You can then fix the rotation by using the Matrix above and calling the following:

IsoFile isoFile = new IsoFile(srcVideo.getAbsolutePath());
FileOutputStream fileOutputStream = new FileOutputStream(destVideo.getAbsolutePath());
FileChannel channel = fileOutputStream.getChannel()

TrackHeaderBox thb = Path.getPath(isoFile, "/moov/trak/tkhd");
//This was called above
thb.setMatrix(mRotation);
isoFile.writeContainer(channel);

---

Note that some encoders do not add the rotate tag, but this question is related to android camera and android always adds this tag.

Also, I'm using the above with a TextureView and not to rotate a video and save it, but it is the same premise.

[buntupana]

@HBiSoft I tried your solution but it does not work when you append the videos. I'm still getting the same result

[jianinz]

@buntupana i tried what you said by using CameraView to record videos and Mp4Parser to append them together, i still get second video that to be concatenated upside down, i was using takeVideo method in CameraView, if i switch to use takeVideoSnapshot the rotation issue will be gone.

Right now, the work around i found is to use another library called https://github.com/natario1/Transcoder, it fixes rotation issue.