Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed issue #87 so that now SRT subtitles are read live and made local and remote file access consistent and easily extendable. #156

Merged
merged 42 commits into from Jul 27, 2016
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
e79ad93
Added TRUNC and FORCE commands so RemoteFileChannel has a full set of…
enternoescape Jul 18, 2016
d5d0398
Created new interfaces and classes for more seamless integration of l…
enternoescape Jul 18, 2016
a0b3486
Updated subtitle handlers to use the new remote file access methods t…
enternoescape Jul 18, 2016
5ee9d52
Merge pull request #2 from google/master
enternoescape Jul 18, 2016
b7d62c8
Fixed read only exceptions to throw a standard NonWritableChannelExce…
enternoescape Jul 18, 2016
573e82c
Fixed a logical problem with limit not being set correctly.
enternoescape Jul 18, 2016
89df89a
Fixed a typo in the test file name.
enternoescape Jul 18, 2016
8cc5abb
Simplified tests and added actual random read and write patterns to t…
enternoescape Jul 18, 2016
9c5b6fa
Fixed some wording the code.
enternoescape Jul 18, 2016
bbc0966
Incremented micro version.
enternoescape Jul 19, 2016
0234aac
Removed executeCommand from interface.
enternoescape Jul 19, 2016
895f15b
Removed isActiveFile() from interfaces.
enternoescape Jul 19, 2016
3aedae8
Lowered visibility of the RandomAccessFile behind LocalSageFile.
enternoescape Jul 19, 2016
30f190c
Removed wild cards from imports.
enternoescape Jul 19, 2016
1d11048
Removed executeCommand and isActiveFile.
enternoescape Jul 19, 2016
9d81f0f
Removed wild cards from imports.
enternoescape Jul 19, 2016
99689c9
synchronized encryptWrite.
enternoescape Jul 20, 2016
e14893d
Fixed read(byte[],int,int) method to read the most possible and corre…
enternoescape Jul 20, 2016
d3afef1
Changed fileChannel when in write mode to open up in read/write mode …
enternoescape Jul 20, 2016
aac89e5
Improved file length/size logic to be less aggressive by default.
enternoescape Jul 20, 2016
1759c6f
Added parameter for a remote file to be read as if it's an active fil…
enternoescape Jul 20, 2016
20894b4
Removed a debugging line accidentally left in and changed check for E…
enternoescape Jul 20, 2016
973a472
Incremented version.
enternoescape Jul 20, 2016
242d2ed
Incremented versions for MiniLauncher and SageLauncher.
enternoescape Jul 20, 2016
c50b743
Fixed upload ID missing, flushing after writes and a lack of optimal …
enternoescape Jul 20, 2016
251ebf5
Fixed some scatter, gather logic.
enternoescape Jul 20, 2016
06b7e5d
Fixed regression after verifying that the original code was working a…
enternoescape Jul 20, 2016
5683b94
Fixed write flush behavior when seeking backwards in the write buffer…
enternoescape Jul 21, 2016
ac52a36
Fixed variable being used to get currPlaylist. Fixed skip method so w…
enternoescape Jul 21, 2016
ac73615
Changed transferFrom(ReadableByteChannel, long) to more accurately ha…
enternoescape Jul 21, 2016
ddc3a3c
Added ensureProperFile(false) to the start of all read methods that d…
enternoescape Jul 21, 2016
e961fd3
Fixed the length not returning correctly when not at the end of the f…
enternoescape Jul 21, 2016
87432c4
Fixed skip on BlurayFile so it returns the number of bytes skipped in…
enternoescape Jul 22, 2016
aec8b6e
Moved flush in read(byte[], int, int) to just before the actual read …
enternoescape Jul 22, 2016
78be558
Wrapped LocalSageFile classes in BufferedSageFile classes.
enternoescape Jul 22, 2016
9fa860c
Fixed version and copyright.
enternoescape Jul 22, 2016
2f46a77
Added constant for determining if the next subtitle wasn't returned a…
enternoescape Jul 22, 2016
e0fde56
Remove remote file from access list before returning.
enternoescape Jul 22, 2016
028ec72
Remove raw file from access list when we close this subtitle handler.
enternoescape Jul 22, 2016
39fa023
Removed encryption from ModuleGroup.
enternoescape Jul 22, 2016
bf6db93
Fixed a few comments and JavaDocs to be more descriptive/accurate.
enternoescape Jul 22, 2016
9f2dec9
Moved test folder and reverted erroneous change in VobSubSubtitleHand…
enternoescape Jul 26, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
20 changes: 18 additions & 2 deletions build.gradle
Expand Up @@ -91,6 +91,14 @@ sourceSets {
]
}
}

test {
java {
srcDirs = [
'src/test/java'
]
}
}

// Miniclient configuration only specifies the Minimal files and
// javac will find and compile all the dependencies based on the
Expand All @@ -111,6 +119,8 @@ sourceSets {
}

dependencies {
testCompile 'org.testng:testng:6.1.1'

compile files(
'third_party/UPnPLib/sbbi-upnplib-1.0.3.jar',

Expand All @@ -124,11 +134,17 @@ dependencies {
compile "org.jogamp.gluegen:gluegen-rt-main:$jogampVer"


// Miniclient dependendencies for compiling
// Miniclient dependencies for compiling
miniclientCompile "org.jogamp.gluegen:gluegen-rt-main:$jogampVer"
miniclientCompile "org.jogamp.jogl:jogl-all-main:$jogampVer"
}

test {
useTestNG()
reports.html.destination = file("$sageBuildDir/reports/testng")
systemProperty 'io_test_file', "$sageBuildDir/delete-io-testng."
systemProperty 'nio_test_file', "$sageBuildDir/delete-nio-testng."
}

/**
* this is sort of a hack... normally we'd never set the -sourcepath on a java compile
Expand Down Expand Up @@ -344,7 +360,7 @@ task (restoreSageConstants) << {

// task dependencies
compileJava.dependsOn updateBuildNumber
sageJar.dependsOn cleanSageJar, classes
sageJar.dependsOn cleanSageJar, classes, test
miniclientJar.dependsOn clean, miniclientClasses, cleanMiniJar
linuxMiniClientRelease.dependsOn miniclientJar
sageJar.finalizedBy restoreSageConstants
Expand Down
51 changes: 27 additions & 24 deletions java/sage/FastMpeg2Reader.java
Expand Up @@ -15,11 +15,18 @@
*/
package sage;

import java.io.*;
import sage.nio.BufferedFileChannel;
import sage.nio.LocalFileChannel;
import sage.nio.RemoteFileChannel;
import sage.nio.SageFileChannel;

import java.io.EOFException;
import java.io.File;
import java.io.IOException;

public final class FastMpeg2Reader
{
private FasterRandomFile ins;
private SageFileChannel ins;
private sage.media.bluray.BluRayStreamer bdp; // for faster access w/out casting
private TranscodeEngine inxs;
private File mpegFile;
Expand Down Expand Up @@ -292,18 +299,15 @@ public void init(boolean findFirstPTS, boolean findDuration, boolean allowRemuxi
if (isBluRaySource)
{
if (Sage.DBG) System.out.println("MPEG2 pusher detected network BluRay source format...");
sage.media.bluray.BluRayNetworkFile bnf = new sage.media.bluray.BluRayNetworkFile(hostname, mpegFile, false, targetBDTitle);
sage.media.bluray.BluRayFile bnf = new sage.media.bluray.BluRayFile(hostname, mpegFile, false, targetBDTitle, 131072);
bdp = bnf;
ins = bnf;
bdp.setBufferSize(131072);
forceTSSource = true;
tsPacketSize = 192;
}
else
{
NetworkChannelFile nrf = new NetworkChannelFile(hostname, mpegFile, "r", null, false);
ins = nrf;
nrf.setBufferSize(131072);
ins = new BufferedFileChannel(new RemoteFileChannel(hostname, mpegFile), 131072, true);
// If we already know it's TS then set it that way.
// This is MUCH more reliable then the TS detection we have below which simply checks the first byte for 0x47
if (sourceFormat != null)
Expand Down Expand Up @@ -375,16 +379,15 @@ else if (streamTranscodeMode != null)
else if (mpegFile.isDirectory() && sage.media.format.MediaFormat.MPEG2_TS.equals(sourceFormat.getFormatName()))
{
if (Sage.DBG) System.out.println("MPEG2 pusher detected BluRay source format...");
sage.media.bluray.BluRayRandomFile bnf = new sage.media.bluray.BluRayRandomFile(mpegFile, false, targetBDTitle);
sage.media.bluray.BluRayFile bnf = new sage.media.bluray.BluRayFile(mpegFile, false, targetBDTitle, 65536);
bdp = bnf;
ins = bnf;
bdp.setBufferSize(65536);
forceTSSource = true;
tsPacketSize = 192;
}
else
{
ins = new BufferedFileChannel(mpegFile, "r", null, false);
ins = new sage.nio.BufferedFileChannel(new LocalFileChannel(mpegFile, true), true);
if (sourceFormat != null && sage.media.format.MediaFormat.VC1.equals(sourceFormat.getPrimaryVideoFormat()))
vc1RemapMode = true;
// If we already know it's TS then set it that way.
Expand Down Expand Up @@ -747,7 +750,7 @@ public long length()
{
try
{
return (ins != null) ? ins.length() : inxs.getVirtualTranscodeSize();
return (ins != null) ? ins.size() : inxs.getVirtualTranscodeSize();
}catch (Exception e){return 0;}
}

Expand All @@ -766,7 +769,7 @@ public boolean lengthGreaterThan(long testLength)

public long availableToRead()
{
return Math.max(0, (ins != null) ? (length() - ins.getFilePointer()) : inxs.getAvailableTranscodeBytes());
return Math.max(0, (ins != null) ? (length() - ins.position()) : inxs.getAvailableTranscodeBytes());
}

// This will return a number less than or equal to the argument; it avoids checking the actual file size if it knows
Expand All @@ -776,7 +779,7 @@ public long availableToRead2(long numBytes)
// Minimize the number of calls to length()
if (ins != null)
{
long currPos = ins.getFilePointer();
long currPos = ins.position();
if (lastCheckSize - currPos >= numBytes)
return numBytes;
lastCheckSize = length();
Expand All @@ -792,7 +795,7 @@ public boolean availableToRead(long numBytes)
// Minimize the number of calls to length()
if (ins != null)
{
long currPos = ins.getFilePointer();
long currPos = ins.position();
if (lastCheckSize - currPos >= numBytes)
return true;
lastCheckSize = length();
Expand All @@ -802,7 +805,7 @@ public boolean availableToRead(long numBytes)
return inxs.getAvailableTranscodeBytes() >= numBytes;
}

public long getReadPos() { return (ins != null) ? ins.getFilePointer() : inxs.getVirtualReadPosition(); }
public long getReadPos() { return (ins != null) ? ins.position() : inxs.getVirtualReadPosition(); }

public boolean canSkipOnNextRead()
{
Expand Down Expand Up @@ -874,7 +877,7 @@ else if (skipRatio > 1.25f && iframeSkipMultiplier4 > 4)
{
if (ins != null)
{
ins.seek(currBytePos);
ins.position(currBytePos);
}
buf.clear().limit(len);
int rv = read(buf, len);
Expand Down Expand Up @@ -913,7 +916,7 @@ else if (skipRatio > 1.25f && iframeSkipMultiplier4 > 4)
long llen = len;
while (llen > 0) // try 16k blocks for the CIFS issue
{
llen -= ins.transferTo(dest, llen);
llen -= ins.transferTo(llen, dest);
}
}
else
Expand Down Expand Up @@ -979,7 +982,7 @@ else if (alignIFrames && lastRawIFramePTS != 0 && lastRawVideoPTS > lastRawIFram
{
if (Sage.DBG) System.out.println("BluRay reseek at end of cell w/ not enough left for buffer; skipping to next cell");
long skipper = bdp.getBytesLeftInClip();
ins.skipBytes((int)skipper);
ins.skip(skipper);
bitsDone += 8*skipper;
}
buf.clear().limit(len);
Expand Down Expand Up @@ -1178,7 +1181,7 @@ else if ((b1 & 0x40) != 0 && (adaptation_field_control == 1 || adaptation_field_
// IDR start code found...good packet :)
waitForIFrame = false;
bytesSkippedForIFrameSync = tsStart;
ins.seek(lastPAT);
ins.position(lastPAT);
bitsDone = lastPAT*8;
numReseeksLeft = 0;
forceAnotherCycle = true;
Expand All @@ -1198,7 +1201,7 @@ else if (mpeg2Video)
// Found the I Frame
waitForIFrame = false;
bytesSkippedForIFrameSync = tsStart;
ins.seek(lastPAT);
ins.position(lastPAT);
bitsDone = lastPAT*8;
numReseeksLeft = 0;
forceAnotherCycle = true;
Expand Down Expand Up @@ -1382,7 +1385,7 @@ else if (mpeg2Video)
// Found the I Frame
waitForIFrame = false;
bytesSkippedForIFrameSync = i;
ins.seek(bitsDone/8 + packStart);
ins.position(bitsDone/8 + packStart);
bitsDone += packStart*8;
numReseeksLeft = 0;
forceAnotherCycle = true;
Expand Down Expand Up @@ -1643,7 +1646,7 @@ public void seekToBeginning() throws IOException
{
if (Sage.DBG) System.out.println("Mpeg2Reader seeking to pos=" + 0);
if (ins != null)
ins.seek(0);
ins.position(0);
else
inxs.seekToPosition(0);
bitsDone = 0;
Expand Down Expand Up @@ -1701,7 +1704,7 @@ private void seek(long seekTime, int minReadRequired) throws IOException
if (Sage.DBG) System.out.println("Mpeg2Reader seeking to pos=" + targetPos + " time=" + Sage.durFormatMillis(seekTime));
if (streamTranscodeMode != null)
transcodePTSOffset = seekTime*90;
ins.seek(targetPos);
ins.position(targetPos);
bitsDone = targetPos*8;
}
else
Expand Down Expand Up @@ -2473,7 +2476,7 @@ void skipBytes(long numBytes) throws IOException
numBytes -= numPeeks;
if (ins != null)
{
ins.seek(ins.getFilePointer() + numBytes);
ins.position(ins.position() + numBytes);
}
else
inxs.seekToPosition(inxs.getVirtualReadPosition() + numBytes);
Expand Down
20 changes: 11 additions & 9 deletions java/sage/FileDownloader.java
Expand Up @@ -15,6 +15,8 @@
*/
package sage;

import sage.io.LocalSageFile;

public class FileDownloader extends SystemTask
{
private static final int MP4_RESEEK_PREROLL_TIME = 1000;
Expand Down Expand Up @@ -239,7 +241,7 @@ public synchronized Object downloadFile(String serverName, String srcFile, java.
}
break;
}
fileOut = new FastRandomFile(myDestFile, "rw", Sage.I18N_CHARSET);
fileOut = new LocalSageFile(myDestFile, false);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FastRandomFile does buffered writes, so you should probably wrap this in the buffering class for equivalence.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

if (captureMode)
fileOut.seek(fileOut.length());
else
Expand Down Expand Up @@ -318,7 +320,7 @@ else if (remoteUIXfer)
outStream.flush();
str = Sage.readLineBytes(inStream);
remoteSize = Long.parseLong(str.substring(0, str.indexOf(' ')));
fileOut = new FastRandomFile(myDestFile, "rw", Sage.I18N_CHARSET);
fileOut = new LocalSageFile(myDestFile, false);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FastRandomFile does buffered writes, so you should probably wrap this in the buffering class for equivalence.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

if (captureMode)
fileOut.seek(fileOut.length());
else
Expand Down Expand Up @@ -616,11 +618,11 @@ else if (isMP4Stream && lastNotifyReadOffset > downloadedBytes + 256*1024)
waitForCircWrite(numRead);
synchronized (notifyRead)
{
if (fileOut.getFilePointer() + numRead < circSize)
if (fileOut.position() + numRead < circSize)
fileOut.write(decryptBuf, 0, numRead);
else
{
int firstWrite = (int)(circSize - fileOut.getFilePointer());
int firstWrite = (int)(circSize - fileOut.position());
fileOut.write(decryptBuf, 0, firstWrite);
fileOut.seek(0);
fileOut.write(decryptBuf, firstWrite, numRead - firstWrite);
Expand Down Expand Up @@ -648,11 +650,11 @@ else if (isMP4Stream && lastNotifyReadOffset > downloadedBytes + 256*1024)
waitForCircWrite(numRead);
synchronized (notifyRead)
{
if (fileOut.getFilePointer() + numRead < circSize)
if (fileOut.position() + numRead < circSize)
fileOut.write(xferBuf, 0, numRead);
else
{
int firstWrite = (int)(circSize - fileOut.getFilePointer());
int firstWrite = (int)(circSize - fileOut.position());
fileOut.write(xferBuf, 0, firstWrite);
fileOut.seek(0);
fileOut.write(xferBuf, firstWrite, numRead - firstWrite);
Expand Down Expand Up @@ -1458,11 +1460,11 @@ private void stvTaskRun()
if (circSize > 0)
{
waitForCircWrite(currRead);
if (fileOut.getFilePointer() + currRead < circSize)
if (fileOut.position() + currRead < circSize)
fileOut.write(xferBuf, 0, currRead);
else
{
int firstWrite = (int)(circSize - fileOut.getFilePointer());
int firstWrite = (int)(circSize - fileOut.position());
fileOut.write(xferBuf, 0, firstWrite);
fileOut.seek(0);
fileOut.write(xferBuf, firstWrite, currRead - firstWrite);
Expand Down Expand Up @@ -1755,7 +1757,7 @@ public DecryptInfo(String algorithm, byte[] key, byte[] iv)
protected java.net.Socket sock = null;
protected java.io.DataOutputStream outStream = null;
protected java.io.DataInputStream inStream = null;
protected FastRandomFile fileOut = null;
protected sage.io.SageFileSource fileOut = null;
protected long remoteSize;
protected boolean isMP4Stream;

Expand Down
14 changes: 9 additions & 5 deletions java/sage/FreetypeFont.java
Expand Up @@ -15,6 +15,10 @@
*/
package sage;

import sage.io.BufferedSageFile;
import sage.io.LocalSageFile;
import sage.io.SageDataFile;

public class FreetypeFont extends MetaFont
{
// Freetype is NOT designed for multi-threading so we need to ensure not more than a single thread goes into the native freetype code at one time
Expand Down Expand Up @@ -460,10 +464,10 @@ public SageRenderer.CachedFontGlyphs loadAcceleratedFont(int maxRequiredGlyphCod
if (cacheFile.isFile())
{
// Verify the number of glyph
FasterRandomFile cacheIn = null;
SageDataFile cacheIn = null;
try
{
cacheIn = new FasterRandomFile(cacheFile, "r", Sage.I18N_CHARSET);
cacheIn = new SageDataFile(new BufferedSageFile(new LocalSageFile(cacheFile, true)), Sage.I18N_CHARSET);
if(numGlyphs==cacheIn.readInt())
{
imageCount=cacheIn.readInt()-1;
Expand Down Expand Up @@ -494,7 +498,7 @@ public SageRenderer.CachedFontGlyphs loadAcceleratedFont(int maxRequiredGlyphCod
catch (Exception e)
{
System.out.println("Error reading font cache : " + e);
e.printStackTrace();
e.printStackTrace(System.out);
}
finally
{
Expand Down Expand Up @@ -575,11 +579,11 @@ public SageRenderer.CachedFontGlyphs loadAcceleratedFont(int maxRequiredGlyphCod
{
if (Sage.DBG) System.out.println("Saving cache version");
cacheFile.getParentFile().mkdirs();
FasterRandomFile cacheOut = null;
SageDataFile cacheOut = null;
try
{
// assumes font name is valid file string
cacheOut = new FasterRandomFile(cacheFile, "rw", Sage.I18N_CHARSET);
cacheOut = new SageDataFile(new BufferedSageFile(new LocalSageFile(cacheFile, false)), Sage.I18N_CHARSET);
cacheOut.writeInt(numGlyphs);
cacheOut.writeInt(imageCount+1);
int j;
Expand Down