Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hard-coded output matrix type and incomplete MatrixType definition #44

Open
i-am-sijia opened this issue Jun 10, 2021 · 1 comment
Open

Comments

@i-am-sijia
Copy link
Collaborator

Describe the bug

Failed to write out trip table matrices from CTRAMP, using the latest mtctm2.jar and common-base.jar.

08-Jun-2021 10:55:16:440, ERROR, InvocationTargetException exception making RMI method call: //172.22.78.49:1191/com.pb.mtctm2.abm.ctramp.MatrixDataServer.writeMatrixFile().
java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at gnu.cajo.invoke.Remote.invoke(Unknown Source)
	at gnu.cajo.invoke.Remote.invoke(Unknown Source)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
	at sun.rmi.transport.Transport$1.run(Unknown Source)
	at sun.rmi.transport.Transport$1.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Unknown Source)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$240(Unknown Source)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$1/1876363557.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(Unknown Source)
	at sun.rmi.transport.StreamRemoteCall.executeCall(Unknown Source)
	at sun.rmi.server.UnicastRef.invoke(Unknown Source)
	at gnu.cajo.invoke.Remote_Stub.invoke(Unknown Source)
	at gnu.cajo.invoke.Remote.invoke(Unknown Source)
	at com.pb.mtctm2.abm.ctramp.UtilRmi.method(UtilRmi.java:123)
	at com.pb.mtctm2.abm.ctramp.MatrixDataServerRmi.writeMatrixFile(MatrixDataServerRmi.java:41)
	at com.pb.mtctm2.abm.application.MTCTM2TripTables.writeMatricesToFile(MTCTM2TripTables.java:546)
	at com.pb.mtctm2.abm.application.MTCTM2TripTables.writeTrips(MTCTM2TripTables.java:503)
	at com.pb.mtctm2.abm.application.MTCTM2TripTables.createTripTables(MTCTM2TripTables.java:282)
	at com.pb.mtctm2.abm.application.MTCTM2TripTables.main(MTCTM2TripTables.java:717)
Caused by: java.lang.RuntimeException: Could not determine type of matrix file, D:\MTC\travel-model-two\ctramp_output\auto_EA_SOV_GP_EA.mat
	at com.pb.common.matrix.MatrixReader.determineMatrixType(MatrixReader.java:193)
	at com.pb.common.matrix.MatrixWriter.createWriter(MatrixWriter.java:115)
	at com.pb.mtctm2.abm.ctramp.MatrixDataServer.writeMatrixFile(MatrixDataServer.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at gnu.cajo.invoke.Remote.invoke(Unknown Source)
	at gnu.cajo.invoke.Remote.invoke(Unknown Source)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
	at sun.rmi.transport.Transport$1.run(Unknown Source)
	at sun.rmi.transport.Transport$1.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Unknown Source)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$240(Unknown Source)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$1/1876363557.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

To Reproduce

Steps to reproduce the behavior:
Run CTRAMP/runtime/runMTCTM2ABM.cmd

rem ### create demand matrices in Cube matrix format - must restart mtx manager before running?
java -Xmx200g -cp "%CLASSPATH%" -Dproject.folder=%PROJECT_DIRECTORY% com.pb.mtctm2.abm.application.MTCTM2TripTables mtctm2 -iteration %iteration% -sampleRate %sampleRate%

Triggering line of code

  1. output matrix file extension is hard-coded as "mat". Matrix writer uses filename extension to determine matrix type:

public void writeMatricesToFile(String fileName, Matrix[] m){
//todo - currently write one matrix to each file since there is a problem
//calling RemoteMatrixDataServer.writeMatrices from RMI if the matrix array has more than 1 matrix
for (int i=0; i<m.length; i++) {
String matFileName = fileName + "_" + m[i].getName() + "." + matrixFileExtension;
Matrix[] temp = new Matrix[1];
temp[0] = m[i];
logger.info( m[i].getName() + " has " + m[i].getRowCount() + " rows, " + m[i].getColumnCount() + " cols, and a total of " + m[i].getSum() );
logger.info("Writing " + matFileName);
ms.writeMatrixFile(matFileName, temp);
}
}

  1. ".mat" is not defined in the common-base DetermineMatrixType method

public static MatrixType determineMatrixType(File file) {
MatrixType type;
String fileName = file.getName();
if (fileName.indexOf(".bin") > 0 || fileName.indexOf(".BIN") > 0 ||
fileName.indexOf(".binary") > 0 || fileName.indexOf(".BINARY") > 0)
{
type = MatrixType.BINARY;
}
else if (fileName.indexOf(".zip") > 0 || fileName.indexOf(".ZIP") > 0
|| fileName.indexOf(".zmx") > 0
|| fileName.indexOf(".compressed") > 0
|| fileName.indexOf(".COMPRESSED") > 0) {
type = MatrixType.ZIP;
}
else if (fileName.indexOf(".csv") > 0 || fileName.indexOf(".CSV") > 0) {
type = MatrixType.CSV;
}
else if (fileName.indexOf(".emme2") > 0 || fileName.indexOf(".EMME2") > 0 ||
fileName.indexOf(".e2ban") > 0 || fileName.indexOf(".E2BAN") > 0 ||
fileName.indexOf(".em2") > 0 || fileName.indexOf(".EM2") > 0 ||
fileName.indexOf("emme2ban") >= 0 || fileName.indexOf("EMME2BAN") >= 0 ||
fileName.equalsIgnoreCase("emme2ban") || fileName.equalsIgnoreCase("emmebank"))
{
type = MatrixType.EMME2;
}
else if (fileName.indexOf(".tpp") > 0 || fileName.indexOf(".TPP") > 0 ||
fileName.indexOf(".tpplus") > 0 || fileName.indexOf(".TPPLUS") > 0)
{
type = MatrixType.TPPLUS;
}
else if (fileName.indexOf(".mtx") > 0 || fileName.indexOf(".MTX") > 0 ||
fileName.indexOf(".transcad") > 0 || fileName.indexOf(".TRANSCAD") > 0)
{
type = MatrixType.TRANSCAD;
}
else if (fileName.indexOf(".omx") > 0 || fileName.indexOf(".OMX") > 0) {
type = MatrixType.OMX;
}
else {
throw new RuntimeException("Could not determine type of matrix file, " + file.getAbsolutePath());
}
return type;
}

Thoughts on resolution

  1. Remove the hard-coded matrix file extension, replace it with reading from properties file
  2. Add if condition for ".mat" in DetermineMatrixType()

Environment

windows 10
java 1.8.0_31

fyi @jfdman @lmz @DavidOry

i-am-sijia added a commit that referenced this issue Jun 16, 2021
@i-am-sijia
Copy link
Collaborator Author

I see in transit-ccr branch, the hard-coded matrix file extension is changed to omx, instead of mat. This is a quick solution to get the omx working, but in the long run, maybe it's better to define the matrix type in properties file.

i-am-sijia added a commit that referenced this issue Aug 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant