https://github.com/nuvolabase/orientdb/pull/1406 #1430

Merged
merged 5 commits into from Apr 12, 2013

Projects

None yet

2 participants

@henryzhao81
Contributor

Fix #1406 which failed in Unittest.
2 Fixes :

  1. Remove set className to previous for loop. Otherwise if in request JSON string, if @class is not first object. In following loop, all fields type which before @class will always be null.
  2. If value instanceof ODocument and type is LINK, then save object.

Passed unit test

Contributor

One more check in :
Check in change for OSchedule (#1226)
In orientdb-server-config.xml :
Add a new handler

< handler class="com.orientechnologies.orient.server.schedule.OScheduleHandler" >
< parameters>
< parameter name="enabled" value="false" />
< parameter name="databaseName" value="databaseName" />
< parameter name="user" value="admin" />
< parameter name="pass" value="admin" />
< /parameters >
< /handler >

A class will be created, each record represent a job : name : job name rule : cron like rule (*/5 \* \* \* *) : http://www.sauronsoftware.it/projects/cron4j/manual.php#p14 - 2. Scheduling patterns arguments : function parameters status : job status function : link to the job function (OFunction) starttime : last start time start : the switch to start or stop job

Once change enabled to true in orientdb-server-config.xml, timer thread will be start, every minute, it will check rule for each job, if rule is qualified, then the job will be started.

  1. Create Job : name, rule and function are mandatory fields. Once job is created, by default start is false, status is STOPPED. Job name should be unique.
  2. Start Job : update start to true, the status will be automatically change to WAITTING, and wait for time thread to check rule to decide if it is ok to run in each minute.
  3. Run Job : if rule is qulified to run. Then the status will automatically change to RUNNING, and execute function. After function exectued, change back to WAITTING.
  4. Stop Job : change start to false, the job will be stopped.

OScheduleHandler.java :
Contains TimerThread, to check each job rule and start the job if rule qualified.

OScheduler.java :
Each job.

OSchedulerListenerImpl.java :
Methods to retrieve, load , add and remove job.

OSchedulerTrigger.java :
Check and change job status.

Contributor

One question : I run ant clean test, unit test passed on my Linux machine (ubuntu), but failed on my macbook.
Following are the error log on my macbook, not sure what's going on.

[testng] index intstanceof class com.orientechnologies.orient.core.index.OIndexTxAwareOneValue

[testng] FAILED CONFIGURATION: @BeforeClass setUp
[testng] java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 10.7.2.31; nested exception is:
[testng] java.net.ConnectException: Operation timed out]
[testng] at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:357)
[testng] at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:267)
[testng] at com.orientechnologies.orient.test.database.auto.AbstractIndexReuseTest.getProfilerInstance(AbstractIndexReuseTest.java:67)
[testng] at com.orientechnologies.orient.test.database.auto.AbstractIndexReuseTest.setUp(AbstractIndexReuseTest.java:33)
[testng] at com.orientechnologies.orient.test.database.auto.SQLIndexWithoutSchemaTest.setUp(SQLIndexWithoutSchemaTest.java:40)
[testng] Caused by: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 10.7.2.31; nested exception is:
[testng] java.net.ConnectException: Operation timed out]
[testng] at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:118)
[testng] at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:203)
[testng] at javax.naming.InitialContext.lookup(InitialContext.java:411)
[testng] at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1924)
[testng] at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1891)
[testng] at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:274)
[testng] ... 26 more
[testng] Caused by: java.rmi.ConnectException: Connection refused to host: 10.7.2.31; nested exception is:
[testng] java.net.ConnectException: Operation timed out
[testng] at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
[testng] at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
[testng] at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
[testng] at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:340)
[testng] at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
[testng] at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:114)
[testng] ... 31 more
[testng] Caused by: java.net.ConnectException: Operation timed out
[testng] at java.net.PlainSocketImpl.socketConnect(Native Method)
[testng] at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[testng] at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[testng] at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[testng] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[testng] at java.net.Socket.connect(Socket.java:579)
[testng] at java.net.Socket.connect(Socket.java:528)
[testng] at java.net.Socket.(Socket.java:425)
[testng] at java.net.Socket.(Socket.java:208)
[testng] at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
[testng] at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:146)
[testng] at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
[testng] ... 36 more
[testng] ... Removed 22 stack frames
[testng] FAILED CONFIGURATION: @BeforeClass setUp

FAILED: insertAvoidingSubQuery
[testng] com.orientechnologies.orient.core.sql.OCommandSQLParsingException: Error on parsing command at position #0: Error on parsing command at position #11: Class TEST not found in database
[testng] Command: INSERT INTO test(text) VALUES ('(Hello World)')
[testng] -------------------^
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:517)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.handleStatus(OChannelBinary.java:470)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:145)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:59)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemote.beginResponse(OStorageRemote.java:1877)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemote.command(OStorageRemote.java:1039)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemoteThread.command(OStorageRemoteThread.java:261)
[testng] at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:59)
[testng] at com.orientechnologies.orient.test.database.auto.SQLInsertTest.insertAvoidingSubQuery(SQLInsertTest.java:238)
[testng] ... Removed 26 stack frames
[testng] FAILED: insertCluster
[testng] com.orientechnologies.orient.core.exception.ODatabaseException: Cannot open database
[testng] at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:117)
[testng] at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
[testng] at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.open(ODatabaseRecordAbstract.java:116)
[testng] at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
[testng] at com.orientechnologies.orient.test.database.auto.SQLInsertTest.insertCluster(SQLInsertTest.java:263)
[testng] Caused by: java.lang.IllegalStateException: Database demo is already open
[testng] at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:94)
[testng] ... 26 more
[testng] ... Removed 22 stack frames
[testng] FAILED: insertSubQuery
[testng] com.orientechnologies.orient.core.sql.OCommandSQLParsingException: Error on parsing command at position #0: Error on parsing command at position #11: Class TEST not found in database
[testng] Command: INSERT INTO test SET names = (select name from OUser)
[testng] -------------------^
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:517)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.handleStatus(OChannelBinary.java:470)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:145)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:59)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemote.beginResponse(OStorageRemote.java:1877)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemote.command(OStorageRemote.java:1039)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemoteThread.command(OStorageRemoteThread.java:261)
[testng] at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:59)
[testng] at com.orientechnologies.orient.test.database.auto.SQLInsertTest.insertSubQuery(SQLInsertTest.java:250)
[testng] ... Removed 26 stack frames
[testng] FAILED: insertWithNoSpaces
[testng] com.orientechnologies.orient.core.exception.ODatabaseException: Cannot open database
[testng] at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:117)
[testng] at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
[testng] at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.open(ODatabaseRecordAbstract.java:116)
[testng] at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
[testng] at com.orientechnologies.orient.test.database.auto.SQLInsertTest.insertWithNoSpaces(SQLInsertTest.java:224)
[testng] Caused by: java.lang.IllegalStateException: Database demo is already open
[testng] at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:94)
[testng] ... 26 more
[testng] ... Removed 22 stack frames

Owner
lvca commented Apr 11, 2013

Hi,
did you include the scheduler sources to avoid the library? Does it license allow it?

Contributor

Yes, scheduler sources are included, total 6 classes.
The license is LGPL so some kind of attribution notice will be required. I leave the file header.

package com.orientechnologies.orient.server.schedule
OSchedulingPattern.java
OAlwaysTrueValueMatcher.java
ODayOfMonthValueMatcher.java
OIntArrayValueMatcher.java
OInvalidPatternException.java
OValueMatcher.jhava

Please check it out.

Owner
lvca commented Apr 11, 2013

Cool! This classes could be used by Automatic Backup too in place of Timer. Backups with a cron expression would be very useful...

Contributor

Hi Lvca, how is my change looks like. I found a potential issue, in distributed mode, if two server running same scheduled job, may cause conflict issue because the status is keep changing, and scheduled jobs on each server may be different for example some node may need backup some not may not. It is not make sense to synchronize or replicate Oschedule class in each node. My suggestion is maybe we can set some class/cluster which are not synchronizable into configuration. I am not sure if it is doable so far. If not, my suggestion is create another field in distributed-db-config.json called excluded, and put the class/cluster name in it which do not want to synchronize. How about that.

Code change are : ODistributedStorage.java

 
For example :
public OStorageOperationResult updateRecord(final ORecordId iRecordId, final byte[] iContent,
      final ORecordVersion iVersion, final byte iRecordType, final int iMode, final ORecordCallback iCallback) {
        //TODO : check configuration here, if class/cluster name in excluded list, just update local
        if (ODistributedThreadLocal.INSTANCE.distributedExecution || isExcludedClass(iRecoredId))
        // ALREADY DISTRIBUTED
      return wrapped.updateRecord(iRecordId, iContent, iVersion, iRecordType, iMode, iCallback);
 ...
}
Owner
lvca commented Apr 12, 2013

Travis can't pass the tests. Is it aligned to the master head?

Contributor

I just did. How about now.

@lvca lvca merged commit 2400ef4 into orientechnologies:master Apr 12, 2013

1 check passed

default The Travis build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment