Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

https://github.com/nuvolabase/orientdb/pull/1406 #1430

Merged
merged 5 commits into from

2 participants

@henryzhao81

Fix #1406 which failed in Unittest.
2 Fixes :
1) Remove set className to previous for loop. Otherwise if in request JSON string, if @class is not first object. In following loop, all fields type which before @class will always be null.
2) If value instanceof ODocument and type is LINK, then save object.

Passed unit test

@henryzhao81

One more check in :
Check in change for OSchedule (#1226)
In orientdb-server-config.xml :
Add a new handler

< handler class="com.orientechnologies.orient.server.schedule.OScheduleHandler" >
< parameters>
< parameter name="enabled" value="false" />
< parameter name="databaseName" value="databaseName" />
< parameter name="user" value="admin" />
< parameter name="pass" value="admin" />
< /parameters >
< /handler >


A class will be created, each record represent a job :
name : job name
rule : cron like rule (*/5 * * * *) : http://www.sauronsoftware.it/projects/cron4j/manual.php#p14 - 2. Scheduling patterns
arguments : function parameters
status : job status
function : link to the job function (OFunction)
starttime : last start time
start : the switch to start or stop job

Once change enabled to true in orientdb-server-config.xml, timer thread will be start, every minute, it will check rule for each job, if rule is qualified, then the job will be started.

  1. Create Job : name, rule and function are mandatory fields. Once job is created, by default start is false, status is STOPPED. Job name should be unique.
  2. Start Job : update start to true, the status will be automatically change to WAITTING, and wait for time thread to check rule to decide if it is ok to run in each minute.
  3. Run Job : if rule is qulified to run. Then the status will automatically change to RUNNING, and execute function. After function exectued, change back to WAITTING.
  4. Stop Job : change start to false, the job will be stopped.

OScheduleHandler.java :
Contains TimerThread, to check each job rule and start the job if rule qualified.

OScheduler.java :
Each job.

OSchedulerListenerImpl.java :
Methods to retrieve, load , add and remove job.

OSchedulerTrigger.java :
Check and change job status.

@henryzhao81

One question : I run ant clean test, unit test passed on my Linux machine (ubuntu), but failed on my macbook.
Following are the error log on my macbook, not sure what's going on.

[testng] index intstanceof class com.orientechnologies.orient.core.index.OIndexTxAwareOneValue

[testng] FAILED CONFIGURATION: @BeforeClass setUp
[testng] java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 10.7.2.31; nested exception is:
[testng] java.net.ConnectException: Operation timed out]
[testng] at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:357)
[testng] at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:267)
[testng] at com.orientechnologies.orient.test.database.auto.AbstractIndexReuseTest.getProfilerInstance(AbstractIndexReuseTest.java:67)
[testng] at com.orientechnologies.orient.test.database.auto.AbstractIndexReuseTest.setUp(AbstractIndexReuseTest.java:33)
[testng] at com.orientechnologies.orient.test.database.auto.SQLIndexWithoutSchemaTest.setUp(SQLIndexWithoutSchemaTest.java:40)
[testng] Caused by: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 10.7.2.31; nested exception is:
[testng] java.net.ConnectException: Operation timed out]
[testng] at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:118)
[testng] at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:203)
[testng] at javax.naming.InitialContext.lookup(InitialContext.java:411)
[testng] at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1924)
[testng] at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1891)
[testng] at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:274)
[testng] ... 26 more
[testng] Caused by: java.rmi.ConnectException: Connection refused to host: 10.7.2.31; nested exception is:
[testng] java.net.ConnectException: Operation timed out
[testng] at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
[testng] at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
[testng] at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
[testng] at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:340)
[testng] at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
[testng] at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:114)
[testng] ... 31 more
[testng] Caused by: java.net.ConnectException: Operation timed out
[testng] at java.net.PlainSocketImpl.socketConnect(Native Method)
[testng] at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[testng] at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[testng] at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[testng] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[testng] at java.net.Socket.connect(Socket.java:579)
[testng] at java.net.Socket.connect(Socket.java:528)
[testng] at java.net.Socket.(Socket.java:425)
[testng] at java.net.Socket.(Socket.java:208)
[testng] at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
[testng] at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:146)
[testng] at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
[testng] ... 36 more
[testng] ... Removed 22 stack frames
[testng] FAILED CONFIGURATION: @BeforeClass setUp

FAILED: insertAvoidingSubQuery
[testng] com.orientechnologies.orient.core.sql.OCommandSQLParsingException: Error on parsing command at position #0: Error on parsing command at position #11: Class TEST not found in database
[testng] Command: INSERT INTO test(text) VALUES ('(Hello World)')
[testng] -------------------^
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:517)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.handleStatus(OChannelBinary.java:470)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:145)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:59)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemote.beginResponse(OStorageRemote.java:1877)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemote.command(OStorageRemote.java:1039)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemoteThread.command(OStorageRemoteThread.java:261)
[testng] at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:59)
[testng] at com.orientechnologies.orient.test.database.auto.SQLInsertTest.insertAvoidingSubQuery(SQLInsertTest.java:238)
[testng] ... Removed 26 stack frames
[testng] FAILED: insertCluster
[testng] com.orientechnologies.orient.core.exception.ODatabaseException: Cannot open database
[testng] at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:117)
[testng] at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
[testng] at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.open(ODatabaseRecordAbstract.java:116)
[testng] at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
[testng] at com.orientechnologies.orient.test.database.auto.SQLInsertTest.insertCluster(SQLInsertTest.java:263)
[testng] Caused by: java.lang.IllegalStateException: Database demo is already open
[testng] at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:94)
[testng] ... 26 more
[testng] ... Removed 22 stack frames
[testng] FAILED: insertSubQuery
[testng] com.orientechnologies.orient.core.sql.OCommandSQLParsingException: Error on parsing command at position #0: Error on parsing command at position #11: Class TEST not found in database
[testng] Command: INSERT INTO test SET names = (select name from OUser)
[testng] -------------------^
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:517)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.handleStatus(OChannelBinary.java:470)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:145)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:59)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemote.beginResponse(OStorageRemote.java:1877)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemote.command(OStorageRemote.java:1039)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemoteThread.command(OStorageRemoteThread.java:261)
[testng] at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:59)
[testng] at com.orientechnologies.orient.test.database.auto.SQLInsertTest.insertSubQuery(SQLInsertTest.java:250)
[testng] ... Removed 26 stack frames
[testng] FAILED: insertWithNoSpaces
[testng] com.orientechnologies.orient.core.exception.ODatabaseException: Cannot open database
[testng] at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:117)
[testng] at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
[testng] at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.open(ODatabaseRecordAbstract.java:116)
[testng] at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
[testng] at com.orientechnologies.orient.test.database.auto.SQLInsertTest.insertWithNoSpaces(SQLInsertTest.java:224)
[testng] Caused by: java.lang.IllegalStateException: Database demo is already open
[testng] at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:94)
[testng] ... 26 more
[testng] ... Removed 22 stack frames

@lvca
Owner

Hi,
did you include the scheduler sources to avoid the library? Does it license allow it?

@henryzhao81

Yes, scheduler sources are included, total 6 classes.
The license is LGPL so some kind of attribution notice will be required. I leave the file header.

package com.orientechnologies.orient.server.schedule
OSchedulingPattern.java
OAlwaysTrueValueMatcher.java
ODayOfMonthValueMatcher.java
OIntArrayValueMatcher.java
OInvalidPatternException.java
OValueMatcher.jhava

Please check it out.

@lvca
Owner

Cool! This classes could be used by Automatic Backup too in place of Timer. Backups with a cron expression would be very useful...

@henryzhao81

Hi Lvca, how is my change looks like. I found a potential issue, in distributed mode, if two server running same scheduled job, may cause conflict issue because the status is keep changing, and scheduled jobs on each server may be different for example some node may need backup some not may not. It is not make sense to synchronize or replicate Oschedule class in each node. My suggestion is maybe we can set some class/cluster which are not synchronizable into configuration. I am not sure if it is doable so far. If not, my suggestion is create another field in distributed-db-config.json called excluded, and put the class/cluster name in it which do not want to synchronize. How about that.

Code change are : ODistributedStorage.java

 
For example :
public OStorageOperationResult updateRecord(final ORecordId iRecordId, final byte[] iContent,
final ORecordVersion iVersion, final byte iRecordType, final int iMode, final ORecordCallback iCallback) {
//TODO : check configuration here, if class/cluster name in excluded list, just update local
if (ODistributedThreadLocal.INSTANCE.distributedExecution || isExcludedClass(iRecoredId))
// ALREADY DISTRIBUTED
return wrapped.updateRecord(iRecordId, iContent, iVersion, iRecordType, iMode, iCallback);
...
}

@lvca
Owner

Travis can't pass the tests. Is it aligned to the master head?

@henryzhao81

I just did. How about now.

@lvca lvca merged commit 2400ef4 into orientechnologies:master
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.