Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENG-10968 run-everywhere #3843

Merged
merged 13 commits into from Aug 24, 2016
14 changes: 8 additions & 6 deletions src/frontend/org/voltdb/client/Client.java
Expand Up @@ -430,10 +430,11 @@ public boolean updateClasses(ProcedureCallback callback,
* </p><p>
* The set of partition values is cached to avoid repeated requests to fetch them. However the database partitions may be changed. The cached set of partition values will
* be updated when the client affinity feature {@link ClientConfig#setClientAffinity(boolean)} is enabled and database cluster topology is updated.
* If the client affinity is not enabled, the cached set of partition values will be synchronized with the database very 5 minutes. The database partitions are usually pretty static.
* But when the cached set of partition values is out sync with the database, a procedure execution may be routed to an incorrect partition.
* If the client affinity is not enabled, the cached set of partition values will be synchronized with the database if the cached set is more than 1 second old. The database partitions are usually pretty static.
* But when the cached set of partition values is out sync with the database, a procedure execution may be routed to a partition zero time or multiple times, exactly once per partition will not be guaranteed.
* </p><p>There may be undesirable impact on latency and throughput as a result of running a multi-partition procedure. This is particularly true for longer running procedures.
* Using multiple, smaller procedures can also help reducing latency and increasing throughput, for queries that modify large volumes of data, such as large deletes.
* Using multiple, smaller procedures can also help reducing latency and increasing throughput, for queries that modify large volumes of data, such as large deletes. For example, multiple smaller single partition procedures are
* particularly useful to age out large stale data where strong global consistency is not required.
* </p><p>
* When creating a single-partitioned procedure, you can use <strong>PARAMETER</strong> clause to specify the partitioning parameter which is used to determine the target partition.
* The <strong>PARAMETER</strong> should not be specified in the stored procedure used in this call since the stored procedure will be executed on every partition. If you only want to
Expand Down Expand Up @@ -479,11 +480,12 @@ public ClientResponseWithPartitionKey[] callAllPartitionProcedure(String procedu
* </p><p>
* The set of partition values is cached to avoid repeated requests to fetch them. However the database partitions may be changed. The cached set of partition values will
* be updated when the client affinity feature {@link ClientConfig#setClientAffinity(boolean)} is enabled and database cluster topology is updated.
* If the client affinity is not enabled, the cached set of partition values will be synchronized with the database very 5 minutes. The database partitions are usually pretty static.
* But when the cached set of partition values is out sync with the database, a procedure execution may be routed to an incorrect partition.
* If the client affinity is not enabled, the cached set of partition values will be synchronized with the database if the cached set is more than 1 second old. The database partitions are usually pretty static.
* But when the cached set of partition values is out sync with the database, a procedure execution may be routed to a partition zero time or multiple times, exactly once per partition will not be guaranteed.
* </p><p>
* There may be undesirable impact on latency and throughput as a result of running a multi-partition procedure. This is particularly true for longer running procedures.
* Using multiple, smaller procedures can also help reducing latency and increasing throughput, for queries that modify large volumes of data, such as large deletes.
* Using multiple, smaller procedures can also help reducing latency and increasing throughput, for queries that modify large volumes of data, such as large deletes. For example, multiple smaller single partition procedures are
* particularly useful to age out large stale data where strong global consistency is not required.
* </p><p>
* When creating a single-partitioned procedure, you can use <strong>PARAMETER</strong> clause to specify the partitioning parameter which is used to determine the target partition.
* The <strong>PARAMETER</strong> should not be specified in the stored procedure used in this call since the stored procedure will be executed on every partition. If you only want to
Expand Down
Expand Up @@ -29,7 +29,7 @@

public class PartitionFailureTestProc extends VoltProcedure {

public final SQLStmt stmt = new SQLStmt("INSERT INTO TABLE_INT_PARTITION VALUES (1001, 'one thoudand and one', 1001, 1001, '2016-09-16 00:00:00.000000');");
public final SQLStmt stmt = new SQLStmt("INSERT INTO TABLE_INT_PARTITION VALUES (1001, 'one thoudand and one', 1001, 1001);");

public VoltTable[] run(int partitionKey) {
voltQueueSQL(stmt);
Expand Down
Expand Up @@ -24,9 +24,6 @@

import java.io.File;
import java.io.IOException;
import java.time.LocalDate;
import java.time.ZoneId;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

Expand All @@ -36,7 +33,6 @@
import org.voltdb.regressionsuites.LocalCluster;
import org.voltdb.regressionsuites.MultiConfigSuiteBuilder;
import org.voltdb.regressionsuites.RegressionSuite;
import org.voltdb.types.TimestampType;
import org.voltdb.utils.VoltFile;

/**
Expand Down Expand Up @@ -122,8 +118,10 @@ public void testSyncCallAllPartitionProcedureWithIntPartition() throws Exception
public void testAsyncCallAllPartitionProcedureWithIntPartition() throws Exception {

Client client = getClient();
client.callAllPartitionProcedure(new CallBack(INT_PARTITION_EXPECTED_COUNTS), "PartitionIntegerTestProc");
CallBack cb = new CallBack(INT_PARTITION_EXPECTED_COUNTS);
client.callAllPartitionProcedure(cb, "PartitionIntegerTestProc");
Thread.sleep(2000);
assertTrue(cb.m_callbackInvoked);
}

public void testSyncCallAllPartitionProcedureWithStringPartition() throws Exception {
Expand All @@ -136,8 +134,10 @@ public void testSyncCallAllPartitionProcedureWithStringPartition() throws Except
public void testAsyncCallAllPartitionProcedureWithStringPartition() throws Exception{

Client client = getClient();
client.callAllPartitionProcedure(new CallBack(STRING_PARTITION_EXPECTED_COUNTS), "PartitionStringTestProc");
CallBack cb = new CallBack(STRING_PARTITION_EXPECTED_COUNTS);
client.callAllPartitionProcedure(cb, "PartitionStringTestProc");
Thread.sleep(2000);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Async test version does not necessary validate the result. It could be the "callAllPartitionProcedure" never return anything. We want to ensure the "clientCallback" has been executed.

assertTrue(cb.m_callbackInvoked);
}

public void testCallAllPartitionProcedureFailuerProc() throws Exception {
Expand Down Expand Up @@ -166,14 +166,10 @@ private void validateResults(ClientResponseWithPartitionKey[] responses, Map<In

private void load(Client client, String tableName) throws NoConnectionsException, IOException, ProcCallException {

LocalDate localDate = LocalDate.parse("2016-08-23");
for (int i = 0; i < 1000; i++) {
StringBuilder builder = new StringBuilder();
builder.append("insert into " + tableName + " values (" + i);
Date date = Date.from(localDate.atStartOfDay(ZoneId.systemDefault()).toInstant());
String currentTime = new TimestampType(date).toString();
localDate = localDate.plusDays(1);
builder.append(", '" + convert(i) + "', " + i + ", " + i + ",'" + currentTime + "')");
builder.append(", '" + convert(i) + "', " + i + ", " + i + ")");
client.callProcedure("@AdHoc", builder.toString());
}

Expand Down Expand Up @@ -211,15 +207,15 @@ static public junit.framework.Test suite() throws Exception {

public static class CallBack implements AllPartitionProcedureCallback {

final Map<Integer, Long> m_expectedCounts;

Map<Integer, Long> m_expectedCounts;
boolean m_callbackInvoked = false;
CallBack(Map<Integer, Long> expectedCounts) {
m_expectedCounts = expectedCounts;
}

@Override
public void clientCallback(ClientResponseWithPartitionKey[] clientResponse) throws Exception {

m_callbackInvoked = true;
if (clientResponse != null) {
for (ClientResponseWithPartitionKey resp: clientResponse) {
VoltTable results = resp.getResponse().getResults()[0];
Expand Down
6 changes: 2 additions & 4 deletions tests/frontend/org/voltdb/client/allpartitioncall.sql
Expand Up @@ -3,16 +3,14 @@ CREATE TABLE TABLE_INT_PARTITION
id bigint NOT NULL,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also fix the indentation ?

value_string varchar(50) NOT NULL,
value_number1 bigint NOT NULL,
value_number2 bigint NOT NULL,
value_ts TIMESTAMP NOT NULL
value_number2 bigint NOT NULL
);

CREATE TABLE TABLE_STRING_PARTITION
(
id bigint NOT NULL,
value_string varchar(50) NOT NULL,
value_number1 bigint NOT NULL,
value_number2 bigint NOT NULL,
value_ts TIMESTAMP NOT NULL
value_number2 bigint NOT NULL
);