Skip to content

Enable larger tables #126

Open
wants to merge 8 commits into from

3 participants

@twilli3141

Need ability to handle larger tables. Convert to JAVA long for row counters and indices.

@busbey busbey commented on an outdated diff May 21, 2015
@@ -1,5 +1,5 @@
-Yahoo! Cloud System Benchmark (YCSB)
-====================================
+
@busbey
Collaborator
busbey added a note May 21, 2015

Leave out the README update

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@busbey busbey commented on an outdated diff May 21, 2015
...om/yahoo/ycsb/generator/ExponentialLongGenerator.java
@@ -0,0 +1,98 @@
+/**
+ * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
@busbey
Collaborator
busbey added a note May 21, 2015

Nit: this should be year of authorship

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@busbey busbey commented on an outdated diff May 23, 2015
.../ycsb/measurements/exporter/MeasurementsExporter.java
* @throws IOException if writing failed
*/
- public void write(String metric, String measurement, int i) throws IOException;
@busbey
Collaborator
busbey added a note May 23, 2015

this change will be breaking. can we do this by adding an additional method with the long parameter and keep the one that's int?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@busbey busbey commented on an outdated diff May 23, 2015
@@ -98,13 +98,13 @@
<configLocation>checkstyle.xml</configLocation>
</configuration>
<executions>
- <execution>
@busbey
Collaborator
busbey added a note May 23, 2015

please don't include disabling checkstyle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@twilli3141 twilli3141 closed this May 26, 2015
@twilli3141

Updated per feedback

@twilli3141 twilli3141 reopened this May 27, 2015
@busbey busbey commented on an outdated diff Jun 6, 2015
...main/java/com/yahoo/ycsb/generator/LongGenerator.java
+
+ /**
+ * Set the last value generated. IntegerGenerator subclasses must use this call
+ * to properly set the last string value, or the lastString() and lastInt() calls won't work.
+ */
+ protected void setLastLong(long last)
+ {
+ lastlong=last;
+ }
+
+ /**
+ * Generate the next string in the distribution.
+ */
+ public String nextString()
+ {
+ return ""+nextLong();
@busbey
Collaborator
busbey added a note Jun 6, 2015

please use Long.toString(long)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@busbey busbey commented on an outdated diff Jun 6, 2015
...main/java/com/yahoo/ycsb/generator/LongGenerator.java
+ */
+ public String nextString()
+ {
+ return ""+nextLong();
+ }
+
+
+ /**
+ * Return the previous string generated by the distribution; e.g., returned from the last nextString() call.
+ * Calling lastString() should not advance the distribution or have any side effects. If nextString() has not yet
+ * been called, lastString() should return something reasonable.
+ */
+ @Override
+ public String lastString()
+ {
+ return ""+nextLong();
@busbey
Collaborator
busbey added a note Jun 6, 2015

please use Long.toString(long)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@busbey busbey commented on an outdated diff Jun 6, 2015
...om/yahoo/ycsb/generator/ExponentialLongGenerator.java
+ /**
+ * Generate the next item as a long.
+ *
+ * @return The next item in the sequence.
+ */
+ @Override
+ public long nextLong()
+ {
+ return (long) (-Math.log(Utils.random().nextDouble()) / _gamma);
+ }
+
+ @Override
+ public double mean() {
+ return 1.0/_gamma;
+ }
+ public static void main(String args[]) {
@busbey
Collaborator
busbey added a note Jun 6, 2015

do not add main() methods for testing, instead add as a junit test if you want to include.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@busbey busbey commented on an outdated diff Jun 6, 2015
...m/yahoo/ycsb/generator/ScrambledZipfianGenerator.java
public long nextLong()
{
long ret=gen.nextLong();
ret=_min+Utils.FNVhash64(ret)%_itemcount;
- setLastInt((int)ret);
+ setLastLong((int)ret);
@busbey
Collaborator
busbey added a note Jun 6, 2015

this shouldn't truncate ret to an int.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@busbey busbey commented on an outdated diff Jun 6, 2015
...a/com/yahoo/ycsb/generator/SkewedLatestGenerator.java
{
- int max=_basis.lastInt();
- int nextint=max-_zipfian.nextInt(max);
- setLastInt(nextint);
- return nextint;
+ long max=_basis.lastLong();
+ long nextlong=max-_zipfian.nextInt((int)max);
@busbey
Collaborator
busbey added a note Jun 6, 2015

shouldn't this use zipfian.nextLong?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@busbey busbey commented on an outdated diff Jun 6, 2015
...om/yahoo/ycsb/generator/ConstantIntegerGenerator.java
@@ -36,7 +36,12 @@ public int nextInt() {
return i;
}
- @Override
+ public long nextLong()
@busbey
Collaborator
busbey added a note Jun 6, 2015

IntegerGenerator doesn't have a nextLong method, so it's a bit odd for these other subclasses to include a nextLong.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@busbey busbey commented on an outdated diff Jun 6, 2015
...main/java/com/yahoo/ycsb/generator/LongGenerator.java
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ * implied. See the License for the specific language governing
+ * permissions and limitations under the License. See accompanying
+ * LICENSE file.
+ */
+
+package com.yahoo.ycsb.generator;
+
+/**
+ * A generator that is capable of generating ints as well as strings
+ *
+ * @author cooperb
+ *
+ */
+public abstract class LongGenerator extends Generator
@busbey
Collaborator
busbey added a note Jun 6, 2015

To maintain compatibility, you could have LongGenerator extend IntegerGenerator so that all the current implementers will stay have IntegerGenerator as an ancestor (and having getInt will make sense)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@busbey
Collaborator
busbey commented Jun 6, 2015

sorry I didn't catch some of these issues on the first review pass.

@twilli3141

Updated per feedback

@busbey busbey commented on the diff Jun 20, 2015
.../main/java/com/yahoo/ycsb/workloads/CoreWorkload.java
do
{
- keynum=transactioninsertkeysequence.lastInt() - keychooser.nextInt();
+ //keynum=transactioninsertkeysequence.lastLong() - keychooser.nextInt();
@busbey
Collaborator
busbey added a note Jun 20, 2015

I started getting this merged in, but testing failed. looks like having this commented out results in an infinite loop.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.