Iterator bug

[chrsan]

I've found a bug while comparing how long it takes to insert 1000000 int keys and values (from 0 to 999999) using this lib and rocksdb. When forward iterating the first key should be 0 but it isn't.

@rnarubin - any thoughts on this? Your fork seem to be the one that's most active.

Here's a test case that verifies the bug.

package se.svt.sandbox;

import java.io.File;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.file.FileVisitResult;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.Map;

import org.iq80.leveldb.CompressionType;
import org.iq80.leveldb.DB;
import org.iq80.leveldb.DBIterator;
import org.iq80.leveldb.Options;
import org.iq80.leveldb.impl.Iq80DBFactory;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;

import static org.hamcrest.CoreMatchers.is;
import static org.junit.Assert.assertThat;

public class LeveldbBugTest {
    private static Path tempDir;

    @BeforeClass
    public static void before() throws IOException {
        tempDir = Files.createTempDirectory(null);
    }

    @Test
    public void verifyBug() throws IOException {
        File file = tempDir.resolve("levelbug").toFile();

        Options options = new Options();
        options.compressionType(CompressionType.NONE);
        options.createIfMissing(true);

        DB db1 = Iq80DBFactory.factory.open(file, options);
        byte[] data = new byte[4];
        ByteBuffer buffer = ByteBuffer.wrap(data);
        for (int i = 0; i < 600_000; ++i) {
            buffer.putInt(i);
            db1.put(data, data);
            buffer.clear();
        }

        db1.close();

        options.createIfMissing(false);

        DB db2 = Iq80DBFactory.factory.open(file, options);
        DBIterator iterator = db2.iterator();
        iterator.seekToFirst();

        assertThat(iterator.hasNext(), is(true));

        Map.Entry<byte[], byte[]> entry = iterator.peekNext();
        int key = ByteBuffer.wrap(entry.getKey()).getInt();
        try {
            assertThat(key, is(0));
        } finally {
            iterator.close();
            db2.close();
        }
    }

    @AfterClass
    public static void after() throws IOException {
        Files.walkFileTree(tempDir, new SimpleFileVisitor<Path>() {
            @Override
            public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
                Files.delete(file);
                return FileVisitResult.CONTINUE;
            }

            @Override
            public FileVisitResult postVisitDirectory(Path dir, IOException e) throws IOException {
                if (e == null) {
                    Files.delete(dir);
                    return FileVisitResult.CONTINUE;
                } else {
                    throw e;
                }
            }
        });
    }
}

[rnarubin]

this looks like a test issue. you're inserting the same byte array repeatedly, overwriting the data with each iteration. The db doesn't make a copy internally, it holds onto the array as given until it's ready to flush the write buffer, so you're mutating the key it's holding. try running the same test with a new byte array for each iteration

[chrsan]

Ah, thanks!