ExcerptTailer.read(ByteBuffer) fails

[jhalterman]

When writing a simple ByteBuffer out and attempting to read it back via ExcerptTailer.read, the ByteBuffer is never read.

String path = new File(System.getProperty("java.io.tmpdir"), "foo").getAbsolutePath() + "/" + UUID.randomUUID().toString();
try (IndexedChronicle chronicle = new IndexedChronicle(path, ChronicleConfig.TEST)) {
  ExcerptAppender appender = chronicle.createAppender();
  appender.startExcerpt(100);
  ByteBuffer entry = ByteBuffer.wrap("1".getBytes());
  appender.write(entry);
  appender.finish();

  ExcerptTailer tailer = chronicle.createTailer();
  ByteBuffer input = ByteBuffer.allocate(1);
  tailer.read(input);

  // Fails
  assert input.equals(entry);
}

This appears to be because ExcerptTailer.remaining() is 0, which causes the read to do nothing.

[peter-lawrey]

You need to use nextIndex () on the Tailer first.
On 17/12/2014 9:20 PM, "Jonathan Halterman" notifications@github.com
wrote:

When writing a simple ByteBuffer out and attempting to read it back via
ExcerptTailer.read, the ByteBuffer is never read.

String path = new File(System.getProperty("java.io.tmpdir"), "foo").getAbsolutePath() + "/" + UUID.randomUUID().toString();
try (IndexedChronicle chronicle = new IndexedChronicle(path, ChronicleConfig.TEST)) {
ExcerptAppender appender = chronicle.createAppender();
appender.startExcerpt(100);
ByteBuffer entry = ByteBuffer.wrap("1".getBytes());
appender.write(entry);
appender.finish();

ExcerptTailer tailer = chronicle.createTailer();
ByteBuffer input = ByteBuffer.allocate(1);
tailer.read(input);

assertEquals(new String(input.array()), new String(entry.array()));
}

This appears to be because ExcerptTailer.remaining() is 0, which causes
the read to do nothing.

—
Reply to this email directly or view it on GitHub
#112.

[jhalterman]

Thanks - I've gone through a few variations of this seeing what works and what doesn't. Using nextIndex(), here's a similar example that fails:

String path = new File(System.getProperty("java.io.tmpdir"), "foo").getAbsolutePath() + "/"
  + UUID.randomUUID().toString();
try (IndexedChronicle chronicle = new IndexedChronicle(path, ChronicleConfig.TEST)) {
  ExcerptAppender appender = chronicle.createAppender();
  appender.startExcerpt(100);
  ByteBuffer entry = ByteBuffer.allocate(4).putInt(5);
  appender.write(entry);
  appender.finish();

  ExcerptTailer tailer = chronicle.createTailer();
  if (tailer.nextIndex()) {
    ByteBuffer input = ByteBuffer.allocate(4);
    tailer.read(input);

    // Fails
    assert input.equals(entry);
  }
}

[peter-lawrey]

After you write to a ByteBuffer you need to flip() it. i.e. this leaves
you with a a ByteBuffer which is entry.remaining() == 0.

ByteBuffer entry = ByteBuffer.allocate(4).putInt(5);

Instead of using ByteBuffers can you not write directly to/from the
chronicle.

File path = File.createTempFile("foo", "");
ChronicleTools.deleteOnExit(path.toString());
try (Chronicle chronicle = ChronicleQueueBuilder.indexed(path).build()) {
ExcerptAppender appender = chronicle.createAppender();
appender.startExcerpt();
appender.writeInt(5);
appender.finish();

ExcerptTailer tailer = chronicle.createTailer();
assertTrue(tailer.nextIndex());
int x = tailer.readInt();

// Fails
assertEquals(5, x);

}

On 17 December 2014 at 22:10, Jonathan Halterman notifications@github.com
wrote:

Thanks - I've gone through a few variations of this seeing what works and
what doesn't. Using nextIndex(), here's a similar example that fails:

String path = new File(System.getProperty("java.io.tmpdir"), "foo").getAbsolutePath() + "/"
  + UUID.randomUUID().toString();
try (IndexedChronicle chronicle = new IndexedChronicle(path, ChronicleConfig.TEST)) {
  ExcerptAppender appender = chronicle.createAppender();
  appender.startExcerpt(100);
  ByteBuffer entry = ByteBuffer.allocate(4).putInt(5);
  appender.write(entry);
  appender.finish();

  ExcerptTailer tailer = chronicle.createTailer();
  if (tailer.nextIndex()) {
    ByteBuffer input = ByteBuffer.allocate(4);
    tailer.read(input);

    // Fails
    assert input.equals(entry);
  }
}

—
Reply to this email directly or view it on GitHub
#112 (comment)
.

[peter-lawrey]

Oops this example doesn't fail. Ignore the comment.

On 17 December 2014 at 22:23, Peter Lawrey <
peter.lawrey@higherfrequencytrading.com> wrote:

After you write to a ByteBuffer you need to flip() it. i.e. this leaves
you with a a ByteBuffer which is entry.remaining() == 0.

ByteBuffer entry = ByteBuffer.allocate(4).putInt(5);

Instead of using ByteBuffers can you not write directly to/from the
chronicle.

File path = File.createTempFile("foo", "");
ChronicleTools.deleteOnExit(path.toString());
try (Chronicle chronicle = ChronicleQueueBuilder.indexed(path).build()) {
ExcerptAppender appender = chronicle.createAppender();
appender.startExcerpt();
appender.writeInt(5);
appender.finish();

ExcerptTailer tailer = chronicle.createTailer();
assertTrue(tailer.nextIndex());
int x = tailer.readInt();

// Fails
assertEquals(5, x);

}

On 17 December 2014 at 22:10, Jonathan Halterman <notifications@github.com

wrote:

Thanks - I've gone through a few variations of this seeing what works and
what doesn't. Using nextIndex(), here's a similar example that fails:

String path = new File(System.getProperty("java.io.tmpdir"), "foo").getAbsolutePath() + "/"
  + UUID.randomUUID().toString();
try (IndexedChronicle chronicle = new IndexedChronicle(path, ChronicleConfig.TEST)) {
  ExcerptAppender appender = chronicle.createAppender();
  appender.startExcerpt(100);
  ByteBuffer entry = ByteBuffer.allocate(4).putInt(5);
  appender.write(entry);
  appender.finish();

  ExcerptTailer tailer = chronicle.createTailer();
  if (tailer.nextIndex()) {
    ByteBuffer input = ByteBuffer.allocate(4);
    tailer.read(input);

    // Fails
    assert input.equals(entry);
  }
}

—
Reply to this email directly or view it on GitHub
#112 (comment)
.

[jhalterman]

We're exposing an API that allows ByteBuffers to be written through to a Chronicle and read back later - so we don't have control over the state of the ByteBuffer or its content when we receive it.

Do you have an opinion on whether this is generally a good or bad idea?

[peter-lawrey]

You need to flip() it and pass the original ByteBuffer to the
write(ByteBuffer) method or you can cast it.

ByteBuffer is a built in library so we have to support how it works
currently. Or we can keep suggestion you use our API instead ;)

On 17 December 2014 at 22:29, Jonathan Halterman notifications@github.com
wrote:

Re: flipping the ByteBuffer, flip() returns Buffer, which cannot be
written via ExcerptAppender.

—
Reply to this email directly or view it on GitHub
#112 (comment)
.

[lburgazzoli]

A new tailer should point to the end of a chronicle-queue so if you want to read the data written before the tailer was created you should move it to the beginning - with toStart() - or set the index from which you want to start from - with index(..)

[jhalterman]

Thanks - We'll use flip() to try and normalize the buffers we receive before attempting to write them out.