Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Updating Stream Metadata causes Exception accessing stream of $streams #141

Closed
ascjones opened this Issue Jun 13, 2014 · 16 comments

Comments

Projects
None yet
3 participants
Contributor

ascjones commented Jun 13, 2014

If I call SetStreamMetadata more than once (thus updating the stream metadata), then I can no longer access the stream of streams ($streams) because of the following exception:

[PID:14524:010 2014.06.13 13:42:46.402 ERROR QueuedHandlerThreadP] Error while processing message ReadStreamEventsBackward InternalCorrId: 7743385e-9b5c-474a-add2-abbe79fa06db, CorrelationId: 7743385e-9b5c-474a-add2-abbe79fa06db, EventStreamId: $streams, FromEventNumber: -1, MaxCount: 20, ResolveLinkTos: True, RequireMaster: False, ValidationStreamVersion:  in queued handler 'StorageReaderQueue #4'.
System.ArgumentNullException: Value cannot be null.
Parameter name: entry
   at EventStore.Common.Utils.Ensure.NotNull[T](T argument, String argumentName) in c:\code\EventStore\src\EventStore\EventStore.Common\Utils\Ensure.cs:line 11
   at EventStore.Transport.Http.Atom.FeedElement.AddEntry(EntryElement entry) in c:\code\EventStore\src\EventStore\EventStore.Transport.Http\Atom\Feed.cs:line 77
   at EventStore.Core.Services.Transport.Http.Convert.ToStreamEventBackwardFeed(ReadStreamEventsBackwardCompleted msg, Uri requestedUrl, EmbedLevel embedContent, Boolean headOfStream) in c:\code\EventStore\src\EventStore\EventStore.Core\Services\Transport\Http\Convert.cs:line 83
   at EventStore.Core.Services.Transport.Http.Format.GetStreamEventsBackward(HttpResponseFormatterArgs entity, Message message, EmbedLevel embed, Boolean headOfStream) in c:\code\EventStore\src\EventStore\EventStore.Core\Services\Transport\Http\Format.cs:line 42
   at EventStore.Core.Services.Transport.Http.Controllers.AtomController.<>c__DisplayClass19.<GetStreamEventsBackward>b__17(HttpResponseFormatterArgs ent, Message msg) in c:\code\EventStore\src\EventStore\EventStore.Core\Services\Transport\Http\Controllers\AtomController.cs:line 720
   at EventStore.Core.Services.Transport.Http.SendToHttpEnvelope.ReplyWith[T](T message) in c:\code\EventStore\src\EventStore\EventStore.Core\Services\Transport\Http\SendToHttpEnvelope.cs:line 73
   at EventStore.Core.Services.Storage.StorageReaderWorker.EventStore.Core.Bus.IHandle<EventStore.Core.Messages.ClientMessage.ReadStreamEventsBackward>.Handle(ReadStreamEventsBackward msg) in c:\code\EventStore\src\EventStore\EventStore.Core\Services\Storage\StorageReaderWorker.cs:line 79
   at EventStore.Core.Bus.MessageHandler`1.TryHandle(Message message) in c:\code\EventStore\src\EventStore\EventStore.Core\Bus\MessageHandler.cs:line 33
   at EventStore.Core.Bus.InMemoryBus.Publish(Message message) in c:\code\EventStore\src\EventStore\EventStore.Core\Bus\InMemoryBus.cs:line 336
   at EventStore.Core.Bus.InMemoryBus.Handle(Message message) in c:\code\EventStore\src\EventStore\EventStore.Core\Bus\InMemoryBus.cs:line 308
   at EventStore.Core.Bus.QueuedHandlerThreadPool.ReadFromQueue(Object o) in c:\code\EventStore\src\EventStore\EventStore.Core\Bus\QueuedHandlerThreadPool.cs:line 106

I had a quick debug and looks like it has a link to the version 0 of the stream metadata which is no longer there:

image

@ascjones ascjones changed the title from Updating Stream Metadata causes NullReferenceException accessing $streams stream to Updating Stream Metadata causes Exception accessing stream of $streams Jun 13, 2014

Owner

gregoryyoung commented Jun 13, 2014

What version is this on?

On Fri, Jun 13, 2014 at 7:05 PM, Andrew Jones notifications@github.com
wrote:

If I call SetStreamMetadata more than once (thus updating the stream
metadata), then I can no longer access the stream of streams ($streams)
because of the following exception:

[PID:14524:010 2014.06.13 13:42:46.402 ERROR QueuedHandlerThreadP] Error while processing message ReadStreamEventsBackward InternalCorrId: 7743385e-9b5c-474a-add2-abbe79fa06db, CorrelationId: 7743385e-9b5c-474a-add2-abbe79fa06db, EventStreamId: $streams, FromEventNumber: -1, MaxCount: 20, ResolveLinkTos: True, RequireMaster: False, ValidationStreamVersion: in queued handler 'StorageReaderQueue #4'.
System.ArgumentNullException: Value cannot be null.
Parameter name: entry
at EventStore.Common.Utils.Ensure.NotNull[T](T argument, String argumentName) in c:\code\EventStore\src\EventStore\EventStore.Common\Utils\Ensure.cs:line 11
at EventStore.Transport.Http.Atom.FeedElement.AddEntry(EntryElement entry) in c:\code\EventStore\src\EventStore\EventStore.Transport.Http\Atom\Feed.cs:line 77
at EventStore.Core.Services.Transport.Http.Convert.ToStreamEventBackwardFeed(ReadStreamEventsBackwardCompleted msg, Uri requestedUrl, EmbedLevel embedContent, Boolean headOfStream) in c:\code\EventStore\src\EventStore\EventStore.Core\Services\Transport\Http\Convert.cs:line 83
at EventStore.Core.Services.Transport.Http.Format.GetStreamEventsBackward(HttpResponseFormatterArgs entity, Message message, EmbedLevel embed, Boolean headOfStream) in c:\code\EventStore\src\EventStore\EventStore.Core\Services\Transport\Http\Format.cs:line 42
at EventStore.Core.Services.Transport.Http.Controllers.AtomController.<>c__DisplayClass19.b__17(HttpResponseFormatterArgs ent, Message msg) in c:\code\EventStore\src\EventStore\EventStore.Core\Services\Transport\Http\Controllers\AtomController.cs:line 720
at EventStore.Core.Services.Transport.Http.SendToHttpEnvelope.ReplyWith[T](T message) in c:\code\EventStore\src\EventStore\EventStore.Core\Services\Transport\Http\SendToHttpEnvelope.cs:line 73
at EventStore.Core.Services.Storage.StorageReaderWorker.EventStore.Core.Bus.IHandle<EventStore.Core.Messages.ClientMessage.ReadStreamEventsBackward>.Handle(ReadStreamEventsBackward msg) in c:\code\EventStore\src\EventStore\EventStore.Core\Services\Storage\StorageReaderWorker.cs:line 79
at EventStore.Core.Bus.MessageHandler`1.TryHandle(Message message) in c:\code\EventStore\src\EventStore\EventStore.Core\Bus\MessageHandler.cs:line 33
at EventStore.Core.Bus.InMemoryBus.Publish(Message message) in c:\code\EventStore\src\EventStore\EventStore.Core\Bus\InMemoryBus.cs:line 336
at EventStore.Core.Bus.InMemoryBus.Handle(Message message) in c:\code\EventStore\src\EventStore\EventStore.Core\Bus\InMemoryBus.cs:line 308
at EventStore.Core.Bus.QueuedHandlerThreadPool.ReadFromQueue(Object o) in c:\code\EventStore\src\EventStore\EventStore.Core\Bus\QueuedHandlerThreadPool.cs:line 106

I had a quick debug and looks like it has a link to the version 0 of the
stream metadata which is no longer there:

[image: image]
https://cloud.githubusercontent.com/assets/75586/3272079/56e15d0a-f314-11e3-989d-40e102b90ca2.png


Reply to this email directly or view it on GitHub
#141.

Studying for the Turing test

Contributor

ascjones commented Jun 13, 2014

3.0.0rc2, and I can also reproduce the error debugging with the dev branch on my machine

Contributor

ascjones commented Jun 13, 2014

fsx script I used to repro:

#r "bin/Debug/EventStore.ClientAPI.dll"

open System
open System.Net
open EventStore.ClientAPI

let conn = EventStoreConnection.Create(new IPEndPoint(IPAddress.Loopback, 1113))
conn.Connect()

let metadata = "{ name: 'Andrew' }"
let metadataBytes = System.Text.Encoding.UTF8.GetBytes(metadata)

conn.SetStreamMetadata("StreamMetadataTest", ExpectedVersion.Any, metadataBytes)
conn.SetStreamMetadata("StreamMetadataTest", ExpectedVersion.Any, metadataBytes)
Owner

gregoryyoung commented Jun 14, 2014

if its not there it should just not show up in the feed. I will take a look through it today or tomorrow. Thanks for the repro.

Owner

gregoryyoung commented Jun 14, 2014

As you can see in your debugged view there the event is actually a pointer $> I will look through the code can you test quickly if forward as the same issue as backwards?

Contributor

ascjones commented Jun 14, 2014

Yeah so it's a pointer to an Event (Number 0 of the $$StreamMetadataTest metastream) which is no longer there having been 'overwritten' by the second SetStreamMetadata?

Reading Forwards synchronously from the TCP API just hangs after setting the stream metadata the second time. No errors show up in the logs until I try to read $streams from the HTTP API. Here's the repro:

#r "bin/Debug/EventStore.ClientAPI.dll"

open System
open System.Net
open EventStore.ClientAPI

let inline (!>) (x:^a) : ^b = ((^a or ^b) : (static member op_Implicit : ^a -> ^b) x) // exlicit operator to return implicit ConnectionSettings
let connSettings = !> ConnectionSettings.Create().SetDefaultUserCredentials(new SystemData.UserCredentials("admin", "changeit"))
let conn = EventStoreConnection.Create(connSettings, new IPEndPoint(IPAddress.Loopback, 1113))
conn.Connect()

let metadata = "{ name: 'Andrew' }"
let metadataBytes = System.Text.Encoding.UTF8.GetBytes(metadata)

conn.SetStreamMetadata("StreamMetadataTest", ExpectedVersion.Any, metadataBytes)
let slice = conn.ReadStreamEventsForward("$streams", StreamPosition.Start, 1, true)  // returns okay
conn.SetStreamMetadata("StreamMetadataTest", ExpectedVersion.Any, metadataBytes)
let slice2 = conn.ReadStreamEventsForward("$streams", StreamPosition.Start, 1, true) // just hangs
Owner

gregoryyoung commented Jun 14, 2014

If you look in the http (atom api there are two types of paging links
forwards and backwards thats what I was interested in)/

Greg

On Sat, Jun 14, 2014 at 11:10 PM, Andrew Jones notifications@github.com
wrote:

Yeah so it's a pointer to an Event (Number 0 of the $$StreamMetadataTest
metastream) which is no longer there having been 'overwritten' by the
second SetStreamMetadata?

Reading Forwards synchronously from the TCP API just hangs after setting
the stream metadata the second time. No errors show up in the logs until I
try to read $streams from the HTTP API. Here's the repro:

#r "bin/Debug/EventStore.ClientAPI.dll"

open System
open System.Net
open EventStore.ClientAPI

let inline (!>) (x:^a) : ^b = ((^a or ^b) : (static member op_Implicit : ^a -> ^b) x) // exlicit operator to return implicit ConnectionSettings
let connSettings = !> ConnectionSettings.Create().SetDefaultUserCredentials(new SystemData.UserCredentials("admin", "changeit"))
let conn = EventStoreConnection.Create(connSettings, new IPEndPoint(IPAddress.Loopback, 1113))
conn.Connect()

let metadata = "{ name: 'Andrew' }"
let metadataBytes = System.Text.Encoding.UTF8.GetBytes(metadata)

conn.SetStreamMetadata("StreamMetadataTest", ExpectedVersion.Any, metadataBytes)
let slice = conn.ReadStreamEventsForward("$streams", StreamPosition.Start, 1, true) // returns okay
conn.SetStreamMetadata("StreamMetadataTest", ExpectedVersion.Any, metadataBytes)
let slice2 = conn.ReadStreamEventsForward("$streams", StreamPosition.Start, 1, true) // just hangs


Reply to this email directly or view it on GitHub
#141 (comment)
.

Studying for the Turing test

Owner

gregoryyoung commented Jun 14, 2014

And read forward in client api just hangs? what does read backwards do?

On Sat, Jun 14, 2014 at 11:11 PM, Greg Young gregoryyoung1@gmail.com
wrote:

If you look in the http (atom api there are two types of paging links
forwards and backwards thats what I was interested in)/

Greg

On Sat, Jun 14, 2014 at 11:10 PM, Andrew Jones notifications@github.com
wrote:

Yeah so it's a pointer to an Event (Number 0 of the $$StreamMetadataTest
metastream) which is no longer there having been 'overwritten' by the
second SetStreamMetadata?

Reading Forwards synchronously from the TCP API just hangs after setting
the stream metadata the second time. No errors show up in the logs until I
try to read $streams from the HTTP API. Here's the repro:

#r "bin/Debug/EventStore.ClientAPI.dll"

open System
open System.Net
open EventStore.ClientAPI

let inline (!>) (x:^a) : ^b = ((^a or ^b) : (static member op_Implicit : ^a -> ^b) x) // exlicit operator to return implicit ConnectionSettings
let connSettings = !> ConnectionSettings.Create().SetDefaultUserCredentials(new SystemData.UserCredentials("admin", "changeit"))
let conn = EventStoreConnection.Create(connSettings, new IPEndPoint(IPAddress.Loopback, 1113))
conn.Connect()

let metadata = "{ name: 'Andrew' }"
let metadataBytes = System.Text.Encoding.UTF8.GetBytes(metadata)

conn.SetStreamMetadata("StreamMetadataTest", ExpectedVersion.Any, metadataBytes)
let slice = conn.ReadStreamEventsForward("$streams", StreamPosition.Start, 1, true) // returns okay
conn.SetStreamMetadata("StreamMetadataTest", ExpectedVersion.Any, metadataBytes)
let slice2 = conn.ReadStreamEventsForward("$streams", StreamPosition.Start, 1, true) // just hangs


Reply to this email directly or view it on GitHub
#141 (comment)
.

Studying for the Turing test

Studying for the Turing test

Contributor

ascjones commented Jun 14, 2014

Ah gotcha. Yeah the Forward also gets the same error:

[PID:04756:029 2014.06.14 21:14:47.152 ERROR QueuedHandlerThreadP] Error while processing message ReadStreamEventsForward InternalCorrId: de3c22a9-797c-41b5-9267-ca74b4626f13, CorrelationId: de3c22a9-797c-41b5-9267-ca74b4626f13, EventStreamId: $streams, FromEventNumber: 0, MaxCount: 20, ResolveLinkTos: True, RequireMaster: False, ValidationStreamVersion:  in queued handler 'StorageReaderQueue #3'.
System.NullReferenceException: Object reference not set to an instance of an object.
   at EventStore.Core.Services.Transport.Http.Convert.ToStreamEventForwardFeed(ReadStreamEventsForwardCompleted msg, Uri requestedUrl, EmbedLevel embedContent) in c:\data\EventStore\src\EventStore\EventStore.Core\Services\Transport\Http\Convert.cs:line 54
   at EventStore.Core.Services.Transport.Http.Format.GetStreamEventsForward(HttpResponseFormatterArgs entity, Message message, EmbedLevel embed) in c:\data\EventStore\src\EventStore\EventStore.Core\Services\Transport\Http\Format.cs:line 78
   at EventStore.Core.Services.Transport.Http.SendToHttpEnvelope.ReplyWith[T](T message) in c:\data\EventStore\src\EventStore\EventStore.Core\Services\Transport\Http\SendToHttpEnvelope.cs:line 101
   at EventStore.Core.Services.Storage.StorageReaderWorker.EventStore.Core.Bus.IHandle<EventStore.Core.Messages.ClientMessage.ReadStreamEventsForward>.Handle(ReadStreamEventsForward msg) in c:\data\EventStore\src\EventStore\EventStore.Core\Services\Storage\StorageReaderWorker.cs:line 90
   at EventStore.Core.Bus.MessageHandler`1.TryHandle(Message message) in c:\data\EventStore\src\EventStore\EventStore.Core\Bus\MessageHandler.cs:line 59
   at EventStore.Core.Bus.InMemoryBus.Publish(Message message) in c:\data\EventStore\src\EventStore\EventStore.Core\Bus\InMemoryBus.cs:line 363
   at EventStore.Core.Bus.QueuedHandlerThreadPool.ReadFromQueue(Object o) in c:\data\EventStore\src\EventStore\EventStore.Core\Bus\QueuedHandlerThreadPool.cs:line 130

And yeah, reading stream events backward also hangs from the Client API. Which is what we were originally experiencing in our app which led me to this issue.

Anyway, Saturday night...time for the footie!

Owner

gregoryyoung commented Jun 15, 2014

So the problem is fairly clear.

    public static EntryElement ToEntry(ResolvedEvent eventLinkPair, Uri requestedUrl, EmbedLevel embedContent, bool singleEntry = false)
    {
        if (eventLinkPair.Event == null || requestedUrl == null)
            return null;

Its not handling the case of a non-resolved event :) I will look at this this week and patch it as this should definitely work (it should write you a uri that you then get a GONE for).

Contributor

ascjones commented Jun 15, 2014

Brilliant thanks Greg.

But I guess this wouldn't fix the hanging client API call (see previous repro)?

Owner

gregoryyoung commented Jun 15, 2014

Yes I havn't looked into that one yet will fix the first one then look at
it.

On Sun, Jun 15, 2014 at 5:42 PM, Andrew Jones notifications@github.com
wrote:

Brilliant thanks Greg.

But I guess this wouldn't fix the hanging client API call (see previous
repro)?


Reply to this email directly or view it on GitHub
#141 (comment)
.

Studying for the Turing test

Contributor

ascjones commented Jul 1, 2014

We are also getting the same error after writing to our own stream with a $maxCount of 1. i.e. there is a NRE accessing both $streams and $ce streams for links to the events which have 'expired'.

Owner

gregoryyoung commented Jul 1, 2014

This would be expected given the issue. We are looking at it.

On Tue, Jul 1, 2014 at 8:11 PM, Andrew Jones notifications@github.com
wrote:

We are also getting the same error after writing to our own stream with a
$maxAge of 1. i.e. there is a NRE accessing both $streams and $ce streams
for links to the events which have 'expired'.


Reply to this email directly or view it on GitHub
#141 (comment)
.

Studying for the Turing test

@jen20 jen20 added this to the v3.0.0 milestone Aug 19, 2014

@jen20 jen20 added the bug label Aug 19, 2014

Contributor

ascjones commented Aug 19, 2014

This is fixed now with Greg's fix for #169

@ascjones ascjones closed this Aug 19, 2014

Owner

jen20 commented Aug 19, 2014

Thanks @ascjones

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment