Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"There is not enough space left in the buffer" when inserting and the number of parameters is several hundred in preview9 #2612

Closed
darkflame0 opened this issue Sep 7, 2019 · 4 comments · Fixed by #2614

Comments

@darkflame0
Copy link

commented Sep 7, 2019

fail: Microsoft.EntityFrameworkCore.Update[10000]
      An exception occurred in the database while saving changes for context type '*****'.
      Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while updating the entries. See the inner exception for details.
       ---> System.InvalidOperationException: There is not enough space left in the buffer.
         at Npgsql.NpgsqlWriteBuffer.ThrowNotSpaceLeft()
         at Npgsql.TypeHandlers.JsonHandler.WriteObjectWithLength(Object value, NpgsqlWriteBuffer buf, NpgsqlLengthCache lengthCache, NpgsqlParameter parameter, Boolean async)
         at Npgsql.NpgsqlConnector.WriteBind(List`1 inputParameters, String portal, String statement, Boolean allResultTypesAreUnknown, Boolean[] unknownResultTypeList, Boolean async)
         at Npgsql.NpgsqlCommand.SendExecute(NpgsqlConnector connector, Boolean async)
         at Npgsql.NpgsqlCommand.ExecuteReaderAsync(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
         at Npgsql.NpgsqlCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.ExecuteAsync(IRelationalConnection connection, CancellationToken cancellationToken)
         --- End of inner exception stack trace ---
         at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.ExecuteAsync(IRelationalConnection connection, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(DbContext _, ValueTuple`2 parameters, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(DbContext _, ValueTuple`2 parameters, CancellationToken cancellationToken)
         at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IList`1 entriesToSave, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while updating the entries. See the inner exception for details.
 ---> System.InvalidOperationException: There is not enough space left in the buffer.
   at Npgsql.NpgsqlWriteBuffer.ThrowNotSpaceLeft()
   at Npgsql.TypeHandlers.JsonHandler.WriteObjectWithLength(Object value, NpgsqlWriteBuffer buf, NpgsqlLengthCache lengthCache, NpgsqlParameter parameter, Boolean async)
   at Npgsql.NpgsqlConnector.WriteBind(List`1 inputParameters, String portal, String statement, Boolean allResultTypesAreUnknown, Boolean[] unknownResultTypeList, Boolean async)
   at Npgsql.NpgsqlCommand.SendExecute(NpgsqlConnector connector, Boolean async)
   at Npgsql.NpgsqlCommand.ExecuteReaderAsync(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.ExecuteAsync(IRelationalConnection connection, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.ExecuteAsync(IRelationalConnection connection, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(DbContext _, ValueTuple`2 parameters, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(DbContext _, ValueTuple`2 parameters, CancellationToken cancellationToken)
   at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IList`1 entriesToSave, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)

In 3.0 preview5,my code works fine when inserting with thousands of parameters

@roji

This comment has been minimized.

Copy link
Member

commented Sep 8, 2019

This indeed looks like a bug, and it's very important we fix this before the 3.0 release. Can you please post a minimal code sample (including schema) which triggers the issue?

@roji roji transferred this issue from npgsql/Npgsql.EntityFrameworkCore.PostgreSQL Sep 8, 2019

@roji roji added the bug label Sep 8, 2019

@roji roji added this to the 4.1 milestone Sep 8, 2019

@roji

This comment has been minimized.

Copy link
Member

commented Sep 8, 2019

Almost certainly introduced in #2306 or #2572, will try to figure it out from the source (but a code sample would still help).

@darkflame0

This comment has been minimized.

Copy link
Author

commented Sep 10, 2019

        [Fact]
        public async Task TestInPreview9()
        {
            var db = new TestContext();
            db.Database.EnsureDeleted();
            db.Database.EnsureCreated();
            var entities = Enumerable.Range(1, 1024).Select(a => new TestEntity() 
            {
                JsonbColumn = Enumerable.Range(1, 1024).Select(a => new KV { Key = a, Value = new string('1', 128) }).ToList(),
                JsonbColumn2 = Enumerable.Range(1, 1024).Select(a => new KV { Key = a, Value = new string('1', 128) }).ToList(),
                JsonbColumn3 = Enumerable.Range(1, 1024).Select(a => new KV { Key = a, Value = new string('1', 128) }).ToList(),
                JsonbColumn4 = Enumerable.Range(1, 1).Select(a => new KV { Key = a, Value = new string('1', 128) }).ToList(),
            });
            db.TestEntity.AddRange(entities);
            await db.SaveChangesAsync();
            // on my environment above code works fine


            db = new TestContext();
// this case won't throw the Exception,the Exception is catched in back,you can debug this test then will see the Exception in IDE
            await Assert.ThrowsAsync<DbUpdateException>(async () => 
            {
                var entities = Enumerable.Range(1, 1024).Select(a => new TestEntity()
                {
                    JsonbColumn = Enumerable.Range(1, 1024).Select(a => new KV { Key = a, Value = new string('1', 128) }).ToList(),
                    JsonbColumn2 = Enumerable.Range(1, 1024).Select(a => new KV { Key = a, Value = new string('1', 128) }).ToList(),
                    JsonbColumn3 = Enumerable.Range(1, 1024).Select(a => new KV { Key = a, Value = new string('1', 128) }).ToList(),
                    JsonbColumn4 = Enumerable.Range(1, 1024).Select(a => new KV { Key = a, Value = new string('1', 128) }).ToList()
                });
                db.TestEntity.AddRange(entities);
                await db.SaveChangesAsync();
            });
        }
        public class TestEntity
        {
            public int Id { get; set; }
            [Column(TypeName = "jsonb")]
            public List<KV> JsonbColumn { get; set; }
            [Column(TypeName = "jsonb")]
            public List<KV> JsonbColumn2 { get; set; }
            [Column(TypeName = "jsonb")]
            public List<KV> JsonbColumn3 { get; set; }
            [Column(TypeName = "jsonb")]
            public List<KV> JsonbColumn4 { get; set; }
        }
        public class KV
        {
            public int Key { get; set; }
            public string Value { get; set; }
        }
        public class TestContext : DbContext
        {
            public DbSet<TestEntity> TestEntity { get; set; }

            protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
            {
                optionsBuilder.UseNpgsql("Host=localhost;Database=TestDb;Username=postgres;Password=postgres");
            }
        }

@roji roji self-assigned this Sep 10, 2019

roji added a commit that referenced this issue Sep 10, 2019
Correct JSON handler write logic
Simplify non-generic parameter handling logic to call into the generic.

Fixes #2612
@roji

This comment has been minimized.

Copy link
Member

commented Sep 10, 2019

@darkflame0 I've pushed a fix, can you please try adding the unstable nuget feed and running with package 5.0.0-ci.2265, and confirm that the bug is gone?

@roji roji closed this in #2614 Sep 10, 2019

roji added a commit that referenced this issue Sep 10, 2019
Correct JSON handler write logic
Simplify non-generic parameter handling logic to call into the generic.

Fixes #2612
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.