Cannot mock DynamoDBv2 AsyncSearch for unit testing AWS calls

[jazpearson]

Would like to be able to properly unit test my code which uses the AWS SDK, and in particular the AsyncSearch class.

Possible Solution

1. Create an interface for this class
2. Create an empty public constructor may also work

[mercury-jin-autodesk]

I have the same issue with AsyncSearch.

[aburgett87]

Any update on this issue?

[ColWhi]

Any update on this, causing us some pain

[assyadh]

Thanks @Supercheez, I'll have a look at the PR.

[arifemre61]

at the PR.

Do you have chance to look at it ?

[0leksii]

Hello,

Any update regarding this?

[james-r-parker]

This would be very nice please

[leninator]

• 1 vote for me. It's painful to test.

[Hardconkers]

Frankly, The TDD approach of this lib is pretty shocking, why have you got interfaces that expose internally constructed objects such as AsyncSearch<T> and Table ? It makes for grim reading when trying to carry out the simplest of mocking in unit tests.

Just adding a public empty constructor would allow a safe non breaking fix for this - albeit not the best, but it should not have taken over 2 years to come to this conclusion.

[kwbauer]

When will we see movement on this issue. Not being able to write proper unit tests is very frustrating.

Just add the empty and public constructor so that we can mock the class.

[Hardconkers]

When will we see movement on this issue. Not being able to write proper unit tests is very frustrating.

Just add the empty and public constructor so that we can mock the class.

@kwbauer - I ended up caving in and wrapping this code in a new class with what should be the appropriate interface in this library. - It means that my unit tests are testing the wrapper, and not the actual call within the AWSSDK which means I'm not 100% happy about it, but it will do. Code snippets below.

    internal interface IAsyncSearch<T>
    {
        bool IsDone { get; }

        Task<IReadOnlyList<T>> GetNextSetAsync(CancellationToken cancellationToken);

        Task<IReadOnlyList<T>> GetRemainingAsync(CancellationToken cancellationToken);
    }

    internal sealed class AsyncSearchWrapper<T> : IAsyncSearch<T>
    {
        private readonly AsyncSearch<T> asyncSearch;

        public AsyncSearchWrapper(AsyncSearch<T> asyncSearch)
        {
            this.asyncSearch = asyncSearch;
        }

        public bool IsDone => this.asyncSearch.IsDone;

        public async Task<IReadOnlyList<T>> GetNextSetAsync(CancellationToken cancellationToken)
        {
            return await this.asyncSearch.GetNextSetAsync(cancellationToken);
        }

        public async Task<IReadOnlyList<T>> GetRemainingAsync(CancellationToken cancellationToken)
        {
            return await this.asyncSearch.GetRemainingAsync(cancellationToken);
        }
    }

    internal sealed class WrappedDbContext : IWrappedDbContext
    {
        private readonly IDynamoDBContext context;

        public WrappedDbContext(IDynamoDBContext context)
        {
            this.context = context;
        }

        public IAsyncSearch<T> Query<T>(object hashKeyValue, DynamoDBOperationConfig operationConfig = null)
        {
            var search = this.context.Value.QueryAsync<T>(hashKeyValue, operationConfig);
            return new AsyncSearchWrapper<T>(search);
        }
    }

I also renamed the context method to Query instead of QueryAsync, as its not actually an Async method.

Hope this helps.

I am not suggesting this as a fix. This needs to be fixed in the library.

[KeithGarry]

Just got burned by this. +1 for some way to mock AsyncSearch. @assyadh Have you had a chance to look at the PR yet?

[KeithGarry]

After following @Hardconkers code I have successfully managed to mock out the IAsyncSearch. Here's an example:

var mockDynamoDbContext = new Mock<IWrappedDbContext>();
var clientsRepo = new ClientsRepo(mockDynamoDbContext.Object, new RepoExceptionFactory());
var asyncSearchWrapper = new Mock<IAsyncSearch<Client>>(); 
var clientResult = new List<Client>() { new Client() { Name = "Boop"} };
             

asyncSearchWrapper.Setup(x => x.GetRemainingAsync(new CancellationToken())).ReturnsAsync(clientResult); 

mockDynamoDbContext.Setup(x => x.FromQueryAsync<Client>(
        It.IsAny<QueryOperationConfig>(), null))
    .Returns(asyncSearchWrapper.Object);


var result = await clientsRepo.GetClientsByNameAsync("test");

I've also included the implementation

public sealed class DynamoDbContextWrapper : IDynamoDbContextWrapper
{
private readonly IDynamoDBContext _context;

public DynamoDbContextWrapper(IDynamoDBContext context)
{
    this._context = context;
}

public async Task<T> LoadAsync<T>(object hashKeyValue, CancellationToken cancellationToken = new CancellationToken())
{
    var item = await _context.LoadAsync<T>(hashKeyValue, cancellationToken);
    return item;
}

public IAsyncSearch<T> Query<T>(object hashKeyValue, DynamoDBOperationConfig operationConfig = null)
{
    var search = _context.QueryAsync<T>(hashKeyValue, operationConfig);
    return new AsyncSearchWrapper<T>(search);
}

public IAsyncSearch<T> FromQueryAsync<T>(QueryOperationConfig queryConfig, DynamoDBOperationConfig opConfig = null)
{
    var query = _context.FromQueryAsync<T>(queryConfig, opConfig);
    return new AsyncSearchWrapper<T>(query);
}
IAsyncSearch<T> IDynamoDbContextWrapper.FromQueryAsync<T>(QueryOperationConfig queryConfig, DynamoDBOperationConfig operationConfig)
{
    return new AsyncSearchWrapper<T>(_context.FromQueryAsync<T>(queryConfig, operationConfig));
}
public IAsyncSearch<T> QueryAsync<T>(object hashKeyValue, DynamoDBOperationConfig operationConfig = null)
{
    return new AsyncSearchWrapper<T>(_context.QueryAsync<T>(hashKeyValue, operationConfig));
}

public IAsyncSearch<T> QueryAsync<T>(object hashKeyValue, QueryOperator op, IEnumerable<object> values, DynamoDBOperationConfig operationConfig = null)
{
    return new AsyncSearchWrapper<T>(_context.QueryAsync<T>(hashKeyValue, op, values, operationConfig));
}

public IAsyncSearch<T> ScanAsync<T>(IEnumerable<ScanCondition> conditions, DynamoDBOperationConfig operationConfig = null)
{
    return new AsyncSearchWrapper<T>(_context.ScanAsync<T>(conditions, operationConfig));
}
public IAsyncSearch<T> FromScanAsync<T>(ScanOperationConfig scanConfig, DynamoDBOperationConfig operationConfig = null)
{
    return new AsyncSearchWrapper<T>(_context.FromScanAsync<T>(scanConfig, operationConfig));
}

public async Task SaveAsync<T>(T item, CancellationToken cancellationToken = new CancellationToken())
{
    await _context.SaveAsync(item, cancellationToken);
}

public BatchGet<T> CreateBatchGet<T>(DynamoDBOperationConfig operationConfig = null)
{
    return _context.CreateBatchGet<T>(operationConfig);
}

public BatchWrite<T> CreateBatchWrite<T>(DynamoDBOperationConfig operationConfig = null)
{
    return _context.CreateBatchWrite<T>(operationConfig);
}

public MultiTableBatchGet CreateMultiTableBatchGet(params BatchGet[] batches)
{
    return _context.CreateMultiTableBatchGet(batches);
}

public MultiTableBatchWrite CreateMultiTableBatchWrite(params BatchWrite[] batches)
{
    return _context.CreateMultiTableBatchWrite(batches);
}

public async Task DeleteAsync<T>(object hashKey, CancellationToken cancellationToken = default)
{
    await _context.DeleteAsync<T>(hashKey, cancellationToken);
}

public async Task DeleteAsync<T>(object hashKey, DynamoDBOperationConfig operationConfig, CancellationToken cancellationToken = default)
{
    await _context.DeleteAsync<T>(hashKey, operationConfig, cancellationToken);
}

public async Task DeleteAsync<T>(object hashKey, object rangeKey, CancellationToken cancellationToken = default)
{
    await _context.DeleteAsync<T>(hashKey, rangeKey, cancellationToken);
}

public async Task DeleteAsync<T>(object hashKey, object rangeKey, DynamoDBOperationConfig operationConfig, CancellationToken cancellationToken = default)
{
    await _context.DeleteAsync<T>(hashKey, rangeKey, operationConfig, cancellationToken);
}

public async Task DeleteAsync<T>(T value, DynamoDBOperationConfig operationConfig, CancellationToken cancellationToken = default)
{
    await _context.DeleteAsync<T>(value, operationConfig, cancellationToken);
}

public async Task DeleteAsync<T>(T value, CancellationToken cancellationToken = default)
{
    await _context.DeleteAsync<T>(value, cancellationToken);            
}

public async Task ExecuteBatchGetAsync(BatchGet[] batches, CancellationToken cancellationToken = default)
{
    await _context.ExecuteBatchGetAsync(batches, cancellationToken);
}

public async Task ExecuteBatchWriteAsync(BatchWrite[] batches, CancellationToken cancellationToken = default)
{
    await _context.ExecuteBatchWriteAsync(batches, cancellationToken);
}

public T FromDocument<T>(Document document, DynamoDBOperationConfig operationConfig)
{
    return _context.FromDocument<T>(document, operationConfig);
}

public T FromDocument<T>(Document document)
{
    return _context.FromDocument<T>(document);
}

public IEnumerable<T> FromDocuments<T>(IEnumerable<Document> documents)
{
    return _context.FromDocuments<T>(documents);
}

public IEnumerable<T> FromDocuments<T>(IEnumerable<Document> documents, DynamoDBOperationConfig operationConfig)
{
    return _context.FromDocuments<T>(documents, operationConfig);
}

       

       

public Table GetTargetTable<T>(DynamoDBOperationConfig operationConfig = null)
{
    return _context.GetTargetTable<T>(operationConfig);
}

public async Task<T> LoadAsync<T>(object hashKey, object rangeKey, DynamoDBOperationConfig operationConfig, CancellationToken cancellationToken = default)
{
    return await _context.LoadAsync<T>(hashKey, rangeKey, operationConfig, cancellationToken);
}

public async Task<T> LoadAsync<T>(T keyObject, CancellationToken cancellationToken = default)
{
    return await _context.LoadAsync<T>(keyObject, cancellationToken);
}

public async Task<T> LoadAsync<T>(object hashKey, object rangeKey, CancellationToken cancellationToken = default)
{
    return await _context.LoadAsync<T>(hashKey, rangeKey, cancellationToken);
}

public async Task<T> LoadAsync<T>(object hashKey, DynamoDBOperationConfig operationConfig, CancellationToken cancellationToken = default)
{
    return await _context.LoadAsync<T>(hashKey, operationConfig, cancellationToken);
}

public async Task<T> LoadAsync<T>(T keyObject, DynamoDBOperationConfig operationConfig, CancellationToken cancellationToken = default)
{
    return await _context.LoadAsync<T>(keyObject, operationConfig, cancellationToken);
}
        
public async Task SaveAsync<T>(T value, DynamoDBOperationConfig operationConfig, CancellationToken cancellationToken = default)
{
    await _context.SaveAsync<T>(value, operationConfig, cancellationToken);
}
        
public Document ToDocument<T>(T value, DynamoDBOperationConfig operationConfig)
{
    return _context.ToDocument<T>(value, operationConfig);
}

public Document ToDocument<T>(T value)
{
    return _context.ToDocument<T>(value);
}

[ahmed-malik-cko]

Hi Guys,
Any chance this issue can move into the main library it's a bit frustrating not being able to write unit tests as part of the main library?

[nukec]

There is a PR here #1187 , but was never merged.

[bsolovij-ebsco]

Bump

[JuFrolov]

Bump

[kidchenko-jq-tw]

Bump

[AviadHasidof]

Bump

[brpyne]

Bump

[joao2391]

Bump

[martinjansa]

Bump

[eric-davis]

BUMP!!!

[osherzhukovebsco]

Bump

[jacob-forbes]

bump

[AdalbertKhoolsaat]

bump

[jakubbujak]

It is ridiculous not to do anything about it for 5 years (sic!) 😡 😡 😡

[jgv115]

bump

Just wondering if there has been any movement on this?

[normj]

I put out this PR to provide a way to mock AsyncSearch. It is not perfect given the constraints we have to avoid making breaking changes but I would appreciate some thoughts on the PR.

#2089

[jakubbujak]

If anyone wants workaround you can use approach propsed in this comment:

#772 (comment)

Works fine 😃 But still it would be better to have that out of the box from library 👎

[normj]

@jakubbujak Are you saying that my PR that would go in the box would not suffice?

[jakubbujak]

I did not write anything like that, it may be, but I did not have time to review it :)

[normj]

Version 3.7.4.9 of the AWSSDK.DynamoDBv2 package was recently released with #2089 that adds the ability to subclass AsyncSearch for mocking purposes.

        public class MockAsyncSearch<T> : AsyncSearch<T>
        {
            public override Task<List<T>> GetNextSetAsync(CancellationToken cancellationToken = default(CancellationToken))
            {
                return Task.FromResult(new List<T>());
            }
        }

        [TestMethod]
        [TestCategory("DynamoDBv2")]
        public async Task TestMockingAsyncSeach()
        {
            var mockDBContext = new Mock<IDynamoDBContext>();
            mockDBContext
                .Setup(x => x.ScanAsync<DataItem>(
                   It.IsAny<IEnumerable<ScanCondition>>(),
                   It.IsAny<DynamoDBOperationConfig>()))
                .Returns(
                   new MockAsyncSearch<DataItem>() // Return mock version of AsyncSearch
                );

            var search = mockDBContext.Object.ScanAsync<DataItem>(new List<ScanCondition>());
            Assert.IsInstanceOfType(search, typeof(MockAsyncSearch<DataItem>));

            var items = await search.GetNextSetAsync();
            Assert.AreEqual(0, items.Count());
        }

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.