ScanResultPage; no mapping for HASH key

[brettcooper]

Hi,

I have been experiencing an odd issue where I am intermittently receiving an "Caused by: com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMappingException: ScanResultPage; no mapping for HASH key" when trying to do paginated queries with & without a query filter. This error is being thrown while running our unit tests in CircleCI for almost every other build.

After some Googling, I have made sure that our table (Audit) has a DynamoDB hash key and public setters & getters, but still can't figure out this issue. We are running AWS SDK 1.11.281 and building on java-1.8.0-openjdk-devel. Thanks for your help and let me know if any more info is needed!

Audit object:

@DynamoDBTable(tableName = "audit")
public class Audit implements ApplicationScoped, VersionedEntity {

    @DynamoDBHashKey(attributeName = "application_id")
    private String applicationId;

    @DynamoDBRangeKey(attributeName = "entity_id")
    private String entityId;

    @DynamoDBAttribute(attributeName = "campaign_id")
    private String campaignId;

    @DynamoDBTypeConvertedEnum
    @DynamoDBAttribute(attributeName = "entity_type")
    private EntityType entityType;

    @DynamoDBAttribute(attributeName = "webhook_received")
    @DynamoDBTypeConverted(converter = ZonedDateTimeConverter.class)
    private ZonedDateTime webhookReceived;

    @DynamoDBAttribute(attributeName = "webhook_transformed")
    @DynamoDBTypeConverted(converter = ZonedDateTimeConverter.class)
    private ZonedDateTime webhookTransformed;

    @DynamoDBAttribute(attributeName = "outbound_email_created_at")
    @DynamoDBTypeConverted(converter = ZonedDateTimeConverter.class)
    private ZonedDateTime outboundEmailCreatedAt;

    @DynamoDBAttribute(attributeName = "outbound_email_sent_at")
    @DynamoDBTypeConverted(converter = ZonedDateTimeConverter.class)
    private ZonedDateTime outboundEmailSentAt;

    @DynamoDBAttribute(attributeName = "email_processed")
    @DynamoDBTypeConverted(converter = ZonedDateTimeConverter.class)
    private ZonedDateTime emailProcessed;

    @DynamoDBAttribute(attributeName = "email_delivered")
    @DynamoDBTypeConverted(converter = ZonedDateTimeConverter.class)
    private ZonedDateTime emailDelivered;

    @DynamoDBAttribute(attributeName = "email_opened")
    @DynamoDBTypeConverted(converter = ZonedDateTimeConverter.class)
    private ZonedDateTime emailOpened;

    @DynamoDBAttribute(attributeName = "processing_time")
    private Integer processingTime;

    private Long version;

    public Audit() {}

    @Override
    public String getApplicationId() {
        return applicationId;
    }

    public void setApplicationId(String applicationId) {
        this.applicationId = applicationId;
    }

    public String getEntityId() {
        return entityId;
    }

    public void setEntityId(String entityId) {
        this.entityId = entityId;
    }

    public EntityType getEntityType() { return entityType; }

    public void setEntityType(EntityType entityType) { this.entityType = entityType; }

    public ZonedDateTime getWebhookReceived() {
        return webhookReceived;
    }

    public void setWebhookReceived(ZonedDateTime webhookReceived) {
        this.webhookReceived = webhookReceived;
    }

    public ZonedDateTime getWebhookTransformed() {
        return webhookTransformed;
    }

    public void setWebhookTransformed(ZonedDateTime webhookTransformed) {
        this.webhookTransformed = webhookTransformed;
    }

    public ZonedDateTime getOutboundEmailCreatedAt() {
        return outboundEmailCreatedAt;
    }

    public void setOutboundEmailCreatedAt(ZonedDateTime outboundEmailCreatedAt) {
        this.outboundEmailCreatedAt = outboundEmailCreatedAt;
    }

    public ZonedDateTime getEmailProcessed() {
        return emailProcessed;
    }

    public void setEmailProcessed(ZonedDateTime emailProcessed) {
        this.emailProcessed = emailProcessed;
    }

    public ZonedDateTime getEmailDelivered() {
        return emailDelivered;
    }

    public void setEmailDelivered(ZonedDateTime emailDelivered) {
        this.emailDelivered = emailDelivered;
    }

    public ZonedDateTime getEmailOpened() {
        return emailOpened;
    }

    public void setEmailOpened(ZonedDateTime emailOpened) {
        this.emailOpened = emailOpened;
    }

    public Integer getProcessingTime() {
        return processingTime;
    }

    public void setProcessingTime(Integer processingTime) {
        this.processingTime = processingTime;
    }

    public ZonedDateTime getOutboundEmailSentAt() {
        return outboundEmailSentAt;
    }

    public void setOutboundEmailSentAt(ZonedDateTime outboundEmailSentAt) {
        this.outboundEmailSentAt = outboundEmailSentAt;
    }

    public String getCampaignId() { return campaignId; }

    public void setCampaignId(String campaignId) { this.campaignId = campaignId; }

    @Override
    @DynamoDBVersionAttribute
    public Long getVersion() {
        return version;
    }

    @Override
    public void setVersion(Long version) {
        this.version = version;
    }

    @Override
    public String toString() {
        return ReflectionToStringBuilder.toString(this);
    }
}

GraphQL Audit Service:

    @GraphQLQuery(name = "paginated_audits")
    public ScanResultPage<Audit> getPaginatedAudits(@GraphQLArgument(name = "filter") PaginationFilter filter) {
        return auditRepository.findAll(filter.getPageSize(), filter.getLastEvaluatedKey(), filter.getQueryFilter());
    }

FindAll method in BaseRepository:

    @Override
    public ScanResultPage<T> findAll(Integer limit, Map<String, AttributeValue> exclusiveStartKey,
                                     Map<String, Condition> queryFilter) {
        if (limit == null || limit <= 0 || limit == Integer.MAX_VALUE) return new ScanResultPage<>();
        if (queryFilter == null || queryFilter.isEmpty()) return findAll(limit, exclusiveStartKey);

        DynamoDBScanExpression dynamoDBScanExpression = new DynamoDBScanExpression()
                .withScanFilter(queryFilter)
                .withExclusiveStartKey(exclusiveStartKey)
                .withLimit(limit);

        ScanResultPage<T> scanResultPage = dbMapper.scanPage(type, dynamoDBScanExpression);

        if (storageHandlerFeature != null) {
            storageHandlerFeature.retrieveAll(scanResultPage);
        }

        return scanResultPage;
    }

Unit tests:

    //    @Ignore("Intermittent failures and not sure why.")
    @Test
    public void test_get_all_paginated() throws JsonProcessingException {
        ZonedDateTime now = ZonedDateTime.now();

        Audit audit1 = new Audit();
        audit1.setApplicationId(UUID.randomUUID().toString());
        audit1.setEntityId(UUID.randomUUID().toString());
        audit1.setCampaignId(UUID.randomUUID().toString());
        audit1.setEntityType(EntityType.Order);
        audit1.setWebhookReceived(now);

        Audit audit2 = new Audit();
        audit2.setApplicationId(UUID.randomUUID().toString());
        audit2.setEntityId(UUID.randomUUID().toString());
        audit2.setCampaignId(UUID.randomUUID().toString());
        audit2.setEntityType(EntityType.Order);
        audit2.setWebhookTransformed(now.plusMinutes(1L));

        Audit audit3 = new Audit();
        audit3.setApplicationId(UUID.randomUUID().toString());
        audit3.setEntityId(UUID.randomUUID().toString());
        audit3.setCampaignId(UUID.randomUUID().toString());
        audit3.setEntityType(EntityType.Order);
        audit3.setOutboundEmailCreatedAt(now.plusMinutes(2L));

        repository.saveAll(audit1, audit2, audit3);

        AuditService auditService = new AuditService(repository);

        List<Audit> audits = new ArrayList<>(3);
        PaginationFilter paginationFilter = new PaginationFilter(1);

        do {
            LOG.debug("Pagination filter: {}", MAPPER.writeValueAsString(paginationFilter));

            ScanResultPage<Audit> paginatedAudits = auditService.getPaginatedAudits(paginationFilter);
            paginationFilter.setLastEvaluatedKey(paginatedAudits.getLastEvaluatedKey());

            audits.addAll(paginatedAudits.getResults());
        }
        while (paginationFilter.getLastEvaluatedKey() != null);

        assertThat(audits.size(), is(3));
        assertThat(audits, hasItems(sameBeanAs(audit1), sameBeanAs(audit2), sameBeanAs(audit3)));
    }

    //    @Ignore("Intermittent failures and not sure why.")
    @Test
    public void test_get_all_paginated_with_query_filter() {
        ZonedDateTime now = ZonedDateTime.now();

        Audit audit1 = new Audit();
        audit1.setApplicationId(UUID.randomUUID().toString());
        audit1.setEntityId(UUID.randomUUID().toString());
        audit1.setCampaignId(UUID.randomUUID().toString());
        audit1.setEntityType(EntityType.Order);
        audit1.setWebhookReceived(now);

        Audit audit2 = new Audit();
        audit2.setApplicationId(UUID.randomUUID().toString());
        audit2.setEntityId(UUID.randomUUID().toString());
        audit2.setCampaignId(UUID.randomUUID().toString());
        audit2.setEntityType(EntityType.Customer);
        audit2.setWebhookTransformed(now.plusMinutes(1L));

        Audit audit3 = new Audit();
        audit3.setApplicationId(UUID.randomUUID().toString());
        audit3.setEntityId(UUID.randomUUID().toString());
        audit3.setCampaignId(UUID.randomUUID().toString());
        audit3.setEntityType(EntityType.Product);
        audit3.setOutboundEmailCreatedAt(now.plusMinutes(2L));

        repository.saveAll(audit1, audit2, audit3);

        AuditService auditService = new AuditService(repository);

        Map<String, Condition> queryFilter = new HashMap<>();
        queryFilter.put("entity_type", createCondition(EQ, EntityType.Order.name()));

        List<Audit> audits = new ArrayList<>(1);
        PaginationFilter paginationFilter = new PaginationFilter(1, queryFilter);

        do {
            ScanResultPage<Audit> paginatedAudits = auditService.getPaginatedAudits(paginationFilter);
            paginationFilter.setLastEvaluatedKey(paginatedAudits.getLastEvaluatedKey());

            audits.addAll(paginatedAudits.getResults());
        }
        while (paginationFilter.getLastEvaluatedKey() != null);

        assertThat(audits.size(), is(1));
        assertThat(audits, hasItem(sameBeanAs(audit1)));
    }

Error message:

[INFO] Running com.unific.graphql.services.AuditServiceTest
23:42:34.811 DEBUG [main] com.unific.graphql.services.AuditServiceTest - Pagination filter: {"page_size":1,"query_filter":{},"last_evaluated_key":null}
[ERROR] Tests run: 3, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.079 s <<< FAILURE! - in com.unific.graphql.services.AuditServiceTest
[ERROR] test_get_all_paginated(com.unific.graphql.services.AuditServiceTest)  Time elapsed: 0.042 s  <<< ERROR!
java.lang.reflect.UndeclaredThrowableException
	at com.unific.graphql.services.AuditServiceTest.test_get_all_paginated(AuditServiceTest.java:114)
Caused by: java.lang.reflect.InvocationTargetException
	at com.unific.graphql.services.AuditServiceTest.test_get_all_paginated(AuditServiceTest.java:114)
Caused by: com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMappingException: ScanResultPage; no mapping for HASH key
	at com.unific.graphql.services.AuditServiceTest.test_get_all_paginated(AuditServiceTest.java:114)

[ERROR] test_get_all_paginated_with_query_filter(com.unific.graphql.services.AuditServiceTest)  Time elapsed: 0.022 s  <<< ERROR!
java.lang.reflect.UndeclaredThrowableException
	at com.unific.graphql.services.AuditServiceTest.test_get_all_paginated_with_query_filter(AuditServiceTest.java:161)
Caused by: java.lang.reflect.InvocationTargetException
	at com.unific.graphql.services.AuditServiceTest.test_get_all_paginated_with_query_filter(AuditServiceTest.java:161)
Caused by: com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMappingException: ScanResultPage; no mapping for HASH key
	at com.unific.graphql.services.AuditServiceTest.test_get_all_paginated_with_query_filter(AuditServiceTest.java:161)

[millems]

Could you enable wire logging and share the results for a failed request/response?

[millems]

Any update?

[brettcooper]

Hi Matt, I haven't had a chance to enable wire logging and reproduce this issue yet. I will try to get to this soon. Thanks, Brett

…

On Mon, May 7, 2018 at 1:26 PM, Matthew Miller ***@***.***> wrote: Any update? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1496 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQBefa4f3ETt4owLuZLZjYQIIzi5aRn_ks5twIPPgaJpZM4SaeFl> .

[brettcooper]

Update: I was able to get around this, but can't quite remember what I did to fix it. If I run into this again, I will try to send over more details. Thanks for responding!