Caching

[mathewmarcus]

Is there any interest in implementing a caching layer in PyanmoDB? Or is this best left to the users of the library?

[jyore]

@mathewmarcus there is now an AWS service addition to Dynamo called DynamoDB Accelerator (DAX) that provides a fully managed caching layer. The best part, your application code does not need to manage it in any way, as it runs off the same API calls as DynamoDB, once enabled.

https://aws.amazon.com/dynamodb/dax/

[mikhailklassen]

After creating a cluster for DAX on the AWS Console, what modifications are needed so that pynamodb accesses the cached data on the cluster? I'm still finding that my application is hitting the DynamoDB table and not the DAX cluster.

[koslib]

Anyone has ideas on @mikhailklassen's question?
The closest I've thought so far about this is to change the host Meta attribute to DAX's endpoint URL. Has anyone tried it before?

[eldos-dl]

Is there support for DAX with PynamoDB?

[gulbinas]

DAX is not supported in eu-central-1 👎

Is there any extension points that could be used to cache data locally?

[devangmehta123]

I am seeing comment by @koslibpro and it would be nice if someone from pynamodb core team would comment on that idea if it works. But I think an extra step might be needed.

It would help if someone from the core team would comment. AWS has published special DAX clients which operate in roughly the same way.

If someone integrates the python client from here, http://dax-sdk.s3-website-us-west-2.amazonaws.com/, into dynamodb replacing boto3, maybe it would work with minimal changes.

I think someone from the core group of authors could do it quite quickly but it may be harder for someone not familiar with the code base.

UPDATE:

There is more useful information here: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DAX.client.run-application-python.html

UPDATE 2:

In the 3 examples, there is a way to just switch the low level dynamodb client on whether DAX endpoint is provided or not like this:

region = os.environ.get('AWS_DEFAULT_REGION', 'us-west-2')

session = botocore.session.get_session()
dynamodb = session.create_client('dynamodb', region_name=region) # low-level client

table_name = "TryDaxTable"

if len(sys.argv) > 1:
    endpoint = sys.argv[1]
    dax = amazondax.AmazonDaxClient(session, region_name=region, endpoints=[endpoint])
    client = dax
else:
    client = dynamodb

[devangmehta123]

@garrettheel and @jpinner-lyft FYI and all the rest on this thread.

I have tried to make a simple change to make this work by replacing the the botocore client with AmazonDaxClient.

And it is blocked at the place where Pynamodb uses the _service_model attribute from inside the client. This is a hidden attribute so may be it is not fully necessary.

My fork of the repo is here: https://github.com/devangmehta123/PynamoDB

Can someone help me move this forward?

[gulbinas]

#470