Is There A Python Api For Submitting Batch Get Requests To Aws Dynamodb?

The package boto3 - Amazon's official AWS API wrapper for python - has great support for uploading items to DynamoDB in bulk. It looks like this: db = boto3.resource('dynamodb', r

Solution 1:

So thankfully there is something that you might find useful - much like the json module which has json.dumps and json.loads, boto3 has a types module that includes a serializer and deserializer. See TypeSerializer/TypeDeserializer. If you look at the source code, the serialization/deserialization is recursive and should be perfect for your use case.

Note: Its recommended that you use Binary/Decimal instead of just using a regular old python float/int for round trip conversions.

serializer = TypeSerializer()
serializer.serialize('awesome') # returns {'S' : 'awesome' }

deser = TypeDeserializer()
deser.deserialize({'S' : 'awesome'}) # returns u'awesome'

Hopefully this helps!

Solution 2:

There's the service resource level batch_get_item. Maybe you could do something like that :

def batch_query_wrapper(table, key, values):

    results = []

    response = dynamo.batch_get_item(RequestItems={table: {'Keys': [{key: val} for val in values]}})
    results.extend(response['Responses'][table])

    while response['UnprocessedKeys']:

        # Implement some kind of exponential back off here
        response = dynamo.batch_get_item(RequestItems={table: {'Keys': [{key: val} for val in values]}})
        results.extend(response['Response'][table])

    return results

It will return your result as python objects.

Solution 3:

I find this to be an effective way to convert a Boto 3 DynamoDB item to a Python dict.

https://github.com/Alonreznik/dynamodb-json

Solution 4:

Somewhat related answer, the docs advertise the query method; their e.g.:

from boto3.dynamodb.conditions import Key, Attr

response = table.query(
    KeyConditionExpression=Key('username').eq('johndoe')
)
items = response['Items']
print(items)