How to use AWS dynamoDB outside AWS Environment

Previously, I saved my json-like data in Firebase realtime database, when I just develop the function, they are all fine. However, once I deployed and started to use in public, I found out that I ran out the limit of free tier in Firebase.

At the same time, I found out that the limit of free tier on AWS dynamoDB is much bigger than firebase realtime database. Therefore, I decided to transfer my data from firebase to dynamoDB.

You can see the difference, 25GB of storage v.s. 1GB storing. That's why I want to transfer data.

Using dynamoDB outside AWS is not that simple

I thought my old article Build an AI Line Chatbot using AWS Bedrock already had the solution to dealing with dynamoDB. However, I was wrong. Under AWS enviroment, you just need to do the two things:

1. Open permission in IAM
2. import boto3

Actually, that is the answer what I saw in most of articles in web, but when you are outside of AWS, the things become a little bit complicated.

Using dynamoDB out of AWS is not so hard

Create Virtual User

• Enter IAM and find Users section.
  
• Press Create User
  • You can decide whether this user can enter console or not. If you just want to this user a virtual one, then do not need to click it.
• Set some permissions. For this example, I just want the permission of dynamoDB and also I am lazy, so I open all permissions on dynamoDB for this virtual user.
• After finishing creating virutal user, you should get access keys:
  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY

Import boto3

I want to use dynamoDB outside AWS, and in this case, I want to use it in Google colab. boto3 is python SDK to connect AWS services, and of course, this package is not installed in colab.

• First, install boto3. !pip install boto3
• After installing, import boto3

Set virtual user's access keys

Before continuing on python, remember create some tables in dynamoDB. For this tutorial, I create a table called 123. At this moment, you can find out lots of articles saying that you just paste all the code belowed, and it can work magically.

import boto3
dynamodb = boto3.resource('dynamodb', region_name=AWS_REGION)

table = dynamodb.Table('123')

table.put_item(
    Item={
        'userId': '1',
        'cartNumber': 'cart#123',
        'name': 'SomeName',
        'inventory': 500,
    }
)

But it is not true, since boto3 cannot know who you are outside AWS. Here, we need to set access keys. There are two ways to set them.

Just hard code anyway

import boto3
client = boto3.client(
    'dynamodb',
    region_name=AWS_REGION,
    aws_access_key_id=ACCESS_KEY,
    aws_secret_access_key=SECRET_KEY,
)

AWS_REGION is the location you put your dynamoDB tables, for example, Oregon is 'us-west-2'.

However, I do not recommend this method if you want to share your code or you want to update to github.

Environment variables

import os
os.environ["AWS_ACCESS_KEY_ID"] = ACCESS_KEY
os.environ["AWS_SECRET_ACCESS_KEY"] = SECRET_KEY

You just save your access key into environment variables, and that's it. When you import boto3, it can find out your access key and start run the code you want to use dynamoDB correctly.