Build an AI Line Chatbot using AWS Bedrock

From the last article, I explained how to use build a simple chatbot with AWS Lambda. However, the chatbot we built last time it can only response what you said, it did not Chat at all. This time, I try to connect that chatbot with AWS bedrock, and choose a perfect LLM so that it can actually chat with us.

Steps

1. Get the access of models from AWS bedrock, and choose the suitable model
2. Create a no sql database from DynamoDB
3. Open the permissions from AWS IAM
4. Install boto layers
5. Connect Bedrock and DynamoDB to AWS Lambda

Get Access of Models from Bedrock

Go to Bedrock console to open the access of models. Scrolling down the side bar to the bottom, you can see Model access. Click on it.

After clicking, you will find tons of model names, on top left of the list of model name, you will find Request access. Or find Modify model access, they are the same link.

What location of server you chose decide which models you can use. For example, models in Oregon is more than models in Tokyo.

You can get the all access in AWS, and AWS will automatically decide which models you can use. Press Next until press Submit.

If you want to use Claude models from Anthropic, you need to input additional information for access.

Once you see the Access status of all models you selected are green, then this step is completed. The process of waiting access needs some time, so you can do the next step while waiting.

Create a No SQL Database from DynamoDB

Now, go to the DynamoDB console. Press Craete tables to create a table.
You can name whatever you want in Table name. Please put userId in Partition key and set it as string.
After creating, click your table which you just created, and under the General information, expand Additional info and get ARN.

Open the Permissions from AWS IAM

Go back to the lambda function you wrote in last article. Under Configuration, click Permissions on the left side bar. Press the link under Role name.
After entering IAM, press Add permissions - Create inline policy.

• Choose service Bedrock. If you are lazy enough (like me), click All Bedrock actions in Actions allowed and click all in Resources.
• Choose service DynamoDB. If you are lazy enough (like me), click All Bedrock actions in Actions allowed and click all in Resources.

Name the policy whatever you want, and create two policies. One is for Bedrock and the other is for DynamoDB.

Connect Bedrock and DynamoDB to AWS Lambda

First, you need to build a layer for boto3, the AWS python SDK. The way how to build a layer is in the last article.

Modify my following example code, I used mistral.mixtral-8x7b-instruct as my model.

from linebot import (
    LineBotApi, WebhookHandler
)
from linebot.exceptions import (
    InvalidSignatureError
)
from linebot.models import (
    MessageEvent, TextMessage, TextSendMessage,
)

import os
import json
import boto3

client = boto3.client('bedrock-runtime')
line_bot_api = LineBotApi(os.environ['Channel_access_token'])
handler = WebhookHandler(os.environ['Channel_secret'])
modelId = "mistral.mixtral-8x7b-instruct-v0:1"
accept = 'application/json'
contentType = 'application/json'
dynamodb = boto3.resource('dynamodb', region_name='us-west-2')
table = dynamodb.Table('tableName')

def lambda_handler(event, context):
    @handler.add(MessageEvent, message=TextMessage)
    def handle_message(event):
        db_res = table.get_item(
            Key={
                'userId': event.source.user_id
            }
        )
        item = db_res.get('Item')

        if item:
            messageText = item['history']
            messageText += "<s>[INST]"+event.message.text+"[/INST]"
        else:
            messageText = "<s>[INST]"+event.message.text+"[/INST]"

        mistral_input = json.dumps({
        "prompt": messageText, 
        "max_tokens":500,
        "temperature":0.5,
        "top_p":0.9,
        "top_k":50
        })

        response = client.invoke_model(body=mistral_input, modelId=modelId, accept=accept, contentType=contentType)
        response_body = json.loads(response.get("body").read())
        response_text = response_body['outputs'][0]['text']

        messageText += response_text + "</s>"
        res_put = table.put_item(
        Item={
            'userId': event.source.user_id,
            'history': messageText
            }
        )

        line_bot_api.reply_message(
            event.reply_token,
            TextSendMessage(text=response_text))

    # get X-Line-Signature header value
    signature = event['headers']['x-line-signature']

    # get request body as text
    body = event['body']

    # handle webhook body
    try:
        handler.handle(body, signature)
    except InvalidSignatureError:
        return {
            'statusCode': 502,
            'body': json.dumps("Invalid signature. Please check your channel access token/channel secret.")
            }
    return {
        'statusCode': 200,
        'body': json.dumps("Hello from Lambda!")
        }

Deploy again, and it should be work.

If you want to change the model, please be careful that models from different companies need different structure of input. You can see it in Bedrock - providers