CDK Entest

Introduction#

Video demo

This repo shows how to get started with Amazon Bedrock in golang through basic examples.

simple chat and prompt
query vector database (opensearch)
simple image analyzing
bedrock knowledge base

For learning purpose, it implement these features using only basic concepts and without relying on framework like LangChain, Streamlit, or React.

basic stream response
basic css and javascript

WebServer#

Project structure

|--image
   |--demo.jpeg
|--static
   |--claude-haiku.html
   |--claude2.html
   |--image.html
   |--opensearch.html
|--aoss.go
|--bedrock.go
|--constants.go
|--go.mod
|--main.go

main.go implement a http server and route request to handlers. bedrock.go and aoss.go are functions to invoke Amazon Bedrock and Amazon OpenSearch Serverless (AOSS), respecitively. static folder contains simple frontend with javascript.

[!IMPORTANT]
To use AOSS, you need create a OpenSearch collection and provide its URL endpoint in constants.go. In addition, you need to setup data access in the AOSS for the running time environment (EC2 profile, ECS taks role, Lambda role, .etc)

Stream Response#

First it is good to create some data structs according to Amazon Bedrock Claude3 API format

type Content struct {
	Type string `json:"type"`
	Text string `json:"text"`
}

type Message struct {
	Role    string    `json:"role"`
	Content []Content `json:"content"`
}

type Body struct {
	MaxTokensToSample int       `json:"max_tokens"`
	Temperature       float64   `json:"temperature,omitempty"`
	AnthropicVersion  string    `json:"anthropic_version"`
	Messages          []Message `json:"messages"`
}

// list of messages
messages := []Message{{
	Role:    "user",
	Content: []Content{{Type: "text", Text: promt}},
}}

// form request body
payload := Body{
	MaxTokensToSample: 2048,
	Temperature:       0.9,
	AnthropicVersion:  "bedrock-2023-05-31",
	Messages:          messages,
}

Then convert the payload to bytes and invoke Bedrock client

payload := Body{
	MaxTokensToSample: 2048,
	Temperature:       0.9,
	AnthropicVersion:  "bedrock-2023-05-31",
	Messages:          messages,
}

// marshal payload to bytes
payloadBytes, err := json.Marshal(payload)

if err != nil {
	fmt.Println(err)
	return
}

// create request to bedrock
output, error := BedrockClient.InvokeModelWithResponseStream(
	context.Background(),
	&bedrockruntime.InvokeModelWithResponseStreamInput{
		Body:        payloadBytes,
		ModelId:     aws.String("anthropic.claude-3-haiku-20240307-v1:0"),
		ContentType: aws.String("application/json"),
		Accept:      aws.String("application/json"),
	},
)

if error != nil {
	fmt.Println(error)
	return
}

Finally, parse the streaming reponse and decode to text. When deploy on a http server, we need to modify the code a bit to stream each chunk of response to client. For example HERE

output, error := BedrockClient.InvokeModelWithResponseStream(
	context.Background(),
	&bedrockruntime.InvokeModelWithResponseStreamInput{
		Body:        payloadBytes,
		ModelId:     aws.String("anthropic.claude-3-haiku-20240307-v1:0"),
		ContentType: aws.String("application/json"),
		Accept:      aws.String("application/json"),
	},
)

if error != nil {
	fmt.Println(error)
	return
}

// parse response stream
for event := range output.GetStream().Events() {
	switch v := event.(type) {
	case *types.ResponseStreamMemberChunk:

		//fmt.Println("payload", string(v.Value.Bytes))

		var resp ResponseClaude3
		err := json.NewDecoder(bytes.NewReader(v.Value.Bytes)).Decode(&resp)
		if err != nil {
			fmt.Println(err)
		}

		fmt.Println(resp.Delta.Text)

	case *types.UnknownUnionMember:
		fmt.Println("unknown tag:", v.Tag)

	default:
		fmt.Println("union is nil or unknown type")
	}
}

Image Analyze#

Similarly, for image analyzing using Amazon Bedrock Claude3, we need to create a correct request format. It is possible without explicitly define structs as above and using interface

// read image from local file
imageData, error := ioutil.ReadFile("demo.jpeg")

if error != nil {
	fmt.Println(error)
}

// encode image to base64
base64Image := base64.StdEncoding.EncodeToString(imageData)

source := map[string]interface{}{
		"type":       "base64",
		"media_type": "image/jpeg",
		"data":       base64Image,
	}

messages := []map[string]interface{}{{
	"role":    "user",
	"content": []map[string]interface{}{{"type": "image", "source": source}, {"type": "text", "text": "what is in this image?"}},
}}

payload := map[string]interface{}{
	"max_tokens":        2048,
	"anthropic_version": "bedrock-2023-05-31",
	"temperature":       0.9,
	"messages":          messages,
}

Then invoke Amazon Bedrock Client like below, and similar for streaming reponse as previous example.

// convert payload struct to bytes
payloadBytes, error := json.Marshal(payload)

if error != nil {
	fmt.Println(error)
}

// invoke bedrock claude3 haiku
output, error := BedrockClient.InvokeModel(
	context.Background(),
	&bedrockruntime.InvokeModelInput{
		Body:        payloadBytes,
		ModelId:     aws.String("anthropic.claude-3-haiku-20240307-v1:0"),
		ContentType: aws.String("application/json"),
		Accept:      aws.String("application/json"),
	},
)

if error != nil {
	fmt.Println(error)
}

// response
fmt.Println(string(output.Body))

OpenSearch#

create OpenSearch client
convert user question to embedding vector
send query or request to OpenSearch

A OpenSearch and Bedrock client can be initialized as below

InitOpenSearchBedrockClient

// opensearch severless client
var AOSSClient *opensearch.Client

// bedrock client
var BedrockClient *bedrockruntime.Client

// create an init function to initializing opensearch client
func init() {

	//
	fmt.Println("init and create an opensearch client")

	// load aws credentials from profile demo using config
	awsCfg, err := config.LoadDefaultConfig(context.Background(),
		config.WithRegion("us-east-1"),
	)

	if err != nil {
		log.Fatal(err)
	}

	// create bedorck runtime client
	BedrockClient = bedrockruntime.NewFromConfig(awsCfg)

	// create a aws request signer using requestsigner
	signer, err := requestsigner.NewSignerWithService(awsCfg, "aoss")

	if err != nil {
		log.Fatal(err)
	}

	uncommen for opensearch client
	create an opensearch client using opensearch package
	AOSSClient, err = opensearch.NewClient(opensearch.Config{
		Addresses: []string{AOSS_ENDPOINT},
		Signer:    signer,
	})

	if err != nil {
		log.Fatal(err)
	}
}

Create a function to convert text to vector by invoking Amazon Bedrock Titan model.

GetEmbedVector

func GetEmbedVector(qustion string) ([]float64, error) {

	// create request body to titan model
	body := map[string]interface{}{
		"inputText": qustion,
	}
	bodyJson, err := json.Marshal(body)

	if err != nil {
		fmt.Println(err)
		return nil, err
	}

	// invoke bedrock titan model to convert string to embedding vector
	response, error := BedrockClient.InvokeModel(
		context.Background(),
		&bedrockruntime.InvokeModelInput{
			Body:        []byte(bodyJson),
			ModelId:     aws.String("amazon.titan-embed-text-v1"),
			ContentType: aws.String("application/json"),
		},
	)

	if error != nil {
		fmt.Println(error)
		return nil, error
	}

	// assert response to map
	var embedResponse map[string]interface{}

	error = json.Unmarshal(response.Body, &embedResponse)

	if error != nil {
		fmt.Println(error)
		return nil, error
	}

	// assert response to array
	slice, ok := embedResponse["embedding"].([]interface{})

	if !ok {
		fmt.Println(ok)
	}

	// assert to array of float64
	values := make([]float64, len(slice))

	for k, v := range slice {
		values[k] = float64(v.(float64))
	}

	return values, nil
}

Then send request or query to AOSS

QueryOpenSearch

func QueryAOSS(vec []float64) ([]string, error) {

	// let query get all item in an index
	// content := strings.NewReader(`{
	//     "size": 10,
	//     "query": {
	//         "match_all": {}
	//         }
	// }`)

	vecStr := make([]string, len(vec))

	// convert array float to string
	for k, v := range vec {

		if k < len(vec)-1 {
			vecStr[k] = fmt.Sprint(v) + ","
		} else {
			vecStr[k] = fmt.Sprint(v)
		}

	}

	// create request body to titan model
	content := strings.NewReader(fmt.Sprintf(`{
		"size": 5,
		"query": {
			"knn": {
				"vector_field": {
					"vector": %s,
					"k": 5
				}
			}
		}
	}`, vecStr))

	// fmt.Println(content)

	search := opensearchapi.SearchRequest{
		Index: []string{"demo"},
		Body:  content,
	}

	searchResponse, err := search.Do(context.Background(), AOSSClient)

	if err != nil {
		log.Fatal(err)
	}

	// fmt.Println(searchResponse)

	var answer AossResponse

	json.NewDecoder(searchResponse.Body).Decode(&answer)

	// first := answer.Hits.Hits[0]

	// fmt.Printf("id: %s\n, index: %s\n, text: %s", first["_id"], first["_index"], first["_source"].(map[string]interface{})["text"])

	// fmt.Println(answer.Hits.Hits[0]["_id"])

	queryResult := answer.Hits.Hits[0]["_source"].(map[string]interface{})["text"]

	if queryResult == nil {
		return []string{"nil"}, nil
	}

	// extract hint text only
	hits := []string{}

	for k, v := range answer.Hits.Hits {

		if k > 0 {
			hits = append(hits, v["_source"].(map[string]interface{})["text"].(string))
		}

	}

	return hits, nil

	// return fmt.Sprint(queryResult), nil

}

Knowledge Based#

create knowledge based
retrieve
retrieve and generate
iam permissions for ECS task

First, go to bedrock console to create a new Knowledge Based and select Amazon OpenSearch Serverless (AOSS) as a vector database. Then let write a simple Handler to query the AOSS collection.

output, error := client.Retrieve(
		context.TODO(),
		&bedrockagentruntime.RetrieveInput{
			KnowledgeBaseId: aws.String(KNOWLEDGE_BASE_ID),
			RetrievalQuery: &types.KnowledgeBaseQuery{
				Text: aws.String(userQuestion),
			},
				RetrievalConfiguration: &types.KnowledgeBaseRetrievalConfiguration{
				VectorSearchConfiguration: &types.KnowledgeBaseVectorSearchConfiguration{
					NumberOfResults: aws.Int32(KNOWLEDGE_BASE_NUMBER_OF_RESULT),
				},
			},
		},
	)

Second, let write a function to retrieve and generate

output, error := client.RetrieveAndGenerate(
		context.TODO(),
		&bedrockagentruntime.RetrieveAndGenerateInput{
			Input: &types.RetrieveAndGenerateInput{
				Text: aws.String(userQuestion),
			},
			RetrieveAndGenerateConfiguration: &types.RetrieveAndGenerateConfiguration{
				Type: types.RetrieveAndGenerateTypeKnowledgeBase,
				KnowledgeBaseConfiguration: &types.KnowledgeBaseRetrieveAndGenerateConfiguration{
					KnowledgeBaseId: aws.String(KNOWLEDGE_BASE_ID),
					ModelArn:        aws.String(KNOWLEDGE_BASE_MODEL_ID),
					RetrievalConfiguration: &types.KnowledgeBaseRetrievalConfiguration{
						VectorSearchConfiguration: &types.KnowledgeBaseVectorSearchConfiguration{
							NumberOfResults: aws.Int32(KNOWLEDGE_BASE_NUMBER_OF_RESULT),
						},
					},
				},
			},
		},
	)

Then response should look like this

POST /knowledgebases/knowledgeBaseId/retrieve HTTP/1.1
Content-type: application/json

{
   "nextToken": "string",
   "retrievalConfiguration": {
      "vectorSearchConfiguration": {
         "filter": { ... },
         "numberOfResults": number,
         "overrideSearchType": "string"
      }
   },
   "retrievalQuery": {
      "text": "string"
   }
}

The response format should look like this

POST /retrieveAndGenerate HTTP/1.1
Content-type: application/json

{
   "input": {
      "text": "string"
   },
   "retrieveAndGenerateConfiguration": {
      "knowledgeBaseConfiguration": {
         "generationConfiguration": {
            "promptTemplate": {
               "textPromptTemplate": "string"
            }
         },
         "knowledgeBaseId": "string",
         "modelArn": "string",
         "retrievalConfiguration": {
            "vectorSearchConfiguration": {
               "filter": { ... },
               "numberOfResults": number,
               "overrideSearchType": "string"
            }
         }
      },
      "type": "string"
   },
   "sessionConfiguration": {
      "kmsKeyArn": "string"
   },
   "sessionId": "string"
}

knowledge-based.go

package bedrock

import (
	"context"
	"encoding/json"
	"fmt"
	"net/http"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/service/bedrockagentruntime"
	"github.com/aws/aws-sdk-go-v2/service/bedrockagentruntime/types"
)

func HandleRetrieve(w http.ResponseWriter, r *http.Request, client *bedrockagentruntime.Client) {

	// parse user messages
	type Content struct {
		Type string `json:"type"`
		Text string `json:"text"`
	}

	type Message struct {
		Role    string    `json:"role"`
		Content []Content `json:"content"`
	}

	var request struct {
		Messages []Message `json:"messages"`
	}

	error := json.NewDecoder(r.Body).Decode(&request)

	if error != nil {
		fmt.Println(error)
	}

	messages := request.Messages

	fmt.Println(messages)

	// pop the last message as user question
	userQuestion := messages[len(messages)-1].Content[0].Text

	// invoke bedrock agent runtime to retreive opensearch
	output, error := client.Retrieve(
		context.TODO(),
		&bedrockagentruntime.RetrieveInput{
			KnowledgeBaseId: aws.String(KNOWLEDGE_BASE_ID),
			RetrievalQuery: &types.KnowledgeBaseQuery{
				Text: aws.String(userQuestion),
			},
				RetrievalConfiguration: &types.KnowledgeBaseRetrievalConfiguration{
				VectorSearchConfiguration: &types.KnowledgeBaseVectorSearchConfiguration{
					NumberOfResults: aws.Int32(KNOWLEDGE_BASE_NUMBER_OF_RESULT),
				},
			},
		},
	)

	if error != nil {
		fmt.Println(error)
	}

	// for k, v := range output.RetrievalResults {
	// 	fmt.Println(k)
	// 	fmt.Println("=======================================")
	// 	fmt.Println(*v.Content.Text)
	// }

	// parse output to []byte and return client
	outputJson, error := json.Marshal(output)

	if error != nil {
		fmt.Println(error)
	}

	w.Write(outputJson)
}

func HandleRetrieveAndGenerate(w http.ResponseWriter, r *http.Request, client *bedrockagentruntime.Client) {

	// parse user messages
	type Content struct {
		Type string `json:"type"`
		Text string `json:"text"`
	}

	type Message struct {
		Role    string    `json:"role"`
		Content []Content `json:"content"`
	}

	var request struct {
		Messages []Message `json:"messages"`
	}

	error := json.NewDecoder(r.Body).Decode(&request)

	if error != nil {
		fmt.Println(error)
	}

	messages := request.Messages

	fmt.Println(messages)

	// pop the last message as user question
	userQuestion := messages[len(messages)-1].Content[0].Text

	// invoke bedrock agent runtime to retrieve and generate
	output, error := client.RetrieveAndGenerate(
		context.TODO(),
		&bedrockagentruntime.RetrieveAndGenerateInput{
			Input: &types.RetrieveAndGenerateInput{
				Text: aws.String(userQuestion),
			},
			RetrieveAndGenerateConfiguration: &types.RetrieveAndGenerateConfiguration{
				Type: types.RetrieveAndGenerateTypeKnowledgeBase,
				KnowledgeBaseConfiguration: &types.KnowledgeBaseRetrieveAndGenerateConfiguration{
					KnowledgeBaseId: aws.String(KNOWLEDGE_BASE_ID),
					ModelArn:        aws.String(KNOWLEDGE_BASE_MODEL_ID),
					RetrievalConfiguration: &types.KnowledgeBaseRetrievalConfiguration{
						VectorSearchConfiguration: &types.KnowledgeBaseVectorSearchConfiguration{
							NumberOfResults: aws.Int32(KNOWLEDGE_BASE_NUMBER_OF_RESULT),
						},
					},
				},
			},
		},
	)

	if error != nil {
		fmt.Println(error)
	}

	fmt.Println(*output.Output.Text)

	// parse output to []byte
	outputJson, error := json.Marshal(output)

	if error != nil {
		fmt.Println(error)
	}

	// write output to client
	w.Write(outputJson)

}

retrieve.html

<html>
  <head>
    <meta name="viewport" content="width=device-width" />
    <style>
      :root {
        box-sizing: border-box;
      }
      *,
      ::before,
      ::after {
        box-sizing: inherit;
      }

      body {
        /* background-color: antiquewhite; */
      }

      .container {
        width: 100%;
        max-width: 500px;
        margin: auto;
        /* background-color: antiquewhite; */
      }

      .button {
        background-color: #43a047;
        padding: 8px 20px;
        border-radius: 5px;
        border: none;
        cursor: pointer;
        position: absolute;
        transform: translateY(-50%);
        top: 50%;
        right: 10px;
        opacity: 0.8;
      }

      .button:hover {
        background-color: orange;
      }

      .text-input {
        padding: 10px 15px;
        width: 100%;
        outline: none;
        border: solid black 1px;
        background-color: #e0e0e0;
        box-shadow: 0 10px 15px -3px #e0e0e0;

        font-family: system-ui, -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto,
          Oxygen, Ubuntu, Cantarell, 'Open Sans', 'Helvetica Neue', sans-serif;
        font-size: medium;
        font-weight: 400;
        letter-spacing: normal;
        line-height: 25px;
      }

      .text-input:focus {
        border: solid #4caf50 1.5px;
        outline: none;
      }

      .container-input {
        position: relative;
      }

      .form {
        margin-top: 20px;
      }

      .text-model {
        /* color: #4caf50; */
        font-family: system-ui, -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto,
          Oxygen, Ubuntu, Cantarell, 'Open Sans', 'Helvetica Neue', sans-serif;
        font-size: medium;
        font-weight: 400;
        letter-spacing: normal;
        line-height: 25px;
      }
    </style>
  </head>
  <body>
    <div class="container">
      <form id="form" onkeydown="return event.key != 'Enter';" class="form">
        <div class="container-input">
          <input class="text-input" type="text" id="text-input" />
          <button id="submit" class="button">retrieve</button>
        </div>
      </form>
      <div id="list" class="text-model"></div>
    </div>
    <script>
      const callBedrockStream = async () => {
        // Get the list container element
        var listContainer = document.getElementById('list')

        // clear content before query
        listContainer.innerHTML = ''

        // conversation turns
        let messages = []

        // get user question
        const userQuestion = document.getElementById('text-input').value

        // push user question to messages
        messages.push({
          role: 'user',
          content: [{ type: 'text', text: userQuestion }]
        })

        if (userQuestion) {
          try {
            const response = await fetch('/knowledge-base-retrieve', {
              method: 'POST',
              headers: {
                'Content-Type': 'application/json'
              },
              body: JSON.stringify({ messages: messages })
            })

            console.log(response)

            const decoder = new TextDecoder()

            // batch processing
            const json = await response.json()
            const items = json['RetrievalResults']

            // update frontend
            for (var i = 0; i < items.length; i++) {
              var listItem = document.createElement('div')
              listItem.style.marginBottom = '15px'
              listItem.style.borderBottom = '1px solid #0000FF'

              var header = document.createElement('h4')
              header.textContent = `Chunk ${i}`

              var itemText = document.createTextNode(
                items[i]['Content']['Text']
              )
              listItem.appendChild(header)
              listItem.appendChild(itemText)
              listContainer.appendChild(listItem)
            }

            // push model answer to converstion turn
            // messages.push({
            //   role: "assistant",
            //   content: [{ type: "text", text: modelAnswer.innerText }],
            // });
          } catch (error) {
            console.log(error)
          }
        } else {
          console.log('Please enter question ...')
        }
      }

      document
        .getElementById('submit')
        .addEventListener('click', async event => {
          event.preventDefault()
          await callBedrockStream()
        })

      document
        .getElementById('text-input')
        .addEventListener('keydown', async event => {
          if (event.code === 'Enter') {
            await callBedrockStream()
          }
        })
    </script>
  </body>
</html>

retrieve-generate.html

<html>
  <head>
    <meta name="viewport" content="width=device-width" />
    <style>
      :root {
        box-sizing: border-box;
      }
      *,
      ::before,
      ::after {
        box-sizing: inherit;
      }

      body {
        /* background-color: antiquewhite; */
      }

      .container {
        width: 100%;
        max-width: 500px;
        margin: auto;
        /* background-color: antiquewhite; */
      }

      .button {
        background-color: #43a047;
        padding: 8px 20px;
        border-radius: 5px;
        border: none;
        cursor: pointer;
        position: absolute;
        transform: translateY(-50%);
        top: 50%;
        right: 10px;
        opacity: 0.8;
      }

      .button:hover {
        background-color: orange;
      }

      .text-input {
        padding: 10px 15px;
        width: 100%;
        outline: none;
        border: solid black 1px;
        background-color: #e0e0e0;
        box-shadow: 0 10px 15px -3px #e0e0e0;

        font-family: system-ui, -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto,
          Oxygen, Ubuntu, Cantarell, 'Open Sans', 'Helvetica Neue', sans-serif;
        font-size: medium;
        font-weight: 400;
        letter-spacing: normal;
        line-height: 25px;
      }

      .text-input:focus {
        border: solid #4caf50 1.5px;
        outline: none;
      }

      .container-input {
        position: relative;
      }

      .form {
        margin-top: 20px;
      }

      .text-model {
        /* color: #4caf50; */
        font-family: system-ui, -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto,
          Oxygen, Ubuntu, Cantarell, 'Open Sans', 'Helvetica Neue', sans-serif;
        font-size: medium;
        font-weight: 400;
        letter-spacing: normal;
        line-height: 25px;
      }
    </style>
  </head>
  <body>
    <div class="container">
      <form id="form" onkeydown="return event.key != 'Enter';" class="form">
        <div class="container-input">
          <input class="text-input" type="text" id="text-input" />
          <button id="submit" class="button">retrieve</button>
        </div>
      </form>
      <div>
        <p id="model-answer" class="text-model"></p>
      </div>
      <div id="list" class="text-model"></div>
    </div>
    <script>
      // get html component for model answer
      const modelAnswer = document.getElementById('model-answer')
      // Get the list container element
      var listContainer = document.getElementById('list')
      // conversation turns
      let messages = []

      const callBedrockStream = async () => {
        // present model answer to frontend
        modelAnswer.innerText = ''

        // clear content before query
        listContainer.innerHTML = ''

        // get user question
        const userQuestion = document.getElementById('text-input').value

        // push user question to messages
        messages.push({
          role: 'user',
          content: [{ type: 'text', text: userQuestion }]
        })

        if (userQuestion) {
          try {
            const response = await fetch(
              '/knowledge-base-retrieve-and-generate',
              {
                method: 'POST',
                headers: {
                  'Content-Type': 'application/json'
                },
                body: JSON.stringify({ messages: messages })
              }
            )

            console.log(response)

            const decoder = new TextDecoder()

            // batch processing
            const json = await response.json()
            console.log(json)

            // update frontend
            modelAnswer.innerText += json['Output']['Text']

            // citations
            citations = json['Citations']

            console.log(citations)

            // update frontend
            for (var i = 0; i < citations.length; i++) {
              // generted part
              var genertedPart =
                citations[i]['GeneratedResponsePart']['TextResponsePart'][
                  'Text'
                ]

              // references
              var retrievedReferences = citations[i]['RetrievedReferences']

              // citation i and generated i
              var listItemC = document.createElement('div')
              listItemC.style.marginBottom = '15px'
              listItemC.style.borderBottom = '1px solid #0000FF'
              listItemC.style.color = 'blue'
              var headerC = document.createElement('h4')
              headerC.textContent = `Citation ${i} Generated Part ${i}`
              var itemTextC = document.createTextNode(genertedPart)
              //
              listItemC.appendChild(headerC)
              listItemC.appendChild(itemTextC)
              listContainer.appendChild(listItemC)

              for (var j = 0; j < retrievedReferences.length; j++) {
                console.log(`citation ${i} and reference ${j}`)

                // reference text
                var refText = retrievedReferences[j]['Content']['Text']
                // reference uri s3
                var refUri =
                  retrievedReferences[j]['Location']['S3Location']['Uri']
                // citation i and reference j
                var listItem = document.createElement('div')
                listItem.style.marginBottom = '15px'
                listItem.style.borderBottom = '1px solid #0000FF'
                var header = document.createElement('h4')
                header.textContent = `Citation ${i} Reference ${j}`
                var itemText = document.createTextNode(refText + refUri)
                // citation i and reference j
                listItem.appendChild(header)
                listItem.appendChild(itemText)
                listContainer.appendChild(listItem)
              }
            }

            // push model answer to converstion turn
            messages.push({
              role: 'assistant',
              content: [{ type: 'text', text: modelAnswer.innerText }]
            })
          } catch (error) {
            console.log(error)
          }
        } else {
          console.log('Please enter question ...')
        }
      }

      document
        .getElementById('submit')
        .addEventListener('click', async event => {
          event.preventDefault()
          await callBedrockStream()
        })

      document
        .getElementById('text-input')
        .addEventListener('keydown', async event => {
          if (event.code === 'Enter') {
            await callBedrockStream()
          }
        })
    </script>
  </body>
</html>

IAM Permissions#

To allow the ECS task to invoke bedrock runtime agent, and retreive AOSS we need to add permissions to task role by below policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream",
        "bedrock:InvokeAgent",
        "bedrock:Retrieve",
        "bedrock:RetrieveAndGenerate"
      ],
      "Resource": [
        "arn:aws:bedrock:us-east-1::foundation-model/*",
        "arn:aws:bedrock:us-west-2::foundation-model/*",
        "arn:aws:bedrock:us-west-2:<ACCOUNT_ID>:knowledge-base/<KNOWLEDGE_BASE_ID>"
      ],
      "Effect": "Allow"
    },
    {
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": ["arn:aws:s3:::<BUCKET_NAME>/*"],
      "Effect": "Allow"
    },
    {
      "Action": "aoss:APIAccessAll",
      "Resource": [
        "arn:aws:aoss:us-east-1:<ACCOUNT_ID>:collection/<AOSS_ID>",
        "arn:aws:aoss:us-west-2:<ACCOUNT_ID>:collection/<AOSS_ID>",
        "arn:aws:aoss:us-west-2:<ACCOUNT_ID>:collection/<AOSS_ID>"
      ],
      "Effect": "Allow"
    }
  ]
}

And we also need to add the task role ARN to data collection of the AOSS collection

UserData#

#!/bin/bash
cd /home/ec2-user/
wget https://go.dev/dl/go1.21.5.linux-amd64.tar.gz
tar -xvf go1.21.5.linux-amd64.tar.gz
export 'PATH=/home/ec2-user/go/bin:$PATH' >> ~/.bashrc

wget https://github.com/cdk-entest/golang-bedrock-demo/archive/refs/heads/main.zip
unzip main
cd golang-bedrock-demo-main
go mod tidy
go run .