Building an AWS-Native Engineering Assistant with Amazon Bedrock AgentCore

Building an AWS-Native Engineering Assistant with Amazon Bedrock AgentCore

Generative AI demonstrations are easy to build. A useful internal assistant is harder.

The difference is not the chat box. It is everything around it: deciding what the model is allowed to know, keeping private documents current, showing where an answer came from, authenticating users, observing failures and deploying the whole system consistently.

At Opsmorph, we built a proof of concept to explore how much of that work can now be handled with AWS-native agent tooling. The result is an engineering knowledge assistant built with Amazon Bedrock AgentCore, Strands Agents, Amazon Bedrock Knowledge Bases and S3 Vectors, with the surrounding infrastructure defined in Terraform.

The assistant answers questions from a private set of engineering documents. For grounded answers, it returns the S3 sources it used. If the available evidence is insufficient, it is instructed to say so rather than fill the gap with a plausible-sounding answer.

This was deliberately a proof of concept, not a claim that one architecture fits every AI application. Our goal was to understand where a managed AWS approach removes engineering work, where custom code is still necessary and how it compares with assembling the same system from components such as FastAPI, Lambda, S3 and LanceDB.

What We Built

The request path is intentionally small. A user signs in through Amazon Cognito and opens a static Next.js interface hosted by AWS Amplify. API Gateway validates the Cognito access token, while a thin FastAPI service on Lambda signs the AgentCore request and streams the response back to the browser.

Inside AgentCore Runtime, a Strands agent uses Amazon Nova 2 Lite and has one purpose-built tool: search the engineering knowledge base. That tool calls the Bedrock Knowledge Bases Retrieve API and returns relevant passages together with their source URIs. Strands remains responsible for orchestration and answer generation; the knowledge base is responsible for retrieval.

The knowledge path is separate from the application path:

1. Private documents are stored in an encrypted, versioned S3 bucket.
2. Bedrock splits them into chunks and creates embeddings with Amazon Titan Text Embeddings V2.
3. The vectors are stored in an S3 Vectors index.
4. Upload and delete events flow through EventBridge and SQS to a small synchronisation Lambda.
5. That function calls the Bedrock direct-ingestion APIs so document changes become searchable without a Terraform deployment.

A scheduled reconciliation job performs a full incremental sync as a safety net in case an event is ever lost.

Why AgentCore Was Interesting

The main attraction of AgentCore Runtime is that it is designed around the execution characteristics of agents rather than treating an agent as an ordinary short-lived HTTP handler.

We still own the agent code and package it as a container, so there is no need to hide the application behind a fully managed abstraction. In return, the runtime handles concerns such as scaling, invocation, streaming and session-level isolation. AgentCore is also framework- and model-flexible. This POC uses Strands Agents, an open-source SDK maintained by AWS, together with Amazon Nova.

That felt like a useful middle ground. We kept control of the orchestration loop, retrieval tool and system prompt, while avoiding the need to design a bespoke runtime platform before we could test the product idea.

There were also constraints to learn. The runtime image must follow AgentCore's container contract, authentication still needs to be designed at the application boundary, and managed infrastructure does not remove the need for careful IAM. We used separate least-privilege roles for the runtime, the knowledge base and the ingestion function so that no component can silently take on another component's responsibilities.

Why Bedrock Knowledge Bases and S3 Vectors

A RAG system needs more than a vector index. It needs document parsing, chunking, embeddings, ingestion, retrieval metadata and a reliable way to keep the index aligned with its source documents.

Bedrock Knowledge Bases gave us those pieces as a managed pipeline. We selected S3 Vectors as the backing store because this is a sporadically used sandbox application. It keeps the data inside an AWS-native, consumption-oriented architecture without introducing continuously running search capacity.

We also chose Retrieve rather than RetrieveAndGenerate. This separation was valuable: Bedrock finds evidence, while the Strands agent decides when to search and how to construct the final answer. It makes the boundary between retrieval and reasoning visible and gives us direct access to the source metadata used for citations.

The managed path is not configuration-free. Chunk size, overlap, embedding dimensions and vector-index metadata all affect whether ingestion succeeds and whether retrieval is useful. Managed services move the abstraction boundary; they do not eliminate the need to understand what is happening underneath it.

Keeping Knowledge Current

One of the most useful lessons came from document ingestion. Uploading a file to an S3 data source does not, by itself, make the change searchable. Something still needs to initiate ingestion.

For this POC, S3 publishes object-created and object-deleted events to EventBridge. SQS buffers them, and Lambda uses the Bedrock direct-ingestion APIs to update the knowledge base.

The Lambda treats each event as a hint, not an instruction. Before acting, it checks the current state of the S3 object. That matters because a standard SQS queue can deliver duplicates or deliver a create and delete event out of order. Looking at the current source of truth makes repeated and reordered events converge on the correct final state.

Failed messages are retried and then moved to a dead-letter queue. CloudWatch alarms cover a non-empty dead-letter queue, an ageing ingestion backlog, agent failures and repeated empty retrievals. This is more operational machinery than a quick demo requires, but it is exactly the machinery that determines whether an internal assistant remains trustworthy after the demo.

Testing Answers, Not Only Infrastructure

Terraform can confirm that a knowledge base and runtime exist. It cannot confirm that the assistant gives a grounded answer.

We added a small behavioural evaluation suite with two main classes of test:

• Grounded questions must cite at least one source, every cited S3 URI must exist, the citation must point to an expected document and the answer must contain the essential facts.
• Questions outside the available knowledge must produce a clear refusal and must not invent company policies, credentials or other plausible internal details.

Checking that a cited object actually exists is particularly important. A fabricated citation can look reassuring while making the answer less trustworthy, not more. The evaluation runner exits with a failure code, so these cases can become a release gate rather than a one-off manual check.

How This Compares with Lambda, FastAPI, S3 and LanceDB

We could build the same product with a more composable stack: FastAPI for the service, Lambda for compute, S3 for source documents and LanceDB for vector search. In fact, this POC still uses FastAPI and Lambda for the thin browser-facing API. The choice is not “managed services or code”; it is where custom ownership provides value.

What we liked about the AgentCore and Knowledge Bases approach was the speed at which we could reach a credible end-to-end architecture. Runtime isolation, managed ingestion primitives and native IAM integration removed several categories of platform work. For an organisation already operating on AWS, that can be a meaningful advantage.

What we like about the composable approach is transparency and control. FastAPI is easy to understand and test. Lambda is a good fit for short-lived APIs and event handlers, and the AWS Lambda Web Adapter can run familiar HTTP applications with response streaming. S3 is an excellent durable source of truth. LanceDB offers direct control of the vector data layer and can use object storage, which is attractive when custom retrieval logic, portability or local development matters more than a fully managed Bedrock workflow.

The trade-off is ownership. With the composable stack, the team must design and operate more of the ingestion lifecycle, embedding pipeline, index consistency, concurrency behaviour, citation mapping and observability. That extra ownership can be exactly what a specialised product needs, but it is not free.

Our conclusion is not that one stack wins. For a focused internal assistant on AWS, AgentCore and Bedrock Knowledge Bases provide a strong route from experiment to operational POC. For a product whose differentiation lives in retrieval, data processing or cross-cloud portability, a custom FastAPI and LanceDB architecture may be the better engineering investment.

What We Would Explore Next

The current system is intentionally stateless at the application level. A production iteration could add durable conversational memory, richer access controls for different document collections and retrieval filters based on user identity. We would also expand the evaluation set using real user questions, measure retrieval quality separately from answer quality and test how alternative embedding and generation models affect the results.

The security model would need the same treatment. The POC uses private encrypted storage, least-privilege IAM roles, Cognito authentication and a non-root runtime container, but production readiness depends on the sensitivity of the documents, retention requirements, network boundaries and organisational controls around access and audit.

Most importantly, we would keep the architecture replaceable. The agent talks to retrieval through a small tool boundary, the model identifier is configuration and documents remain in S3 as the source of truth. Those choices make it possible to test a managed implementation today without making every future decision irreversible.

Final Thoughts

This POC showed that AWS's newer agent tooling can remove a meaningful amount of undifferentiated platform work while leaving the core application logic in our hands.

The most valuable outcome was not the chat interface. It was a deployable reference architecture that connects private knowledge, grounded answers, automated ingestion, identity, observability and behavioural evaluation. It also gave us a clearer decision framework: use managed agent services where they accelerate delivery and operational confidence, and choose custom components where control is part of the product's value.

That is a much more useful starting point for a real engineering assistant than a demo that only knows how to answer a prompt.