From Prototype to Production With AWS AgentCore

This post was co-written with Staff Software Engineer Martin Holman.

"Hello world, this is your agent speaking!"

The agent loop! The LLM is calling tools, the answers are sensible, and the sky's the limit. Now, as you look forward to production, you look for a composable toolset, something that can grow with your use case and system needs. That's what we created with Honeycomb Canvas: a collaborative investigation space where AI agents help you understand, fix, and learn about your system.

Canvas runs on AWS Bedrock AgentCore. We picked AgentCore because it supports the ramp described in this post: it’s quick to get running, it’s modular, and it has production-grade underpinnings. But, like any platform, it has some traps to be aware of. In this post, we share some of the most important things we learned while building on AWS AgentCore.

A quick primer on AWS AgentCore

AWS AgentCore is a runtime platform for agentic applications. The core primitive is a Runtime, which is basically a bag of configuration pointing to the code (container or zip file) that runs your agent. The contract is straightforward. You write an HTTP server that exposes an /invocations endpoint, and invoking the runtime proxies the request to this endpoint. Your code invokes your agent and streams the response. Behind the scenes, AWS AgentCore provisions a unique, isolated compute environment per session. Your code runs in its own process with its own filesystem, memory, and network.

Into this process, you can layer an assortment of capabilities and features, provided out of the box by AWS AgentCore. But the toolkit doesn't make the design decisions for you. There are fundamental choices to make as you wire together your system. To create the best experience possible, you need to think through a few practical requirements for your agent: how much and what kinds of context you will need, how long you anticipate sessions lasting, and how to maintain continuity across deployments. Let’s walk through some of the design decisions we made when building Honeycomb Canvas.

Building a memory palace

For Canvas, an investigation can last minutes, hours, or days. The agents in that investigation can't forget what they were doing. They need accurate, in-the-moment context, and they must pick up where they left off when the user does. The decisions about where various kinds of state should live ultimately emerged as a standard data modeling problem, and that's how we recommend you treat them too: a standard data modeling problem. In our case, we found we needed to distribute state across three buckets in a multi-layer strategy to meet our system's needs. AgentCore helped us with two of them.

• Per-invocation context - Current UI state, active selections, real-time status. Anything that changes faster than the agent's turn cycle and can be efficiently/cheaply read from an authoritative source is reasonable for context injection. We built this ourselves.
• Runtime/environment-scoped state - Cross-agent coordination, in-progress work tracking, peer awareness, and other information extrinsic to an individual agent with value tied to momentary conditions may be a good fit for runtime-scoped state. AWS AgentCore's isolated execution environment is the perfect setup for this.
• Durable, customer-scoped state through Agentic Memory - User preferences, snippets of team-specific domain knowledge, and learned patterns that apply across sessions are a good fit for durable, customer-scoped state. AWS AgentCore's built-in Agent Memory feature—especially long-term memories—is a relatively turnkey solution.

"What goes where" has always been hard to change after the fact, and agentic systems are no different. AWS AgentCore's Runtime session primitive and memory features provide solid building blocks. Spending some time laying this out up front will pay dividends down the line. Deciding what gets stored at which layer should be based on factors such as how long it's needed, who needs access when, and how frequently it changes. If this sounds oddly familiar, you're not wrong: these are the same principles we've been using to design data access in applications for years. We'll explore the consequences of session state in particular in part 2 of this series.

Deployments and session continuity

Agents accumulate valuable context over the life of a conversation. We encountered some surprising deployment and CI gotchas in AWS AgentCore that made it difficult to retain the context and achieve the continuity we desired. This section is about the challenge we faced with AgentCore in making the agent's state outlive its infrastructure.

Deployments on AWS AgentCore

AWS AgentCore has three primitives for version management: Runtimes (the agent's configuration), Runtime versions (specific builds), and endpoints (named aliases that route to a version). The standard deployment approach is to create a new Runtime version and update an endpoint to address it. Endpoints are always available during the update process, so an application automatically transitions from old to new code with no downtime.

This model works well generally, but it has a scary property: when an endpoint invocation for an existing session is routed to a new Runtime version, it goes to a brand-new execution environment. And we deploy a lot. Since Canvas's active coordination state lives in memory, we were losing history on every deploy. We solved this by inverting the deployment model with "client chooses the version" semantics. We create a new endpoint for every version. On first invocation for an investigation, an internal gateway service records the endpoint it routed through. Then we route subsequent requests to the same endpoint. This pins active investigations to their current version while new investigations always start at latest.

With deployments handled, we turned to longer-term continuity. Since AWS AgentCore's Runtime session execution environment stays alive a maximum of 8 hours, we needed a solution for session persistence.

Owning session persistence with S3 Files

AWS AgentCore has a managed session storage feature. But it has the same limitation we encountered earlier: the session storage is tied to the Runtime version; it doesn't survive version updates. So we looked elsewhere for persistence and found it with S3 Files. S3 Files enable a fully POSIX-compliant file system mount backed by S3 storage, and AWS AgentCore recently announced support for it.

When an investigation continues in a new Runtime session environment, we mount the session history directly from the S3 Files system. This allows us to continue sessions regardless of Runtime version or age. Of course, the trade-off for control is increased complexity. Taking this approach means managing the storage layer. For us that trade-off was necessary to achieve the experience we wanted.

Observe the agent and the application

Agents are the least predictable component we've ever added to software. Surprising no one, we found we needed to observe our agent's behavior with greater rigor than we'd ever applied to another production service.

For Canvas, we rely on Honeycomb's LLM observability stack to measure a variety of performance dimensions. We track tool call durations, rejections, content truncation events (are we losing data to size limits?), and token spend per invocation. We run online and offline evaluations on agent responses. Observing agents is a deep topic, so we won't treat it superficially here. Our practical advice is to start instrumenting your agent from day one.

What's next

These questions of relationship cardinality, session lifecycle, state layering, deployment and operational limits, and observability form the foundation of a production agentic application. In part 2, we'll go deeper on one of these topics: how we designed Canvas to enable both directed and cooperative agentic investigation. This is the architecture that makes Canvas collaborative, built on the AWS AgentCore foundation we just described.