Hey Amara is an agent platform for recruitment agencies: recruiters talk to it in plain language, and it reads and writes their applicant tracking system on their behalf. I was the founding engineering hire on the backend underneath that — the queue it runs on, the tenant boundary around it, and the memory that lets a conversation outlive its own context window.
- 2Memory tiers behind a conversationIn-thread compaction for the live conversation, and a separate long-term memory service reached over an internal API. · as of 1 Jul 2026
- 3Router stages before a model is calledA rule-based pattern match that calls no model, then a fast classifier, then a router model — in that order, so the cheapest stage answers first. · as of 1 Jul 2026
The problem
A recruiter's question does not fit inside one request and one response. A single answer can mean a research agent reading a company for a minute, a write to the applicant tracking system that a human has to approve first, and a document generated and uploaded before anything can be shown — all while a browser tab waits. Wiring agents to each other over synchronous HTTP made every one of those a place where a slow call holds its caller open and a failure travels back upstream.
Everything said in those conversations has to be usable next month and has to stay inside one agency, and those two constraints pull apart. A thread outgrows the context window, so it has to be compacted without losing what the agent was in the middle of doing, and every query, cache read and retrieval has to carry a tenant identifier so the boundary holds even when the calling code asks for the wrong thing.
The system
A queued agent backend with two tiers of memory.
What it is made of
- A gateway that authenticates the caller and validates the tenant, then hands the request to the orchestrator as a queued message instead of a blocking call.
- A three-layer router — a rule-based pattern match with no model call, then a fast classifier, then a router model — that picks an agent, which assembles its context, runs a bounded tool loop and streams back to the client.
- Two tiers of memory: in-thread compaction for the live conversation, and a separate long-term memory service reached over an internal API that returns hybrid vector and graph retrieval, gated on consent.
- Queue plumbing under all of it: durable topic exchanges per agent category, dead-letter queues with bounded retries and backoff, circuit breakers around failing consumers, and an explicit degraded mode when a dependency is unreachable.
My role
I was the founding engineering hire and built the backend and AI infrastructure. My own work was the migration of the message-queuing layer from Redis to RabbitMQ; the authentication middleware and the hardening pass that followed it; the conversation compaction system; the email generation pipeline, with four retrieval sources feeding a composition service; the streaming research agent; the human-in-the-loop confirmation cards for actions that write to the applicant tracking system; and the integration test suite. The long-term memory service and the routing broker were principally other engineers' work — I built against them from the consumer side, and it matters to say which is which.
The decisions
RabbitMQ instead of Redis for the agent work queue. I moved the message-queuing layer off Redis and rewrote the queue service around a broker, along with the circuit breaker, the gateway and the service launcher, so a chat request is acknowledged at the edge and handed to the orchestrator as a durable message. Staying on Redis lists already worked and cost nothing new to operate. What the move bought was load levelling, dead-lettering and per-queue back-pressure, and one slow agent no longer holding its caller open. The price is a broker to run and at-least-once delivery, which pushes idempotency and cancellation into the application rather than removing them.
Compaction that preserves a schema, not a paragraph. When a thread crosses the token threshold, the compactor summarises earlier turns into a validated structure — task state, personalisation, technical context, artifact references — and rewrites the conversation behind a lock with crash recovery, instead of producing prose. It replaced a free-text summarisation prompt, and the cheaper option was simply dropping the oldest turns once the window filled. Structured state survives the compaction boundary, so an agent in the middle of a task still knows which artifact it was working on. In exchange the schema became an interface: it has to be versioned, and a malformed model response has to be caught by validation instead of absorbed silently.
The outcome
The platform runs in production, with the request path, the auth and tenant middleware, the compaction system, the email pipeline and the research agent all in it. The queue migration is the change that shaped everything after it. On scale I can honestly describe the shape and not a number: the bounded pools, the iteration caps and the back-pressure exist because coupling agents synchronously had already failed once, not because a benchmark asked for them.
What is unresolved
Performance was asserted rather than measured. What existed were local benchmarks against a single test user and design targets phrased as what the system is optimised for, so nobody could tell you what a p99 costs under real load. Test coverage was thin, and several modules had grown past the size anyone would defend. At-least-once delivery is the other open edge: it moves idempotency into every consumer, and a consumer that is not idempotent fails in a way the queue cannot see.