Manifesto

Artificial Intelligence should let more people be able to do more things.

We believe agentic workflows have the potential to augment people’s abilities, taking time off the mundane to free up time for more important tasks.

One of the biggest problems with running agentic workflows is the creep up of extra technologies as you scale up to production. Initially, a framework for your application, and then an infrastructure manager. And then you would need something to observe all the calls being made. All of these steps make what's supposed to be a simple process into a multi-week slog.

At the end, you also had to verify that your agent is properly managed, unable to burn through your entire limit in an hour, or to access confidential information without permission.

95% of agentic workflows never reached production stated by MIT’s NANDA initiative. The problem isn't the model intelligence though, but brittle workflows, poor integration, limited contextual learning, and systems that couldn't adapt to feedback.

New products and frameworks have been getting released constantly for this, helping you out with some aspects of agentic development, but also increasing the complexity and lock-in of your application.

We would rather allow you to build and deploy fast, to be able to see the results of your workflow immediately and begin to iterate, rather than being stuck in integration purgatory.

We approached CanyonOS from three key principles:

• Simplicity. Developing workflows should be the same as writing in Python, no new framework needed. If you want to add your own MCPs or tools in it should be seamless, not requiring patching code.
• Modularity. Components should be separately managed and scaled based on SLAs and such.
• Completely Agnostic. You shouldn’t be locked into a service that can become outdated.

CanyonOS wants you to write your workflow in plain Python, no need for any external agent framework, and we would be able to handle all the underlying deployment infrastructure, allowing you to spend your time iterating and improving workflows.

With CanyonOS, each agent runs in its own container with a small controller beside it, and one central controller managing the whole workflow. These controllers manage all the underlying distributed systems (scaling, routing, replica placement) for you. Agents call each other over gRPC, and Redis holds the shared state: work queues, results, and the routing table. You run canyonos deploy, and your workflow comes up as an API you can call.

The usual answers are Kubernetes for infrastructure and Ray for distributed compute.

Why not AWS AgentCore?

How about LangGraph Platform? LangGraph Platform is a really strong option if your whole workflow is built on LangGraph, but using that locks you into the LangChain ecosystem.

Why not Kubernetes or Ray? Kubernetes and Ray are two of the absolute best tools for running distributed compute, but neither was built for agentic workloads, and have gaps as a result.

Kubernetes is excellent at running containers, and has the most configurability out there. But it's also meant purely for infrastructure; it has no idea what a workflow is. Out of the box, it scales on CPU and memory, which tells you almost nothing about an agent that spends most of its time waiting on an LLM.

To run a real agent workflow on Kubernetes, teams end up assembling a stack: Kubernetes for orchestration, one tool for LLM tracing, another for metrics, often Ray on top for task execution. Kubernetes may be the most battle-tested option on the market, but if you want quick deployment, effortless management, and a single dependency, CanyonOS replaces that stack.

One of the biggest challenges for using Ray is the massive learning curve it takes to serve your workflow on it, let alone optimize it. But for a fast-moving team, the time needed to set up a cluster and manage it could outweigh the benefits.

CanyonOS is built specifically for long-running agentic workflows. It understands the shape of your workflow and sees every trace, metric, and LLM call it makes. Ray can run dynamic tasks, but it sees each one as a generic function call. It doesn't know which call was an LLM request, what it cost, or how long it sat in a queue.

That is our foundation for scaling and policy decisions, something a general-purpose engine simply can't make.

Both Ray and K8s have steep learning curves for massive controllability, but the knobs they control and tune are not what is needed. CanyonOS runs on your laptop or on EC2 with the same three commands.

When you deploy your workflow, all observability is automatically there, nothing else.

The decorator approach. The biggest LLM observability tools, like Langfuse and Braintrust, rely on decorators: you mark every function you want traced. It's quick to set up, but it has two costs:

• It misses key telemetry. A decorator only sees inside your function. It can't tell you how long a request waited before an agent picked it up, or what the machine running it cost.
• Code has to change. You will have to add decorators to every function you want to trace, with forgetting to mark a function resulting in a dark spot in observability and every one you mark tying your code to that tool.

What about zero-code OpenTelemetry tracing? Zero-code tracing patches common libraries so outgoing requests, like HTTP and LLM calls, are traced automatically. That's useful, but it only sees the requests, not your workflow: which agent handled each step, which step called which, or how long work sat waiting.

How it works with us. CanyonOS doesn't need any of that, because it's already in the path. Every call from one agent to another is managed by us, and every LLM call goes through our gateway. Recording each step is a side effect of running the workflow.

Each agent call becomes a span with its parent call, how long it sat in the queue, and how long it ran. When the call reaches an LLM, the gateway adds the model, token counts, and cost to that same span.

What you see:

• Traces, metrics, and logs. Machine metrics, agent metrics, and your agents' logs are all collected, and each log is connected to the agent that produced it.
• True asynchronous analysis. Async workflows have been notoriously hard to debug, with work fanning out in parallel, and results returning at any time later, with it being hard to tell which branch held everything up. We built our entire platform on the basis of everything being async and managed that way. There is no need to define async functions or code, as everything has been designed to handle it.
• All errors tracked. Each error is recorded with the inputs and outputs of the step that failed.
• Full costs. The cost of each LLM call and the server cost of each request are calculated, using current token prices and EC2 hourly rates.

We are fully compatible with OpenTelemetry. Everything is recorded in OpenTelemetry. View it in our dashboard or in the monitoring tools you already use. Any OTLP receiver works over HTTP or gRPC, and a batch is retried until every destination has accepted it.

Configuration. All of your configuration lives in one folder, and that would be the only folder you have to touch after deploying to manage it. We offer the ability to change the config via CLI, dashboard, or even through the file itself. It lists your agents, how many copies of each to run, what resources they need, and where they run. The folder is modular, so you can add pieces like scaling.yaml and policy.yaml for more functionality only as you need them.

We value speed and simplicity, so to help you get started fast, we added a skill to help jumpstart the configuration file process too. canyonos build downloads our skill file and gives your coding agent (Claude Code or Codex) everything it needs to draft this setup for you. It can help you onboard and port your workflow, and you can read and edit everything it writes.

Scaling. Today, CanyonOS scales agents based on queue length, but unlike other autoscalers, our entire platform is asynchronous with bounded concurrency on each agent. With that base, all queue backups would always happen at the correctly bottlenecked agent, leading to our simple autoscaling policy scaling far better than the same autoscaling policy on K8s.

What also sets it apart is what it can see. Because CanyonOS runs your whole workflow, it collects every trace, metric, and log in one place. All of this information gets fed into our control plane, which makes it possible to scale based on what's actually slow in your workflow, and eventually to get ready for the next step before the work arrives.

Policy.

Every LLM call your agents make goes through CanyonOS's built-in LLM gateway, which records token usage and cost for each request. We support all OpenAI-compatible endpoints, allowing you to plug in and use any provider without changing a single line of code.

You can also set rules, for which parts of your workflow an agent is allowed to access. A rule matches the request's context, like origin: admin, and lists the agents it may call. On that foundation, we're building spending limits and user/agent authentication.

CanyonOS is new and still growing. Below are some of our current limitations:

• It isn't battle-tested yet. We haven't run it at large production scale.
• Only supports Docker. Everything runs in containers, with no current options to use Podman or microVMs.
• It runs locally, on AWS EC2 and on Kubernetes today. Serverless, Azure, and GCP are planned for the future, but not supported. The goal is for you to pick any amount of providers and use whatever, no need to be locked in to just one.
• There are sharp edges. A few rules about how projects must be structured will go away as we smooth them out. For example, agents can't run Docker commands themselves, and databases can't scale.