Agents CLI: Teaching AI Assistants to Build AI Agents for You

Here's the meta layer we've reached in 2025: I'm using an AI coding assistant (Claude, Copilot, etc.) to write code. That AI assistant can now install skills that teach it how to build other AI agents — scaffold them, evaluate them, deploy them to production, and publish them to an enterprise catalog. I don't learn the platform APIs or CLI commands. The AI learns them for me, and I just describe what I want in plain language.

Google's agents-cli is purpose-built for this workflow. It's a CLI tool and skill library that teaches coding agents (Claude Code, GitHub Copilot, Codex, or any other) how to build, evaluate, and deploy agents on Google's Gemini Enterprise Agent Platform using the ADK (Agent Development Kit) framework. I haven't installed it yet, but the concept is too compelling to ignore: AI assistants teaching AI assistants how to build AI agents. It's turtles all the way down, and I'm bookmarking it to see if the abstraction actually works in practice.

What Agents CLI Actually Does

Agents CLI is both a command-line tool and a collection of "skills" (structured knowledge bundles) that you install into your coding agent. Once installed, your coding agent understands:

• How to scaffold agent projects — creating new ADK agents, adding deployment configs, setting up CI/CD, upgrading existing projects.
• How to write ADK code — the Python API for agents, tools, orchestration, callbacks, state management, and patterns specific to Google's Agent Development Kit.
• How to evaluate agents — generating test datasets, running evaluations, grading with LLM-as-judge, analyzing failure modes, and optimizing prompts based on eval results.
• How to deploy agents — pushing to Google Cloud (Agent Runtime, Cloud Run, GKE), managing secrets, setting up observability (Cloud Trace, logging).
• How to publish agents — registering them in the Gemini Enterprise agent catalog for org-wide discovery and governance.

You don't run most of these commands yourself. You ask your coding agent to do it. For example, you might say: "Use agents-cli to build a support agent that answers questions from our documentation, evaluate it on 20 test cases, and deploy it to Cloud Run." The coding agent — now equipped with the agents-cli skills — knows how to execute that entire workflow.

The Meta Loop: AI Building AI

The traditional workflow for building an AI agent looks like this:

1. I read the ADK documentation.
2. I learn the agent framework API (how to define tools, orchestration, state).
3. I write Python code manually or with Copilot autocomplete.
4. I read the deployment docs for Cloud Run or GKE.
5. I write Dockerfiles, CI/CD configs, and infrastructure-as-code.
6. I set up evaluation pipelines, learn the eval CLI, write test cases.
7. I deploy, monitor, debug, iterate.

That's weeks of ramp-up time, even for experienced developers. The agents-cli approach inverts this:

1. I install agents-cli skills into my coding agent (one command: uvx google-agents-cli setup).
2. I describe what I want in plain language: "Build a customer support agent that reads from our FAQ Markdown files and answers questions."
3. The coding agent uses the skills to scaffold the project, write the ADK code, generate eval cases, run evals, and deploy to Google Cloud.
4. I review the output, refine the prompt, and iterate.

I never read the ADK docs. I never write Dockerfiles. I never manually configure Cloud Run. The coding agent — now an expert in Google's agent platform because it absorbed the skills — does that work. I operate at the intent level ("build this agent"), not the implementation level ("import this module, call this API").

It's a higher abstraction layer. Whether it actually saves time or just adds indirection is the open question I'm testing by bookmarking this.

The Skills: What the AI Learns

Agents CLI packages its knowledge as seven "skills" that coding agents can consume:

• Workflow: The development lifecycle for ADK agents — when to scaffold, when to eval, when to deploy, and rules for preserving user code during upgrades.
• ADK Code: The Python API patterns for building agents with ADK — defining tools, orchestration strategies, callbacks, state management, error handling.
• Scaffold: How to create new projects, enhance existing ones (add deployment, CI/CD, RAG), and upgrade to newer agents-cli versions.
• Eval: Evaluation methodology — generating synthetic multi-turn test cases, running agents on datasets, grading with LLM-as-judge, analyzing failure clusters, auto-tuning prompts.
• Deploy: Deployment to Google Cloud — Agent Runtime, Cloud Run, GKE, secrets management, CI/CD pipelines, staging/prod environments.
• Publish: Registering agents in Gemini Enterprise for org-wide discoverability and governance.
• Observability: Setting up Cloud Trace, logging, and third-party observability integrations.

These aren't just CLI command references. They're context documents that teach the coding agent how to think about agent development — what questions to ask, what tradeoffs to consider, what best practices to follow. It's like giving the AI a crash course in being a Google Cloud agent platform engineer.

The Commands: What Gets Automated

Here are the core CLI commands that the coding agent can now run on your behalf:

# Scaffold a new agent project agents-cli scaffold my-support-agent # Run the agent locally with a test prompt agents-cli run "What's our refund policy?" # Generate evaluation dataset (synthetic multi-turn conversations) agents-cli eval dataset synthesize # Run agent on eval cases and produce traces agents-cli eval generate # Grade the traces against quality metrics agents-cli eval grade # Auto-tune agent prompts based on eval failures agents-cli eval optimize # Deploy to Google Cloud agents-cli deploy # Publish to Gemini Enterprise catalog agents-cli publish gemini-enterprise

The coding agent strings these together based on your high-level request. You say "build and deploy a billing FAQ agent," and the agent internally decides: scaffold → write ADK code → generate evals → run evals → grade → optimize → deploy. You review at checkpoints, but you're not manually typing each command.

The Use Cases I'm Planning to Test

Here's what I want to try agents-cli for, once I have time to set it up:

• Rapid prototyping of task-specific agents — Ask Claude Code to "build an agent that summarizes Slack threads and posts the summary back to the channel." Let it scaffold the project, write the ADK code, and deploy to Cloud Run. See if the result is good enough to iterate on or if I still need to rewrite most of it manually.
• Evaluation-driven development — Build an agent, generate 50 synthetic test cases, run evals, see where it fails, ask the coding agent to "fix the top 3 failure modes," re-eval, iterate. Test whether the eval loop actually improves agent quality faster than manual testing.
• Multi-agent systems — Scaffold several specialized agents (one for docs lookup, one for code search, one for task routing), deploy them, and orchestrate them with a meta-agent. See if agents-cli makes this easier or if it's still too complex even with AI assistance.
• Enterprise agent catalog — Build a few internal-use agents (HR FAQ bot, eng onboarding bot, release notes summarizer), publish them to Gemini Enterprise, and see if discoverability and governance actually work at the org level.

The common thread: tasks where I know what I want but don't want to spend days learning how to build it on Google's platform. If agents-cli collapses "idea to deployed agent" from weeks to hours, it's transformative. If it just adds a layer of prompting on top of the same manual work, it's not worth the overhead.

How It Compares to Other AI Agent Tools

I've bookmarked or used Aider for AI pair programming, Continue for IDE-based AI assistance, and various agent frameworks (LangChain, AutoGPT, etc.). Here's how agents-cli differs:

• vs. LangChain / LlamaIndex: Those are agent frameworks you code against. Agents-cli is a meta-tool that teaches coding agents how to use ADK (Google's agent framework). You're not writing framework code yourself; the AI is.
• vs. Aider / Continue: Those are general-purpose coding assistants. Agents-cli is domain-specific: it only knows about building agents on Google Cloud. It's deeper expertise in a narrower area.
• vs. AutoGPT / AgentGPT: Those are autonomous agent runners. Agents-cli is a development toolkit for building, evaluating, and deploying your own agents, not running pre-built ones.
• vs. using ADK directly: You could learn ADK yourself and write agents manually. Agents-cli lets you offload that learning to your coding agent. The tradeoff is control vs. speed: you give up low-level control but (theoretically) ship faster.

The closest analogy: agents-cli is to ADK what ReviewDog is to linters. ReviewDog doesn't replace linters; it makes them easier to use in CI. Agents-cli doesn't replace ADK; it makes ADK easier to use via AI assistance.

Potential Rough Edges (From the Docs and GitHub Issues)

I haven't tested this yet, but based on the README and issue tracker, here are the likely friction points:

• Google Cloud dependency: For deployment, you need a Google Cloud project. Local dev works with just an AI Studio API key, but the full workflow (eval, deploy, publish, observability) requires GCP. That's infrastructure overhead and cost.
• Preview/Pre-GA status: The README explicitly says this is a preview feature subject to limited support and potential breaking changes. Production use is risky until it hits GA.
• Coding agent quality variance: The skills teach the coding agent what to do, but execution quality depends on the underlying model (Claude, GPT-4, etc.). A weaker model might misinterpret the skills or generate broken code even with perfect instructions.
• Trust and verification: When the coding agent scaffolds a project, writes ADK code, configures deployment, and sets up CI/CD, I still need to review every file to ensure it's correct and secure. If I don't understand ADK or Google Cloud well enough to review, I'm blindly trusting AI output — that's dangerous for production systems.
• Skill bloat: Installing multiple skill libraries (agents-cli, agent-skills, google/skills, etc.) into one coding agent might cause conflicts or confusion. The agent needs to know when to use which skill set, and that context management isn't always reliable.

Why This Bookmark Is High-Priority (Despite the Risks)

Two months ago, I wanted to build a simple agent that reads our internal docs (Markdown files in a Git repo) and answers questions. I spent four hours reading LangChain docs, trying different vectorstore setups, debugging embedding issues, and finally got a prototype working locally. Deploying it to production would have been another day of work (Dockerize it, set up Cloud Run, configure auth, add monitoring).

If agents-cli works as advertised, that entire workflow — from "I want a docs Q&A agent" to "it's deployed and monitored on Cloud Run" — could be a 30-minute conversation with Claude Code. I describe what I want, the agent scaffolds it, I review the code, the agent runs evals, I verify quality, the agent deploys it. The four-hour research tax and day-long deployment work collapse into review and verification tasks.

That's the promise. The risk is that I trade "learning ADK and Google Cloud myself" for "blindly trusting an AI that learned it from skill documents." If the AI makes mistakes I can't catch because I never learned the underlying platform, I'm in a worse position than if I'd just read the docs myself.

The right mental model is probably: use agents-cli to go from zero to working prototype in hours, then learn the underlying platform to go from prototype to production-ready. Let the AI handle the scaffolding and boilerplate. I handle the review, refinement, and hardening. That division of labor makes sense. Letting the AI do everything end-to-end without human expertise in the loop is probably too risky for anything mission-critical.

The Test Plan

I'm planning to test agents-cli on a low-stakes internal tool first:

1. Install: Run uvx google-agents-cli setup to install the CLI and push skills to Claude Code (or whichever coding agent I'm using that week).
2. Scaffold: Ask the agent to build a simple FAQ agent that reads from a local Markdown file and answers questions.
3. Review: Examine the generated ADK code, project structure, and dependencies. Verify nothing is broken, insecure, or nonsensical.
4. Eval: Ask the agent to generate 20 synthetic test cases, run evals, and show me the results. Check if the eval methodology makes sense or if it's just theater.
5. Iterate: If evals show failures, ask the agent to fix them. See if it can actually improve the agent based on eval feedback or if I need to manually debug.
6. Deploy (local): Run the agent locally, test it interactively, verify it works as expected.
7. Deploy (Cloud): Ask the agent to deploy to Cloud Run. Review the generated Dockerfile, deployment config, and infrastructure code before applying it.
8. Monitor: Check if observability (Cloud Trace, logs) is set up correctly and actually useful for debugging.

If the output at each step is high-quality and requires only minor tweaks, agents-cli moves from "Bookmarked" to "Using" and I start building more complex agents with it. If the output is frequently broken, insecure, or requires extensive rewriting, I fall back to learning ADK directly and using agents-cli only for boilerplate scaffolding.

Pairing It With the Rest of the AI Toolchain

Agents CLI fits into a broader AI-assisted development workflow:

• Agents CLI: Teaches coding agents how to build, eval, and deploy other AI agents on Google Cloud.
• Aider: Edits code files with AI assistance and manual commit control. Good for refactoring agents-cli-generated code.
• Cody: Codebase-aware AI assistant for exploring and understanding the ADK code that agents-cli generates.
• ReviewDog + ESLint: Lints the Python/TypeScript code that agents-cli scaffolds, catching style and safety issues before deployment.
• PR-Agent: Reviews the logic and architecture of agents-cli-generated code before you deploy it to production.

The stack would be: agents-cli generates the agent code → I review it with Cody → Aider helps me refactor → ReviewDog lints it in CI → PR-Agent reviews the PR → I deploy. Each tool operates at a different layer of abstraction, and together they cover the full agent development lifecycle.

The Bigger Question: Is This the Future?

Agents CLI represents a possible future where developers don't learn platforms directly — they teach AI assistants to learn platforms, then work at the intent level. Instead of reading Google Cloud docs, I read what the AI built and verify it's correct. Instead of writing Dockerfiles, I review the Dockerfile the AI generated. Instead of configuring CI/CD, I approve the pipeline the AI scaffolded.

That future is compelling if the AI is reliable and I can verify its output without deep platform expertise. It's dangerous if the AI makes subtle mistakes I can't catch, or if "reviewing AI output" takes longer than "doing it myself." The only way to know is to test it on real projects and measure: does agents-cli make me faster, or does it just add a layer of prompting on top of the same work?

I'm bookmarking it to find out. If it works, it's transformative — the abstraction layer we've been waiting for. If it doesn't, it's an interesting experiment that highlights the limits of current AI coding assistance. Either way, it's worth testing. The meta is too strong to ignore.