In the current landscape of AI engineering, multi-agent frameworks like CrewAI and AutoGen are highly celebrated. The promise is tempting: take a complex process like reviewing a pull request, decompose it into specialized roles (Security, Architecture, Code Quality, Documentation), assign each to an autonomous agent, and let a Lead Reviewer consolidate their findings. On paper, this mimics a senior human engineering review panel. It guarantees depth, isolation of concerns, and structured feedback.
The reality is messier. The moment we moved our CrewAI-based GitLab agentic code reviewer from a proof of concept into anything resembling production testing, we ran straight into LLM rate limits that stopped our entire test suite in its tracks. This post traces that journey: the initial five-agent architecture, the daily token quota failure that broke our pipeline, the architectural pivot that fixed it, and the production engineering (webhooks, background tasks, and CI mocking) that made the whole system reliable.
Here’s what this article covers:
- Why a naive multi-agent design burns through tokens far faster than most tutorials admit
- The exact error and math behind a LLM Tokens Per Day (TPD) failure
- How we collapsed five LLM calls into one without losing review quality
- How the GitLab webhook and background task architecture fits together
- How we kept our test suite fast and immune to rate limits and network failures
- A deep-dive Q&A on what actually makes a system “agentic”
The Promise and Pitfalls of Multi-Agentic Code Reviewer
Our initial architecture defined five specialized agents:
- Security Reviewer – enforces secure coding standards (OWASP Top 10) and detects leaked credentials.
- Architecture Reviewer – catches coupling, dependency layering violations, and circular imports.
- Code Quality Reviewer – identifies code smells, nested logic, and cognitive complexity.
- Documentation Reviewer – enforces PEP-257 docstring compliance and monitors README updates.
- Lead Reviewer – consolidates the outputs of the four reviews into a prioritized markdown report.
This structure was clean and modular on a whiteboard. In practice, it immediately exposed the real bottleneck of multi-agent engineering: token consumption and sequential HTTP overhead. Every additional agent is not just a persona – it’s a full LLM round trip, with its own latency, its own cost, and its own contribution to the rate limit ceiling.
Designing the Multi-Agentic Code Reviewer Crew: Personas, Prompts, and the ReAct Loop
To build this crew, we wrapped our LLM connections inside CrewAI’s Agent and Task structures. In CrewAI, an agent is essentially a combination of a system prompt, a backstory, and a model configuration, executing under the ReAct (Reason + Action) reasoning framework. Here’s a simplified look at how these agents are defined in agents.py:
from crewai import Agent, LLM
def create_security_reviewer(llm: LLM) -> Agent:
return Agent(
role="Security Reviewer",
goal="Identify security vulnerabilities, hardcoded secrets, and cryptographic weaknesses.",
backstory=(
"An expert cybersecurity analyst specializing in static application security testing (SAST). "
"You check git diffs for exposed API keys, credentials, SQL injection, and OWASP vulnerabilities."
),
verbose=True,
llm=llm
)Under the hood, when crew.kickoff() is called, CrewAI launches an execution loop. If we associate tools (like running a local linter or reading files) with an agent, it enters a dynamic loop where it reasons, executes a tool, observes the results, and refines its answer.
Since our review task is purely analysis based on a pull request diff, we opted not to equip our agents with tools. This revealed an interesting boundary: without tools, the ReAct loop resolves immediately to a “Final Answer.” The agent reasons that it can respond directly using the context already provided in its task description. Functionally, this behaves like a standard sequential LLM chain – CrewAI’s real value here is in its structured state management, which maps the output of one agent’s task as input context for the next.
The First Major Failure: Tokens Per Minute vs. Daily Quotas (TPD)
As soon as we implemented the five-agent sequential crew and ran our test suite, we hit a hard blocker: litellm.exceptions.RateLimitError (HTTP 429). Many developers assume rate limits are purely transactional or minute-based (Tokens Per Minute, or TPM). We learned the hard way that public and free-tier API endpoints, like Groq, also enforce strict Tokens Per Day (TPD) quotas – and those are much easier to exhaust than a per-minute cap.
Here’s the math that broke our initial build:
- Our PR diffs and description prompts averaged roughly 2,500 tokens of input context.
- In the sequential workflow, four agentic code reviewers evaluated the diff independently. That’s four separate LLM requests, each producing around 1,000 output tokens.
- The Lead Reviewer task then ingested all four reports (another 4,000 tokens of context) plus the original diff, making a fifth LLM request to compile the final comment.
- That’s five heavy LLM calls per review, averaging over 15,000 tokens in total, just for one pull request.
When we ran our test suite of five sample pull requests (covering security flaws, architecture violations, quality smells, doc gaps, and a clean run), pytest executed these workflows back-to-back. Total requested tokens exceeded 75,000 within a few minutes, completely exhausting a daily TPD limit of 100,000 tokens. The tests failed with:
Rate limit reached for model llama-3.3-70b-versatile ... on tokens per day (TPD): Limit 100000, Used 99765, Requested 636. Please try again in 16h35m.
That was our first real, production-shaped encounter with CrewAI rate limits, and it taught us a critical lesson: applying a multi-agent sequential design by default, without measuring its token footprint, is inefficient and economically unviable the moment you hit a real quota – whether that quota comes from a free tier or a paid one with daily caps.
The Architectural Pivot: Optimizing to a Single-Call Workflow
To resolve the daily token exhaustion while preserving the ability to run a detailed, multi-dimensional review, we introduced an optimization layer. We refactored main.py to support an OPTIMIZED_WORKFLOW environment toggle.
Instead of running five separate sequential tasks across five different agents, our optimized mode consolidates the criteria of all four reviews – Security, Architecture, Code Quality, and Documentation – into a single, multi-disciplinary task executed by the Lead Reviewer. This reduces LLM calls from five down to one single execution per PR.
Here’s the task implementation in tasks.py:
def create_optimized_review_task(agent: Agent) -> Task:
return Task(
description=(
"Conduct a comprehensive code review of this PR across four areas: Security (secrets/injections), "
"Architecture (layering/decoupling), Code Quality (smells/complexity), and Documentation (docstrings/README).\n"
"PR: {title}\nDescription: {description}\nDiff:\n{changed_files}\n\n"
"Expected Output: A concise Markdown report containing Summary, Findings Grouped by Severity, and Verdict. "
"Keep the output compact, direct, and actionable to save token space."
),
expected_output="Optimized Markdown review comment containing the summary, severity-grouped findings, and verdict.",
agent=agent
)And in main.py, we check the environment and execute the appropriate path:
optimized = os.getenv("OPTIMIZED_WORKFLOW", "true").lower() == "true"
if optimized:
lead_reviewer = create_lead_reviewer(llm)
review_task = create_optimized_review_task(lead_reviewer)
crew = Crew(agents=[lead_reviewer], tasks=[review_task], process=Process.sequential)
else:
# Fallback to the 5-agent sequential crew
...By switching to this single-call model, we reduced token consumption by 80%, slashed execution time, and eliminated the rate-limiting failures entirely, all while keeping the consolidated feedback just as actionable as the original five-agent version. We kept the old five-agent path behind a flag rather than deleting it, since it’s still useful for local experimentation or for teams running on a quota generous enough to support it.
Building the Webhook Receiver and REST API Client
An agentic code reviewer is only useful if it integrates smoothly with existing developer workflows. We built a production-ready GitLab integration consisting of two main parts: a FastAPI webhook receiver and an asynchronous GitLabClient wrapper.
When a developer creates or updates a Merge Request, GitLab fires a webhook event. Our FastAPI server listens on the /webhook/gitlab endpoint:
@app.post("/webhook/gitlab")
async def handle_gitlab_webhook(
request: Request,
background_tasks: BackgroundTasks,
x_gitlab_token: Optional[str] = Header(None)
):
# Validate secret webhook token
expected_token = os.getenv("WEBHOOK_SECRET")
if expected_token and x_gitlab_token != expected_token:
raise HTTPException(status_code=401, detail="Invalid token")
payload = await request.json()
# Check that this is a merge request event and is open/updated
...
# Retrieve changes using the GitLab REST API
client = GitLabClient()
changes_data = client.get_merge_request_changes(project_id, mr_iid)
# Queue the heavy CrewAI execution as a background task to respond instantly to GitLab
background_tasks.add_task(
execute_review_in_background,
project_id=project_id,
mr_iid=mr_iid,
title=title,
description=description,
changes=changes_data.get("changes", [])
)
return {"status": "queued"}Decoupling the webhook response from the LLM execution is essential. GitLab expects a webhook endpoint to respond within a few seconds, but running a multi-agent crew can take anywhere from 10 to 45 seconds depending on LLM latency. By leveraging FastAPI’s BackgroundTasks, we immediately acknowledge the webhook with a 200 OK (or 202 Accepted), then spin up the CrewAI workflow asynchronously. Once it completes, the background task uses GitLabClient.post_merge_request_comment to publish the markdown report back to the MR notes.
Defensive Testing: Handling Rate Limits and Mocks in CI/CD
In modern backend systems, CI/CD pipelines need to be fast and reliable every time. If your test suite makes live API calls to third-party LLMs, two things go wrong:
- Tests fail in isolated build environments where API secrets aren’t exposed.
- Tests exhaust quotas, triggering the same CrewAI rate limits we hit in section three, and slowing down every developer’s feedback loop for hours afterward.
To guarantee a green, dependable test suite, we built a defensive mocking strategy in test_main.py, using an autouse pytest fixture that intercepts Crew.kickoff calls:
@pytest.fixture(autouse=True)
def mock_crew_kickoff():
"""Mock Crew.kickoff agentic code reviewer globally to avoid external LLM API calls and daily TPD limits in tests,
unless REAL_LLM=true is specified.
"""
if os.getenv("REAL_LLM", "false").lower() == "true":
yield
return
def mock_kickoff(self, inputs=None):
inputs = inputs or {}
changed_files = inputs.get("changed_files", "")
# Inspect inputs and return structured mock reports matching the test cases for agentic code reviewer
if "session" in changed_files:
report = "... Found critical security vulnerabilities ... Verdict: Needs Changes"
elif "controller" in changed_files:
report = "... Architectural violations ... Verdict: Needs Changes"
else:
report = "... Excellent clean code ... Verdict: Approved"
mock_output = MagicMock()
mock_output.__str__.return_value = report
mock_output.raw = report
return mock_output
with patch("crewai.Crew.kickoff", mock_kickoff):
yieldBy default, this mock intercepts the network call, returns the expected review comment instantly, and lets pytest validate parsing, headers, webhook queueing, and validation rules in under 2.5 seconds – with zero risk of tripping a rate limit.
If a developer wants to run live integration tests against the actual model (for example, gemini/gemini-1.5-flash), they simply prefix the pytest command with REAL_LLM=true. This design isolates the pipeline from network failures and rate limits while keeping a clean, testable integration boundary intact.
Deep-Dive Q&A: Agents, Orchestration, and the “Agentic” Boundary
During our review and debugging sessions, several conceptual questions came up about code structure, agent definitions, and where the boundary of “agentic” execution actually sits. We’ve gathered the most useful ones here.
Why Import Placement Matters (PEP 8)
During development, we briefly imported litellm locally inside the get_llm() function in agents.py. Local imports are sometimes used in Python to resolve circular dependencies, but they violate PEP 8 style standards when used without a specific reason. In production-ready code, imports should sit at the top of the file, organized into three blocks: standard library, third-party packages, and local modules. We refactored this to keep the codebase clean and idiomatic.
Are CrewAI Agents Just Packaged Prompts?
Fundamentally, yes. In most LLM-based agent frameworks, an Agent is a structured wrapper packaging three components:
- System persona – the
role(defines identity), thegoal(defines target output), and thebackstory(sets tone, background expertise, and reasoning style). - LLM connection – model choice (for example,
gemini-1.5-flash), temperature, key mapping, and token caps. - Execution loop and memory – the runtime memory and the capability to run a loop or use tools.
The Task, separately, represents the actual execution instructions: input placeholders (like {changed_files}) and the expected output layout.
What Does crew.kickoff() Do, and Is It Always Sequential?
crew.kickoff() is the entry point that initializes the runtime state. It maps initial inputs, manages context memory, forwards inputs and outputs between tasks, calls LLM wrappers, and handles tool errors.
Execution isn’t limited to a sequential process. CrewAI supports:
Process.sequential– executes tasks in a predefined order.Process.hierarchical– sets up a manager agent that autonomously decomposes the goal, delegates sub-tasks to specialized agentic code reviewers, reviews their answers, and demands revisions if they fail to meet quality criteria.
What Makes an Approach Truly “Agentic”?
If a script simply passes text from one LLM prompt to the next, it’s a linear pipeline, not an agent. What actually makes a system agentic is:
- The ReAct loop – the agent runs an internal reasoning cycle (Thought → Action → Observation → Thought) to decide for itself when its output is complete.
- Autonomous tool usage – giving agents tools, like a Python compiler or a linter, and letting the agent decide how to call the tool and how to resolve errors, without hardcoded if/else rules.
- Autonomous collaboration – in hierarchical crews, the manager agent can reject outputs and re-delegate tasks back to agentic code reviewers on its own.
By this definition, our optimized single-call reviewer is closer to a well-structured prompt than a “true” agent – and that’s fine. Matching the architecture to the actual complexity of the task, rather than defaulting to the most agentic option available, is exactly the kind of decision that keeps a system inside its rate limits.
Key Takeaways for Building Production-Ready CrewAI Systems
- More agents isn’t automatically better. Every additional agent is a full LLM round trip. Measure the token cost of your architecture before you scale the number of roles.
- Watch for CrewAI LLM rate limits at the daily quota level, not just per-minute. TPD limits are exhausted quietly by test suites and batch jobs long before anyone notices a TPM warning.
- Collapse redundant agent calls into one well-structured task when quality allows it. We cut token usage by 80% by merging four agentic code reviewer roles into a single prompt, with no meaningful drop in review quality.
- Decouple slow agent execution from time-sensitive webhooks. GitLab, and most webhook providers, expect a fast response; background tasks keep that contract intact.
- Mock LLM calls in CI by default. A fast, deterministic test suite that never touches a live model protects both your build times and your daily token quota.
- “Agentic” is a spectrum, not a label. ReAct loops, autonomous tool use, and hierarchical delegation are what make a genuine agentic code reviewer – a single well-crafted prompt can still be the right engineering choice.
Multi-agent frameworks like CrewAI are genuinely useful, but the tutorials rarely mention what happens once you point them at a real API quota. Our experience building this PR agentic code reviewer suggests a simple rule: start with the architecture that actually fits your rate limits, not the one that looks best in a diagram, and add agentic complexity only where it earns its keep.