How Agentic Search Unlocks Legal Research Intelligence at :Harvey:

Legal research has always been iterative: searching case law, discovering a new angle, refining your query, checking another jurisdiction, cross-referencing with statutes, and repeat. This natural back-and-forth, or reasoning loop, is fundamental to how lawyers work. Traditional retrieval systems, however, reduce this nuanced process to a single query-and-retrieve operation.

At Harvey, we built Agentic Search to mirror how legal professionals actually conduct research. The key was moving from static, one-shot retrieval to a system that reasons about what information it needs, where to find it, and when to dig deeper. This capability helps raise the baseline quality of queries in Harvey Assistant, while also serving as the technical foundation for Deep Research — our most comprehensive mode for multi-source legal analysis.

In this post, we'll walk through what agentic search is, why it matters for legal research, and how we implemented it to handle the complex, multi-source queries that define legal work.

To understand why we needed agentic search, it’s helpful to walk through an example of a type of query that traditional retrieval systems struggle with. Let’s focus on due diligence.

Imagine conducting due diligence on an acquisition target and using this prompt in Harvey: "Review all employment agreements in the data room and identify any non-compete provisions that may be unenforceable under California law. For each provision, cite the relevant California statute or case law."

This requires:

The agent must move fluidly between document analysis and legal research, understanding how initial diligence findings trigger the need for additional statutory or case law research, and then applying those legal standards back in order to analyze the specific provisions it found necessitating the research. This requires reasoning about what information is needed, where it lives, and how to synthesize it — capabilities that sit at the core of agentic Retrieval Augmented Generation (RAG).

Traditional one-shot retrieval fails here because it can't adapt its strategy as it discovers information; it can't decide which knowledge sources are relevant; and it can't determine when it has gathered sufficient context. The query demands an intelligent system that reasons about the research process itself.

Agentic Search represents a paradigm shift in how we approach complex information retrieval.

Naive RAG follows a straightforward pattern: embed the query, retrieve relevant chunks, generate an answer. This works for simple queries but breaks down when research requires multiple rounds of information gathering, synthesis across sources, or reasoning about missing information — exactly like the due diligence example above.

Agentic search transforms this into an iterative reasoning process. The system plans its approach, decides which sources to query, evaluates whether it has sufficient information, and refines its strategy based on discoveries. Here’s a quick comparison between naive RAG and agentic search:

Given the characteristics above, there are a few key reasons why agentic search matters for legal research:

Building an agentic RAG system requires careful orchestration between reasoning and retrieval. At Harvey, we designed our system around a workflow that allows the agent to make intelligent decisions about when and where to search, evaluate the sufficiency of retrieved information, and determine when it has enough context to answer.

Our implementation follows a reasoning-driven retrieval loop inspired by the ReAct (Reasoning and Acting) paradigm, where the agent interleaves reasoning traces with tool calls:

In the Harvey platform, this workflow enables faster Assistant queries and comprehensive Deep Research reports — all powered by agents that can seamlessly query, reconcile, and synthesize across multiple knowledge sources.

Building this system revealed three fundamental challenges that went beyond standard RAG implementation. Here’s a quick overview of each before we dig into how we solved these challenges.

Challenge 1: Source Selection in the Wild

Our initial prototype worked well on carefully constructed test queries, but real user queries proved more challenging. Lawyers phrase queries with implicit context — a question like "What's our exposure on the termination provisions?" might require searching internal deal documents, checking market standards, and consulting case law, but doesn't explicitly enumerate these sources. The model would often query only one or two sources when broader coverage was needed.

Challenge 2: Scaling Tool Calls to Query Complexity

Simple questions like "What's the notice period in section 8.2?" should resolve quickly with minimal retrieval, while complex questions should trigger extensive research. Initially, the agent treated nearly every query the same way — making one or two tool calls and moving on. The agent lacked a calibrated sense of when "good enough" was sufficient and when a query demanded exhaustive coverage.

Challenge 3: Multi-Source Citations

Our citation system had historically been built and benchmarked on single-source queries. In order to support multi-source queries, we had to ensure our system built upon the foundational improvements made for each source’s citation quality, while remaining flexible enough to adapt to context from multiple sources. We also invested in citation benchmarking sets that captured the full spectrum of multi-source queries — from those that specified exactly which sources were preferred to those that were far more open-ended.

These challenges — source selection and scaling effort to complexity — required a systematic approach to improvement. Since Harvey's privacy guarantees mean no human has access to or reviews actual customer queries, we faced the question: How do you improve a system when you can't directly observe its real-world performance?

Our solution was to build a systematic evaluation process powered by Harvey's in-house legal experts. Applied Legal Researchers (ALRs) and Strategic Business Development Leads (SBDLs) are former practicing lawyers who collaborate closely with Engineering on product development. In partnership with these teams, we developed a data flywheel that continuously improves the system:

This approach solved our source selection challenge by giving the agent clearer signals about when to use each knowledge source — improving tool selection precision from near zero to 0.8-0.9. For the tool-calling intelligence challenge, we used eval data to calibrate the agent's effort level. Complex queries that initially resolved in a single tool call now appropriately scale to 3-10 retrieval operations based on query demands.

Agentic Search brings reasoning to retrieval, enabling Harvey to handle a wide range of tasks, from quick document lookups to comprehensive multi-source legal research. By building a privacy-preserving evaluation system powered by legal experts, we've created a system that continuously improves while still maintaining the data protections fundamental to Harvey.

This represents a significant step forward in legal research intelligence, but there's more work ahead. Our team is focused on these key areas:

If these challenges sound interesting to you, we're hiring for roles across the Engineering team.