Langchain RAG is not retrieving any document
11:22 29 Oct 2025

This is my embedding code, which I run once only:

embeddings = OpenAIEmbeddings(model="text-embedding-3-large")

vector_store = MongoDBAtlasVectorSearch.from_connection_string(
  connection_string = DB_CONNECTION,
  namespace = "gpt.embeddings",
  embedding = embeddings,
  index_name = "vector_index",
  relevance_score_fn="cosine"
)

loader = PyPDFLoader("./manual.pdf")
docs = loader.load()

text_splitter = RecursiveCharacterTextSplitter(chunk_size=100, chunk_overlap=50, add_start_index=True)
all_splits = text_splitter.split_documents(docs)

document_ids = vector_store.add_documents(documents=all_splits)

This is my agent code:

model = init_chat_model("openai:gpt-4.1")

retriever = vector_store.as_retriever(
    searchtype="similarity_score_threshold",
    search_kwargs={
        "k": 3,
        "score_threshold": -99999999999999999999999999999999999999999999999999999999,
    }
)

@tool(response_format="content_and_artifact")
def retrieve_context(query: str):
    """Retrieve information to help answer a query."""
    retrieved_docs = retriever.invoke(query)
    print(retrieved_docs)
    serialized = "\n\n".join([
        (f"Source: {doc.metadata}\nContent: {doc.page_content}")
        for doc in retrieved_docs
    ])
    return serialized, retrieved_docs

tools = [retrieve_context]
prompt = """
Always use the `retrieve_context` tool first. Append the retrieved context to the user prompt and answer the user's question.
"""
agent = create_agent(model, tools, system_prompt=prompt)

The problem is that whenever I invoke agent.invoke, the print statement always prints [] (i.e. no documents have been found). I have checked my MongoDB collection, and it has over a hundred chunks. I even set the threshold to -9999999999999999999999999999 to guarantee the retrieval of 3 documents. I have set the OpenAI key, as well as the database connection string. I can query the database as well. But still, the problem persists. No document is ever retrieved. It's clear the agent is calling the function, since it's printing [], but the retrieval gives nothing.

python langchain large-language-model rag