Introduction to NoSQL and Vector Databases for AI Agents

AI agents need more than just structured tables. When dealing with unstructured content, semantic search, or large-scale embeddings, traditional relational databases fall behind. This is where NoSQL document stores and vector databases come in.

Document databases like MongoDB store flexible JSON-like records. They are ideal when your agent handles varied payloads — for instance, user profiles with optional fields, scraped web pages, or chat histories where each conversation has a different structure.

Vector databases like Pinecone, Weaviate, or Chroma store high-dimensional embeddings and let agents perform similarity search. Instead of matching exact keywords, the agent retrieves items that are semantically closest to a query. This powers RAG pipelines, memory systems, and recommendation features.

Connecting Agents to Document Stores

The typical workflow involves an agent calling a tool or function that wraps database operations. Instead of writing raw queries in the prompt, you expose safe, parameterized actions like get_user_profile or log_conversation.

from pymongo import MongoClient

client = MongoClient("mongodb://localhost:27017")
db = client["agent_store"]

def log_conversation(session_id: str, messages: list):
    db.conversations.update_one(
 {"session_id": session_id},
        {"$set": {"messages": messages}},
        upsert=True
    )

def get_user_context(user_id: str) -> dict:
    return db.users.find_one({"user_id": user_id})

In this pattern, the agent decides which function to call based on the user's request, and the functions handle all database interactions behind the scenes. The database schema stays flexible — MongoDB will accept any shape you store.

Using Vector Databases for Semantic Retrieval

Vector databases turn text into numerical arrays (embeddings) using a model like OpenAI's text-embedding-3-small or a local sentence-transformer. Once stored, similarity is measured with cosine distance or dot product.

from openai import OpenAI
import chromadb

client = OpenAI()
chroma = chromadb.PersistentClient(path="./vector_store")
collection = chroma.get_or_create_collection("docs")

def embed(text: str) -> list:
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=text
    )
    return response.data[0].embedding

def add_document(doc_id: str, content: str):
    collection.add(ids=[doc_id], embeddings=[embed(content)], documents=[content])

def search_similar(query: str, top_k: int = 3) -> list:
    results = collection.query(query_embeddings=[embed(query)], n_results=top_k)
    return results["documents"][0]

The agent calls search_similar when the user's question requires knowledge beyond its training cutoff. The retrieved chunks are then injected into the prompt as context for the language model.

Choosing Between Document and Vector Stores

Document databases are the right choice when you need exact lookups, flexible schemas, and traditional filtering. For example, fetching an order by ID or listing all sessions for a tenant.

Vector databases are the right choice when you need fuzzy, meaning-based retrieval. Examples include finding help articles relevant to a user's phrasing, matching resumes to job descriptions, and locating past conversations that resemble the current topic.

In practice, most production agents combine both. The vector database finds semantically related items, and the document database fetches full structured records related to those items — for example, retrieving the full user profile after a similarity match on past tickets.

Common Pitfalls and Best Practices

A frequent mistake is storing large embeddings inside a document database. While possible, it hurts indexing speed and query performance. Keep embeddings in a dedicated vector store and keep references (like a document ID) in your document store.

Another pitfall is forgetting to handle embedding model changes. If you switch models, you must re-embed your entire corpus — old and new vectors are not comparable.

Finally, always set metadata filters in your vector queries. Searching across millions of vectors is expensive, so narrow by date, user, or category before doing the similarity step.

results = collection.query(
    query_embeddings=[embed(query)],
    n_results=5,
    where={"user_id": "u_123", "created_at": {"$gt": 1700000000}}
)

Summary

NoSQL document databases give AI agents schema flexibility for varied, real-world data. Vector databases add semantic retrieval that traditional SQL cannot match. For production agents, the winning pattern is a hybrid: use a document store for structured state and exact lookups, and a vector store for meaning-based search. Keep embeddings in their native store, filter early, and plan ahead for model upgrades.