Memory & Knowledge Management

DataStax's serverless vector database built on Apache Cassandra for planet-scale AI applications. Provides multi-cloud deployment, zero-TTL cold starts, and native LangChain/LlamaIndex integrations. Generous free tier with automatic scaling.

Real-time distributed SQL database with native vector search, full-text search, and JSON support in a single engine. Eliminates the need for a separate vector database in latency-sensitive AI applications. Used by enterprises needing HTAP plus vector capabilities.

SQLite extension for vector similarity search using Facebook FAISS under the hood. Adds vector search to SQLite databases without any additional infrastructure. Perfect for local, edge, or embedded agent deployments where a full vector database is overkill.

Elasticsearch's native vector search capabilities enabling approximate nearest-neighbour search using HNSW. Integrates vector search with full Elasticsearch query DSL for powerful hybrid queries. Powers search and RAG in thousands of enterprise applications.

Open-source end-to-end vector search engine that handles both embedding generation and storage. Send raw text or images and Marqo embeds and indexes them automatically. Simplifies the RAG stack by eliminating the separate embedding step.

Python client for building AI applications on Redis with vector similarity search. Enables semantic caching, semantic routing, and RAG pipelines on top of Redis. Leverages Redis speed for real-time vector search at low latency.

Managed Postgres with pgvector, storage, and auth — providing a complete AI backend in one platform. Includes a Vector Toolkit for loading, transforming, and querying embeddings. Used by thousands of startups as the all-in-one backend for AI applications.

Facebook AI Similarity Search — the most widely used library for efficient similarity search and clustering of dense vectors. CPU and GPU implementations. Supports billion-scale indexes with IVF, HNSW, and product quantization. The backbone of many production vector search systems.

Spotify's Approximate Nearest Neighbors library optimized for memory usage and read-only workloads. Builds indexes that can be memory-mapped and shared across processes. Ideal for serving pre-built embedding indexes in agent applications with high query throughput.

Next-generation vector search engine with 10x higher indexing speed than FAISS. Supports 8-bit and 16-bit quantization, custom distance functions, and multi-vector storage. Available for Python, C++, JavaScript, and Rust. Designed for multi-billion scale production deployments.

Open-source embedding database designed specifically for LLM applications. Simplest API in the category — add documents, query by text, done. Embedded mode for prototyping, client-server for production. Default choice for most LangChain and LlamaIndex demos.

Browser extension storage APIs (chrome.storage.local and IndexedDB) for persisting agent memory client-side. Enables browser extension agents to maintain context across sessions with no backend required. Chrome Storage Sync allows memory to roam across devices automatically.

Managed Qdrant vector database with a free forever cluster (1GB RAM, 0.5 vCPU). No credit card required. Ideal for prototyping RAG pipelines and agent memory.

Intelligent memory layer for AI assistants that stores, retrieves, and updates memories across sessions. Automatically extracts relevant information from conversations and surfaces it at the right time. SDKs for Python and TypeScript with a managed cloud service.

Long-term memory service for AI applications with automatic summarization, entity extraction, and temporal context. Messages are continuously enriched with facts and summaries. Both self-hosted and cloud options. Designed as a drop-in memory layer for LangChain and LlamaIndex.

Standard memory utilities for tracking conversation history in LangChain agents. Provides a variety of memory types — buffer, summary, entity, conversation kg — that can be mixed and matched to build agents that maintain context across arbitrarily long conversations.

Stateful LLM agents with self-editing memory inspired by OS virtual memory management. Agents manage their own context window, deciding what to keep in working memory and what to offload to long-term storage. Supports persistent personas, tools, and multi-agent networks.

Rust-powered memory and information retrieval server for LLM applications. Acts as a drop-in memory backend that stores session history, provides semantic search over past messages, and delivers relevant context on every new turn with sub-millisecond latency.

AI memory management library with semantic similarity search, context window management, and configurable forgetting mechanisms. Supports short-term and long-term memory stores separately. Lightweight alternative to heavier memory frameworks for simple use cases.

Deterministic memory layer for AI using knowledge graphs. Processes and stores information as interconnected entities rather than flat vector embeddings. Enables more structured reasoning about relationships between stored facts. Supports multiple LLM and vector database backends.

Event-driven agent infrastructure for building persistent, observable agentic workflows. Step-based execution model with built-in state management between steps. Provides a visual debugger and replay capability for understanding agent behavior in development.

Open-source memory module for autonomous agents combining knowledge graphs and vector stores. Agents update their own memories based on conversations. Includes an analytics dashboard showing memory growth over time and retrieval patterns.

Lightweight memory library for LLM agents using ChromaDB as backend. Provides create, search, update, delete operations on agent memories with metadata support. Simple interface designed to be used alongside any agent framework.

Microsoft Research's graph-based RAG system that extracts a knowledge graph from documents, then queries across it for global question answering. Significantly outperforms standard RAG on questions requiring synthesis across many documents. Widely cited research.

Universal data connector API for syncing documents from Google Drive, Notion, Dropbox, OneDrive, and 30+ sources into your RAG pipeline. Handles OAuth, incremental sync, and document parsing automatically. One API key to access all user data sources.

Vector database and data lake for AI, storing embeddings alongside original data (images, text, audio). Supports streaming data from cloud storage and has native integrations with LangChain and LlamaIndex. Open-source with a managed cloud offering.

Proprietary document parsing API from LlamaIndex specialized in extracting structured data from complex PDFs, including tables, charts, and multi-column layouts. Produces clean markdown output optimized for RAG ingestion. The parsing backbone of many production RAG systems.

IBM's open-source document parsing library that converts PDFs, DOCX, PPTX, and HTML into clean markdown or JSON. Uses AI models for accurate layout detection, table extraction, and formula parsing. The recommended document preprocessing layer for RAG pipelines.

Fast, high-accuracy PDF to markdown converter using vision models for layout understanding. Handles complex academic papers, equations, and multi-column documents better than most alternatives. 40x faster than nougat with comparable accuracy.

Cohere's reranking API for improving RAG retrieval quality by re-scoring retrieved documents against the query. Significantly boosts precision without changing the retrieval pipeline. One API call that often outperforms more complex hybrid retrieval strategies.

Neo4j's solution for combining knowledge graphs with vector search to build more accurate RAG systems. Graph structure captures entity relationships that flat vector search misses. Official Python and TypeScript SDKs for building GraphRAG pipelines.

Unified framework for building enterprise RAG pipelines with models specifically trained for document parsing. Includes a model catalog, vector store, and prompt history management. Optimized for CPU inference on commodity hardware.

Document retrieval system using vision language models to index and retrieve document pages as images. Eliminates complex PDF parsing by treating documents as visual objects. Outperforms text-based retrieval on charts, tables, and mixed-layout documents common in enterprise RAG.

Ultra-fast, lightweight Python reranker for RAG pipelines. Runs cross-encoder models locally without GPU. Sub-100ms reranking of 100+ documents. Integrates with LangChain and LlamaIndex. Dramatically improves RAG precision by reranking initial retrieval results before feeding to LLM.

Unified API for multiple reranking models (Cohere, ColBERT, cross-encoders, LLM rerankers). One interface to compare reranking strategies. Supports local and API-based models. Essential for production RAG systems where retrieval quality directly impacts agent accuracy.

Evaluation framework specifically for RAG pipelines. Provides metrics like faithfulness, answer relevancy, context precision, and context recall. Generates synthetic test sets from documents automatically. The de facto standard for measuring RAG quality in production.

Enterprise document parsing and chunking platform that transforms PDFs, Word docs, PowerPoints, and HTML into LLM-ready structured data. Preserves document structure (titles, tables, figures). Handles complex layouts that break naive text extraction.

Fast BM25 implementation in Python using sparse matrices for 10-20x speedup over standard implementations. Drop-in replacement for rank_bm25. Essential for hybrid search RAG pipelines that combine BM25 keyword retrieval with dense vector search.

Specialized PDF-to-text API that preserves layout information for LLM processing. Maintains table structure, multi-column layouts, and headers. Outperforms generic OCR for complex financial, legal, and scientific documents that agents need to reason over.

The leading data framework for connecting private data sources to LLMs. Provides sophisticated indexing, chunking, retrieval, and query engine abstractions — from simple Q&A over documents to complex multi-step reasoning across heterogeneous data sources.

The most comprehensive data framework for RAG and knowledge-augmented LLM applications. Ingests 130+ data sources, provides 50+ retrieval strategies, and powers advanced RAG patterns like sub-question decomposition, recursive retrieval, and query routing.

Weaviate's open-source retrieval-augmented generation application. Ships as a complete, deployable RAG system with a polished web UI, hybrid search, and pluggable reader/chunker/embedder components — a great starting point for production document Q&A agents.

Battle-tested RAG chains and LCEL (LangChain Expression Language) pipelines. Includes document loaders for 100+ sources, text splitters, embedding models, and retrievers. The reference implementation for most RAG tutorial content on the web.

No-code workflow builder with built-in RAG memory and vector search. Provides a visual canvas for assembling agent pipelines that combine document retrieval, LLM calls, web search, and code execution — enabling non-engineers to build sophisticated knowledge agents.

Data-agnostic RAG framework that makes it trivial to turn any data source — PDFs, websites, YouTube videos, databases — into a searchable knowledge base for your agent. Handles chunking, embedding, and retrieval with a clean two-line API.

Deep document understanding RAG engine with support for rich layouts including tables, charts, and scanned PDFs. Goes beyond naive chunking by preserving document structure during indexing, dramatically improving retrieval accuracy for complex enterprise documents.

AI document engineering platform that creates fine-grained semantic chunks from business documents. Preserves semantic coherence better than naive chunking strategies. Particularly strong on contracts, reports, and structured business documents.

All-in-one embeddings database for semantic search, LLM orchestration, and language model workflows. Can run entirely local with no external dependencies. Supports workflows, pipelines, and a broad range of NLP tasks beyond just RAG.

Microsoft Research's graph-based RAG system that builds a knowledge graph from documents before retrieval. Enables global queries across entire document corpora — not just local context windows. Dramatically improves answers to complex analytical questions.

Microsoft utility for converting files (PDF, DOCX, XLSX, PPTX, images, audio) to Markdown for LLM ingestion. Preserves structure in Markdown format that LLMs understand. Simple CLI and Python API — great for building document ingestion pipelines for agents.

Open-source RAG-based document QA tool with a clean Gradio UI. Supports multiple LLMs and embedding models, hybrid search, multi-modal documents, and citation-backed answers. Self-hostable alternative to commercial document intelligence services.

Full-stack application for running a private ChatGPT over your documents with RAG. Desktop app or self-hosted Docker deployment. Supports local models via Ollama, custom embeddings, workspaces for different document collections, and multi-user management.

Production-ready RAG engine with multimodal ingestion, hybrid search, knowledge graphs, and user/document management built in. REST API-first design. Supports advanced features like GraphRAG and agentic RAG out of the box with sensible defaults.

Open-source RAG framework for building modular, production-ready RAG applications. Organizes RAG into collections, data sources, parsers, embedders, and retrievers as independent components. REST API and web UI included. Strong support for custom deployments.

In-memory data structure store used as database, cache, and message broker. Sub-millisecond read/write latency makes it ideal for storing agent session state, conversation history, and short-term working memory. Pub/Sub support enables reactive agent architectures.

The most advanced open-source relational database. Widely used for persisting agent conversation history, user profiles, and structured memory. JSONB columns store flexible agent state. Robust transaction support ensures memory consistency across concurrent agent sessions.

Google's NoSQL document database with real-time sync and offline support. Scales automatically from zero to global traffic. Used for persistent agent memory in applications that need real-time state sharing across multiple users or agent instances simultaneously.

AWS's fully managed NoSQL database with single-digit millisecond latency at any scale. Key-value and document model with auto-scaling. Used by enterprise agent systems for session state and memory storage with DynamoDB Streams enabling event-driven memory processing.

The world's most deployed database engine — a self-contained, serverless, zero-configuration SQL database. Perfect for local agent memory storage in desktop applications and edge deployments. WAL mode and full-text search extension (FTS5) make it surprisingly capable.

The AI-native open-source embedding database. ChromaDB makes it effortless to store, query, and filter embedding vectors locally or in the cloud — a go-to choice for prototyping RAG applications and production deployments that don't need managed infrastructure.

Fully managed vector database built for production ML applications. Serverless tier with no infrastructure to manage. Supports metadata filtering, namespaces, sparse-dense hybrid search, and real-time updates. Used by thousands of production RAG and agent systems.

Open-source vector database with built-in vectorization modules for text, images, and code. GraphQL and REST APIs. Supports hybrid search combining BM25 and vector similarity. Available as managed cloud or self-hosted with Kubernetes helm charts.

High-performance vector similarity search engine written in Rust. Provides rich filtering capabilities, payload storage, and quantization for memory efficiency. Named vectors enable multiple vector spaces per point. Available as managed cloud or open-source Docker image.

Vector similarity search extension for PostgreSQL. Adds approximate nearest neighbor (ANN) and exact vector search to your existing Postgres database — perfect for teams who want production RAG without adopting a separate vector store infrastructure.

Cloud-native open-source vector database built for billion-scale vector similarity search. Supports multiple index types (HNSW, IVF, DiskANN) for different performance-accuracy tradeoffs. Zilliz provides a managed cloud version. CNCF sandbox project with strong community.

Open-source PostgreSQL extension for vector similarity search. Store embeddings in Postgres alongside relational data. Supports exact and approximate nearest neighbor search with IVFFlat and HNSW indexes. The simplest path to vector search for teams already using PostgreSQL.

Vector similarity search built into Redis via the RediSearch module. Sub-millisecond query latency leveraging Redis's in-memory architecture. Supports HNSW and flat index types, hybrid queries with filtering, and JSON/Hash data structures. Ideal for low-latency agent memory.

Yahoo's open-source search and recommendation engine with first-class vector search. Combines lexical, semantic, and structured search in one query. Handles billion-scale datasets with real-time updates. Unique tensor computing model for ML feature transformations at query time.

Next-generation vector database built for cost efficiency at scale. Stores vectors on object storage (S3) with fast query cache. Claims 10x cost reduction vs. in-memory databases for large-scale vector workloads. API-compatible with common vector DB interfaces.

Developer-friendly embedded vector database built on the Lance columnar format. Serverless — runs in-process without a separate server. Supports versioning, full-text search, and can persist to S3 or local disk. Great for local development and edge deployments.

K-nearest neighbor search plugin for OpenSearch enabling semantic search alongside full-text search. Supports HNSW, IVF, and exact algorithms. Can combine vector similarity with BM25 in hybrid queries. Strong AWS managed service integration via Amazon OpenSearch Service.

Vector search built into MongoDB Atlas allowing semantic search over documents alongside regular MongoDB queries. Store embeddings as fields in existing collections. Supports approximate nearest neighbor with HNSW indexing and filter expressions.

Serverless PostgreSQL with pgvector pre-installed. Combine relational data and vector embeddings in the same database with branching for safe schema experiments. Zero cold starts and auto-scaling make it ideal for production AI applications with variable load.