Vector search at scale: pgvector vs dedicated stores

A while back I made the case that for adding semantic search to a Rails app, pgvector — the extension that lets Postgres store and search embedding vectors — is very often all you need, because keeping your vectors in the same database as the rest of your data is a huge operational simplification. I stand by that for the common case. But “very often” isn’t “always,” and as vector search has taken off, a whole category of dedicated vector databases — Pinecone, Qdrant, Weaviate, Milvus and others — has matured into a real alternative. So the question that arrives once your semantic search outgrows a toy dataset is a genuine one: do you stay with pgvector, or move to a dedicated store? This is a deep-dive on what actually changes at scale, what leaving Postgres costs you, and how to decide without talking yourself into infrastructure you don’t need.

What the two options actually are

The distinction is straightforward, and naming it clearly helps the decision:

pgvector is an extension to a database you almost certainly already run. Your embeddings live in a Postgres table, as a column, right next to the rows they describe. You query them with SQL, index them with Postgres indexes, back them up with your Postgres backups. It’s not a separate system; it’s a capability added to your existing one.
A dedicated vector database is a purpose-built system whose entire job is storing vectors and doing nearest-neighbour search over them, fast, at large scale. It’s separate infrastructure you run (or pay for) alongside your main database — another service, another integration, another thing to operate — built and optimized specifically for this one workload.

That framing already tells you the shape of the trade: pgvector buys operational simplicity by reusing what you have; a dedicated store buys specialized performance and scale at the cost of running more infrastructure. Everything else is detail on top of that core tension.

What actually changes at scale

The honest question is: at what point does the dedicated store’s specialization start to matter? Because below that point you’re paying its operational cost for a benefit you can’t yet use. The factors that actually move the needle:

Number of vectors. This is the big one. pgvector handles modest-to-substantial vector counts perfectly well — comfortably into the hundreds of thousands and beyond with a good index. Dedicated stores are built to handle tens or hundreds of millions, or billions, with search performance that stays fast where a general-purpose database would struggle. If you have a few hundred thousand document chunks, this difference is theoretical. If you have a hundred million, it’s the whole decision.
Query latency at high concurrency. Dedicated stores invest heavily in approximate-nearest-neighbour (ANN) algorithms and the engineering around them to keep latency low under heavy concurrent search load. pgvector has ANN indexing too (HNSW and IVFFlat), and it’s good — but a system built solely for this will generally push further on the latency-at-scale frontier.
Specialized features. Dedicated stores often ship features that are specifically about vector workloads: sophisticated metadata filtering combined with vector search, multi-tenancy designed for embeddings, horizontal scaling tuned for this access pattern. If you need those specific capabilities, building them on Postgres is swimming upstream.

Notice that all three only bite at scale or with specific demanding requirements. None of them is a reason for a typical application with a moderate corpus to leave Postgres. The specialization is real, but it’s specialization for a problem most applications don’t have yet.

What you give up by leaving Postgres

This is the side of the ledger the “use the best tool for vectors” framing tends to skip, and it’s substantial. When your vectors live in pgvector, you get things for free that become real work the moment they move to a separate system:

One source of truth. Your vectors and the data they describe live in the same database, so there’s no synchronization problem. With a separate vector store, every time a document changes you must update two systems and keep them consistent — and “keep two datastores in sync” is a classic, perennial source of bugs (the dreaded “the vector store says this document exists but the database deleted it last week”).
Transactional consistency. In Postgres, you can write a row and its embedding in the same transaction — they commit together or not at all. Across two systems you lose that atomicity, and you inherit all the eventual-consistency and partial-failure handling that comes with coordinating writes to two stores.
Joins and SQL. Your vectors sit alongside the rest of your relational data, so you can filter vector search by ordinary SQL conditions in one query — “find similar documents that belong to this user and are published” is a single statement. With a separate store, combining vector similarity with relational filters means querying two systems and reconciling the results yourself.
One thing to operate. One database to back up, monitor, secure, and reason about, versus two. This is the operational simplicity argument, and it is not minor — every additional piece of stateful infrastructure is a standing tax on your team forever.

These aren’t edge concerns; they’re the daily texture of running the system. Leaving Postgres doesn’t just add a database — it adds a synchronization problem, a consistency problem, a cross-system-query problem, and an ops burden, all at once. A dedicated store has to be worth all of that, not just faster on a benchmark.

How to decide

The decision framework is the same one this series keeps arriving at, because it keeps being right: start with pgvector, and move to a dedicated store only when you have a concrete, measured reason to. Concretely:

Default to pgvector if you’re already on Postgres and your vector count is moderate (which, for most applications, it is). You get semantic search with no new infrastructure and all the consistency and simplicity benefits above. This is the right answer far more often than the vector-database hype suggests.
Consider a dedicated store when you hit a real limit: your vector count is in the many millions and climbing, your search latency under load is measurably failing your requirements despite proper pgvector indexing, or you need specialized vector features that are genuinely hard to build on Postgres. The key word in every case is measured — you should be able to point at a number that pgvector can’t meet, not a fear that it might not.
Don’t move pre-emptively. Adopting a dedicated vector database “because we’ll need it eventually” or “because it’s the proper tool for vectors” is the over-engineering trap exactly. You take on the sync problem, the consistency problem, and the ops burden now, in exchange for scale you don’t yet use — and you can always migrate later, when you actually have the volume, with real numbers guiding the design instead of guesses.

Verdict

As vector search has matured, dedicated vector databases — Pinecone, Qdrant, Weaviate, Milvus — have become a real, capable alternative to pgvector, and at genuine scale they’re the right tool: they handle tens or hundreds of millions of vectors with low latency under heavy concurrency, and ship specialized features that are hard to replicate on a general-purpose database. But that specialization only matters at a scale or with specific demanding requirements that most applications don’t have, and leaving Postgres is far from free — you give up a single source of truth, transactional consistency between a row and its embedding, the ability to combine vector similarity with SQL filters in one query, and the simplicity of operating one datastore instead of two. So the decision is the familiar match-the-tool-to- the-need one: default to pgvector, take semantic search for the price of an extension and keep all the consistency and operational wins, and move to a dedicated store only when a measured limit forces it — a vector count in the many millions, latency that’s failing despite proper indexing, or a specialized feature you genuinely need. Don’t adopt a vector database pre-emptively for scale you don’t have; you’ll pay its full cost today for a benefit you can’t yet use, and you can always migrate later with real numbers in hand. Postgres for everything, until Postgres genuinely can’t — and for vectors, that “can’t” arrives much later than the hype implies.