Skip to content
← All posts
13 min read Dawid Skłodowski

Mongoid in anger: modelling documents when you come from ActiveRecord

We have been shipping MongoDB with Mongoid on real projects. Here is how document modelling really differs from ActiveRecord — embedded vs referenced, the missing joins, indexes, atomic writes, consistency boundaries, and where schemaless quietly bites.

If you have spent years with ActiveRecord, your first weeks with Mongoid feel deceptively familiar. You write field :name, you call where, you get back something that quacks like a model. Then you try to think in tables and join your way out of a problem, and MongoDB quietly refuses to play along.

We have now run a few projects on MongoDB, and the lesson is always the same: the ORM is not the hard part — the data model is. Mongoid does an excellent job of making Mongo feel like ActiveRecord, and that is precisely the trap, because the two reward opposite instincts. ActiveRecord rewards normalisation: split your data into clean tables and let the database reassemble it with joins at read time. MongoDB rewards the inverse: decide how you read the data, then store it in roughly that shape, joins be damned. Bring AR habits to Mongo unexamined and you will produce a schema that works in development and falls over in production.

Here is everything we wish we had internalised on day one.

Documents are not rows — they are aggregates

In ActiveRecord a model maps to a table, and a row is a flat list of typed columns. In Mongoid a model maps to a collection, and a document is a nested tree. That nesting is the whole point. A relationship that would force a has_many and a join in SQL can simply live inside the parent document:

class Order
  include Mongoid::Document

  field :placed_at, type: Time
  field :total,     type: BigDecimal
  field :currency,  type: String, default: "GBP"

  embeds_many :line_items
end

class LineItem
  include Mongoid::Document

  field :sku,      type: String
  field :name,     type: String
  field :quantity, type: Integer
  field :price,    type: BigDecimal

  embedded_in :order
end

Loading an order now loads its line items in a single read — no join, no second query, no N+1. The items have no independent existence; they are born and die with the order, and on disk they are literally a sub-array inside the order document.

The mental model that makes all of this click is the aggregate from domain-driven design: a cluster of objects that you treat as a single unit for the purpose of loading, saving, and maintaining invariants. An order and its line items are one aggregate. A blog post and its tags-as-text are one aggregate. MongoDB is, at heart, an aggregate store — it is superb at fetching and persisting one of these trees in a single round trip, and indifferent to everything you might want to do across them. Once you start drawing aggregate boundaries deliberately instead of normalising by reflex, most modelling decisions answer themselves.

Embedded vs referenced: the one decision that matters

Almost every modelling mistake we have made on Mongo comes down to choosing embed when we should have referenced, or the reverse. Get this right and the rest is detail; get it wrong and you will be fighting the database for the life of the project. The rule of thumb we settled on:

  • Embed when the child is owned by exactly one parent, is bounded in size, and is read together with the parent. Line items on an order. Addresses on a user. Variants and options on a product. The test is: would this data ever be queried or displayed without its parent? If no, embed it.
  • Reference when the child is shared between parents, queried on its own, or grows without limit. A product referenced by many orders. Comments that might number in the thousands. Anything you need to paginate independently or hand to another part of the system.
class Product
  include Mongoid::Document
  field :name,  type: String
  field :price, type: BigDecimal
  has_many :reviews            # referenced: reviews are queried on their own
end

class Review
  include Mongoid::Document
  field :body,   type: String
  field :rating, type: Integer
  belongs_to :product          # stores product_id, not the product
end

Two traps lurk here, and both are invisible until they are not.

The 16 MB document limit. Every document has a hard ceiling of 16 megabytes. An embeds_many with no natural bound — activity log entries, comments on a viral post, events on a long-lived account — is a time bomb. It works for months because your test data is small, then one document crosses the line in production and every write to that document fails, often in a code path far from where the growth happened. The fix is not “make it bigger”; the fix is to recognise at design time that an unbounded collection of children must be referenced, living in its own collection, paginated on read.

Cross-collection references give you no real join. A belongs_to / has_many across collections is resolved by Mongoid with a second query. There is no server-side join in MongoDB (not in 2013, anyway — there is no $lookup yet). So this is the N+1 you already know, except crueller:

# Looks innocent. Issues one query for the orders, then one query
# PER review collection touched — across collections, there is no
# `includes` that fully saves you the way it does in ActiveRecord.
Product.all.each do |product|
  puts "#{product.name}: #{product.reviews.count} reviews"
end

Mongoid can eager-load referenced associations in some cases, but you cannot lean on it the way you lean on includes in AR. The honest answer is that if you find yourself constantly walking references in loops, the data wanted to be embedded, or it wanted to be in Postgres.

There are no joins, so denormalise on purpose

Coming from SQL, the absence of joins feels like a missing feature. It is not — it is a different contract. MongoDB’s answer to “I need data from two places at once” is blunt: store it in one place. You denormalise deliberately, copying the handful of fields you need to read together into the document that needs them:

class Order
  include Mongoid::Document
  belongs_to :customer

  # copied from the customer at write time so the order list and the
  # printed invoice never have to touch the customers collection
  field :customer_name,  type: String
  field :customer_email, type: String

  before_save :denormalise_customer

  private

  def denormalise_customer
    return unless customer
    self.customer_name  = customer.full_name
    self.customer_email = customer.email
  end
end

Now the order list renders without ever touching the customers collection, and an invoice from three years ago still shows the name the customer had at the time — which for an invoice is exactly correct. The cost is obvious and must be owned: when a customer changes their name, the copies on past orders go stale unless you decide to propagate the change.

That decision is a modelling choice, not an accident. Sometimes you want the frozen copy (invoices, audit records, anything legal). Sometimes you want propagation, in which case you update the copies in a background job when the source changes. The discipline is simply to write down which fields are denormalised and what the propagation rule is, because the next developer will not guess that customer_name on an order is a cache rather than the source of truth. Denormalisation trades read simplicity for write complexity; on read-heavy screens — order history, dashboards, catalog listings — that trade is almost always worth it.

Querying: the DSL is lovely, the atomic operators are the point

Mongoid’s query DSL will feel immediately at home:

Order.where(:placed_at.gte => 1.month.ago)
     .and(status: "paid")
     .order_by(placed_at: :desc)
     .limit(50)

The symbol-method operators (.gte, .in, .ne) compile down to Mongo’s query operators ($gte, $in, $ne). Pleasant, but not the part that matters. The part that matters is atomic update operators, because they let you change a document on the server without the read-modify-write race that plagues the naive ActiveRecord pattern:

# DON'T: read, mutate in Ruby, write back — two concurrent requests
# both read 5, both write 6, you lost an increment.
product.stock -= 1
product.save

# DO: atomic decrement on the server, race-free
product.inc(stock: -1)

# append to an embedded array atomically
order.push(line_items: { sku: "ABC", quantity: 1 })

# add to a set only if not already present
user.add_to_set(roles: "admin")

$inc, $push, $addToSet, $pull, and the positional $ operator (update the matching element of an embedded array) are the tools that make a document store safe under concurrency. Reach for them whenever more than one process might touch the same document — counters, stock levels, membership sets. They are also dramatically faster than load-mutate-save because the document never makes the round trip to Ruby.

Indexes: Mongo will scan a million documents and never complain

This is the failure mode that has bitten us hardest, because it is silent. MongoDB will happily perform a full collection scan on an unindexed query. On your 200-document development database it is instant. On the two-million-document production collection it is a multi-second query that pins a CPU, and Mongo’s only protest is a line in the slow-query log you are not yet reading.

Declare indexes explicitly in the model and check them in:

class Order
  include Mongoid::Document
  field :status,    type: String, default: "pending"
  field :placed_at, type: Time

  index({ placed_at: -1 })                       # recent-orders queries
  index({ status: 1, placed_at: -1 })            # compound: filter + sort
  index({ "line_items.sku" => 1 })               # multikey: into the array
  index({ customer_email: 1 }, { unique: true }) # uniqueness at the db level
end

A few things worth internalising. Compound index order matters: an index on { status: 1, placed_at: -1 } serves a query that filters on status and sorts by placed_at, but not the reverse — the prefix rule is the same as SQL composite indexes. Indexes into embedded arrays ("line_items.sku") are “multikey” and just work, which is one of the quiet joys of embedding. And uniqueness must be enforced by the database, not only by a Mongoid validation — a validation has a check-then-write race that lets duplicates slip in under concurrency; a unique index does not.

Then actually build them on deploy:

rake db:mongoid:create_indexes

Make this part of the deploy, not a thing someone remembers to run. Use .explain in a console to confirm a query uses the index you think it does — the difference between COLLSCAN and IXSCAN in the output is the difference between an incident and a non-event.

Schemaless does not mean structureless

The seductive pitch of MongoDB is “no migrations”. It is half true, and the other half is where teams hurt themselves.

You really can add a field by declaring it in the model, and yesterday’s documents simply will not have it. When you are moving fast and the shape of the data is still in flux, that is a genuine accelerant — no migration to write, no table to lock, no deploy ordering to choreograph. But “the database will accept anything” is not the same as “your code will cope with anything”. Old documents return nil for fields added later, and nil propagates silently into views, calculations, and serialised JSON until something explodes far from the cause.

The discipline that keeps schemaless from turning into chaos:

class Order
  include Mongoid::Document
  field :status,   type: String, default: "pending"
  field :channel,  type: String, default: "web"   # new field, safe default

  validates :status, presence: true, inclusion: %w[pending paid shipped]
end
  • Always give new fields a sensible default, or guard every read of them.
  • Validate at the model layer on purpose. The database will not. If status must be one of a fixed set, say so in the model — Mongo certainly will not stop you writing "banana".
  • Run a one-off data-migration script anyway when meaning changes. Schemaless removes the forced migration, not the need for one. If you rename a field or change how a value is interpreted, write a script that walks the collection and rewrites old documents. “The old documents are still in the old shape” is a bug waiting for the unlucky read.
  • Consider a schema_version field on long-lived, much-changed documents so you can branch on it during a gradual migration.

Consistency boundaries: design around the lack of transactions

Here is the constraint that most shapes a Mongo data model, and it is easy to miss until it bites: in 2013 there are no multi-document transactions. A single document update is atomic — all of it lands or none of it does — but the moment an operation spans two documents (or two collections), there is no BEGIN ... COMMIT to make them succeed or fail together.

This is not a flaw so much as a forcing function, and it is the deepest reason to take aggregate boundaries seriously. If two pieces of data must change together atomically, that is a strong signal they belong in the same document. An order and its line items change together, so they live together, and a single atomic update keeps them consistent. Conversely, if two things genuinely live in separate documents, you must design for the possibility that one write succeeds and the other does not — with idempotent operations, a reconciliation job, or a deliberate “eventually consistent” stance — exactly the work a SQL transaction would have done for you for free.

When we find a domain riddled with invariants that span many entities — “this is only valid if these five things across four collections all agree” — that is the clearest signal we have left MongoDB’s comfort zone and should be in Postgres.

Reporting: the aggregation pipeline

The other thing you give up with no joins is ad-hoc relational reporting. Mongo’s answer is the aggregation pipeline — a sequence of stages ($match, $group, $sort, $project) that the server executes:

Order.collection.aggregate([
  { "$match"  => { status: "paid" } },
  { "$unwind" => "$line_items" },
  { "$group"  => { _id: "$line_items.sku",
                   units: { "$sum" => "$line_items.quantity" } } },
  { "$sort"   => { units: -1 } },
  { "$limit"  => 10 }
])

It is powerful and it runs server-side, but it is its own language with its own quirks, and complex reporting in it is markedly harder than the SQL GROUP BY you already know. This, too, is a useful signal: if the bulk of a system’s value is in cross-entity reporting, that system probably wanted a relational database underneath it.

What we actually like

After the adjustment, there is a lot to love. Embedded documents map beautifully to aggregates — load and save the whole tree as a unit, with the invariants intact. The atomic operators make concurrent counters and sets genuinely easy. The schemaless freedom is a real accelerant in the early, exploratory phase of a project, when you do not yet know what the data wants to be and an over-eager migration is just friction. For storing and serving self-contained documents — user profiles, content, event payloads, anything you read as a whole — Mongo is fast, pleasant, and a good fit.

What we watch for

We reach for Postgres the moment a domain is fundamentally relational: when the questions we ask span many entities, when reporting needs ad-hoc joins, when we want real foreign-key integrity, and when correctness depends on transactions across more than one document. MongoDB is excellent at storing and serving aggregates; it is not a relational database wearing a costume, and pretending otherwise is how teams end up reimplementing joins, transactions, and constraints by hand in Ruby — slower, buggier, and with none of the guarantees the database would have given them.

The headline lesson is a single mental flip. In SQL you normalise first and optimise the reads later; in MongoDB you model the read first and accept the write cost. Internalise that inversion and Mongoid is a genuine pleasure to work with. Fight it — drag your table-and-join instincts across unchanged — and you will spend your evenings debugging a join the database was never going to do for you.

#ruby #rails #mongodb