MongoDB Bulk Writes

Writing 200,000 documents one updateOne at a time means 200,000 round trips. At 1 ms each that’s over three minutes spent waiting on the network before the server does any real work. bulkWrite lets you send the same operations in large batches, and the job finishes in seconds. The speed is the easy part. What interviewers probe is what happens when operation 73,412 fails.

The shell output in this note is illustrative: it follows what mongosh prints against MongoDB 7.0 and 8.0, but it wasn’t captured from a live server.

What one bulkWrite call contains

db.collection.bulkWrite() takes an array of operations, each wrapped in its type:

db.products.bulkWrite([
  { insertOne:  { document: { sku: "A-1", price: 1200, stock: 4 } } },
  { updateOne:  { filter: { sku: "A-2" }, update: { $set: { price: 950 } } } },
  { updateMany: { filter: { discontinued: true }, update: { $set: { stock: 0 } } } },
  { replaceOne: { filter: { sku: "A-3" }, replacement: { sku: "A-3", price: 400, stock: 10 } } },
  { deleteOne:  { filter: { sku: "A-4" } } },
  { deleteMany: { filter: { stock: 0, discontinued: true } } }
], { ordered: true })

The result tells you what each kind of operation did:

{
  acknowledged: true,
  insertedCount: 1,
  insertedIds: { '0': ObjectId('...') },
  matchedCount: 3,
  modifiedCount: 3,
  deletedCount: 2,
  upsertedCount: 0,
  upsertedIds: {}
}

Notice that insertedIds is keyed by position in the array ('0'), not by anything in the document. Positions matter a lot once errors appear.

Under the hood, the driver or shell turns your array into insert, update and delete commands. Each server command accepts up to 100,000 operations (maxWriteBatchSize) and a message of up to 48 MB, and the driver splits larger arrays automatically. In ordered mode it groups only consecutive operations of the same type, so an array that alternates insert, update, insert, update becomes many small commands. If you control the order, group operations by type.

Ordered versus unordered

The ordered option decides what happens after a failure.

Ordered (the default) executes operations in sequence and stops at the first error. Everything before it has been applied; everything after it is never attempted.

Unordered (ordered: false) lets the server execute operations in any order, possibly in parallel on a sharded cluster, and keeps going after errors. You get a report of every failure at the end.

Take five inserts into a collection with a unique index on sku, where A-2 already exists:

db.products.createIndex({ sku: 1 }, { unique: true })

const ops = ["A-1", "A-2", "A-3", "A-4", "A-5"].map((sku) => ({ insertOne: { document: { sku } } }))

The two modes end very differently:

• Ordered: A-1 is inserted, A-2 fails with E11000, and A-3, A-4 and A-5 are skipped. One document written.
• Unordered: A-1, A-3, A-4 and A-5 are inserted, and the error for index 1 is reported. Four written.

In mongosh, the unordered case throws something like:

Uncaught:
MongoBulkWriteError: E11000 duplicate key error collection: shop.products index: sku_1 dup key: { sku: "A-2" }
Result: BulkWriteResult {
  insertedCount: 4,
  matchedCount: 0,
  ...
  writeErrors: [ WriteError { err: { index: 1, code: 11000, errmsg: 'E11000 duplicate key error ...' } } ]
}

Choose ordered when later operations depend on earlier ones, for example inserting a document and then updating it in the same batch. Choose unordered when operations are independent, which is the usual case for imports and sync jobs. It’s faster and one bad row doesn’t block the rest.

A bulk write is not a transaction

People new to MongoDB often assume bulkWrite is all-or-nothing. It isn’t. Each individual operation is atomic on its own document, but there’s no rollback across operations. If the call fails halfway, the first half stays written. And if the connection drops mid-batch, you may not know exactly which operations ran.

If you truly need all-or-nothing, run the bulk write inside a multi-document transaction. That brings its own limits: transactions default to a 60-second lifetime (transactionLifetimeLimitSeconds) and hold locks and cache pressure for the whole duration, so a 200,000-operation transaction is a bad idea. For large jobs, the better design is to make the operations safe to repeat.

A worked example: syncing supplier prices every night

A supplier sends a file of 240,000 SKUs with prices and stock. The job must add new products, update existing ones, and be safe to rerun if it crashes. Here’s the core with the Node.js driver (v6):

import { MongoClient, MongoBulkWriteError } from 'mongodb';

const client = new MongoClient(process.env.MONGO_URL);
const products = client.db('shop').collection('products');
await products.createIndex({ sku: 1 }, { unique: true });

async function syncBatch(rows) {
  const ops = rows.map((r) => ({
    updateOne: {
      filter: { sku: r.sku },
      update: {
        $set: { price: r.price, stock: r.stock, syncedAt: new Date() },
        $setOnInsert: { createdAt: new Date() },
      },
      upsert: true,
    },
  }));
  try {
    const res = await products.bulkWrite(ops, { ordered: false });
    return { upserted: res.upsertedCount, modified: res.modifiedCount, failed: [] };
  } catch (err) {
    if (!(err instanceof MongoBulkWriteError)) throw err; // network error, auth failure, etc.
    const failed = err.writeErrors.map((e) => ({ sku: rows[e.index].sku, code: e.code, msg: e.errmsg }));
    return { upserted: err.result.upsertedCount, modified: err.result.modifiedCount, failed };
  }
}

for (let i = 0; i < allRows.length; i += 1000) {
  const summary = await syncBatch(allRows.slice(i, i + 1000));
  if (summary.failed.length) await logFailures(summary.failed);
}

The decisions:

• Upsert on a unique key makes every operation idempotent. Running the same batch twice leaves the same final state, so a crashed job can simply restart from the beginning or from the last batch it logged. The unique index matters: two concurrent upserts on a field without one can both insert, leaving duplicate products.
• $setOnInsert sets createdAt only when the upsert creates a document, so reruns don’t overwrite it.
• ordered: false because rows are independent. A malformed price on one row shouldn’t stop the other 999.
• Batches of 1,000 keep each call’s memory, latency and failure scope small, and give natural progress checkpoints. The driver would split a single 240,000-operation array correctly, but you’d lose the checkpoints.
• Map e.index back to the input row so the failure log names the SKU, not “operation 412”. This is the step most implementations skip.
• Separate per-operation errors from total failures. MongoBulkWriteError means the server processed the batch and some operations failed. Any other error (a timeout, a failover) means you don’t know the outcome, which is exactly when idempotency pays off: retry the whole batch.

Retryable writes help too. With them enabled (the default in modern drivers), a bulk write made only of single-document operations (insertOne, updateOne, replaceOne, deleteOne) is retried once automatically after a transient error such as a primary election. A batch containing updateMany or deleteMany isn’t retryable.

Writing to several collections in one call (MongoDB 8.0)

MongoDB 8.0 added a cluster-level bulkWrite command, exposed in mongosh as Mongo.bulkWrite(), where each operation names its own namespace:

db.getMongo().bulkWrite([
  { insertOne: { namespace: "shop.orders", document: { _id: 9001, sku: "A-1", qty: 2 } } },
  { updateOne: { namespace: "shop.products", filter: { sku: "A-1" }, update: { $inc: { stock: -2 } } } }
], { ordered: true })

That saves round trips when one logical change touches several collections. It’s still not a transaction: if the second operation fails, the order exists and stock wasn’t decremented. On 7.0 you’d send one bulkWrite per collection, and for a pair like this you’d usually want a transaction anyway.

Interview exercise: the half-finished import

An import job calls insertMany with 50,000 new customer documents, using the default options. It fails with an E11000 error on email at index 18,204. The developer fixes the duplicate and reruns the whole job, which now fails at index 0. Explain both failures and how you’d redesign the job.

Answer

insertMany is ordered by default, so the first run inserted documents 0 to 18,203, failed on 18,204, and never attempted the remaining 31,795. The rerun starts again at document 0, which now already exists, so it fails immediately on the unique index (on _id if the IDs are fixed, or on email otherwise). Nothing was rolled back, which is the misunderstanding behind the rerun.

The redesign makes the job idempotent and failure-tolerant. Use bulkWrite with updateOne plus upsert: true keyed on the unique email, using $setOnInsert for the fields that should only be written once. Run it with ordered: false so one bad record doesn’t block the others, and map each writeErrors[i].index back to its input row to log genuinely bad data. Process in batches of a few thousand and record progress, so a crash can resume or simply rerun safely. If the business needs “all 50,000 or none”, that’s a staging-collection problem: import into a new collection, validate it, then swap it in, rather than a 50,000-document transaction.

Related reading

• MongoDB interview questions
• MongoDB quiz questions
• Write concern in MongoDB
• Unique indexes in MongoDB
• db.collection.bulkWrite() in the MongoDB manual