Posts tagged:


"Partial Object Updates" will be an Important NoSQL Feature

Dec 30 • Posted 4 years ago

It’s nice that in SQL we can do things like


We term this a “partial object update”: we updated the value of X without sending a full row update to the server.

Seems like a very simple thing to be discussing, yet some nosql solutions do not support this (others do).

In these new datastores, the average stored object size (whether it be a document, a key/value blob, or a row) tends to be larger than the traditional database row.  The data is not fully normalized, so we are packing more data into a single storage object than before.

This means the cost of full updates is higher.  If we have a 100KB document and want to set a single value within it, passing the full 100KB in both directions over the network for the operation is expensive.

MongoDB supports partial updates in its update operation via a set of special $ operators: $inc, $set, $push, etc.  More of these operators will be added in the future.

There are further benefits to the technique too.  First, we get easy (single document) atomicity for these operations (consider $inc).  Second, replication is made cheaper: when a partial update occurs, MongoDB replicates the partial update rather than the full object changed.  This makes replication much less expensive and network intensive.

Fast Updates with MongoDB (update-in-place)

Nov 18 • Posted 4 years ago

One nice feature with MongoDB is that updates can happen “in place” — the database does not have to allocate and write a full new copy of the object.

This can be highly performant for frequent update use cases.  For example, incrementing a counter is a highly efficient operation.  We need not fetch the document from the server, we can simply send an increment operation over:

db.my_collection.update( { _id : ... }, { $inc : { y : 2 } } ); // increment y by 2

MongoDB disk writes are lazy.  If we receive 1,000 increments in one second for the object, it will only be written once.  Physical writes occur a couple of seconds after the operation.

One question is what happens when an object grows.  If the object fits in its previous allocation space, it will update in place.  If it does not, it will be moved to a new location in the datafile, and its index keys must be updated, which is slower.  Because of this, Mongo uses an adaptive algorithm to try to minimize moves on an update.  The database computes a padding factor for each collection based on how often items grow and move.  The more often the objects grow, the larger the padding factor will be; when less frequent, smaller.

See also:

blog comments powered by Disqus