$push to sorted array

Aug 20 • Posted 11 months ago

By Sam Weaver, MongoDB Solutions Architect and Alberto Lerner, MongoDB Kernel Lead

MongoDB 2.4 introduced a feature that many have requested for some time - the ability to create a capped array.

Capped arrays are great for any application that needs a fixed size list. For example, If you’re designing an ecommerce application with MongoDB and want to include a listing of the last 5 products viewed, you previously had to issue a $push request for each new item viewed, and then a $pop to kick the oldest item out of the array. Whilst this method was effective, it wasn’t necessarily efficient. Let’s take an example of the old way to do this:

First we would need to create a document to represent a user which contains an array to hold the last products viewed:

db.products.insert({last_viewed:["bike","cd","game","bike","book"]})
db.products.findOne()
{
    "_id" : ObjectId("51ff97d233c4f2089347cab6"),
    "last_viewed" : [
        "bike",
        "cd",
        "game",
        "bike",
        "book"
    ]
}

We can see the user has looked at a bike, cd, game, bike and book. Now if they look at a pair of ski’s we need to push ski’s into the array:

db.products.update({},{$push: {last_viewed: "skis"}})
db.products.findOne()
{
    "_id" : ObjectId("51ff97d233c4f2089347cab6"),
    "last_viewed" : [
        "bike",
        "cd",
        "game",
        "bike",
        "book",
        "skis"
    ]
}

You can see at this point we have 6 values in the array. Now we would need a separate operation to pop “bike” out:

db.products.update({},{$pop: {last_viewed: -1}})
db.products.findOne()
{
    "_id" : ObjectId("51ff97d233c4f2089347cab6"),
    "last_viewed" : [
        "cd",
        "game",
        "bike",
        "book",
        "skis"
    ]
}

In MongoDB 2.4, we combined these two operations to maintain a limit for arrays sorted by a specific field.

Using the same example document above, it is now possible to do a fixed sized array in a single update operation by using $slice:

db.products.update({},{$push:{last_viewed:{$each:["skis"],$slice:-5}}})

You push the value ski’s into the last_viewed array and then slice it to 5 elements. This gives us:

db.products.findOne()
{
    "_id" : ObjectId("51ff9a2d33c4f2089347cab7"),
    "last_viewed" : [
        "cd",
        "game",
        "bike",
        "book",
        "skis"
    ]
}

Mongo maintains the array in natural order and trims the array to 5 elements. It is possible to specify to slice from the start of the array or the end of the array by using positive or negative integers with $slice. It is also possible to sort ascending or descending by passing $sort also. This helps avoid unbounded document growth, and allows for the event system to guarantee in-order delivery.

There are lots of other applications for this feature:

  • Keeping track of the newest messages in a messaging system
  • Managing recent clicks on a website Last accessed/viewed products
  • Top X users/comments/posts

The list goes on.

This feature is available in MongoDB 2.4, but there are many extensions requested, such as full $sort and $slice semantics in $push (SERVER-8069), and making the $slice operation optional (SERVER-8746). Both of these are planned for the 2.5 development series.

Special thanks to Yuri Finkelstein from eBay who was very enthusiastic about this feature and inspired this blog post.

blog comments powered by Disqus
blog comments powered by Disqus