The MongoDB NoSQL Database Blog

Operations in the New Aggregation Framework

Tue, 17 Jan 2012 14:04:00 -0500

Available in 2.1 development release. Will be stable for production in the 2.2 release

Built by Chris Westin (@cwestin63)

MongoDB has built-in MapReduce functionality that can be used for complex analytics tasks. However, we’ve found that most of the time, users need the kind of group-by functionality that SQL implementations have. This can be implemented using map/reduce, but doing so is more work than it was in SQL. In version 2.1, MongoDB is introducing a new aggregation framework that will make it much easier to obtain the kind of results SQL group-by is used for, without having to write custom JavaScript.

MongoDB provides an easy way to store documents that aren’t just flat records like SQL, but that also contain arrays and nested documents. Another common problem users have told us about is that it can be difficult to manipulate these sub-parts of documents easily. The new aggregation mechanism is also taking on making this easier, by providing operators that can be used to disassemble and re-assemble document sub-parts.

After many requests, we have also included expression evaluation to return computed values—for virtual fields.

The new aggregation framework is declarative. Rather than writing code to aggregate data, you specify a chain of operations, called a “pipeline,” to apply to your documents. This is similar to a pipe in a unix shell.

We’re now going to show you a few short examples (using the mongo shell) of this new functionality to get you started.

$match is a pipeline operator that filters out documents that don’t match the specified condition. For example


db.runCommand({ aggregate : "article", pipeline : [
    { $match : { author : "dave" } }
]});

This very simple one step pipeline doesn’t introduce any new functionality yet, but shows how to put a query at the beginning of your pipeline. For those of you familiar with MongoDB, this is the equivalient of


> db.article.find({ author : "dave" });

$project is a pipeline operator that let’s you select which fields you want to include or exclude from a result. You can also create new virtual fields from computed values. Here is a simple example, building on the previous one:


db.runCommand({ aggregate : "article", pipeline : [
    { $match : { author : "dave" } },
    { $project : {
        _id : 0,
	author : 1,
        tags : 1
    }}
]});

As before, this doesn’t introduce anything radically new; this is the equivalent of


> db.article.find({ author : "dave" }, { _id : 0, author : 1, tags : 1);

While this functionality was available before from find(), what it does demonstrate here is how to put together more than one aggregation pipeline operator. The value of pipeline is an array consisting of the operators to apply. Conceptually, the aggregation framework behaves as if it was feeding the result of scanning the target collection through that pipeline.

Now let’s take a look at some new functionality that wasn’t available before.

$unwind hands out the elements of an array one at a time; each element is surrounded by the document that contains the original array. For this example, we’ll first show what a document from the article collection looks like. Suppose we had saved this:


db.article.save( {
    title : "this is your title" , 
    author : "dave" , 
    posted : new Date(4121381470000) , 
    pageViews : 7 , 
    tags : [ "fun" , "nasty" ] ,
    comments : [ 
        { author :"barbara" , text : "this is interesting" } , 
        { author :"jenny" , text : "i like to play pinball", votes: 10 } 
    ],
    other : { bar : 14 }
});

Now, we can unwind the tags array as follows:


db.runCommand({ aggregate : "article", pipeline : [
    { $unwind : "$tags" }
]});

The result looks like this:


{
        "result" : [
                {
                        "_id" : ObjectId("4eeeb5fef09a7c9170df094b"),
                        "title" : "this is your title",
                        "author" : "dave",
                        "posted" : ISODate("2100-08-08T04:11:10Z"),
                        "pageViews" : 7,
                        "tags" : "fun",
                        "comments" : [
                                {
                                        "author" : "barbara",
                                        "text" : "this is interesting"
                                },
                                {
                                        "author" : "jenny",
                                        "text" : "i like to play pinball",
                                        "votes" : 10
                                }
                        ],
                        "other" : {
                                "bar" : 14
                        }
                },
                {
                        "_id" : ObjectId("4eeeb5fef09a7c9170df094b"),
                        "title" : "this is your title",
                        "author" : "dave",
                        "posted" : ISODate("2100-08-08T04:11:10Z"),
                        "pageViews" : 7,
                        "tags" : "nasty",
                        "comments" : [
                                {
                                        "author" : "barbara",
                                        "text" : "this is interesting"
                                },
                                {
                                        "author" : "jenny",
                                        "text" : "i like to play pinball",
                                        "votes" : 10
                                }
                        ],
                        "other" : {
                                "bar" : 14
                        }
                }
        ],
        "ok" : 1
}

Notice that the single document has been duplicated (see the identical _id values). However, the tags array in each copy of the document has been replaced with one value from the original tags array.

$unwind isn’t useful on its own, but is useful when combined with $match, which can be used to filter out copies of the document without values of interest, and then the array can be reassembled using other operators such as $push or $addToSet. $unwind is also useful when combined with $group.

$group groups elements with a common key together, and allows the application of aggregation functions. For the purposes of a small example, let’s add a second article to our collection:


db.article.save( {
    title : "this is some other title" , 
    author : "jane" , 
    posted : new Date(978239834000) , 
    pageViews : 6 , 
    tags : [ "nasty" , "filthy" ] ,
    comments : [ 
        { author :"will" , text : "i don't like the color" } , 
        { author :"jenny" , text : "can i get that in green?" } 
    ],
    other : { bar : 14 }
});

If these articles represent content on a web site, I might want to know what tags users are applying to these articles. I can use $group to find that out like so:


db.runCommand({ aggregate : "article", pipeline : [
    { $unwind : "$tags" },
    { $group : {
	_id : "$tags",
        count : { $sum : 1 },
	authors : { $addToSet : "$author" }
    }}
]});

The grouping key for a $group is defined by specifying a field (or subdocument of multiple fields) for the _id. The grouping key is used to define buckets into which documents are put. Other fields are aggregate functions that can compute values across the entries in each bucket.

In this case, one is added to count for each document that matches the group key. That counts up the number of items in each bucket. authors will be an array, and each unique author in the bucket will be added to it.

The result of all this is


{
        "result" : [
                {
                        "_id" : "filthy",
                        "count" : 1,
                        "authors" : [
                                "jane"
                        ]
                },
                {
                        "_id" : "fun",
                        "count" : 1,
                        "authors" : [
                                "dave"
                        ]
                },
                {
                        "_id" : "nasty",
                        "count" : 2,
                        "authors" : [
                                "jane",
                                "dave"
                        ]
                }
        ],
        "ok" : 1
}

One document has the “filthy” tag, and it was written by jane. One document has the “fun” tag, and it was written by dave. Two documents have the “nasty” tag, and both jane and dave wrote documents with that tag.

The new aggregation functionality is available only by compiling from source. It is available in the 2.1 development release but is not meant for production. MongoDB 2.2 will be the first stable release with the aggregation framework. For more examples and information, check out the talk and demo from MongoSV 2011, or the documentation on the MongoDB wiki.

by Chris Westin

MongoSV Recap

Fri, 16 Dec 2011 14:57:00 -0500

Last week over 1,100 developers came together for MongoSV, the largest MongoDB conference to date. 10gen kicked off MongoSV with our inaugural MongoDB Masters program, which brought together MongoDB evangelists from around the world.

At the opening keynote, 10gen CTO Eliot Horowitz demoed a twitter app for #mongoSV tweets, featuring the new aggregation framework expected for the MongoDB 2.2 release. These gather all the tweets sent out with the hashtag #mongoSV and organizes them in by recency and most retweets. Get the source code for the demo app here

Highlights from MongoSV include presentations on X.commerce’s new open source developer platform, MongoDB’s integration with Azure, MongoDB’s new aggregation framework, How Disney manages their deployment of 1400 Mongo instances and more

the 10gen booth at MongoSV

10gen President Max Schireson welcomes the Speakers and Masters to MongoSV

MongoDB On Microsoft Azure

Thu, 01 Dec 2011 12:54:00 -0500

A new preview release of the MongoDB controller for Azure is available. This release includes support for replica sets, and over the coming months, we’ll be adding support for MongoDB’s sharding facilities. We’ll also be working to more tightly integrate MongoDB with the features of Azure platform.

Each member of a replica set is hosted by an instance of an Azure worker role, so the size of the replica set is determined by the number of instances configured for the the replica set worker roles. Each replica set worker role creates a child process to run the mongod server process.

The controller defines an Azure worker role which represents a MongoDB cluster. Currently the cluster consists of a single MongoDB replica set. Each role instance participates as a member of the replica set. On deployment the controller configures the replica set and initiates it. Role instances handle restarts, including assuming identity for a replica set member and replacing old instances. The worker role also integrates with Azure Diagnostics to capture trace and performance counter information.

The MongoDB Azure wrapper release is currently delivered as a Visual Studio 2010 solution with associated source files, and we’ll follow that up in the future with packaging and deployment tools. To get started, take a look at the documentation or browse through the source. We’ve included a small sample application to get you up and running quickly. If you have questions, head over to our user mailing list or file a bug report on our JIRA system.

Microsoft will be presenting at the MongoSV conference in Santa Clara on December 9th. It will be a great opportunity to understand the solution in depth, as well as understand the future direction of this project. We hope to see you there.

MongoDB Monitoring Service Docs Available

Thu, 03 Nov 2011 14:53:48 -0400

MongoDB Monitoring Service (MMS) documentation now available: http://mms.10gen.com/help/

Mongo Boston Recap

Tue, 11 Oct 2011 13:42:56 -0400

Last week 250 developers converged at the Microsoft New England Research and Design Center for Mongo Boston. Highlights from the event include presentations on MongoDB 2.0, how MTV leverages MongoDB for CMS, rapid prototyping, and more.

More photos from the event are available on the MongoDB Flickr page.

If you missed the event, join the Boston MongoDB User Group, which also meets at NERD. The next meeting is on November 15.

2.0 Presentation at New York MongoDB User Group

Tue, 20 Sep 2011 15:46:32 -0400

On Thursday MongoDB core committer Eliot Horowitz presented to the New York MongoDB User Group on the latest features in v2.0. The event was hosted by Sailthru, a MongoDB-powered startup doing intelligent email marketing. The meetup was announced Monday night and within a day was oversubscribed. After the presentation, we all went out for drinks to celebrate the release.

The NY MUG has over 1,000 members and meets monthly. There are also MongoDB user groups in San Francisco, Washington DC, London, Boston, Japan, and more. And if there isn’t a MongoDB meetup in your city, we’re happy to support you if you would like to start one.

In case you missed the meetup, video and photos are posted below. Enjoy!

Cache Reheating - Not to be Ignored

Mon, 19 Sep 2011 13:56:05 -0400

An important aspect to keep in mind with databases is the cost of cache reheating after a server restart. Consider the following diagram which shows several cache servers (e.g., memcached) in front of a database server.

This sort of setup is common and can work quite well when appropriate; it removes read load from the database and allows more RAM to be utilized for scaling (when the database doesn’t scale horizontally). But what happens if all the cache servers restart at the same time, say, on a power glitch in a data center?

We then have a cache reheating scenario. After the bounce the full load of all read requests will hit the database server (for a while) given the caches are empty. The server won’t be able to handle it (that’s why the cache servers were there in the first place). Now if the reheat time is short, this is not a big problem : we did go down after all. But if reheat takes a long time, it’s a big problem. Imagine 20 cache servers with 64GB RAM each. 1.2 terabytes of data must be queried from the database to be fully reheated!

Even without a cache server, the same issue exists for almost all databases. Imagine a server restart on a system with 64GB RAM. If loading 100MB/sec, reheat will take 10 minutes. However, if the queries are coming in randomly, it could take much much longer — only with sequential I/O can we get anywhere near that speed.

One could just sequentially read data in and fill the cache. However for databases much larger than RAM, loading in the right portion is difficult. “Hot” data could be at very different locations in the terabyte(s) of on disk data.

A very nice attribute of the MongoDB storage engine is its use of memory-mapped files. In this model the cache is the operating system’s file system cache. Restart the mongod process, and there is no reheat issue at all. Some databases use their own page cache which causes a reheat scenario even on just a process restart. Of course on a full server reboot MongoDB must reheat too.

A few points to keep in mind:

Think about reheating and how you will operationally handle it if a scenario involving it occurs. For schedules OS maintenance restart the server during off peak hours to minimize the load during reheat.
Restarting the mongod process is safe with respect to reheating.
Remounting a volume likely loses all file system cache info for the volume.
On a server restart, copy datafiles to /dev/null to force reheating to be sequential and thus much faster. This can be done even if the mongod process is already running. If the database is larger than RAM, copy only the newest datafiles (ones with the highest numbers); while this isn’t perfect, the latest files likely contain the largest percentage of frequently used data.

MongoDB 2.0 Released

Mon, 12 Sep 2011 11:32:22 -0400

The MongoDB development team is pleased to announce the release of version 2.0.0. Version 2.0 is the latest stable release, following the March 2011 release of version 1.8. This release includes many new features, improvements to existing features, and performance enhancements.

Please note version 2.0 is a significant new release, but is 2.0 solely because 1.8 + 0.2 = 2.0; for example the upgrade from 1.6 to 1.8 was similar in scope.

Highlights of the 2.0 release:

Concurrency improvements
Index enhancements to improve size and performance
Authentication with sharded clusters
Replica Set Data Center Awareness
Journaling is now enabled by default. Journal files are now compressed.
User defined getLastError options
Smaller default stack size to accommodate more connections
Compact Command
Geo-spatial features
- Multi-location documents
- Polygon searches
Output map/reduce results to sharded collection

Concurrency improvements in 2.0 are just the start of a much larger concurrency roadmap we are working on. In 2.0, we are beginning to address one of the biggest issues: holding locks during a page fault. 2.0 tracks memory caching and has the ability to yield locks and fault outside. This is hooked in a number places, notably: updates by _id, remove, and long table scans.

Being able to keep the working index set in memory is an important factor in overall performance, and we overhauled indexes to make this easier. Indexes in 2.0 are about 25% smaller and faster, meaning that you can fit more in memory.

Replica sets get two new important features in 2.0: priorities and tagging. Priorities let you have nodes that you prefer to be primary if you have a non homogeneous environment. Tagging lets you guarantee writes hit certain groups of servers. One use case for this is guaranteeing a new user registration is written to two data centers before acknowledging to a user.

There were many other improvements, so we encourage those interested to look at the change log. Overall, this release added a large number of small performance and concurrency improvements, greater stability to sharding, and better replica set operations.

Downloads: http://www.mongodb.org/downloads

Release Notes: http://www.mongodb.org/display/DOCS/2.0+Release+Notes

Full change log: https://jira.mongodb.org/secure/IssueNavigator.jspa?requestId=10140

For the full scoop on what’s new in MongoDB version 2.0, register for our live webinar on Thursday, September 15th. We will have two sessions: the first at 10am GMT, and another at 1:30pm ET/10:30am PT.

Thank you to the MongoDB community for your continued feedback and testing through the 1.9 development series.

-Eliot and the MongoDB Team

BSON and Data Interchange

Wed, 24 Aug 2011 09:52:00 -0400

There’s a lot of good things about JSON — it’s a standards based, language independent, representation of object-like data. Also, it’s easy to read (for users and programmers alike). Each document is only about data, not complex object graphs and links. Thus it’s easy to inspect without knowing all the code of an application.

Further, JSON is “schemaless”. We do not have to predefine our (protocol) schema. This can be quite helpful: imagine RPC’ing data from client A to server B with a fixed schema for the messages. On a schema change both need to be ‘updated’ with the new schema. If there are many components to the system it’s even more complicated of course. There is some analogy here to XML, which can (optionally) be schemaless.

It would be nice to have a binary representation of JSON. That is what BSON is all about.

So what are the goals of BSON? They are:

Fast scan-ability. For very large JSON documents, scanning can be slow. To skip a nested document or array we have to scan through the intervening field completely. In addition as we go we must count nestings of braces, brackets, and quotation marks. In BSON, the size of these elements is at the beginning of the field’s value, which makes skipping an element easy.
Easy manipulation. We want to be able to modify information within a document efficiently. For example, incrementing a field’s value from 9 to 10 in a JSON text document might require shifting all the bytes for the remainder of the document, which if the document is large could be quite costly. (Albeit this benefit is not comprehensive: adding an element to an array mid-document would result in shifting.) It’s also helpful to not need to translate the string “9” to a numeric type, increment, and then translate back.
Additional data types. JSON is potentially a great interchange format, but it would be nice to have a a few more data types. Most importantly is the addition of a “byte array” data type. This avoids any need to do base64 encoding, which is awkward.

One thing that is not a goal of BSON: compactness. The JSON document {“field”:7} represents the number seven as a single byte. That’s pretty good.

Perhaps the best example to date of usage of BSON is MongoDB. MongoDB uses it heavily — for sending documents over the network, persisting them to disk, as well as for internal data manipulations. In fact this is where BSON originated, although today it is a separate spec that should not be considered coupled to any one particular project.

There is BSON serialization and deserialization code for most languages; implementations are open source and mostly available under Apache 2 license. Quite a few implementations originated from a MongoDB drivers; work is underway in most drivers to fully decouple, although independent use works fine today.

If you or someone you know is using BSON in a project, please let us know by posting on the BSON mailing list. Check out bsonspec.org for more information.

— Spencer & Dwight

Master Detail Transactions in MongoDB

Mon, 11 Jul 2011 11:17:00 -0400

In relational databases, transactions let you make reliable atomic updates to your data. Because relational schemas are often highly normalized, most logical transactions span multiple tables, so it is important to be able to do multiple updates atomically (all or nothing).

While MongoDB does not have multi-document transactions, it makes up for this in many use cases through its document oriented data model. In this post, we’ll talk about the Master-Detail design pattern that comes up very often in data modelling that almost always requires multi-statement transactions in an RDBMS, but is easily handled without cross-statement transactions in MongoDB.

Master-Detail transactions in an RDBMS

As an example of the Master-Detail pattern, consider a Purchase Order with multiple line items. In an RDBMS, we might model this as a Purchase Order table (the Master) and a Line Item table (the Detail). To get a purchase order, I need to join Purchase Order and Line Item tables together to get all of the info in the purchase order.

I might model my Purchase Orders as follows in an RDBMS:

If I want to make atomic updates to a purchase order and its line items, I need a multi-statement transaction. For example, if I am going to create a purchase order, I might follow these steps:

With this update, there is never a time where the Purchase Order exists but has no Line Items in it. The whole object and its details are committed in a single transaction.

Now if I need to update that Purchase Order, say to add a few more line items, then I would perform another transaction.

This time I’ve ensured that my two new Line Items appear at the same time (or not at all) and that the total field of the Purchase Order is updated at the same time. No client will ever see a state in which only one of those Line Items exists nor any state where the total does not match the sum of the line items.

Master-Detail in MongoDB

Working with MongoDB is a bit different. While we don’t have the ability to perform multi-documents transactions (at least so far). However this Master-Detail pattern can be handled without multi-statement transactions thanks to MongoDB’s richer data model.

MongoDB can store data as documents with nested objects, arrays, and other rich data types. Thus we don’t need to store Master-Detail entities in multiple collections, or even in more than one document. A common way of modeling such objects with MongoDB is using nested documents. So our Purchase Order model might look like this:

Let’s look at how we can get the same level of atomicity from MongoDB without needing multi-statement transactions!

First, we want to be able to create a new purchase order and its first line items atomically.

This atomically creates the purchase order and its initial items in a single operation. Just as with the SQL scenario, clients will never see a point in time where the purchase order is empty. It all succeeds in a single step.

Now what about modifying that purchase order? If we want to add some items to the PO, we can do so like this:

The $pushAll operator atomically appends values onto an array attribute. Just as with our RDBMS scenario, this update is atomic and the whole command either succeeds or fails. Meanwhile the $inc operator atomically increments the “total” field of the purchase order. All of these updates happen atomically and they succeed or fail as a group so another client will never see an inconsistent state of the order.

Summary

It turns out that most of the time where you find yourself with a Master-Detail pattern in an RDBMS, you can achieve the same level of consistency in MongoDB by modelling your object as a rich, nested document, rather than multiple joined tables. Combine this with MongoDB’s atomic update operators, and you can solve most of what you would traditionally do with multi-statement transactions in an RDBMS.

An RDBMS needs multi-statement transactions for these scenarios because the only way it has to model these types of objects is with multiple tables. By contrast, MongoDB can go much further with single-statement transactions because there’s no need to join on simple updates like this.

This is not to say that multi-statement transactions are not useful. If you need to perform cross-entity transactions (e.g. move a line item from one purchase order to another) or if you need to modify a purchase order and inventory objects in a single step, then you may still need multi-statement transactions. Or else you would have to take some alternate approach.

But it turns out that many of the cases where we traditionally need multi-statement transactions go away when we can model objects as documents and perform atomic updates on those documents.

Another aspect of transactions and ACID is isolation. MongoDB does not support fully generalized snapshotting. We haven’t discussed that here; it’s probably a good topic for another blog post in the future.

— Jared Rosoff (@forjared)

Design of the Erlang Driver for MongoDB

Tue, 05 Jul 2011 14:27:00 -0400

Since November 2010, I have been writing an Erlang driver for MongoDB. After many months of work, I would consider the driver production-ready, and wanted to take this opportunity to introduce the driver and highlight a few of the design decisions. For detailed documentation with code examples please see the links at the end of this article.

BSON
At the highest level, the driver is divided into two library applications, mongodb and bson. BSON is defined independently of MongoDB at bsonspec.org. One design decision was how to represent BSON documents in Erlang. Conceptually, a document is a record, but unlike an Erlang record, a BSON document does not have a single type tag. Furthermore, the same MongoDB collection can hold different types of records. So I decided to represent a BSON document as a tuple with labels interleaved with values, as in {name, <<"Tony">>, city, <<"New York">>}. An alternative would have been to represent a document as a list of label-value pairs, but I wanted to reserve lists for BSON arrays.

A BSON value is one of several types. One of these types is the document type itself, making it recursive. Several value types are not primitive, like objectId and javascript, so I had to define a tagged tuple type for each of them. I defined them all to have an odd number of elements to distinguish them from a document which has an even number of elements. For example, a javascript value looks like {javascript, {x,2}, <<"x + 1">>}. Finally, to distinguish between a string and a list of integers, which is indistinguishable in Erlang, I require BSON strings to be binary (UTF-8). Therefore, a plain Erlang string is interpreted as a BSON array of integers, so make sure to always encode your strings, as in <<"hello">> or bson:utf8("hello")

Var
The mongodb driver has a couple of objects like connection and cursor that maintain mutable state. The only way to mutate state in Erlang is to have a process that maintains its own state that it updates when it receives messages from other processes. The Erlang programmer typically creates a process for each mutable object and defines the messages it may receive and the action to take for each message. They usually provide helper functions for the clients to call that hide the actual messages being sent and returned. Erlang OTP provides a small framework called gen_server to facilitate this process definition but it is still non-trivial. To alleviate this burden I created another abstraction on top of gen_server called var. A var is an object (process) that holds a value of some type A that may change. Its basic operation is modify (var(A), fun ((A) -> {A, B})) -> B which applies the function to the current value of the var then changes the var’s value to the first item of the result while returning the second item of the result to the client. This is done atomically thanks to the sequential nature of Erlang processes. The function may perform side effects (sending/receiving messages or doing IO), in which case the var acts like a mutex since only one function can execute against the var at a time. In essence, using var or even just gen_server changes the programming paradigm from message passing to protected shared state, which is more like Haskell for example.

With var it is now very easy to create objects that protect a shared resource or have internal mutable state. A TCP connection to a MongoDB server is one such resource that needs protection. The connection object in mongodb is implemented as a var holding a TCP connection. Every read and write operation gets exclusive access to the TCP connection when sending and receiving its messages to and from the server. This allows multiple user processes to read and write to the same mongodb connection concurrently.

DB Action
Every read/write operation may throw a DB exception. Furthermore, every read/write operation requires a DB context that includes the connection to use, the database to access, whether slave is ok (read_mode), and whether to confirm writes (write_mode). We group a series of read/write operations that perform a single high-level task into a function called a DB action. We then execute the action within a single exception handler and with a single DB context in dynamic scope (using Erlang’s process dictionary). mongo:do (write_mode(), read_mode(), connection(), db(), action(A)) -> {ok, A} | {failure, failure()} sets up the context, runs the action, and catches and returns any DB failure. Note, it will not catch and return other types of exceptions like programming errors.

You may notice that a DB action is analogous to a DB transaction for a relational database in that the action aborts when one of its operations fails. However, for scalability reasons, MongoDB does not support ACID across multiple read/write operations, so the operations before the failed operation remain in effect. Your failure handler must be prepared to recover from this intermediate state. If your DB action is conceptually a single high-level task, then it should not be too hard to undo and redo that task even from an intermediate state.

Documentation
Detailed documentation with examples can be found in the ReadMe’s of the two libraries, mongodb and bson, and in their source code comments and test modules.

- Tony Hannan

Getting started with VMware CloudFoundry, MongoDB and Node.js

Thu, 16 Jun 2011 10:00:00 -0400

Listen to the recording of the Node.js Panel Discussion webinar.

Overview

Following up from our previous post we’re posting up a quick how-to for using Node.JS, CloudFoundry and MongoDB together.

Our end goal here is to build a simple web app that records visits and provides a reporting screen for the last 10 visits.

Tools We Need

Sign up for a Cloud Foundry account.
Local installation of MongoDB & Node.JS.
Cloud Foundry VMC tools.
All of the code is available on github.

Follow the links to install & configure the various tools.

Getting Started

Start the mongod process on your local computer. Use the default port.
Confirm that node.js is correctly installed. You should be able to run node from the command-line and receive a basic javascript shell. You should also have NPM (node package manager) installed.
Make a directory for the project and then ensure that CloudFoundry is correctly configured. Mine looked as follows:

mongo@ubuntu:~$ sudo gem install vmc
mongo@ubuntu:~$ vmc target api.cloudfoundry.com
Succesfully targeted to [http://api.cloudfoundry.com]

mongo@ubuntu:~$ vmc login
Email: gates@10gen.com
Password: ********
Successfully logged into [http://api.cloudfoundry.com]

Step 1: Hello world

In your directory create a file called app.js. In that file, type the following code. This will create a basic web server on localhost:3000 or on the assigned host:port combination on the CloudFoundry server.

var port = (process.env.VMC_APP_PORT || 3000);
var host = (process.env.VCAP_APP_HOST || 'localhost');
var http = require('http');

http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
}).listen(port, host);

Test our file locally:

$ node app.js
$ curl localhost:3000
Hello World
# kill node with CTRL+C

Push our file to CloudFoundry and test. CloudFoundry automatically picks up that we’re using node.js,but it will ask some other configuration questions, including a name and the services we want to have running. I have named mine gvp_node_test and requested that MongoDB be run as a service.

The commands & output:

$ vmc push 
Would you like to deploy from the current directory? [Yn]: Y
Application Name: gvp_node_test
Application Deployed URL: 'gvp_node_test.cloudfoundry.com'? 
Detected a Node.js Application, is this correct? [Yn]: Y
Memory Reservation [Default:64M] (64M, 128M, 256M, 512M, 1G or 2G) 
Creating Application: OK
Would you like to bind any services to 'gvp_node_test'? [yN]: y
The following system services are available::
1. mongodb
2. mysql
3. redis
Please select one you wish to provision: 1
Specify the name of the service [mongodb-55516]: 
Creating Service: OK
Binding Service: OK
Uploading Application:
  Checking for available resources: OK
  Packing application: OK
  Uploading (0K): OK   
Push Status: OK
Staging Application: OK                                                         
Starting Application: OK

At this point you should have a simple working web app. Note the URL: your_app_name.cloudfoundry.com, we can test it is working with curl.

$ curl your_app_name.cloudfoundry.com
Hello World

Step 2: getting mongo configs

CloudFoundry has now configured a MongoDB service with its own user name, password, ip and port. To access these on the server, we will need to parse the environment variables coming into the node process.

To do this we do the following, note that we’ve added an else clause so that we can run this locally.

if(process.env.VCAP_SERVICES){
  var env = JSON.parse(process.env.VCAP_SERVICES);
  var mongo = env['mongodb-1.8'][0]['credentials'];
}
else{
  var mongo = {"hostname":"localhost","port":27017,"username":"",
    "password":"","name":"","db":"db"}
}

The code wraps this up in a generate_mongo_url function that provides a “connection string” of the form mongodb://username:password@host:port/db_name.

Copy in the rest of the code from step 2 on github and test locally.

$ node app.js
$ curl localhost:3000
# connection string for localhost

Once that’s working push the update to the cloud. Notice that we add the name of the project and we don’t get asked any questions:

$ vmc update your_app_name
Uploading Application:
...
Stopping Application: OK
Staging Application: OK                                                         
Starting Application: OK 
# test again
$ curl your_app_name.cloudfoundry.com
# bunch of environment variables

Step 3: now with drivers

First we need to install the node-mongodb-native driver. To do this, we use NPM.

$ npm install mongodb

You should see a new directory at the end of this process: node_modules. To enable use to include these module on the cloud we add this path to the require variable at the top of the code.

require.paths.unshift('./node_modules');

if(process.env.VCAP_SERVICES){
...

Our goal here is to build a function for recording a visit. Let’s build that function.

var record_visit = function(req, res){
  /* Connect to the DB and auth */
  require('mongodb').connect(mongourl, function(err, conn){
    conn.collection('ips', function(err, coll){
      /* Simple object to insert: ip address and date */
      object_to_insert = { 'ip': req.connection.remoteAddress, 'ts': new Date() };

      /* Insert the object then print in response */
      /* Note the _id has been created */
      coll.insert( object_to_insert, {safe:true}, function(err){
        res.writeHead(200, {'Content-Type': 'text/plain'});
        res.write(JSON.stringify(object_to_insert));
        res.end('\n');
      });
    });
  });
}

Notice the .connect and .collection('ips'...). We’re telling it to store data in the ips collection.

Another nice feature is the object_to_insert. Saving a document with Node+MongoDB is as simple as creating the object and inserting it.

Let’s fix up the main createServer function.

http.createServer(function (req, res) {
  record_visit(req, res);
}).listen(port, host);

Then we can test locally and push with vmc. If this is working locally, you should be able to connect to your local mongod instance and see some data in the ips collection.

$ mongo localhost:27017/db
> db.ips.find()
...

Step 4

At this point, you’ve probably tested a few times and you’ve successfully put data in the database. Now it’s time to get that data out.

Let’s create a function to print the last 10 visits:

var print_visits = function(req, res){
  /* Connect to the DB and auth */
  require('mongodb').connect(mongourl, function(err, conn){
    conn.collection('ips', function(err, coll){
      /*find with limit:10 & sort */
      coll.find({}, {limit:10, sort:[['_id','desc']]}, function(err, cursor){
        cursor.toArray(function(err, items){
          res.writeHead(200, {'Content-Type': 'text/plain'});
          for(i=0; i < items.length; i++){
            res.write(JSON.stringify(items[i]) + "\n");
          }
          res.end();
        });
      });
    });
  });
}

Let’s update the createServer method to print when we request /history.

http.createServer(function (req, res) {
  params = require('url').parse(req.url);
  if(params.pathname === '/history') {
    print_visits(req, res);
  }
  else{
    record_visit(req, res);
  }
}).listen(port, host);

Again, we test locally and then upload with vmc. If it all works, you should be able to do this:

$ vmc update your_app_name
...
$ curl your_app_name.cloudfoundry.com
{"ip":"172.30.49.42","ts":"2011-06-15T20:14:18.977Z","_id":"4df9129af354f8682d000001"}
$ curl your_app_name.cloudfoundry.com
{"ip":"172.30.49.43","ts":"2011-06-15T20:14:21.745Z","_id":"4df9129df354f8682d000002"}

# now let's test history
$ curl gvp_node_test.cloudfoundry.com/history
{"ip":"172.30.49.43","ts":"2011-06-15T20:14:21.745Z","_id":"4df9129df354f8682d000002"}
{"ip":"172.30.49.42","ts":"2011-06-15T20:14:18.977Z","_id":"4df9129af354f8682d000001"}
...

Going further

Check out our upcoming Node.js Panel Discussion webinar.
For some MongoDB wrappers take a look at
- Mongoose, an ORM / ODM wrapper
- MongoSkin, a layer over node-mongodb-native to help reduce callbacks.
For building more complex web sites take a look at the Express framework.

— Gates Voyer-Perrault
@gatesvp

A reminder about MongoDB "office hours"

Wed, 08 Jun 2011 11:21:00 -0400

Very casual whiteboard type chat sessions on a regular basis in SF, NYC, Redwood Shores, Mountain View, and Atlanta. Stop by!

Also check out the meetup users groups in lots of other cities.

How Journaling and Replication Interact

Mon, 06 Jun 2011 14:15:50 -0400

Version 1.8 of MongoDB supports journaling in the storage engine for crash safety and fast recovery. An interesting question arises then regarding how journaling interacts with replication.

A traditional approach might be to wait for the commit (i.e., journal physical write confirmed) before replicating any data. MongoDB does not do this. Instead, it allows data to replicate even if the journal write has yet to occur or be confirmed.

We then must ask “but what happens if we crash before journaling but the data replicated out?” With replica sets, it turns out this is ok. In a replica set the rule is that the freshest node will be elected primary. Thus if the crashed node comes back up, but the node which received the unjournaled data is ahead, it will be primary.

We might then ask about a cascade of failures. This is ok too as replica sets have a notion of rolling back to a consistent point of view.

How do we know our data won’t be rolled back? The answer is that a write is truly committed in a replica set when it has been written at a majority of set members. We can confirm this with the getLastError command. For example, if our write has made it to the journal on two out of three total set members, we know the data is committed even if nodes fail in a cascading sequence, and even if a minority of nodes are permanently lost.

Why bother replicating so quickly? It lets us minimize latency between secondaries and primaries. In addition more writes will be successful than traditionally when a crash occurs. Yet the latency reduction is the biggest advantage: fsyncing to disks can be slow — replication lag (if on a LAN) can be less than the time to fsync to disk. Plus both can then be underway concurrently.

MongoDB live at Craigslist

Mon, 16 May 2011 11:00:00 -0400

Update: You can view a video of Jeremy Zawodny’s talk at MongoSF on 10gen.com.

MongoDB is now live at Craigslist, where it is being used to archive billions of records.

Craiglist has kept every post anyone has ever made in a large MySQL cluster. A few months ago, they began looking for alternatives: schema changes were taking forever (Craigslist’s schema has changed a couple times since 1995) and it wasn’t really relational information. They wanted to be able to add new machines without downtime (which sharding provides) and route around dead machines without clients failing (which replica sets provide), so MongoDB was a very strong candidate. After looking into a few of the most popular non-relational database systems, they decided to go with MongoDB.

Jeremy Zawodny is a software engineer at Craigslist and an author of High Performance MySQL (O’Reilly). He kindly agreed to answer some questions about their MongoDB cluster (editor’s comments in italics).

Any numbers you can give us?

We’re sizing the install for around 5 billion documents. That’s from the initial 2 billion document import we need to do plus room to grow for a few years to come. Average document size is right around 2KB. (Five billion 2KB documents is 10TB of data.) We’re getting our feet wet with MongoDB so this particular task isn’t high throughput or growing in unpredictable ways.

We can put data into MongoDB faster than we can get it out of MySQL during the migration.

What does your cluster topology look like?

We have several three-machine replica sets, each set serving a shard of our “archive” database cluster. The configuration is three replica sets in each colo (two total) to handle our initial build out. Obviously there will be a set of config servers and routing processes in the mix as well.

Craigslist is using the MongoDB Perl driver.

Did you find any stumbling blocks relational database developers should watch out for?

Oh, yeah. :-) I’m planning to write up a blog post on that and I talked about it a bit at MongoSV (watch the video), too. The short version is that you have to think differently about indexing and do a bit more bookkeeping of your own. But on the plus side, you don’t have to pay the join penalty, so you can get your data back a lot faster.

Character set issues come up as well, since we’re a Latin-1 or Windows-1252 shop currently (but really need to go UTF-8 across the board). That means some upfront work, but it’s good that MongoDB is UTF-8 end-to-end already.

Got future plans for your MongoDB cluster?

Too soon to tell! But I have a few ideas about ways we can use MongoDB to supplement other needs and possibly replace other data stores. But I really need to think more about them before I go spouting off.

Looking into the future, Zawodny hopes for more polish on sharding and replica sets and that some good front-end admin tools will be developed. “But that will come in time, I’m sure.”

If you’re interested in more details, check out the video’s from Jeremy’s talks at MongoSV or MongoSF.

MongoDB Powering MTV's Web Properties

Tue, 10 May 2011 09:21:00 -0400

An interview with Jeff Yemin, Director of Content Management Systems at MTV Networks and a presenter at the upcoming MongoNYC conference.

What do you work on at MTV Networks?
I manage the backend development of our next-generation content management system.

Tell us how MTV Networks is using MongoDB. What is the size and scope of the CMS application that you have built?
MongoDB is the repository that powers our CMS, which will eventually be used to manage and serve content for all of MTV Networks’ major websites.

Can you tell us why you chose MongoDB?
I have to save something for my actual talk! But briefly, MongoDB is a good fit for a lot of reasons. The schema-less document structure provides a lot of flexibility, while the search capabilities provide a lot more value than you get with key-value stores. At the same time, from an operational perspective, it feels very much like MySQL, so our systems and DBA groups are quite comfortable managing the deployment.

What were some of the challenges you faced using MongoDB for a large enterprise application?
The biggest was probably convincing people that it was a good idea. Like most enterprises that have been around for a while, we have a long history of using relational databases, so this is a pretty big change for us.

What’s next for MongoDB at MTV?
Right now MongoDB is powering the recently re-launched spike.com, and we are going to be rolling it out on many other major sites within the next year, most likely including gametrailers.com, thedailyshow.com, comedycentral.com, nick.com, and numerous international properties. We’ll also be rolling out new MongoDB-based applications for social activity tracking and online polls.

Learn more at Jeff’s talk at MongoNYC on June 7. Register before May 17 to take advantage of early bird pricing, which is only $50.

Java is on the Rise, Be-aware!

Thu, 05 May 2011 08:45:00 -0400

Improving scalable Java application development with MongoDB + Morphia:

Over the last year I have seen a significant rise in the number of questions and interest from both the greater Java community and enterprise Java shops about MongoDB. Coming from the MongoDB and Java worlds (among others), this is something I have watched with great interest and excitement.

As one of the authors and project leads for Morphia (MongoDB Java ORM) I have seen a lot of questions relating to both the core driver and how to build Java applications with MongoDB. A lot of these questions arise from the paradigm shift users experience when moving from the standard SQL/JPA/Hibernate platforms/frameworks to the document oriented world of MongoDB.

For the last year or so, Morphia has — how long that seems looking back — bridged the gap between the document/map/dictionary world of the core MongodDB Java driver with the cleaner POJO (domain objects) design pattern, which is much friendlier for business logic.

Over the last few months, the Java MongoDB ecosystem has started to drastically evolve. New features in the core MongoDB server continue to distinguish the product from the crowd and new engineers are joining the MongoDB open source community by contributing Java persistence (mappers) frameworks and libraries.

As one of the main authors of Morphia, I envision many of the projection/mapping features which we have implemented being included in the core MongoDB Java driver, similar to what other drivers support (.Net/C#, and dynamic languages).

Even before I joined 10gen I had been in discussions of how to best integrate the user experience from high-level abstractions such as the JPA EntityManager and DAO pattern into the features supplied with the basic driver. Now that I am a member of the team, and have direct access to the driver (and the great team that has been maintaining it internally, along with the numerous community patches), I am finally in a great position to help make that integration possible. I will in fact be in the best position of all to make this vision come true: I get to do the integration :-)

Major Version Changes

Let’s talk a little about the changes coming in the 3.0* Java driver. First, we have some great ideas of how to improve overall performance for frameworks; these ideas primarily have come from the need to reduce the impedance mismatch of the current driver when being used from frameworks like Morphia, Casbah, and many others.

One of the performance improvements we have in mind will allow us to remove the need to decode binary BSON, to a temporary java map (DBObject) and then to the final format. In some of the very high volume and real-time analytics deployments, this will be an amazing performance improvement. In addition to reduced latency, the change will also reduce resource allocation as well.

A little code:

With Morphia, and soon the driver, you can do things like this:

@Entity
class Customer {
   @Id ObjectId id;
   String name;
   String[] projects; }

Save, and find, a single Model/Entity

Datastore ds = …;
ds.save(new Customer(“Scott Hernandez”, 
                     new String[]{“java”, “morphia”, “casbah”}));
// QBE(query by example)
Customer scott = ds.find(Customer.class, new Customer(“Scott Hernandez”)).get();
//Iterate over all customers
for(Customer customer : ds.find(Customer.class))
     print(customer);

Use the fluent query interface.

//Use the fluent query interface
Query findMe = ds.find(Customer.class).field(“projects”).equal(“java”);
Customer scott = findMe.get(); //The query is validated by your java class

scott.name = “Scott D. Hernandez”; //change a field 
ds.save(scott); //save the whole object

Update just a field or two at a time.

//Update on one field, without needing the full customer object
UpdateOperations updates = ds.createUpdateOperations();

updates.add(“projects”, “support”);
UpdateResults res = ds.updateFirst(findMe, updates);

Coming very soon we will have better support for mock frameworks and testing isolation by providing interfaces to the major components in the driver. Many (open source) projects built on the driver already have support for mock and isolated testing so having the driver support is a great benefit for all.

Compatibility

On the path to these changes we have many other milestones to hit as well. Some of these are internal changes while others will have some outwardly facing impact. The premier goal we have is to make sure that the MongoDB Java driver stays backwards compatible; this will reduce the need for existing applications and deployments to make any changes as we make improvements. There will be some changes that might not fit in this goal but we will keep them as small as possible and highlighted in the release notes.

Events and More Content

Java Web Apps Webinar, May 12, 1:30p ET/ 10:30a PT

We will be hosting an online webinar to talk about one way to build webapps using Java and some common open source frameworks (Guice, Stripes and Morphia) in about week.

You can learn more and register here: http://www.10gen.com/webinars/javawebapps

MongoSF + MongoNYC

MongoSF is a one day conference dedicated to MongoDB on May 24th. There will be loads of talks, including a bunch about MongoDB with Java and Morphia. Find out more about the event and registration at www.mongosf.com. Register soon, because it’s nearly sold out already.

MongoNYC will be held on June 7th and will contain a very similar number of tracks for those on the other coast.

More Blogs

This is just the start of the blogging about the changes coming to the Java driver. There will be many more in-depth posts regarding parts of the driver and interfaces changes.

Feedback

I can be contacted via email as scott -at- 10gen (dotcom).

Please feel free to submit patches or file issues:

http://github.com/mongodb/mongo-java-driver
http://jira.mongodb.org/browse/Java

—Scott

MongoDB on EC2 Best Practices

Wed, 27 Apr 2011 08:00:00 -0400

In light of last week’s EBS issues, we wanted to make sure MongoDB users on EBS are configured to be as robust as possible.

A basic setup would consist of a 3 node replica set. The nodes would be roughly laid out like this:

* A: us-east-1a priority 1
* B: us-east-1b priority 1
* C: us-west-1a priority 0

During steady state, either A or B would be primary. If the primary went down for any reason (system crash, or loss of one availability zone) the other node in us-east would take over. This is guaranteed because the west coast node has a priority of 0.

If instead the entire east coast region were lost, then you would still have a ful copy of data on C. If you decided that you were going to make the west coast your primary data center for the duration, you would just bring up a couple more nodes there, and make a new replica set with the data from C.

More information about running MongoDB on ec2 is available from our recent ec2 webinar. We are big fans of cloud computing in general and want MongoDB to only get more and more cloud-friendly over time.

-Eliot

Getting started with VMware CloudFoundry, MongoDB and Rails

Mon, 18 Apr 2011 10:58:00 -0400

Listen to Jared Rosoff’s June 2 webinar “VMware Cloud Foundry with MongoDB”.
Read about getting started with Cloud Foundry, MongoDB, and Node.js.

Last week, VMware launched Cloud Foundry: an open-source platform as a service. It’s pretty radical in that not only can you run your apps on infrastructure operated by VMware, you can also download Cloud Foundry itself and run it on your own machines!

But what’s most awesome about Cloud Foundry is that it supports MongoDB right out of the box! In today’s post, we’re going to walk through the creation of a Rails application using MongoDB and Cloud Foundry.

Here’s what we’re going to need to do:

Create a new rails project (skipping Active Record)
Add the dependencies for MongoMapper
Build a simple app that interacts with the database
Set up our credentials so we can talk to Cloud Foundry’s MongoDB
Push our application to CloudFoundry

Create our rails project
First we’ll create our new Rails project. My preference is for MongoMapper, so I’m going to skip the ActiveRecord generators when I initialize my project.

$ rails new my_app --skip-active-record

Add the dependencies for MongoMapper
Next we need to setup our dependencies. Cloud Foundry will automatically detect that we’re a Rails app, and it will process our Gemfile if we’ve got one. So we’re need to add any dependencies to our Gemfile so they’ll get installed when we deploy. This is what our Gemfile looks like:

source "http://rubygems.org"
gem "rails", "3.0.5"
gem "mongo_mapper"
gem "bson_ext"

I’m assuming you’re using Rails 3 here, but you can easily adapt these instructions for other versions.

Build a simple app that interacts with the database
Now we’ll write a little code to interact with the DB so we can be sure everything’s working.

$ script/rails generate scaffold messages message:string --orm mongo_mapper

I’m also going to set the root of my rails app to be our new messages controller and remove our public/index.html. Here’s my routes file:

CloudFoundryRailsTutorial::Application.routes.draw do
  resources :messages
  root :to => "messages#index"
end

Be sure to delete your public/index.html!!

Set up our credentials so we can talk to Cloud Foundry’s MongoDB
MongoMapper comes with a generator that will create a basic config/mongo.yml configuration file for us.

$ script/rails generate mongo_mapper:config

The file looks like this:

defaults: &defaults
  host: 127.0.0.1
  port: 27017

development:
  <<: *defaults
  database: myapp_development

test:
  <<: *defaults
  database: myapp_test

# set these environment variables on your prod server
production:
  <<: *defaults
  database: myapp
  username: <%= ENV['MONGO_USERNAME'] %>
  password: <%= ENV['MONGO_PASSWORD'] %>

We need to modify this so that it can talk to Cloud Foundry’s infrastructure. When CloudFoundry runs your app, it passes in a bunch of information through an environment variable. We need to pull the host, port, username, and password for MongoDB out of this environment variable. After some modification, the production section of your config/mongo.yml should look something like this:

production:
  host: <%= JSON.parse( ENV['VCAP_SERVICES'] )['mongodb-1.8'].first['credentials']['hostname'] rescue 'localhost' %>
  port: <%= JSON.parse( ENV['VCAP_SERVICES'] )['mongodb-1.8'].first['credentials']['port'] rescue 27017 %>
  database:  <%= JSON.parse( ENV['VCAP_SERVICES'] )['mongodb-1.8'].first['credentials']['db'] rescue 'cloud_foundry_mongodb_tutorial' %>
  username: <%= JSON.parse( ENV['VCAP_SERVICES'] )['mongodb-1.8'].first['credentials']['username'] rescue '' %>
  password: <%= JSON.parse( ENV['VCAP_SERVICES'] )['mongodb-1.8'].first['credentials']['password'] rescue '' %>

Note: The “rescue” clauses are so that you can run this app locally. If you don’t include this and you try to run this app outside of cloud foundry, you’ll get an exception because there’s no VCAP_SERVICES environment variable passed into your app.

Push our application to CloudFoundry
Now we’re ready to test things out. If you haven’t already, you should go ahead and install the vmc command line tool. There’s a getting started with VMC guide here. Here’s what it looked like when I deployed my app:

redeye:myapp jsr$ vmc push --runtime ruby19
Would you like to deploy from the current directory? [Yn]: y
Application Name: mongodb-on-cf-demo
Application Deployed URL: 'mongodb-on-cf-demo.cloudfoundry.com'? 
Detected a Rails Application, is this correct? [Yn]: y
Memory Reservation [Default:256M] (64M, 128M, 256M or 512M) 256M
Creating Application: OK
Would you like to bind any services to 'mongodb-on-cf-demo'? [yN]: y
Would you like to use an existing provisioned service [yN]? n
The following system services are available::
1. mysql
2. mongodb
3. redis
Please select one you wish to provision: 2
Specify the name of the service [mongodb-a8a43]: 
Creating Service: OK
Binding Service: OK
Uploading Application:
  Checking for available resources: OK
  Processing resources: OK
  Packing application: OK
  Uploading (5K): OK   
Push Status: OK
Starting Application: OK                                                        

redeye:myapp jsr$

Now you can point your browser to http://mongodb-on-cf-demo.cloudfoundry.com/ and you should see the list of messages!

Congrats! You’ve got your first Rails app up and running on Cloud Foundry and MongoDB!

The code for this mongodb + rails + Cloud Foundry app is up on github

Happy coding and be sure to tell us about all the awesome apps you build on Cloud Foundry!

— Jared Rosoff

Introduction to the official C# driver from 10gen

Wed, 23 Mar 2011 19:35:48 -0400

10gen is happy to announce support for the official C# driver for MongoDB. Several preview releases have already been made available, and the latest, Version 0.11, was released January 25, 2011. Version 1.0 has just been released and includes support for the new features in MongoDB 1.8.

The official C# driver is designed to be fast and efficient, is fully supported by 10gen, and will have full support for new MongoDB features as new server versions are released. Version 1.0 is compatible with Visual Studio 2008 and 2010 and .NET 3.5. The driver is well suited for use in high load environments as the main classes are all thread safe and connections are efficiently and automatically managed by a connection pool. The driver can connect directly to a particular server, or can connect to a replica set and automatically find the current primary (as well as automatically rollover to a new primary as needed). SafeMode can be enabled to automatically call getLastError after every update operation to check for errors. Full serialization support for C# classes is provided to make integrating your domain model classes with MongoDB easy (including keeping your domain classes persistent ignorant if desired). The serialization support provides several ways for you to customize serialization of particular classes or data types should your classes require special handling. Interoperability with JavaScript and other languages is enhanced by the included JSON reader and writer. GridFS support is provided to deal with large documents (or files) that are too big to store as a single BSON document.

More information about the official C# driver is available at:

http://www.mongodb.org/display/DOCS/CSharp+Language+Center

A webinar, “Introduction to the New Official C# Driver Developed by 10gen”, is available at:

http://www.10gen.com/webinars/csharp

We are excited to be officially supporting the .NET community with this new driver.

-Robert Stam