January 2012
1 post
Operations in the New Aggregation Framework
Available in 2.1 development release. Will be stable for production in the 2.2 release
Built by Chris Westin (@cwestin63)
MongoDB has built-in
MapReduce
functionality that can be used for complex analytics tasks. However, we’ve
found that most of the time, users need the kind of group-by functionality that
SQL implementations have. This can be implemented using map/reduce, but doing
so...
December 2011
2 posts
6 tags
MongoSV Recap
Last week over 1,100 developers came together for MongoSV, the largest MongoDB conference to date. 10gen kicked off MongoSV with our inaugural MongoDB Masters program, which brought together MongoDB evangelists from around the world.
At the opening keynote, 10gen CTO Eliot Horowitz demoed a twitter app for #mongoSV tweets, featuring the new aggregation framework expected for the MongoDB 2.2...
MongoDB On Microsoft Azure
A new preview release of the MongoDB controller for Azure is available. This release includes support for replica sets, and over the coming months, we’ll be adding support for MongoDB’s sharding facilities. We’ll also be working to more tightly integrate MongoDB with the features of Azure platform.
Each member of a replica set is hosted by an instance of an Azure worker role, so the size of the...
November 2011
1 post
MongoDB Monitoring Service Docs Available
MongoDB Monitoring Service (MMS) documentation now available: http://mms.10gen.com/help/
October 2011
1 post
Mongo Boston Recap
Last week 250 developers converged at the Microsoft New England Research and Design Center for Mongo Boston. Highlights from the event include presentations on MongoDB 2.0, how MTV leverages MongoDB for CMS, rapid prototyping, and more.
More photos from the event are available on the MongoDB Flickr page.
If you missed the event, join the Boston MongoDB User Group, which also meets at NERD....
September 2011
3 posts
2.0 Presentation at New York MongoDB User Group
On Thursday MongoDB core committer Eliot Horowitz presented to the New York MongoDB User Group on the latest features in v2.0. The event was hosted by Sailthru, a MongoDB-powered startup doing intelligent email marketing. The meetup was announced Monday night and within a day was oversubscribed. After the presentation, we all went out for drinks to celebrate the release.
The NY MUG has over...
Cache Reheating - Not to be Ignored
An important aspect to keep in mind with databases is the cost of cache reheating after a server restart. Consider the following diagram which shows several cache servers (e.g., memcached) in front of a database server.
This sort of setup is common and can work quite well when appropriate; it removes read load from the database and allows more RAM to be utilized for scaling (when the...
MongoDB 2.0 Released
The MongoDB development team is pleased to announce the release of version 2.0.0. Version 2.0 is the latest stable release, following the March 2011 release of version 1.8. This release includes many new features, improvements to existing features, and performance enhancements.
Please note version 2.0 is a significant new release, but is 2.0 solely because 1.8 + 0.2 = 2.0; for example the...
August 2011
1 post
BSON and Data Interchange
There’s a lot of good things about JSON — it’s a standards based, language independent, representation of object-like data. Also, it’s easy to read (for users and programmers alike). Each document is only about data, not complex object graphs and links. Thus it’s easy to inspect without knowing all the code of an application.
Further, JSON is “schemaless”. We do not have to predefine...
July 2011
2 posts
Master Detail Transactions in MongoDB
In relational databases, transactions let you make reliable atomic updates to your data. Because relational schemas are often highly normalized, most logical transactions span multiple tables, so it is important to be able to do multiple updates atomically (all or nothing).
While MongoDB does not have multi-document transactions, it makes up for this in many use cases through its document...
Design of the Erlang Driver for MongoDB
Since November 2010, I have been writing an Erlang driver for MongoDB. After many months of work, I would consider the driver production-ready, and wanted to take this opportunity to introduce the driver and highlight a few of the design decisions. For detailed documentation with code examples please see the links at the end of this article.
BSON
At the highest level, the driver is divided into...
June 2011
3 posts
5 tags
Getting started with VMware CloudFoundry, MongoDB...
Listen to the recording of the Node.js Panel Discussion webinar.
Overview Following up from our previous post we’re posting up a quick how-to for using Node.JS, CloudFoundry and MongoDB together.
Our end goal here is to build a simple web app that records visits and provides a reporting screen for the last 10 visits.
Tools We Need
Sign up for a Cloud Foundry account.
Local...
A reminder about MongoDB "office hours"
Very casual whiteboard type chat sessions on a regular basis in SF, NYC, Redwood Shores, Mountain View, and Atlanta. Stop by!
Also check out the meetup users groups in lots of other cities.
How Journaling and Replication Interact
Version 1.8 of MongoDB supports journaling in the storage engine for crash safety and fast recovery. An interesting question arises then regarding how journaling interacts with replication.
A traditional approach might be to wait for the commit (i.e., journal physical write confirmed) before replicating any data. MongoDB does not do this. Instead, it allows data to replicate even if the...
May 2011
3 posts
MongoDB live at Craigslist
Update: You can view a video of Jeremy Zawodny’s talk at MongoSF on 10gen.com.
MongoDB is now live at Craigslist, where it is being used to archive billions of records.
Craiglist has kept every post anyone has ever made in a large MySQL cluster. A few months ago, they began looking for alternatives: schema changes were taking forever (Craigslist’s schema has changed a couple...
MongoDB Powering MTV's Web Properties
An interview with Jeff Yemin, Director of Content Management Systems at MTV Networks and a presenter at the upcoming MongoNYC conference.
What do you work on at MTV Networks?
I manage the backend development of our next-generation content management system.
Tell us how MTV Networks is using MongoDB. What is the size and scope of the CMS application that you have built?
MongoDB is the...
5 tags
Java is on the Rise, Be-aware!
Improving scalable Java application development with MongoDB + Morphia:
Over the last year I have seen a significant rise in the number of questions and interest from both the greater Java community and enterprise Java shops about MongoDB. Coming from the MongoDB and Java worlds (among others), this is something I have watched with great interest and excitement.
As one of the authors and...
April 2011
2 posts
MongoDB on EC2 Best Practices
In light of last week’s EBS issues, we wanted to make sure MongoDB users on EBS are configured to be as robust as possible.
A basic setup would consist of a 3 node replica set. The nodes would be roughly laid out like this:
* A: us-east-1a priority 1
* B: us-east-1b priority 1
* C: us-west-1a priority 0
During steady state, either A or B would be primary. If the primary went...
Getting started with VMware CloudFoundry, MongoDB...
Listen to Jared Rosoff’s June 2 webinar “VMware Cloud Foundry with MongoDB”. Read about getting started with Cloud Foundry, MongoDB, and Node.js.
Last week, VMware launched Cloud Foundry: an open-source platform as a service. It’s pretty radical in that not only can you run your apps on infrastructure operated by VMware, you can also download Cloud Foundry itself and run...
March 2011
2 posts
Introduction to the official C# driver from 10gen
10gen is happy to announce support for the official C# driver for MongoDB. Several preview releases have already been made available, and the latest, Version 0.11, was released January 25, 2011. Version 1.0 has just been released and includes support for the new features in MongoDB 1.8.
The official C# driver is designed to be fast and efficient, is fully supported by 10gen, and will have full...
MongoDB 1.8 Released
We are happy to announce that MongoDB v1.8.0 is now available. 1.8 is the stable follow-up release to 1.6, which came out in August of 2010. Version 1.8 introduces many new features, along with bug fixes and other improvements. Some of the highlights:
Journaling
Sharding performance improvements
Replica set enhancements, including support for authentication
Spherical geo search
Covered and...
January 2011
1 post
The State of MongoDB and Ruby
The state of Ruby and MongoDB is strong. In this post, I’d like to describe some of the recent developments in the Ruby driver and provide a few notes on Rails and the object mappers in particular.
The Ruby Driver
We just released v1.2 of the MongoDB Ruby driver. This release is stable and supports all the latest features of MongoDB. If you haven’t been paying attention to the...
December 2010
1 post
Five New Replica Set Features in 1.7.x
Here’s a rundown of some of the most useful features added recently. These are all available in 1.7.4 and will, of course, be in 1.8.
Initial sync from a secondary
You can now set an initialSync source for each member, which controls where the new guy will sync from. For example, if you wanted to add a new node and force it to sync from a secondary, you could do:
> rs.add({"_id" :...
September 2010
1 post
Archiving - a good MongoDB use case?
Someone recently pointed out to me, rather insightfully, that MongoDB is a good fit for archival of relational data.
I had not really considered this before, but it is a good point : flexible schemas are very helpful for archival. How do we keep an archive of data, say, 10 years or more of data history, when over that time period the schema will undergo significant changes? It is not so...
August 2010
1 post
MongoDB 1.6 Released
MongoDB 1.6.0 is the fourth stable major release (even numbers are “stable” : 1.0, 1.2, 1.4, …) and is the culmination of the 1.5 development series.
Scale-out
The focus of the 1.6 release is scale-out. Sharding is now production-ready. The combination of sharding and replica sets allows one to build out horizontally scalable data storage clusters with no single points...
July 2010
1 post
Node.js and MongoDB
Visit the more recent post, Getting Started with VMware Cloud Foundry, MongoDB, and Node.js.
Listen to the recorded Node.js Panel Discussion webinar.
Node.js is turning out to be a framework of choice for building real-time applications of all kinds, from analytics systems to chat servers to location-based tracking services. If you’re still new to Node, check out Simon Willison’s...
June 2010
3 posts
Blog Contest Winners!
We’re pleased to announce the winner’s of the MongoDB blogging contest!
Grand Prize
Why (and How) I Replaced Amazon SQS with MongoDB - Matt Insler
Runners Up
Running MongoDb on Microsoft Windows Azure with CloudDrive - Simon Green
Using MongoDB for CodeIgniter Logs - Tom Schlick
Integrating MongoDB with Spring Batch - Baruch Sadogursky
The winners should contact meghan@10gen.com...
Highlights from MongoNYC
On May 21, 10gen organized the second conference dedicated to MongoDB. Like MongoSF, MongoNYC included a great line-up of speakers. One of the more popular talks was Kyle Banker’s Schema Design session, which was so crowded that many attendees sat on the floor! Both the video and slides from the talk are now available.
Also interesting were the many talks on MongoDB production...
5 tags
Holy Large Hadron Collider, Batman!
Valentin Kuznetsov just presented a paper at the International Conference on Computational Science on CERN’s use of MongoDB for Large Hadron Collider data. The paper, The CMS Data Aggregation System, is available as a PDF at ScienceDirect.
A summary
“CMS” stands for Compact Muon Solenoid, a general-purpose particle physics detector built on the Large Hadron Collider. The CMS project posted a...
May 2010
3 posts
Write a blog post on MongoDB for a chance to win a...
10gen has a ticket to OSCON that we’d like to give to a MongoDB user.
How to Enter
Write a blog post. It has to be about MongoDB, but within that it can be anything: a how-to, an experience you had, a review, a rant, a rave, a technical piece, a humorous piece… whatever you want.
Post your blog at mongodb.slinkset.com and include your contact information in the description. You...
MongoSF Slides & Video; Discounts on upcoming...
MongoSF, the first full-day conference dedicated to MongoDB, featured 35+ sessions and even produced a few surprises along the way. Over 200 people attended the April 30 conference. Slides and video from many sessions now available on the 10gen website.
The Sharding presentation was one of the major highlights of the event. Eliot Horowitz, the CTO of 10gen, demoed a 25-node cluster on EC2. Check...
MongoDB Conferences in London and Paris in June
MongoDB conferences are coming to Europe! MongoUK is on Friday, June 18 at Skills Matter and MongoFR is on Monday, June 21 at La Cantine. Each conference will feature sessions from the 10gen team on schema design, replication, sharding, indexing, and map/reduce. In addition, attendees will learn about MongoDB in production through presentations by companies like Boxed Ice, Silentale, OCW Search,...
April 2010
6 posts
MongoDB Q1 Download Numbers
The MongoDB team is very excited about how the developer community is building around MongoDB, and we wanted to share some numbers.
These are download numbers for the core server for January through March. It is exactly the number of downloads of the core database from downloads.mongodb.org minus all bots (all known plus anything with bot in the user-agent) and all other crawlers we...
5 tags
On Distributed Consistency - Part 6 - Consistency...
See also:
Part 1 - Introduction and CAP
Part 2 - Eventual Consistency
Part 3 - Network Partitions
Part 4 - Multi Data Center
Part 5 - Multi Writer Eventual Consistency
The following diagram (click for large version) shows the various consistency models that have been discussed in this blog post series. Stronger consistency modes generally meet the requirements of weaker modes, and are thus...
3 tags
On Distributed Consistency - Part 5 - Many Writer...
See also:
Part 1 - Introduction and CAP
Part 2 - Eventual Consistency
Part 3 - Network Partitions
Part 4 - Multi Data Center
Part 6 - Consistency Chart
In part 2 we primarily discussed “single writer” eventual consistency. Here we will discuss many-writer, and define that term more precisely.
By many-writer, we mean a system where different data servers can receive writes...
2 tags
On Distributed Consistency - Part 4 - Multi Data...
See also:
Part 1 - Introduction and CAP
Part 2 - Eventual Consistency
Part 3 - Network Partitions
Part 5 - Multi Writer Eventual Consistency
Part 6 - Consistency Chart
Eventual consistency makes multi-data center data storage easier. There are reasons eventual consistency is helpful for multi-data center that are unrelated to availability and CAP. And as mentioned in Part 3, some common...
On Distributed Consistency - Part 3 - Network...
See also:
Part 1 - Introduction and CAP
Part 2 - Eventual Consistency
Part 4 - Multi Data Center
Part 5 - Multi Writer Eventual Consistency
Part 6 - Consistency Chart
It’s fascinating that the formal theorem statement for CAP, in the first proof (that I know of), doesn’t use the word partition!
Theorem 1 It is impossible in the asynchronous network model to implement a...
On Distributed Consistency - Part 2 - Some...
See Also
Part 1 - Introduction and CAP
Part 3 - Network Partitions
Part 4 - Multi Data Center
Part 5 - Multi Writer Eventual Consistency
Part 6 - Consistency Chart
In Part 1 we discussed C-class and A-class behaviors. For A-class, we need to weaken consistency constraints. This does not mean the system need be completely inconsistent, but it does mean we will need to relax the consistency...
March 2010
12 posts
On Distributed Consistency -- Part 1
See also:
Part 2 - Eventual Consistency
Part 3 - Network Partitions
Part 4 - Multi Data Center
Part 5 - Multi Writer Eventual Consistency
Part 6 - Consistency Chart
For distributed databases, consistency models are a topic of huge importance. We’d like to delve a bit deeper on this topic with a series of articles, discussing subjects such as what model is right for a particular use case. ...
MongoDB 1.4 Ready for Production
The MongoDB team is very excited to announce the release of MongoDB 1.4.0. This is the culmination of 3 months of work in the 1.3 branch and has a large number of very important changes.
Many users have been running 1.3 in production, so this release is already very thoroghly vetted both by our regressions systems and by real users.
Some highlights:
Core server enhancements
concurrency...
MongoDB 1.4 Performance
We generally avoid posting benchmarks and suggest people create their own targeting their use cases. However, we have decided to publish a few of our internal micro-benchmarks comparing 1.2 with 1.4RC2 (aka 1.3.5) to show that in almost all cases performance is the same or better (sometime significantly so), even though we’ve added many new features.
The test works by spawning N threads and...
MongoDB Day Austin coming up on Saturday, March 27...
If you’re in the Austin area on March 27, you won’t want to miss MongoDB Day at Cospace in Austin. MongoDB Day Austin will be hosted by GeekAustin, a Slashdot style news site, and sponsored by 10gen.
This conference will have something for anyone interested in using MongoDB, from introductory sessions to more advanced discussions on sharding and MapReduce. Presenters at the...
Are you going to Structure?
GigaOm’s Structure is one of the most interesting conferences in the Bay Area this year. In 2010, we’re excited that Eliot Horowitz from 10gen / MongoDB will be speaking. GigaOm, who were media sponsors at the recently completed NoSQL Live event in Boston, has provided a special discount code for friends of MongoDB to register for the conference at a $100 savings. Hope to see you...
Announcing MongoSF
Please join us for MongoSF, a full-day conference on Friday, April 30 at Bently Reserve in San Francisco.
MongoSF will include sessions on database features, development with MongoDB in a wide range of dynamic languages, in-depth examples of production deployments, and development workshops. In addition to several of the MongoDB developers from 10gen, confirmed speakers include John Nunemaker and...
NoSQL Live Boston Recap
Check out the 10gen blog for a good recap of NoSQL Live Boston.
Should MongoDB Use SQL as a Query Language?
MongoDB does not use SQL as a query language. Why not? This is a very good question and we have discussed it on the project for a long time. There are a few reasons for this.
Given the document-oriented nature of the storage, if we were to do SQL, it really world be a variant, not true SQL. There would be no joins, and we would need extensions to handle the nested constructs involved in JSON...
State of MongoDB March, 2010
Every once in a while, I think its important for us (the core MongoDB team) to give a broad picture of where we think MongoDB is and where we’re hoping to take it. This is useful both as a gut check for us, to give the community some insight into what we’re thinking, and to make sure we’re all on the same page.
MongoDB has made great strides in the last year. The first public...
You need to learn MongoDB
You need to learn MongoDB. We’re offering an informative, hands-on training session to help you do just that. From document-based data modeling to high-performance optimizations, we’ll answer your questions and prepare you for the move to Mongo. Among the topics we’ll cover:
How to use the language drivers, and how they work
How to make the most of atomic updates
Data-modeling...
2d geospatial indexing
We have now added geospatial indexing to the product. Our approach has been to make something simple but fast: 2d only, and effective for common real world use cases such as lat/long location searches.
Would love to get some feedback on features people would like to see, how its working, etc…
Geospatial docs
More notes on 1.3.3 release
MongoDB March Events and NYC Office Hours
Upcoming MongoDB Events
MongoDB will be featured at several events, conferences, and meetups in March, including a webinar on MongoDB internals, Mountain West Ruby Conference in Salt Lake City, NoSQL Live Boston, QCon London, and Cloud Connect in Santa Clara. There’s a MongoDB training session in San Francisco and there will even be a MongoDB Day in Austin! Check the Events page for a...