Posts tagged:

open source

The MongoDB Open Source Hack Contest

Apr 28 • Posted 4 months ago

Some of the best MongoDB tools come from the Open Source community. Projects like the Node.js Driver, Mongoose and Meteor have become the backbone of many MongoDB apps and have helped support the developer community all over the world. We want to see more of what the community has built.

For the month of May, we’ll be hosting a worldwide hack contest for Open Source tools built on or connected to MongoDB. The winner of the contest will receive a ticket to OSCON, furnished by O’Reilly.

Guidelines:

  • All projects must be built with the MongoDB source code or on top of a MongoDB API, community or MongoDB supported driver
  • Any new drivers created should abide by the driver requirements listed in the MongoDB Manual
  • All hacks will be judged by MongoDB engineers

All entries can be submitted to community@MongoDB.com with the following information before May 31

  • Github or Bitbucket URL
  • Description of the project
  • How do users benefit from this application?
  • Why did you choose to contribute to MongoDB?

We’re looking forward to seeing your hacks come in!

Want to keep up-to-date on MongoDB news and events? Sign up for the MongoDB Monthly Newsletter”

Bug Hunt Winners for MongoDB 2.6-rc0

Mar 13 • Posted 5 months ago

We would like to thank everyone who participated in MongoDB’s inaugural Bug Hunt. Your efforts have helped to improve the 2.6 release.

The MongoDB Bug Hunt for 2.6-rc0 uncovered a number of bugs. From this pool, we’ve selected 5 winning Bug Reporters based on three criteria: user impact, severity and prevalence.

The Winners

First Prize: Roman Kuzmin, SERVER-12846
  • 1 ticket to MongoDB World — with a reserved front-row seat for keynote sessions
  • $1000 Amazon Gift Card
  • MongoDB Contributor T-shirt
Second Prize: Mark Callaghan, Server 13060 & 13041
  • 1 ticket to MongoDB World — with a reserved front-row seat for keynote sessions
  • $500 Donation to Khan Academy (at Mark’s request)
  • MongoDB Contributor T-shirt
Honorable Mentions: Tim Callaghan (Server 12878), David Glasser (Server 12981) and Jeff Lee (Server 12925)
  • 1 ticket to MongoDB World — with a reserved front-row seat for keynote sessions
  • $250 Amazon Gift Card
  • MongoDB Contributor T-shirt

Congratulations to the five winners of the first MongoDB Bug Hunt and thanks to everyone who downloaded, tested and gave feedback on the release candidates.

-Eliot, Dan, Alvin and the MongoDB team

Announcing the MongoDB Bug Hunt 2.6.0-rc0 

Feb 21 • Posted 6 months ago

The MongoDB team released MongoDB 2.6.0-rc0 today and is proud to announce the MongoDB Bug Hunt. The MongoDB Bug Hunt is a new initiative to reward our community members who contribute to improving this MongoDB release. We’ve put the release through rigorous correctness, performance and usability testing. Now it’s your turn. Over the next 10 days, we challenge you to test and uncover any lingering issues in MongoDB 2.6.0-rc0.

How it works

You can download this release at MongoDB.org/downloads. If you find a bug, submit the issue to Jira (Core Server project) by March 4 at 12:00AM GMT. Bug reports will be judged on three criteria: user impact, severity and prevalence.

We will review all bugs submitted against 2.6.0-rc0. Winners will be announced on the MongoDB blog and user forum by March 8. There will be one first place winner, one second place winner and at least two honorable mentions.

The Rewards
First Prize:
  • 1 ticket to MongoDB World — with a reserved front-row seat for keynote sessions
  • $1000 Amazon Gift Card
  • MongoDB Contributor T-shirt
Second Prize:
  • 1 ticket to MongoDB World — with a reserved front-row seat for keynote sessions
  • $500 Amazon Gift Card
  • MongoDB Contributor T-shirt
Honorable Mentions:
  • 1 ticket to MongoDB World — with a reserved front-row seat for keynote sessions
  • $250 Amazon Gift Card
  • MongoDB Contributor T-shirt

How to get started:

  • Deploy in your test environment: Software is best tested in a realistic environment. Help us see how 2.6 fares with your code and data so that others can build and run applications on MongoDB 2.6 successfully.
  • Test new features and improvements: Dozens of new features were added in 2.6. See the 2.6 Release Notes for a full list.
  • Log a ticket: If you find an issue, create a report in Jira. See the documentation for a guide to submitting well written bug reports.

If you are interested in doing this work full time, consider applying to join our engineering teams in New York City, Palo Alto and Austin, Texas.

Happy hunting!

Eliot, Dan and the MongoDB Team

3D Repo Runs MongoDB

Dec 12 • Posted 1 year ago

If you’re an architectural or engineering firm, you’ve undoubtedly confronted the difficulty of managing and collaborating on 3D assets like CAD drawings.  The act of sharing massive files is hard but feasible, but it is significantly complicated by the inability to determine that you’re using the latest version.  For the CAD-inclined, there’s hope. Jozef Dobos, a doctoral student at University College London (UCL), has applied the geospatial indexing capabilities of MongoDB a version control system for 3D assets called 3D Repo.  Sponsored by Arup Foresight, the built environment innovation division of Arup Group Limited, a global design and business consulting firm with offices in over 30 countries, 3D Repo leverages the flexibility of MongoDB’s data model, not to mention its geospatial capabilities, to make collaboration on 3D assets easy.

Read more

How MongoDB makes custom e-commerce easy

Sep 17 • Posted 1 year ago

The market for open source e-commerce software has gone through a lot of stages already, as you might know it by popular platforms like osCommerce, Magento, Zen Cart, PrestaShop, Spree, just to name a few. These platforms are frequently used as a basis for custom e-commerce apps, and they all require a SQL database. Given the inherent challenge in adapting open source software to custom features, it would seem that MongoDB is poised to play an important role in the next wave of e-commerce innovation.

Kyle Banker was one of the first to blog about MongoDB and e-commerce in April 2010, and there’s been surprisingly little written about it since then. In his blog, Kyle writes about Magento and other SQL based platforms: “What you’ll see is a flurry of tables working together to provide a flexible schema on top of a fundamentally inflexible style of database system.”

To this we must ask, why is a flexible schema so important in e-commerce?

Open source platforms are meant to be adapted to many different designs, conversion flows, and business processes. A flexible schema helps by giving developers a way to relate custom data structures to the platform’s existing model. Without a flexible schema, the developer has to get over high hurdles to make a particular feature possible. When the cost of creating and maintaining a custom feature is too high, the options are: give up the feature, start over with a different platform, or build a platform from scratch. That’s an expensive proposition.

There is a better way

For the past year we’ve been developing Forward, a new open source e-commerce platform combined with MongoDB. It’s been in production use since March 2012, and finally reached a point where we can demonstrate the benefits that MongoDB’s schema-less design brings to custom feature development.

The following examples demonstrate Forward’s REST-like ORM conventions, which are only available in the platform itself, but the underlying concepts map directly to MongoDB’s document structure. In this case, think of get() as db.collection.find() — put() as insert/update() — post() as insert() — and delete() as… delete().

Prototype faster

The majority of e-commerce sites represent small businesses, where moving fast can be the most important aspect of a web platform. When the flexible document structure of MongoDB is carried through the platform’s model interface, adding custom fields becomes easier than ever.

For example, let’s say you need a simple administrative view for adding a couple custom attributes to a product. Here’s a basic example for that purpose, written in Forward’s template syntax:

{args $product_id}

{if $request.post}
    {$product = put("/products/$product_id", [
        spec => $params.spec,
        usage => $params.usage
    ])}
    {flash notice="Saved" refresh=true}
{else}
    {$product = get("/products/$product_id")}
{/if}

<for method="post">
    <div class="field">
        <label>Product specification</label>
        <textarea name="spec">{$product.spec|escape}</textarea>
    </div>
    <div class="field">
        <label>Product usage instructions</label>
        <textarea name="usage">{$product.usage|escape}</textarea>
    </div>
    <button type="submit">Save product</button>
</form>

It might be obvious what this template does, but what might be less obvious is that the platform knows nothing about the “spec” or “usage” fields, and yet they are treated as if the e-commerce data model was designed for them. No database migration necessary, just code.

You may argue this can be accomplished with a fuzzy SQL database structure, and you would be correct, but it’s not pretty, or readable with standard database tools. Ad-hoc queries on custom fields would become difficult.

Query on custom fields

If all we needed were custom key/value storage, you might not benefit that much from of a flexible schema. Where MongoDB really shines is in its ability to query on any document field, even embedded documents.

{get $oversized_products from "/products" [
    oversized => true,
    active => true
]}

There are {$oversized_products.count} active oversized products

These fields may or may not be known by the e-commerce API, but in this case MongoDB’s query syntax finds only the documents with matching fields.

No more relational complexity

For those who spent years writing relational SQL queries, this is a big change. How do we create data relationships without joins? There are many different strategies, but Forward defines a field as either a static value or a callback method. This allows a field to return another document or collection based on a query. The result is a data model that can walk through relationships without joins. For example (PHP):

// class Accounts extends AppModel
...
$this->fields => array(
    ...
    'orders' => function ($order) {
        return get("/orders", array('account_id' => $account['id']));
    }
);

This relationship can be used in a template like this:

{get $account from "/accounts/$session.account_id"}

You’ve placed

<table>
    {foreach $account.orders as $order}
        <tr>
            <td>#{$order.id}</td>
            <td>${$order.sub_total}</td>
            <td>${$order.grand_total}</td>
            <td>{$order.items|count} item(s)</td>
        </tr>
    {/foreach}
</table>

Relationships can be defined by simple or complex queries. Results are lazy-loaded, making this example possible:

{get $order from "/orders/123"}

{$order.account.name} placed {$order.account.orders.count} orders since {$order.account.orders.first.date_created|date_format}

// Output: John Smith placed 3 orders since Jun 14, 2012

What about transactions?

Many people bring up MongoDB’s lack of atomic transactions across collections as evidence that it’s not suitable for e-commerce applications. This has not been a significant barrier in our experience so far.

There are other ways to approach data integrity. In systems with low-moderate data contention, optimistic locking is sufficient. We’ll share more details about these strategies as things progress.

In conclusion

The future of e-commerce software looks bright with MongoDB. It’s time to blaze new trails where convoluted schemas, complex relational queries, and hair raising database migrations are a thing of the past. If you’re interested in working with Forward before public release, please consider joining the private beta and help us reinvent open source e-commerce as the world knows it.

A guest post from Eric Ingram, developer/founder @getfwd

MongoDB Sharding Visualizer

Sep 14 • Posted 1 year ago

We’re happy to share with you the initial release of the MongoDB sharding visualizer. The visualizer is a Google Chrome app that provides an intuitive overview of a sharded cluster. This project provides an alternative to the printShardingStatus() utility function available in the MongoDB shell.

Features

The visualizer provides two different perspectives of the cluster’s state.

The collections view is a grid where each rectangle represents a collection. Each rectangle’s area is proportional to that collection’s size relative to the other collections in the cluster. Inside each rectangle a pie chart shows the distribution of that collection’s chunks over all the shards in the cluster.

The shards view is a bar graph where each bar represents a shard and each segment inside the shard represents a collection. The size of each segment is relative to the other collections on that shard.

Additionally, the slider underneath each view allows rewinding the state of the cluster. select and view the state of the cluster at a specific time.

Installation

To install the plugin, download and unzip the source code from 10gen labs. In Google Chrome, go to Preferences > Extensions, enable Developer Mode, and click “Load unpacked extension…”. When prompted, select the “plugin” directory. Then, open a new tab in Chrome and navigate to the Apps page and launch the visualizer.

Feedback

We very much look forward to hearing feedback and encourage everyone to look at the source code which is available https://github.com/10gen-labs/shard-viz .

Motor: Asynchronous Driver for MongoDB and Python

Sep 5 • Posted 1 year ago

Tornado is a popular asynchronous Python web server. Alas, to connect to MongoDB from a Tornado app requires a tradeoff: You can either use PyMongo and give up the advantages of an async web server, or use AsyncMongo, which is non-blocking but lacks key features.

I decided to fill the gap by writing a new async driver called Motor (for “MOngo + TORnado”), and it’s reached the public alpha stage. Please try it out and tell me what you think. I’ll maintain a homepage for it here, including basic documentation.

Status

Motor is alpha. It is certainly buggy. Its implementation and possibly its API will change in the coming months. I hope you’ll help me by reporting bugs, requesting features, and pointing out how it could be better.

Advantages

Two good projects, AsyncMongo and APyMongo, took the straightforward approach to implementing an async MongoDB driver: they forked PyMongo and rewrote it to use callbacks. But this approach creates a maintenance headache: now every improvement to PyMongo must be manually ported over. Motor sidesteps the problem. It uses a Gevent-like technique to wrap PyMongo and run it asynchronously, while presenting a classic callback interface to Tornado applications. This wrapping means Motor reuses all of PyMongo’s code and, aside from GridFS support, Motor is already feature-complete. Motor can easily keep up with PyMongo development in the future.

Installation

Motor depends on greenlet and, of course, Tornado. It is compatible with CPython 2.5, 2.6, 2.7, and 3.2; and PyPy 1.9. You can get the code from my fork of the PyMongo repo, on the motor branch:

pip install tornado greenlet pip install git+https://github.com/ajdavis/mongo-python-driver.git@motor To keep up with development, watch my repo and do

pip install -U git+https://github.com/ajdavis/mongo-python-driver.git@motor when you want to upgrade.

Example

Here’s an example of an application that can create and display short messages:

Other examples are Chirp, a Twitter-like demo app, and Motor-Blog, which runs this site.

Support For now, email me directly if you have any questions or feedback.

Roadmap In the next week I’ll implement the PyMongo feature I’m missing, GridFS. Once the public alpha and beta stages have shaken out the bugs and revealed missing features, Motor will be included as a module in the official PyMongo distribution.

A. Jesse Jiryu Davis

MongoDB on Windows Azure

Jul 19 • Posted 2 years ago

This post originally appeared on the Microsoft Interoperability Blog.  

Do you need to build a high-availability web application or service? One that can scale out quickly in response to fluctuating demand? Need to do complex queries against schema-free collections of rich objects? If you answer yes to any of those questions, MongoDB on Windows Azure is an approach you’ll want to look at closely.

People have been using MongoDB on Windows Azure for some time (for example), but recently the setup, deployment, and development experience has been streamlined by the release of the MongoDB Installer for Windows Azure. It’s now easier than ever to get started with MongoDB on Windows Azure!

MongoDB

MongoDB is a very popular NoSQL database that stores data in collections of BSON (binary JSON) objects. It is very easy to learn if you have JavaScript (or Node.js) experience, featuring a JavaScript interpreter shell for administrating databases, JSON syntax for data updates and queries, and JavaScript-based map/reduce operations on the server. It is also known for a simple but flexible replication architecture based on replica sets, as well as sharding capabilities for load balancing and high availability. MongoDB is used in many high-volume web sites including Craigslist, FourSquare, Shutterfly, The New York Times, MTV, and others.

If you’re new to MongoDB, the best way to get started is to jump right in and start playing with it. Follow the instructions for your operating system from the list of Quickstart guides on MongoDB.org, and within a couple of minutes you’ll have a live MongoDB installation ready to use on your local machine. Then you can go through the MongoDB.org tutorial to learn the basics of creating databases and collections, inserting and updating documents, querying your data, and other common operations.

MongoDB Installer for Windows Azure

The MongoDB Installer for Windows Azure is a command-line tool (Windows PowerShell script) that automates the provisioning and deployment of MongoDB replica sets on Windows Azure virtual machines. You just need to specify a few options such as the number of nodes and the DNS prefix, and the installer will provision virtual machines, deploy MongoDB to them, and configure a replica set.

Once you have a replica set deployed, you’re ready to build your application or service. The tutorial How to deploy a PHP application using MongoDB on Windows Azure takes you through the steps involved for a simple demo app, including the details of configuring and deploying your application as a cloud service in Windows Azure. If you’re a PHP developer who is new to MongoDB, you may want to also check out the MongoDB tutorial
on php.net
.

Developer Choice

MongoDB is also supported by a wide array of programming languages, as you can see on the Drivers page of MongoDB.org. The example above is PHP-based, but if you’re a Node.js developer you can find a the tutorialNode.js Web Application with Storage on MongoDB over on the Developer Center, and for .NET developers looking to take advantage of MongoDB (either on Windows Azure or Windows), be sure to register for the free July 19 webinar that will cover the latest features of the MongoDB .NET driver in detail.

The team at Microsoft Open Technologies is looking forward to working closely with 10gen to continue to improve the MongoDB developer experience on Windows Azure going forward. We’ll keep you updated here as that collaboration continues!

Revamp of MongoDB’s Documentation

May 1 • Posted 2 years ago

We’re revamping MongoDB’s documentation. The new design in the MongoDB Manual has an improved reference section and an index for simplified search. It will also eventually support multiple MongoDB versions at the same time.

This project is a work in progress, and things are changing quickly. Our goal is to consolidate, sharpen, organize, and continue to improve the documentation in support of MongoDB. For now, the new docs will live alongside the original MongoDB Wiki. But over the next few months, we’ll be transitioning everything to the new manual.

In the spirit of open source, the docs are housed on Github. Feedback is welcome! Feel free to fork the repository and issue pull requests. You can also open tickets in JIRA, and we’ll promptly address any suggestions.

blog comments powered by Disqus