Performance Tips: MongoDB at Firescope

Dec 19 • Posted 1 year ago

Guest post by Pete Whitney

Starting to work with any new technology or new API is always challenging at first. You’re often not quite sure of the best ways to get things done or if you’re are using the new technology in the most efficient manner. Furthermore, the early learning process is often littered with trial and error improvements that unfortunately take time to rework and reengineer into more optimal solutions. It sure would be nice if we could short circuit this learning process and simply arrive at nirvana on the first cut. While I won’t proclaim that the destination of the blog is nirvana, I will try to short circuit the learning process by sharing four specific performance related tips that we learned at FireScope Inc. when we transitioned from MySQL to MongoDB for our improved cloud based Stratis product. This blog will share the shorthand version of these tips and point the reader to a more in depth rendering if further understanding is desired.

  1. Through a comparative analysis FireScope found that accessing MongoDB via the MongoDB java driver was three times faster than doing the same access using SpringData. While SpringData yields many benefits it accomplishes its job using a reflection based solution that is performed on top of the native MongoDB java driver. So for FireScope’s performance centric considerations paying a 3X performance penalty for its benefits was not a tradeoff we were willing to make.
Read more

3D Repo Runs MongoDB

Dec 12 • Posted 1 year ago

If you’re an architectural or engineering firm, you’ve undoubtedly confronted the difficulty of managing and collaborating on 3D assets like CAD drawings.  The act of sharing massive files is hard but feasible, but it is significantly complicated by the inability to determine that you’re using the latest version.  For the CAD-inclined, there’s hope. Jozef Dobos, a doctoral student at University College London (UCL), has applied the geospatial indexing capabilities of MongoDB a version control system for 3D assets called 3D Repo.  Sponsored by Arup Foresight, the built environment innovation division of Arup Group Limited, a global design and business consulting firm with offices in over 30 countries, 3D Repo leverages the flexibility of MongoDB’s data model, not to mention its geospatial capabilities, to make collaboration on 3D assets easy.

Read more

November Driver Releases

Dec 10 • Posted 1 year ago

On November 27, all 10gen supported drivers were updated with new error checking and reporting defaults. Each driver now has a MongoClient connection class to handle the error checking. On the same day there was also a server release with fixes on 2.2

MongoQP: MongoDB Slow Query Profiler

Dec 5 • Posted 1 year ago

Two times a year 10gen’s Drivers and Innovations team gather together for a face to face meeting to work together and setting goals for the upcoming six months. This year the team broke up into teams for an evening hackathon. MongoQP, a query profiler, was one of the hacks presented by Jeremy Mikola, PHP Evangelist at 10gen.

Logging slow queries is essential for any database application, and MongoDB makes doing so relatively painless with its database profiler. Unfortunately, making sense of the system.profile collection and tying its contents back to your application requires a bit more effort. The heart of mongoqp (Mongo Query Profiler) is a bit of map/reduce JS that aggregates those queries by their BSON skeleton (i.e. keys preserved, but values removed). With queries reduced to their bare structure, any of their statistics can be aggregated, such as average query time, index scans, counts, etc.

image

Read more

Lessons Learnt Building mongoengine

Nov 29 • Posted 1 year ago

Recently, I attended both Pycon UK and Pycon Ireland to talk about the lessons I have learnt while maintaining mongoengine. The conferences were both excellent and surprisingly different. Pycon UK had quite an “unconference” feel, with some exciting sprint rooms - I wish I had more time as by all reports the educational jam was inspirational. Pycon Ireland in contrast felt more slick with booths from DemonWare, Amazon and Facebook. If you can, I’d advise going to both conferences as they really complement each other.

Read more

Introducing MongoClient

Nov 27 • Posted 1 year ago

Today we are releasing updated versions of most of the officially supported MongoDB drivers with new error checking and reporting defaults. See below for more information on these changes, and check your driver docs for specifics.

Over the past several years, it’s become evident that MongoDB’s previous default behavior (where write messages did not wait for a return code from the server by default) wasn’t intuitive and has caused confusion for MongoDB users. We want to rectify that with minimal disruption to the MongoDB apps already in production.

Read more

Storage-viz: Storage Visualizers and Commands for MongoDB

Nov 20 • Posted 1 year ago

Storage-viz is a suite of web-based visualizers and new experimental database commands that may help you understand how MongoDB utilizes storage and organizes btrees. Storage-viz is now available in the MongoDB Nightly builds.

image

When a MongoDB collection is created, an on-disk extent is allocated to store the documents. Each time a newly created or updated document cannot fit into the existing collection’s extents, a new extent is created. Each document occupies a contiguous storage area - a record - in one of the collection’s extents. Storage-viz’ experimental storageDetails command extracts information about how the disk storage is used and the web-based visualizer generates an easy-to-read graphical representation. Storage-viz also showcases which parts of the collection’s extents are currently in RAM [NOTE: the visualizer doesn’t display how much memory is available].

Read more

Interview with Joe Devon, Organizer of the Los Angeles MongoDB User Group

Nov 12 • Posted 1 year ago

The Los Angeles MongoDB User Group was founded in the summer of 2011 and, thanks to the leadership of Joe Devon, has grown to over 400 members in the past year. Joe Devon, a long-time member of the Silicon Beach tech community, shares his insights from a year of working with the MongoDB community in Los Angeles. 

You’re a User Group Veteran. How did you get started organizing user groups?

I was living in NY, with several meetups to choose from every night. Then moved to Los Angeles, where there were none. In the words of Cal Evans, if you don’t know who your local organizer is, you’re looking at him :) 

So I started a bunch of meetups, told people to gather at Panera Bread once a month in one big, joint set of meetups…. And couldn’t get 10 people to show. But after awhile, there were some regulars. Some of whom agreed to take over a group here, a group there. Fast forward to today and there’s a ton of tech meetups in Los Angeles, with sometimes 100-200 people showing up on good nights. 

Read more

Data Science on MongoDB…At Last!

Nov 7 • Posted 1 year ago

Today, I’m excited to announce the launch of Precog for MongoDB, a release that bundles all of the really cool Precog technology into a free package that anyone can download and deploy on their existing MongoDB database.

Precog is a data science platform that lets developers and data scientists do advanced analytics and statistics using Quirrel, the “R for big data” language. You can analyze data programmatically with a REST API (or client library) or interactively with Labcoat, an easy-to-use HTML5 application built on the REST API. We provide a cloud-hosted version of Precog, but we’ve known for a long time that we were going to bring a standalone version of our data science Precog to some NoSQL database.
Read more

MongoDB for the PHP Mind, Part 3

Nov 6 • Posted 1 year ago

This is part 3 in a series, which will focus on the data modeling aspect of working with document databases. The previous parts are also available for reading: Part 1: Getting Started, and Part 2: Queries and Indexes.

The Usual Suspects

Although there are plenty of existing articles, presentations and webcasts about modeling your data to take advantage of a document database, this post is taking a slightly PHP-centric position as a part of this series. This information is useful to anyone though, regardless of their chosen programming language.

We’re going to use two different scenarios to look at data modeling in the document world, chosen as common examples to illustrate differences in implementation between relational and document databases:

  • Blog. Your garden variety of data, covering posts, comments and tags
  • Private Sale / E-Commerce. Taking a look at needs for orders, users and products

Read more

October MongoDB Blogroll and Releases

Nov 5 • Posted 1 year ago

Password Authentication with Mongoose (Part 2): Account Locking

Oct 24 • Posted 1 year ago

This post is Part 2 (of 2) on implementing secure username/password authentication for your Mongoose User models. In Part 1 we implemented one-way password encryption and verification using bcrypt. Here in Part 2 we’ll discuss how to prevent brute-force attacks by enforcing a maximum number of failed login attempts. This was originally posted on the DevSmash Blog 

Quick Review

If you haven’t done so already, I recommend you start with reading Part 1. However, if you’re like me and usually gloss over the paragraph text looking for code, here’s what our User model looked like when we left off:

As can be seen, there’s not much too it - we hash passwords before documents are saved to MongoDB, and we provide a basic convenience method for comparing passwords later on.

Why do we Need Account Locking?

While our code from Part 1 is functional, it can definitely be improved upon. Hashing passwords will save your bacon if a hacker gains access to your database, but it does nothing to prevent brute-force attacks against your site’s login form. This is where account locking comes in: after a specific number of failed login attempts, we simply ignore subsequent attempts, thereby putting the kibosh on the brute-force attack.

Unfortunately, this still isn’t perfect. As stated by OWASP:

Password lockout mechanisms have a logical weakness. An attacker that undertakes a large numbers of authentication attempts on known account names can produce a result that locks out entire blocks of application users accounts.

The prescribed solution, then, is to continue to lock accounts when a likely attack is encountered, but then unlock the account after some time has passed. Given that a sensible password policy puts the password search space into the hundreds of trillions (or better), we don’t need to be too worried about allowing another five guesses every couple of hours or so.

Read more

Online MongoDB Classes Start Today

Oct 23 • Posted 1 year ago

A guest post from Andrew Erlichson, 10gen’s VP of Education

Free, Online MongoDB classes start this week and there is still time to join one of the two courses: MongoDB for Developers and MongoDB for DBAs.

I am teaching the developer course along with Richard Kreuter. Richard is a lead consulting engineer and has significant experience with MongoDB users around the world. Dwight Merriman, founder of 10gen and one of the creators of MongoDB, will be teaching the MongoDB for Administrators course.


Right now we over 20,000 registrations for the classes. Lots of folks have contacted us asking what they should expect from these courses, so we thought we’d share some information on what we’re planning. 

Read more

ReactiveMongo for Scala: Unleashing MongoDB Streaming capabilities for Realtime Web

Oct 18 • Posted 1 year ago

This is a guest post by Stéphane Godbillon, software architect at Zenexity _ I am very excited to introduce ReactiveMongo, a brand new Scala driver for MongoDB. More than just yet-another-async-driver, it’s a _reactive driver that allows you to design very scalable applications unleashing MongoDB capabilities like streaming infinite live collections and files for modern Realtime Web applications.

What does reactive mean?

I/O operations may be handled in the following ways: * synchronously: each time a request is sent, the running thread is blocked, waiting for the response to arrive. When the response is received, the execution flow resumes. * asynchronously: the code handling the response may be not run simultaneously (often in a closure). Still, a thread may be blocked, but it may not be the same thread that did the request. * non-blocking: sending a request does not block any thread.

Read more

Planet MongoDB

Oct 17 • Posted 1 year ago

To read up on the most up-to-date blog posts on MongoDB check out Planet MongoDB, a newly-released aggregator of the best blogs on MongoDB. MongoDB community members share tips, ticks and best practices on using MongoDB on a regular basis, and Planet MongoDB will bring that expertise into a single feed.

If you have a blog you would like to see added to the aggregator let us know so we can add you in.

blog comments powered by Disqus