Cloud Storage: 8 Ways You're Wasting Money

MongoDB on Tuesday announced that it has acquired WiredTiger, a provider of a high-scale storage engine used by myriad vendors including Amazon Web Services. WiredTiger is also used as an alternative to the InnoDB storage engine used with Oracle MySQL.

MongoDB had already announced plans to include the WiredTiger storage engine with its coming 2.8 release, currently available in beta and expected in general release early next year. The company said that adding the technology to its portfolio and its veteran senior executives to its engineering team will help it address performance constraints that have always limited the use of its database.

MongoDB is hugely popular and supports many high-scale applications, but the database-level locking supported by its incumbent MMAP storage engine is best suited to read-intensive applications. MongoDB hasn't performed or scaled as well in write-intensive and mixed write-and-read-intensive applications.

[Want more on database choices? Read NoSQL Database Choices: Weather Co. CIO's Advice.]

In contrast to MMAP, WiredTiger supports more granular, document-level locking for optimized writing efficiency. WiredTiger is also more efficient than MMAP in harnessing the power of large, multi-core servers. That means customers using this alternative will be able to support bigger apps with less hardware, according to Kelly Stirman, MongoDB's director of products.

"A lot of the applications that companies are building today try to take a fire hose of data and index it and query it efficiently," said Stirman in a phone interview with InformationWeek. "Those apps need really fast, efficient writes."

In competitive bids for such apps, Cassandra is usually MongoDB's biggest competitor, according to Stirman. Cassandra was inspired by Amazon Dynamo, which was developed by the retailer first and foremost to accept database writes at high scale no matter what. Accepting writes is what enables customers to place items in virtual shopping baskets and then pay for their goods. The DynamoDB cloud service inspired by Dynamo and Cassandra favor write performance over consistency and read performance. (WiredTiger is used with separate AWS cloud services, not DynamoDB.)

MongoDB will continue to offer MMAP, which will gain more granular locking in the 2.8 release, but it's still best suited to read-intensive use cases where information doesn't change a lot, such as content-management, customer-lookups, and caching apps.

With WiredTiger available as a no-cost option, customers will be able to support e-commerce uses cases, Internet-of-Things-style sensor apps, log-data analysis, and other write-intensive needs. the engine also supports data compression, which saves on storage space and thereby reduces hardware costs in high-scale deployments.

Stirman predicted that MongoDB 2.8 and subsequent releases will blow away old performance and scalability concerns. "Before we had workarounds to improve performance [with MMAP], but with WiredTirger, we won't have any of the old limitations," he said. "We'll be able to scale dramatically, and you'll see much stronger performance."

Joining MongoDB from WiredTiger will be co-founders Keith Bostic and Dr. Michael Cahill. Bostic was a founder of Sleepycat, which developed the open-source Berkeley DB key-value-store database before it was acquired by Oracle in 2006. Bostic will become a senior staff engineer at MongoDB. Cahill will become MongoDB's director of engineering.

"Our focus at WiredTiger has been to rethink data management and create high-performance software that solves the challenges of the world's most demanding applications," said Cahill in a statement from MongoDB. "Joining [MongoDB's] vast community is a huge opportunity for WiredTiger to more broadly benefit organizations of all sizes, in all industries, around the globe."

MongoDB will continue to develop the WiredTiger storage engine and offer it to other software vendors. Meanwhile, the MongoDB database might eventually support "dozens of different storage engines," said Stirman, citing in-memory engines, a Hadoop file-system engine, and SSD options among the possibilities. "The rest of MongoDB is all the same, including security features, query language, index support, and everything else, but you'll be able to pick the storage engine that gives you the best balance of capabilities for your application."

The terms of the WiredTiger acquisition were not disclosed.

Apply now for the 2015 InformationWeek Elite 100, which recognizes the most innovative users of technology to advance a company's business goals. Winners will be recognized at the InformationWeek Conference, April 27-28, 2015, at the Mandalay Bay in Las Vegas. Application period ends Jan. 16, 2015.