MongoDB performance best practices

MongoDB performance best practices

Published on Author admin

MongoDB performance best practicesMongoDB performance best practices

MongoDB is a very high-performance, scalable NoSQL database
designed for a large array of modern applications. Due to its high throughput
and continuous availability, it is used by many organizations of all sizes to power on-line,
operational applications.

This article will guides you through MongoDB performance best practices to tune your performance. All the recommendations are made by me based on successful completion of M101(MongoDB for DBAs) and M201(MongoDB Advanced Deployment and Operations) MongoDB University Courses, running MongoDB in real high-load production and other articles like Amazon AWS MongoDB performance best practices white paper.

 

Ulimit settings:

At your production environment you may not have problems with system limits. The following thresholds and settings are particularly important for mongod and mongos deployments:

-f (file size): unlimited
-t (cpu time): unlimited
-v (virtual memory): unlimited
-n (open files): 999999
-m (memory size): unlimited

You must restart your mongod and mongos instances after changing the ulimit settings to make sure that the settings change takes effect.

 

Network Setting:

Change default keep-alive time to 300
echo 300 > /proc/sys/net/ipv4/tcp_keepalive_time

 

Engines:

MongoDB has several engines available. The most common are MMAPv1 and WiredTiger.

  • MMAPv1:
    • Was the default engine for all version up to 3.x.x.
  • WiredTiger:
    • New default engine for version starting 3.x.x
    • It is better to use WiredTiger(compression, no collection locks)
    • WiredTiger can be run without journaling, as it takes consistent snapshot of the data in about 1 minute

To enable WiredTiger engine you should set this option in MongoDB config:

Take a look at the cacheSizeGB parameter. Naturally you will not mix your DB service with other services on the same machine in production, so it is recommended to set cacheSizeGB to 70-80% of your RAM size. Well, this also depends on the size of your RAM. You can subtract from the total RAM the amount that is needed for OS, leave some reasonable gap for who-knows-what and the rest define as cacheSizeGB. I should also point that there is no need to leave a big amount of RAM useless. OS will use it as a file cache without performance profit for you. So, this is the first practice from my MongoDB performance best practices.

 

CPU:

MongoDB will deliver better performance on faster CPUs. So, one of the MongoDB performance best practices is that the core speed is more important than the number of cores. And the MongoDB WiredTiger storage engine is better able to saturate multi-core processor resources than the MMAPv1 storage engine.

  • Clock speed more important rather than number of cores(faster math for ‘count’, distinct, etc operations)

 

SWAP:

  • Strongly recommended for OOM killer avoidance
  • To keep the kernel from killing MongoDB when running short on memory
  • To give you as an administrator greater control over how memory is used on a system running MongoDB

Here you can find quick example how to enable SWAP on your Linux machine – SWAP

 

Hardware Memory(RAM):

Running MongoDB on a system with Non-Uniform Access Memory (NUMA) can cause a number of operational problems, including a slow performance for periods of time and high system process usage. MongoDB doesn’t support NUMA.

  • NO NUMA. MongoDB doesn’t support it for now(version 3.2)
    • Running MongoDB on a system with Non-Uniform Access Memory (NUMA) can cause a number of operational problems, including slow performance for periods of time and high system process usage.When running MongoDB servers and clients on NUMA hardware, you should configure a memory interleave policy so that the host behaves in a non-NUMA fashion. MongoDB checks NUMA settings on start up when deployed on Linux (since version 2.0) and Windows (since version 2.6) machines. If the NUMA configuration may degrade performance, MongoDB prints a warning.  Ho handle NUMA in Debian your need ‘numactl’ to be installed:

The Debian MongoDB upstart script now checks for the presence of the numactl binary and adjusts accordingly.

 

  • Disable Transparent Huge Pages (THP)
    • Transparent Huge Pages (THP) is a Linux memory management system that reduces the overhead of Translation Lookaside Buffer (TLB) lookups on machines with large amounts of memory by using larger memory pages. However, database workloads often perform poorly with THP, because they tend to have sparse rather than contiguous memory access patterns. You should disable THP on Linux machines to ensure best performance with MongoDB. To disable THP put this to your /etc/rc.local, for example:

  • 1000 connections = 1GB RAM memory. So, keep in mind this number.

 

HDD and FileSystems:

  • ext4, xfs are tested and recommended by MongoDB
  • no atime(updates even on reads)
  • no ReadAhead for SSDs. ReadAhead good only for OpLog and CappedCollections
  • Use SSDs for write-heavy applications
  • raid-10 and raid-0 show the best performance

 

Virtualization:

If your are running your MongoDB using any kind of containers it is strongly recommended to:

  • Disable memory ballooning

 

Replica Set:

  • Changed in version 3.0.0: A replica set can have up to 50 members but only 7 voting members. In previous versions, replica sets can have up to 12 members.

 

Short Summary of the MongoDB performance best practices:

CPU:

  • Clock speed more important rather than number of cores(faster math for ‘count’, distinct, etc operations)

SWAP:

  • Strongly recommended for OOM killer avoidance
  • To keep the kernel from killing MongoDB when running short on memory
  • To give you as an administrator greater control over how memory is used on a system running MongoDB

Engine:

  • It is better to use WiredTiger(compression, no collection locks)
  • WiredTiger can be run without journaling, as it takes consistent snapshot of the data in about 1 minute

RAM:

  • NO NUMA. MongoDB doesn’t support it for now(version 3.2)
  • Turn off ‘Transparent Huge Pages (THP)’
  • 1000 connections = 1GB RAM memory. So, keep in mind this number.

HDD and FileSystems:

  • ext4, xfs are tested and recommended by MongoDB
  • no atime(updates even on reads)
  • no ReadAhead for SSDs. ReadAhead good only for OpLog and CappedCollections
  • Use SSDs for write-heavy applications
  • raid-10 and raid-0 show best performance

Virtualization:

  • Disable memory ballooning
  • Disabled overcommit feature in VMWare machinesOvercommitting resources is very bad, particularly memory. This is because VMWare will swap them around guest VMs on the host and your guest VMs will suddenly not have memory available.

Replica Set:

  • Changed in version 3.0.0: A replica set can have up to 50 members but only 7 voting members. In previous versions, replica sets can have up to 12 members.

 

Additional information:

Amazon MongoDB Performance white paper

MongoDB production notes