MongoDB Atlas with Meteor: A Step-By-Step Guide
Blog

MongoDB Atlas with Meteor: A Step-By-Step Guide

MongoDB Atlas loves Meteor

Last updated: 3rd February 2017

MongoDB Atlas which launched on June 28, 2016, is a Database-as-a-Service platform created by the same team that builds MongoDB.

Here at OK GROW! we have been relying on mLab and Compose for hosting our databases.

With the welcome update of Meteor’s mongodb driver in the release of Meteor 1.4 (July 25, 2016). Meteor has now been shipping, by default, Mongo 3.2 alongside MongoDB’s WiredTiger Storage Engine.

When we looked into upgrading from MMAPv1 to WiredTiger (on mLab and compose), we were surprised to see a significant increase in cost. We decided to investigate the most cost effective Database-as-a-Service platform for Mongo 3.2+ with WiredTiger.

Hosting costs:

Note: Costs were evaluated at time of writing.

MongoDB Atlas:

  • MMAPv1: MongoDB Atlas has no MMAPv1 plans, Atlas only supports the WiredTiger Storage Engine.
  • WiredTiger: The smallest deployment (M10 Instance, 2GB RAM, 10GB Storage, 3 Data Bearing Nodes) comes in at about $56.94 a month. Automatic backups will increase your costs, but you get your first 1GB per Replica Set for free, then it’s $2.50 per GB/month.

Compose.io:

  • MMAPv1: Cheapest option starts at $31 a month with (102MB RAM, 1GB Storage). Scaling costs $18 per block of 1 GB Storage, which comes with an additional 102MB of RAM.
  • WiredTiger: Cheapest option starts at $133 a month with 1GB RAM, 4GB Storage, 3 Data Bearing Nodes (1 Node is hidden and used for backups only). Each additional gigabyte of storage includes 256MB of RAM and costs $30 to scale.

mLab:

  • MMAPv1: Offers a Single-node free Sandbox option (shared memory, 0.5GB of Storage, no oplog access).
  • WiredTiger: At the time of writing, not all of mLab’s plans support WiredTiger. Currently their cheapest production option is the Standard Line: M3 Cluster (7.5GB RAM, 120GB Storage, 2 Data Bearing Nodes with 1 Arbiter Node) coming in at $720 a month.

NOTE: mLab does offer WiredTiger for $420 a month on a M3 Single-node (7.5GB, 120GB Storage) but they are only suitable for development/testing/staging environments. Do not use this plan for production deployments.

Our findings:

MongoDB Atlas is currently (at the time of writing) the cheapest and most attractive Database-as-a-Service for hosting WiredTiger with Mongo 3.2+. A detailed comparison can be found here.

This made it an ideal opportunity to evaluate MongoDB Atlas for usage with a Meteor 1.4+ web app hosted on Meteor Galaxy.

Let’s get started!


What follows is a step-by-step guide to getting started with MongoDB Atlas, and connecting your Meteor app (hosted on Meteor Galaxy) to your hosted MongoDB with oplog tailing.

Step 1: Sign up for Atlas

Sign up for a MongoDB Atlas Account by completing your profile. Don’t worry; the sign up process is relatively quick and pain-free.

Register for MongoDB Atlas

Step 2: Create your first Atlas Group

Create a name for your MongoDb Atlas Group. Do pay attention to the name you set. You are unable to edit the name or delete the original Atlas Group you created when signing up.

Check out the Q & A: What is an Atlas Group ?

Name your default Atlas Group

It is recommended that you setup 2 Factor Authentication (2FA) before deploying your first cluster. You can find the 2FA option under Settings -> Account.

Enable 2FA

Step 3: Create your first Cluster

Create your first cluster

Step 3.1: Cluster Name

Name your first cluster. Be careful: you cannot rename this!

Step 3.2: AWS Region

Next select your AWS region:

  • If deploying for Galaxy in North America select us-east-1 (N. Virginia), this is Galaxy’s default region.
  • If deploying for Galaxy in Europe select eu-west-1 (Ireland).

Check out the Q & A: What regions are supported ?

Step 3.3: Instance Size

Next select the best instance size for your app. You are able to upgrade easily without any downtime so don’t worry about being conservative to begin with.

At this point you can also select to encrypt the storage volumes on the database. This decision is left up to you as it’s a trade-off between performance and security.

Any sensitive data that you store in your database (e.g - passwords) should have already been encrypted before storing in your database. Sensitive data should never be stored unencrypted (e.g - plain-text) in your database.

Creating your cluster

Step 3.4: Replication Factor

MongoDB Atlas forces you to deploy with Replica Sets. This is actually good! Replica Sets are the foundation for providing high availability and redundancy in a production deployment. This is also great for Meteor deployments as we do not have to do any extra setup to enable Meteor’s “oplog tailing”.

Now select from 3, 5 or 7 Nodes. You should select the number that best fits your apps requirements and tolerance for redundancy.

Step 3.5: Do you want a sharded cluster ?

Next: Sharding - ensure this is disabled (only enable if you know and are confident in what you are doing).

Check out the Q & A: What is Sharding ?

Step 3.6: Do you want to enable backup ?

By default, backups are enabled. It is recommended that you keep backups enabled for production databases.

Step 3.7: Admin Username & Password

Set the cluster’s username & password.

  • Atlas provides a handy password generator. Use it!
  • Make sure to record the password securely, it is not recoverable if you lose it.
  • An important caveat: Setting the cluster’s “Admin Username & Password” is only available when creating your first cluster in your Group. It is important to note that this user is the default admin for the Atlas Group.

Final cluster settings

Step 3.8: Confirm & Deploy

When you’re happy with your cluster’s settings click “Confirm & Deploy”. Your cluster is now being deployed, it can take up to 8 minutes to complete.

Atlas's cluster widget

Atlas provides a handy widget for viewing and accessing your cluster once it has been deployed. You will use this widget in later steps.

Step 5: Security - IP WhiteList & Peering.

In order to connect to your database on Atlas you must create a whitelist of IP addresses or ranges in CIDR notation, or if deployed on AWS you can use AWS VPC Peering.

IP whitelist overview

Meteor Galaxy

If deploying with Galaxy, you will need to provide the IP addresses for the region you are deployed to. At the time of writing Galaxy has two clusters us-east-1 and eu-west-1. The IP addresses for these clusters can be found here in Galaxy’s documentation. You must add all the IP addresses to your whitelist on Atlas for the region where your Meteor app is deployed (us-east-1 or/and eu-west-1).

To read about Galaxy’s Static IP Addresses and Whitelisting on Galaxy check out their guide here.

Custom Deployment

If deploying your Meteor app to a hosting service that doesn’t have static IP addresses to support IP Whitelisting then you will need to whitelist all IP addresses. You can whitelist all IP addresses with this range: 0.0.0.0/0. You can learn more here. Please note that by whitelisting all IP addresses your database will not become inherently insecure (Atlas requires all connections are authenticated). IP address whitelisting is mainly done to mitigate the database’s exposure to a DDoS attack.

AWS VPC Peering

If your deploying your Meteor app directly to AWS, MongoDB Atlas supports AWS VPC Peering. Check out their detailed guide on how to set Peering up here.

Step 6: Security - Users

We need to create at least two users. A “Meteor User” who has read/write abilities & an “Oplog User” for “oplog tailing”.

Security users overview

Step 6.1: Create the Meteor User

This user will be used for the "MONGO_URL".

  • Select: “Create new user”.
  • Set the “User Name”.
  • Select “Read/Write any Database”.
  • Create a password and save it! We need it for our MongoDB connection url.

    Creating the Meteor User for your MONGO_URL

Step 6.2: Create the Oplog User

This user will be used for the "MONGO_OPLOG_URL".

  • Select: “Create new user”.
  • Click “advanced”.
  • Set access to “read” @ “local”.
  • Create a password and save it! We need it for our MongoDB connection url.

Creating the Oplog User for your MONGO_OPLOG_URL

A note on users & their permissions:

  • Users are assigned to the Atlas Group and can have access to any cluster in the Group.
  • If you wish to restrict a user’s access to a different cluster, the recommended way is to create a new Atlas Group.
  • You can learn more about adding a user in MongoDB Atlas here.

OPTIONAL:

You can verify that you have created your two users correctly by connecting to the database via the mongo shell. You can do this by following the below instructions:

  • Click on the “Connect” button for your cluster.
  • Follow the instructions under the “Connect with the Mongo Shell” header.
  • Atlas by default enforces TLS/SSL. TLS/SSL cannot be switched off.
  • If you have installed Mongo via Meteor you may be required to reinstall with -ssl binary package
  • On MacOS you can reinstall mongo shell with the follow brew command: brew install mongodb --with-openssl. For other OS’s follow the instructions provided by Atlas.
  • Copy & paste the mongo shell string from Atlas into your terminal.
    • Ensure that you add the correct user’s password to end of your mongo query string.
    • Now you can connect to your database.
    • Follow the below commands to verify user’s and their roles.

Mongo shell commands to help:

// List all the databases on our primary node.
show dbs

// Meteor apps will normally only have 3 dbs
admin   0.000GB // For dbs authentication
local   0.000GB // For oplog
meteor  0.000GB // To store our Meteor collections. Won't exist until it's created

// Make sure you are connected to the primary node, and also on the admin database.
Use admin
db.system.users.find().pretty()
  • “Oplog User” should have: { “role”: “read”, “db”: “local” }
  • “Meteor User” should have: { "role" : "readWriteAnyDatabase", "db" : "admin" }

A word of caution, there are many commands that are restricted via the console. Even when logged in with full admin privileges. For example, creating a user with roles: is restricted via the Mongo shell. These restricted mongo shell commands are generally accessible through either Atlas’s UI or their public api.

Step 7: Connect your Meteor app to Atlas

We now need to create our "MONGO_URL" and "MONGO_OPLOG_URL" by retrieving our cluster’s “URI Connection String” from Atlas. This will be how we connect our Meteor app with our Mongo database.

It is recommended to have oplog enabled by default. You can disable oplog for specific cases where oplog is negatively effecting your app. You can do this by setting disableOplog to false when querying your collection.

Step 7.1: Create our “MONGO_URL”

To find your "MONGO_URL" click the “Connection” button located on your cluster’s UI widget. Copy the string located under the “URI Connection String:” label. This string will become your "MONGO_URL". Make sure to replace PASSWORD with the password of the user who has the readWriteAnyDatabase role. This is all we have to do for creating our "MONGO_URL".

Create our MONGO_URL

Step 7.2: Create our “MONGO_OPLOG_URL”

We create our "MONGO_OPLOG_URL" by first copying our "MONGO_URL" string. We now replace the existing username and password with our oplog user’s username and password. Next we modify the query located at the end of the "MONGO_OPLOG_URL" string.

Our "MONGO_OPLOG_URL" should contain at the end something looking like this: /meteor?authSource=admin&ssl=true&replicaSet=example-staging-shard-0. We need to replace this end section with this: /local?authSource=admin&ssl=true&replicaSet=example-staging-shard-0.

You may have noticed that we swapped /meteor? with /local?. This string is the name of the db where your collections and documents are stored. In the case of /local?, we wish to connect to the local database which is where Meteor reads the oplog stream from. Below is an explanation on the rest of the connection string:

  • authSource: is the database we wish to authenticate our username:password against. By default this will normally be the admin database.
  • ssl: Specify if connection to database is over TLS/SSL. Atlas does not allow connections without SSL.
  • replicaSet: Specifies the name of your replica set of which to connect to.

NOTE: The value assigned to replicaSet is an example. Yours will be different and depends on the name you set for your cluster. You can learn more about the URL connection string format and options here.

That’s it! We now have the two URLs needed to connect our Meteor app with our Database.

A final note on "MONGO_OPLOG_URL". Previous to Meteor 1.4 the "MONGO_OPLOG_URL" did not contain the replicaSet=replicaSetName parameter. For “oplog tailing” to function in Meteor 1.4+ the replicaSet=replicaSetName must be included.

For deploying on Galaxy you will need to set your "MONGO_URL" & "MONGO_OPLOG_URL" in your settings.json file as seen below. Once your new settings.json file is saved, your ready to redeploy to Galaxy and connect your Meteor app to your database.

Example of your Meteor settings.json file for Galaxy.

{
  "galaxy.meteor.com": {
    "env": {
      "ENV": "staging",
      "MONGO_URL": "mongodb://okgrow-staging-meteor:fakePassword1234@okgrow-staging-shard-00-00-fxnj9.mongodb.net:27017,okgrow-staging-shard-00-01-fxnj9.mongodb.net:27017,okgrow-staging-shard-00-02-fxnj9.mongodb.net:27017/meteor?authSource=admin&ssl=true&replicaSet=okgrow-staging-shard-0",
      "MONGO_OPLOG_URL": "mongodb://okgrow-staging-oplog:fakePassword5678@okgrow-staging-shard-00-00-fxnj9.mongodb.net:27017,okgrow-staging-shard-00-01-fxnj9.mongodb.net:27017,okgrow-staging-shard-00-02-fxnj9.mongodb.net:27017/local?authSource=admin&ssl=true&replicaSet=okgrow-staging-shard-0",
    }
  }
}

NOTE: If deploying on another platform you will normally have the option to set the "MONGO_URL" & "MONGO_OPLOG_URL" as environment variables.

Congrats! We are now finished. Our Meteor app (with oplog tailing) is now connected to our cluster hosted on MongoDB Atlas. I hope the process was as relatively pain-free as promised.

We have been using MongoDB Atlas since August 2016, and so far we have been very happy with our experience.

You can follow us to catch our next MongoDB Atlas blog post which will cover our experiences to date, nifty features, and our evaluation of MongoDB Atlas.

Q & A:


What is an Atlas Group ?

A MongoDB Atlas Group is an AWS region-specific Virtual Private Cloud (VPC). You can create as many VPC’s as you like. MongoDB Deployments or “clusters” live inside an Atlas Group which provide a way to manage and secure access to your MongoDb clusters.

All clusters for an Atlas Group will deploy inside this region-specific VPC. The first cluster added to the group determines the VPC region.

At the time of writing, clusters in multi-regions within the same Group were not yet supported. If you want to deploy a cluster to a different region you will need to create a new Atlas Group.


What regions are supported ?

At the time of writing (updated on Feb 3rd 2017), Atlas supports 5 AWS Regions:

  • us-east-1 (N. Virginia)
  • us-east-2 (Ohio)
  • us-west-2 (Oregon)
  • eu-west-1 (Dublin)
  • ap-southeast-2 (Sydney)

What is Sharding ?

Sharding is MongoDb’s way of providing horizontal scaling.

You should not enable sharded clusters by default as they come with certain restrictions. To enable their benefits sharded clusters require careful planning, execution and maintenance.

You should only consider sharding when your DB has become your application’s bottle neck and vertical scaling is no longer feasible.

The two main causes for sharding are:

  • High throughput operations and/or,
  • Data sets larger than the available RAM on your server.
We'll share what we've learned, get tips and info in your inbox occasionally