Automate Deployment with CloudFormation

Security¶

EC2 instances in AWS must have security groups designated during creation that specify firewall rules for each instance. The templates create a simple security group MongoSecurityGroup that opens up port 22 (SSH) to the outside world and a separate rule (AWS::EC2::SecurityGroupIngress) that is used to open up the standard mongod port (27017) to other instances within that group:

"MongoSecurityGroup" : {
    "Type" : "AWS::EC2::SecurityGroup",
    "Properties" : {
        "GroupDescription" : "MongoDB security group",
        "SecurityGroupIngress" : [
            {
                "IpProtocol" : "tcp",
                "FromPort" : "22",
                "ToPort" : "22",
                "CidrIp" : "0.0.0.0/0"
            }
        ]
    }
},
"MongoSecurityGroupIngress" : {
    "Type" : "AWS::EC2::SecurityGroupIngress",
    "Properties" : {
        "GroupName" : { "Ref" : "MongoSecurityGroup" },
        "IpProtocol" : "tcp",
        "FromPort" : "27017",
        "ToPort" : "27017",
        "SourceSecurityGroupName" : { "Ref" : "MongoSecurityGroup" }
    }
},

Depending on the type of deployment you are creating, you may need to add additional security group ingress rules to open up additional ports, available to your instances or to the outside world (e.g. port 27018 for sharding).

Storage Configuration¶

When using MongoDB on AWS, it is recommended that you use multiple EBS volumes configured as a single RAID10 storage device for your data. Configuring storage via EBS volumes using CloudFormation requires multiple steps. First the EBS volume must be created, using the VolumeSize input value and the same availability zone to be used for the EC2 instance (see Instance Configuration).

"MongoVolume1" : {
    "Type" : "AWS::EC2::Volume",
    "Properties" : {
        "Size" : { "Ref" : "VolumeSize" },
        "AvailabilityZone" : { "Fn::GetAtt" : [ "MongoInstance", "AvailabilityZone" ]}
    }
},

The next step is to attach the volume to an EC2 instance. Referencing the instance name in InstanceId ensures that the EBS volumes will be created after the instance is created.

"MongoVolumeMount1" : {
    "Type" : "AWS::EC2::VolumeAttachment",
    "Properties" : {
        "InstanceId" : { "Ref" : "MongoInstance" },
        "VolumeId" : { "Ref" : "MongoVolume1" },
        "Device" : "/dev/sdh1"
    }
},

The templates use four EBS volumes as the basis for the RAID10 configuration. If you are interested in increasing the number of EBS volumes, you will need to add additional AWS::EC2::Volume and AWS::EC2::VolumeAttachment resources inside of your template. See Customize Templates for more information on the steps required.

Instance Configuration¶

The centerpiece of the CloudFormation template is the creation of the EC2 instances that will be used as the MongoDB server. There are two main sections to be configured for your instance, the instance metadata and instance properties. In the sample provided, the metadata contains information about the packages to be installed (mdadm and sysstat) and any files to be created within the instance. In the sample, the only file to be created is a yum repository entry to facilitate MongoDB being installed via yum after the instance has booted.

"MongoInstance" : {
    "Type" : "AWS::EC2::Instance",
    "Metadata" : {
        "AWS::CloudFormation::Init" : {
            "config" : {
                "packages" : {
                    "yum" : {
                        "mdadm" : [],
                        "sysstat" : []
                    }
                },
                "files" : {
                    "/etc/yum.repos.d/10gen.repo" : {
                        "content" : { "Fn::Join" : ["", [
                            "[10gen]n",
                            "name=10gen Repositoryn",
                            "baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64n",
                            "gpgcheck=0n"
                        ] ] },
                        "mode" : "000644",
                        "owner" : "root",
                        "group" : "root"
                    }
                }
            }
        }
    },

For more information about the possible options for the metadata section, refer to the CloudFormation documentation for the AWS::EC2::Instance resource. The properties section in the template is uses to specify things like the instance type, AMI, security groups and a script to be run after boot (found in the UserData section):

"Properties" : {
    "InstanceType" : { "Ref" : "InstanceType" },
    "ImageId" : { "Fn::FindInMap" : [ "RegionImageZone", { "Ref" : "AWS::Region" },
        { "Fn::FindInMap" : [ "InstanceTypeArch", { "Ref" : "InstanceType" }, "Arch" ] } ] },
    "SecurityGroups" : [ { "Ref" : "MongoSecurityGroup" } ],
    "KeyName" : { "Ref" : "KeyName" },
    "UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
        "#!/bin/bashn",
        ...

The instance type is determined by the InstanceType input parameter. The ImageId specifies the AMI to use, which is determined by the chosen instance type (e.g. m1.large) and region in which the instance is launched (e.g. us-east-1). Refer to the Mappings section inside the sample templates for a list of the available AMIs.

The UserData section shown above contains a bash script that is executed once the instance is launched and available. The first step is to install the aws-cfn-bootstrap tools which are used in the script to initialize the instance, signal when errors have occurred, and signal when instance creation is complete:

"yum update -y aws-cfn-bootstrapn",

"## Error reporting helper functionn",
"function error_exitn",
"{n",
"   /opt/aws/bin/cfn-signal -e 1 -r "$1" '", { "Ref" : "WaitHandleMongoInstance" }, "'n",
"   exit 1n",
"}n",

"## Initialize CloudFormation bitsn",
"/opt/aws/bin/cfn-init -v -s ", { "Ref" : "AWS::StackName" }, " -r MongoInstance",
"   --access-key ",  { "Ref" : "HostKeys" },
"   --secret-key ", {"Fn::GetAtt": ["HostKeys", "SecretAccessKey"]},
"   --region ", { "Ref" : "AWS::Region" }, " > /tmp/cfn-init.log 2>&1 || error_exit $(</tmp/cfn-init.log)n",

Next include a series of sleep conditions in case the EBS volumes are not yet available. If you plan to use more than 4 EBS volumes in your CloudFormation template, you should add additional sleep conditions here:

"## Waiting for EBS mounts to become availablen",
"while [ ! -e /dev/sdh1 ]; do echo waiting for /dev/sdh1 to attach; sleep 10; donen",
"while [ ! -e /dev/sdh2 ]; do echo waiting for /dev/sdh2 to attach; sleep 10; donen",
"while [ ! -e /dev/sdh3 ]; do echo waiting for /dev/sdh3 to attach; sleep 10; donen",
"while [ ! -e /dev/sdh4 ]; do echo waiting for /dev/sdh4 to attach; sleep 10; donen",

Then install MongoDB and create the RAID10 device:

"yum -y install mongo-10gen-server > /tmp/yum-mongo.log 2>&1n",

"## Create RAID10 and persist configurationn",
"mdadm --verbose --create /dev/md0 --level=10 --chunk=256 --raid-devices=4 /dev/sdh1 /dev/sdh2 /dev/sdh3 /dev/sdh4 > /tmp/mdadm.log 2>&1n",
"echo '`mdadm --detail --scan`' | tee -a /etc/mdadm.confn",

With the RAID10 created, set some block device attributes (read-ahead) for each storage device:

"## Set read-ahead on each devicen",
"blockdev --setra 128 /dev/md0n",
"blockdev --setra 128 /dev/sdh1n",
"blockdev --setra 128 /dev/sdh2n",
"blockdev --setra 128 /dev/sdh3n",
"blockdev --setra 128 /dev/sdh4n",

Now use LVM to create a series of logical volumes for data, journal and logs. The values used below for each volume are specified as percentages, those may need to be changed for your deployment. After creating the volumes, next create the filesystem, mount points and entries in the filesystem table. The last storage-related step is to set the user:group ownership of each mount point to mongod:mongod.

"## Create physical and logical volumesn",
"dd if=/dev/zero of=/dev/md0 bs=512 count=1n",
"pvcreate /dev/md0n",
"vgcreate vg0 /dev/md0n",
"lvcreate -l 90%vg -n data vg0n",
"lvcreate -l 5%vg -n log vg0n",
"lvcreate -l 5%vg -n journal vg0n",

"## Create filesystems and mount point infon",
"mke2fs -t ext4 -F /dev/vg0/data > /tmp/mke2fs1.log 2>&1n",
"mke2fs -t ext4 -F /dev/vg0/log > /tmp/mke2fs2.log 2>&1n",
"mke2fs -t ext4 -F /dev/vg0/journal > /tmp/mke2fs3.log 2>&1n",

"mkdir /datan",
"mkdir /logn",
"mkdir /journaln",

"echo '/dev/vg0/data /data ext4 defaults,auto,noatime,noexec 0 0' | tee -a /etc/fstabn",
"echo '/dev/vg0/log /log ext4 defaults,auto,noatime,noexec 0 0' | tee -a /etc/fstabn",
"echo '/dev/vg0/journal /journal ext4 defaults,auto,noatime,noexec 0 0' | tee -a /etc/fstabn",

"mount /data > /tmp/mount1.log 2>&1n",
"mount /log > /tmp/mount2.log 2>&1n",
"mount /journal > /tmp/mount3.log 2>&1n",

"ln -s /journal /data/journaln",

"chown -R mongod:mongod /data > /tmp/chown1.log 2>&1n",
"chown -R mongod:mongod /log > /tmp/chown2.log 2>&1n",
"chown -R mongod:mongod /journal > /tmp/chown3.log 2>&1n",

Next proceed to creating a MongoDB configuration file, specifying the logpath and data directory (among others), and then start MongoDB:

"## Update mongod configurationn",
"cat <<EOF > /etc/mongod.confn",
"logpath=/data/log/mongod.logn",
"logappend=truen",
"fork = truen",
"dbpath=/datan",
"rest=truen",
"EOFn",

"## Start mongodn",
"/etc/init.d/mongod start > /tmp/mongod-start.log 2>&1n",

The final step is to signal the previously created WaitCondition that setup is complete:

"## CloudFormation signal that setup is completen",
"/opt/aws/bin/cfn-signal -e 0 -r "MongoInstance setup complete" '", { "Ref" : "WaitHandleMongoInstance" }, "'n"

Once this script has completed executing, the instance and associated resources have been created and the CloudFormation stack is ready to go.

Operating System¶

The sample templates provided use Amazon Linux as the base operating system. If you are interested in using a recent release of Ubuntu (9.10 or greater), please use the following steps to customize the templates. The steps refer to the line numbers for the “MongoDB_ReplicaSetStack.template” but changes should also be made in “MongoDB_ReplicaSetMember.template.”

First, update the files content of the EC2 instance Metadata section to use the 10gen apt source:

"files" : {
    " /etc/apt/sources.list.d/10gen.list" : {
        "content" : { "Fn::Join" : ["", [
            "deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10genn"
        ] ] },
        "mode" : "000644",
        "owner" : "root",
        "group" : "root"
    }
}

Then in the UserData section of the EC2 instance configuration, make the following changes:

• Line 220: change yum -y install mongo-10gen-server... to apt-get install mongodb-10gen
• Line 221: add service mongodb stop
• Line 260, 261, 262: update to use mongodb:mongodb as the owner:group
• Line 265: update to cat <<EOF > /etc/mongodb.conf
• Line 275: change /etc/init.d/mongod start... to service mongodb start

Storage¶

The sample templates provided use four EBS volumes as the basis for the RAID10 configuration. If you are interested in using additional volumes you will need to update the following items for each new volume you want to add:

• Add an “AWS::EC2::Volume” resource
• Add an “AWS::EC2::VolumeAttachment” resource and mount point
• Add an additional sleep condition
• Update the call to mdadm to include your additional volumes

Instances¶

The sample “ReplicaSetStack” template creates three instances, one from the “stack” template and two additional “ReplicaSetMember” instances, via the “AWS::CloudFormation::Stack” resource in the “ReplicaSetStack” template. If you are interested in adding additional replica set members, you will need to create an additional member instances and edit the replicaSetConfigInit.js found within the “ReplicaSetStack” template. Refer to Replica Set Stack for information about the additional resources and config file to be updated. When creating the replica set the templates spread the created instances across multiple availability zones (e.g. us-east-1a or us-east-1b). When adding additional instances be sure to specify your desired Availability Zone for increase redundancy.