To help Database Administrators feel confident about data integrity through a late-night master/slave swap, regardless of potentially mis-behaving applications and clients, it’s important to remember the “read-only” global variable.  The setting can be toggled on/off at runtime, and except for users who have the SUPER privilege, MySQL will forcefully deny any changes to data unless those changes are requested by a replication thread.

However, keep in mind:

• Many application users, especially at young start-ups, will have GRANT ALL privileges which include SUPER, thereby rendering the read-only setting useless.
• Use of automation (such as with puppet or chef and a template-generated my.cnf file) is critical to avoid human error.  Appropriate monitoring around the read-only setting is a huge plus.
• Even when used consistently, the read-only setting must be switched on and off at necessary times.  For example, during a master/slave swap, you’ll want to set the “old” master to read-only=on BEFORE you set the “new” master to read-only=off.  Otherwise, there’s a solid chance cached DNS (or some other unexpected behavior) will cause writes to both servers, even if during a small window of opportunity.

No matter what hiccup avoidance technique(s) are used (our favorite is eating granulated sugar), it is very important to maintain data integrity at all costs.  No one wants to replay binary logs and migrate misplaced data.  To optimize your next master/slave swap, utilize the read-only setting of MySQL consistently, and be sure to avoid having application users with the SUPER privilege.

Did you know we have a MySQL-specific tips and tricks guide?  Download it now.
We offer on-site and remote MySQL replication training.  Inquire today.

For many, the cloud is a quick and easy solution to many infrastructure and platform challenges. Engineers can focus entirely on application development without ever having to worry about stability and scalability. The control over their infrastructure is surrendered for freedom from full responsibility. That lack of access to their hardware is not missed until a problem occurs, like an outage.  At that point, they can’t do anything about it. They have to wait for someone else to.

How are public cloud services provided to end users: by who, from where, and using what equipment? It’s common to not give these questions any thought.  Before the cloud, everyone always knew where their data and applications were physically located.  They understood the infrastructure behind the platform. They either built their own data center, or rented space from one with trust. Either way, this required an in-house maintenance team providing on-site or remote administration.

In all fairness, this occurs with public cloud computing as well. Your data and applications still sit on a server (or multiple servers) in a data center. They are connected to the Internet by network engineers: people you may never meet but will appreciate nonetheless, being available 24 hours a day to deal with any potential problems. This doesn’t mean, however, that every system in the cloud will always run without trouble. Outages are not unusual, even with top-tier providers, and when such issues arise, it can be frustrating not knowing what’s being done to resolve or prevent them.

It’s important to remember that, while typically more robust than stand-alone servers, public cloud services are still subject to the same frailties as any other network system: human error, crackers, crashes and outages.  It’s also important to remember that not all applications are fit for the public cloud, and private clouds or hybrid solutions are great alternatives for those on the fence.  While some applications are better hosted internally, others are nicely buoyed and flourish under the multi-faceted support the cloud offers. Its recent popularity, after all, isn’t due entirely to its catchy name.

Let Bitlancer help you succeed in the public cloud with our Guide to Engineering in the Cloud.

To be frank, some awesome things to mention about Rackspace Cloud:

• Your CPU and I/O usage is burst-able no matter what instance you have, and under the hood of every instance is RAID 10 disk.  With the cheapest instance at less than 2 cents/hour, you can launch a small or horizontally scaled application with a very small budget while still taking advantage of excellent CPU and I/O performance.  Oh, and yes, you can get started for 1.5 cents/hour.
• The instances are persistent, so you can choose to use them however you’d like: very good for traditional folks used to traditional hosting, who want to name servers, use static public and private IP addressing, update reverse DNS, etc., while also good for those folks looking to spin up and tear down instances dynamically and often.
• Free DNS hosting, an excellent DNS api, and the ability to set reverse DNS.
• Dynamic resizing of instances on the fly (with API support for this and many other operations).
• Console access and system recovery using their simple and steadily advancing control panel.
• Great image snapshot capabilities and automated backups (though still slightly limited in the current implementation).
• Excellent (Fanatical, actually!) support.

Every implementation has its downfalls.  Rackspace Cloud has no EBS solution, so disk space is limited to the amount included with your instance (though it is persistent).  The API is growing, so for now, there are still a few bugs and features missing. They have multiple data centers, but you need two separate accounts to use them.

I could keep going on both positives and negatives.

However, my largest disappointment with Rackspace Cloud is the lack of FreeBSD support.  As an avid FreeBSD user, I look forward to the day when Rackspace Cloud supports this fantastic Operating System.  My assumption is that the version of Xen Rackspace Cloud is using still does not support FreeBSD, as mentioned in the comments of Why the Rackspace Cloud Control Panel is the Best.  One could also argue that FreeBSD does not support the older versions of Xen.  Either way, it’ll be great to see this released later this year if that’s true.

Let Bitlancer help you succeed in the public cloud with our Guide to Engineering in the Cloud.

Rackspace Cloud is a strong offering, and I can understand why it’s quickly becoming a market leader.  Their support is top-notch and offer an enterprise-level product.  Like Amazon EC2, they have an intuitive web interface, growing API, and competitive pricing.  Here are some things I’d like to see improved:

• Speed & usability of the web interface: the web interface is quite slow on my Ubuntu Linux box with Firefox.  It seems to vary in speed based on time of day.  Also, it would be nice to have a few more features in the web interface, such as the ability to launch multiple servers at once.
• Multiple data center support within one account: the ability to choose a specific data center for each instance that is provisioned.
• A directly attached storage solution: although the disk space alloted to each instance is reasonable, non-volatile, and fast, there’s no ability to expand for increased storage or setup logical RAID volumes for increased throughput.
• Custom image support: it would be nice to run FreeBSD (version 8.x has good support for the Xen hypervisor).

Amazon EC2 is the most well-known provider, and for good reason.  They have a strong API, flexible implementation options, “external” storage, and a slew of other features.  Here are some things I’d like to see improved:

• Static internal IP addressing: unlike Rackspace Cloud, EC2 doesn’t guarantee that your internal IP address and/or DNS entry won’t change.  This can be a hurdle for those that want to implement internal DNS.  However, EC2 is smart enough to re-route traffic to other EC2 instances locally at all times, even if the public IP address is used.
• Security groups can’t change: once an instance is launched, you can not change its security group configuration.  This hinders flexibility.
• Buggy AWS console: the console that is provided by Amazon is buggy.  One example: sometimes security group rules will disappear if you attempt to access a security group you’ve modified during the same session.
• Lack of clarity on data center/zone locations: this can hurt standard naming conventions (such as after airport codes) when you don’t know exactly where your instances are located.

Let Bitlancer help you succeed in the public cloud with our Guide to Engineering in the Cloud.

Example:

You launch 20 MySQL servers in a single security group called “MySQL Servers”.  Somewhere down the line (and this does happen), someone needs port 3306 open on one specific server to a random subnet (let’s call it 192.168.0.0/16).

You now have three options:

• Re-launch the one server in its own security group, and grant access.  While launch automation and central management should be one of an EC2 user’s top priorities, this option may not always be practical.
• Open up port 3306 to 192168.0.0/16 inside the “MySQL Servers” security group, and then use local iptables on the remaining servers to block off access on port 3306.
• Open up port 3306 to 192.168.0.0/16 inside the “MySQL Servers” security group, and be okay with the fact that all 20 of your MySQL servers are now open to that subnet (and at that point, put your faith in your MySQL users configuration that access is configured appropriately).

In order to avoid encountering this issue in the future, I’ve recently been experimenting with putting every EC2 instance in its own security group.  Not only can this be used to easily “tag” an instance with a name, but it allows you to individually control security rules for each server independently.  Note that this does increase overhead, and most rules will be repeated for all servers of the same type, so I only recommend this solution if you’re automatically managing your instances via the EC2 API.

Let Bitlancer help you succeed in the public cloud with our Guide to Engineering in the Cloud.