Open Source by FiveRuns

We are proud to release FiveRuns TuneUp v0.8.15.

This release fixes the issue that users have been having related to the TuneUp panel appearing, but no data showing up.

For the curious, this bug seems to appear when used TuneUp is used alongside plugins that alter Rails’ javascript generator (like jrails). We removed all javascript generator usage from TuneUp to prevent this bug from occurring. We want to extend a hearty thanks to Chris Zelenak for leading us in the right direction towards a bugfix by providing reproduction steps and a sample Rails app that demonstrated the bug.

This version is recommended for all new and existing TuneUp users.

Like most servers, Apache allows you to capture details about its performance and status. These metrics are not published by default however and can be tricky to configure securely, especially in an environment using virtual hosts. This article describes how to configure Apache2 to enable its server-status statistics which can be viewed directly via web browser or collected by monitoring tools such as FiveRuns Manage. Additional metrics are published by enabling ExtendedStatus which we will also enable.

In its simplest implementation, there are just a few steps to enable Apache’s mod_status. In the Apache2 configuration file, (often: /etc/apache2/apache2.conf for Apache2 and /etc/httpd/conf/http.conf for Apache v.1.x) this is enabled by adding or uncommenting the following lines:

LoadModule status_module  libexec/apache2/mod_status.so 
ExtendedStatus On

<Location /server-status>
     SetHandler server-status
     order deny,allow
     deny from all
     allow from 127.0.0.1
</Location>

The above edits and a restart of Apache will publish the status metrics to clients on the local system. As in the Apache.org example, security can be lowered to allow remote clients access the server statistics. Since that can be a security concern and because monitoring tools can be installed locally, our examples restrict access to the local web server.

One challenge we often encounter is Apache environments using virtual hosts. The <Location /server-status> directive needs to either appear in the parent apache.conf/http.conf file or in the virtual host that is loaded first. When multiple .conf files are loaded via the Include directive, it is done alphabetically, so consider this when inserting the server-status Location block.

Alternately, you could dedicate a virtual host to serve the status information and bypass the load ordering issue above. The example below details a NameVirtualHost configuration utilizing the localhost hostname. A similar approach could be used to dedicate a vhost using IP-based Virtual Hosts and 127.0.0.1.

<VirtualHost *:80>
  ServerName localhost
  ServerAlias 127.0.0.1
  DocumentRoot /var/www/

  <Location />
     order deny,allow
     deny from all
     allow from 127.0.0.1
  </Location>

  <Location /server-status>
     SetHandler server-status
     order deny,allow
     deny from all
     allow from 127.0.0.1
  </Location>

  <Directory /var/www/>
    Options FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
  </Directory>

</VirtualHost>

Hot on the heels of the ‘Run Your Way to RailsConf Europe’ contest announcement, we’ve released version 0.8.11 of FiveRuns TuneUp. Since our last publicly-announced release, we’ve made a few substantial fixes and improvements. Most notably, these include:

• Signup and login for the TuneUp service is now only required if you want to share a run. If you just want to use TuneUp locally for your own personal development, you don’t need to sign up at all. If, however, you decide that you’d like to use the (free) TuneUp service to share your runs with colleagues or the Rails community, registration is simple.
• Instrumentation for partial rendering under Rails 2.1 now works properly.
• Fixed some lingering issues with XHTML Strict 1.0 compliance.

Grab the latest gem and get to tuning up your Rails application!

One of the lingering issues we’ve had to deal with over the last year in the Manage service is ActiveRecord’s reluctance to talk to more than one database. It’s really quite stubborn in that regard and while there are a few solutions out there, none of them worked quite like we wanted. Some required intrusive application-level code changes, some didn’t offer the features we needed, so we bit the bullet and wrote what we needed.

Specifically we needed two features to scale our mysql database: application-level sharding and master/slave replication. Sharding is the process of splitting a dataset across many independent databases. This often happens based on geographical region (e.g. craigslist) or user account (e.g. flickr). Replication provides a near-real-time copy of a database which can be used for fault tolerance and to reduce load on the master node. Combined, you get a scalable database solution which does not require huge hardware to scale to huge volumes. DataFabric extends ActiveRecord’s standard connection handling to provide these two features.

To install, invoke the usual magic:

gem install data_fabric

Add DataFabric to your Rails 2.1 gems listing in config/environment.rb:

config.gem "data_fabric"

Annotate your sharded models with your desired sharding and replication setup:

class Auction < ActiveRecord::Base
  data_fabric :shard_by => :city, :replicated => true

Let’s assume we are sharding based on the city associated with the request (i.e. the Craigslist model). You’ll need to add the necessary database connections to your config/database.yml for each city based on the naming convention DataFabric uses. See the README for details.

Finally your application controller will activate the city’s shard for each request:

class ApplicationController < ActionController::Base
  around_filter :select_shard

  private
  def select_shard(&block)
    DataFabric.activate_shard(:city => @current_city, &block)
  end
end

Now, whenever you do anything with the Auction model, it will only affect the current shard. Auction.find(:all) will find all auctions within that shard. The converse is also true: you can’t do anything with the Auction model until you set a shard. Note that you can just set the replicated flag without the shard_by flag; DataFabric will act just like Rick Olson’s Masochism plugin.

We’re releasing DataFabric on github for others to use as they see fit. Take a look at the README on github for technical details and code samples. We’ve used it successfully with ActiveRecord 2.0.2 and 2.1. There are some areas which can be painful to deal with, notably migrations and fixtures, but we have both working in production here so you can overcome. :-) I’ll give you a hint: the example application might help.

Good luck and let us know what you think!

The FiveRuns dev and support teams live in Campfire.

As I wrote a while back, our Campfire ‘Development’ room is constantly filled with team chat, subversion/git commit messages, and CruiseControl build statuses. Piping all of this information into one centralized team-viewable location has been awesomely valuable. That being said, we’ve kicked it up a notch.

Say hi to CampTweet.

This simple Ruby script broadcasts your Twitter statuses (tweets), Summize Twitter search results, and RSS/Atom feed items into the Campfire room of your choice.

Here’s how you can use it on your team.

1. Install the gem

sudo gem install camptweet

2. Generate the camptweet daemon script

camptweet .

This will generate a camptweetd file in the current directory. By default, the meat of it looks like this:

Camptweet::Bot.new do |camptweet|
  camptweet.twitter_users = ['bdainton']
  camptweet.twitter_search_terms = ['fiveruns']
  camptweet.feed_urls = ['http://github.com/repositories.atom']
  camptweet.campfire_subdomain = 'mycompany'
  camptweet.campfire_use_ssl = true
  camptweet.campfire_room = 'Room Name'  
  camptweet.campfire_email = 'foo@mycompany.com'
  camptweet.campfire_password = 'foo_password'
  camptweet.verbose = false
  camptweet.logfile = 'camptweet.log'
end.run

Update the above config to list the Twitter screen names of your team members, the simple (one-word) terms for which you’d like to search on Summize, and any RSS/Atom feeds for which you’d like the latest items in the feed to be pumped into Campfire.

3. Start up the script

nohup ./camptweetd &

That’s it.

Here at FiveRuns, we’ve been using this tool to centralize our team tweets, product and company mentions on Twitter, and a number of dev-and-support-relevant feeds (including project card updates in Mingle, forum and GetSatisfaction activity, and build system status).

If your team is using Campfire, give CampTweet a try. It’s neat.

We’re happy to release version 0.8.8 of the TuneUp panel.

• Incompatibility with other js libraries thanks to js sandboxing code from Howard Rauscher
• Incompatibility with xhtml pages
• Issue with tuneup password being saved in the tuneup log. It is now filtered.

This is a recommended upgrade for all TuneUp users with a previous version. Enjoy!

Questions, comments, or problems? Drop us an email or GitHub message.

We’re happy to release version 0.8.6 of the TuneUp panel. This release mainly fixes rendering issues that we’ve encountered across different apps and an important memory issue that affects Firefox users with especially large TuneUp runs. We also made an effort to keep TuneUp-related activity out of your Rails development log. Enjoy!

Questions, comments, or problems? Drop us an email or GitHub message.

In order to scale our service, a large portion of our Manage service runs asynchronously from the http://manage.fiveruns.com website. We use Starling, the fantastic queuing system behind Twitter, to manage jobs to be processed. One interesting twist to Starling is that it uses the memcached client API so any memcached client library can access Starling without any extra effort.

Since our service is 24/7, we needed fault tolerance, high availability and other such high-falootin’ ideals. Our plan was to have two Starling servers, one on each app server, with the job processors using the local Starling server on their own machine by default, falling back to the server on the other machine if the local Staring daemon died.

Seattle.rb’s memcache-client 1.5.0 library supports multiple servers and does make an effort to switch between them but it has three known problems:

1. set() fails when the socket has been disconnected (via Twitter)
2. it does not retry if a socket operation fails (via Will Bryant)
3. it does not retry the current operation on another server if a server dies

The latter issue means that data will be lost when a server dies. We’ve fixed all three issues in our Github repository for memcache-client and Zachary Pinter is working on getting our updated code pushed into Edge Rails. Hopefully we’ll see an updated memcached client in Rails soon. In the mean time, please feel free to fork or clone our repository and use it in your own project!

Soon after releasing v0.8.4, a TuneUp user and community member let us know that the TuneUp Panel wasn’t showing up for him. It turns out he was serving up his pages as XHTML strict with a content type of application/xhtml+xml. TuneUp was only injecting its panel into text/html pages.

The fiveruns_tuneup v0.8.5 gem fixes this problem.

For more information, see the release’s README.rdoc, available at TuneUp’s GitHub repository. Questions, comments, or problems? Drop us an email or GitHub message.

We’re happy to announce the release of the FiveRuns TuneUp Panel gem v0.8.4, which fixes a number of functional and display-related bugs present in the initial beta release, adds support for use as a gem dependency in Rails 2.1 development environments, and allows for environment-specific configuration.

For more information, see the release’s README.rdoc, available at TuneUp’s GitHub repository

We’ll be spinning up an issue tracker soon to handle future bug reports. Until then, feel free to drop us an email or GitHub message.

My favorite development tools are the snoopy kind. I love tools like tcpdump(1) and FireBug that let me look at what’s going on inside the software I’m developing. Its fun to just watch the mechanisms work. Further, they can really save your bacon if your expectations of how something should work diverge from reality.

We’ve always had great tools for this in Ruby. Of course there’s the development log, which is chock full of interesting information. There are great open source tools like pl_analyze that parse that information and give you interesting on what is going on.

However, I haven’t found tools that give me two things:

• The ability to understanding a request in its entirety. Rails dumps a lot of great info in the development log. Out of the box you’ve got queries generated by ActiveRecord, information on the request parameters, template render timings and other useful nuggets. However, trying to figure out exactly what models, filters and templates are involved in rendering a request can prove trying.
• Performance peripheral vision. As I’m developing an application, I’d love to see information on how I’m spending my time in each request. Finding places where a model needs some performance attention or a view is taking a long time to render before they slow down my production servers would make my life much shinier.

Luckily for all of us, we’ve just released a tool for doing both of these things¹. Its callled TuneUp and we think its pretty damn cool.

What is FiveRuns TuneUp?

TuneUp consists of two components; a developer panel plugin that you install in your applications for use in development, and the hosted TuneUp service where you can upload information about your app’s performance and share it with others.

The plugin adds a panel to the top of your application that displays information about how long it takes to execute an action, exactly what happened while it was executing, and how this compares with previous requests with the same URI. The information includes the filters that were called, model activity that occurred, and views that were rendered. You can see in-depth information about your database schema, SQL queries, and if you’re using MySQL (right now), EXPLAIN information on those queries. Everything’s contextual, so you can drill down and see exactly where things are happening… and even jump to the related line of code in TextMate.

How/Why would you use the TuneUp panel?

There are two core uses for the TuneUp panel. The first is simply as a casual source of information as you develop you application. It’s a handy tool to have around to tell you what’s happening in your action, and as a quick reference for schema information. It’s also a compelling way to incrementally improve performance, with basic metrics available at all times to prevent gradual response time creep, helping you see performance bottlenecks early. Premature optimization is the devil, but that doesn’t mean it makes sense to hide your head in the sand and stay ignorant of problems you’ll need to solve around the corner.

How about the service?

Sometimes you run into issues you don’t know how to resolve by yourself, or have information about the performance or makeup on an action that you’d like to share with others. The TuneUp panel includes an easy upload facility that lets you send the information on an action to http://tuneup.fiveruns.com, where you can make it public (or just send a quick link to someone).

Maybe you have a lot of Rails experience that could benefit others in the community. With more call stack and performance context to work from, it’s even easier for you to contribute your know-how to help other people solve problems. TuneUp is meant as a to be a service to the community, so get involved!

How do I get it and install it?

  sudo gem install fiveruns_tuneup
  fiveruns_tuneup /path/to/application
  cd /path/to/application
  ruby script/server

For more information, see the help documentation.

It’s Open Source

We’d love your help in making TuneUp better. You can find us up at at GitHub (with TuneUp’s repo right here).

Fork away. Send pull requests often!

¹ I’ll pretend this surprised you

“Instrument” is a little utility library to get useful information out of methods with a minimum of fuss, and without having to dig into code to add debugging output. We’ve released it as a gem, and it’s being developed on GitHub.

Here’s a simple example. Let’s say you have a [contrived] class like this:

  class Car
    # ...
    def drive(miles)
      # ...
    end
  end

.. and you’d like to get some basic statistics whenever the drive method is called on an instance. With instrument, there’s no need to modify the actual Car class directly; you can just wrap the method with a handler that’s called after every invocation with the instance, time taken to invoke, and any arguments:

  require 'instrument'

  instrument 'Car#drive' do |car, time, miles|
    puts "#{car} was driven #{miles}mi (#{time})" 
  end
  # Drive a lot of cars

If you’d also like to know track the number of cars instantiated, just increment a counter:

  cars = 0
  instrument 'Car.new' do |klass, *args|
    cars += 1
  end
  # Make a lot of cars
  puts "Made #{cars} car(s)!" 

You can use the library to get basic performance numbers, to output tracing information, or just to do a bit of exploratory poking around.

Play with it, fork it, and send pull requests!