Parse & sort unique hits in logfiles

If you want to know the exact hits on your website (or whatever logfile you want) for a specific date without duplicates, here's how.
"Unique" means you don't want to count hits to an URL originating from the same IP twice.

You can use the attached script to do so:

# ./log_parser.rb 2011-10-04

27 hits on /rss.xml
36 hits on /stylesheets/fonts/slkscr-webfont.woff
37 hits on /stylesheets/fonts/slkscrb-webfont.woff
37 hits on /images/bullet.png
38 hits on /images/download.png
38 hits on /images/play.png
39...

Rails logs are not flushed automatically (in Rake tasks)

The Rails logger will store its content in a buffer and write it into the file system every 1000 lines. This will come back to bite you when using Rails.logger.info to write log output during Rake tasks or on a production console.

You often won't notice this because for the development and test environments auto_flushing is set to write after each line. On production environments the Rails logger writes only every 1000 lines -- and not upon shell or script ter...

Monitor a Rake task with God

In order to monitor a Rake task using God your Rake file must write a file with its process ID (PID) to a path determined by God. This way God can check whether the Rake process is still alive.

Here is how to do this: In your God config, call the Rake task with an environment variable PIDFILE. This variable should equal the PID file path desired by God:

God.watch do |w|
  w.dir = "#{rails_root}"
  w.name = "my_task"
  w.interval = 10.seconds
  w.pid_file = "#{rails_root}/tmp/pids/#{w.name}...

Change / Update SSL certificate for Amazon Elastic Load Balancer

There is a new card about how to do this with the new AWS Command Line Interface


At first you need the IAM Cli Tools.
-------------------------------------------------------------------------------------------------------------...

Disable output when using cURL

cURL makes a web request and shows you the response body.

You can redirect the response body to /dev/null just like for many other programs. But if you do that, cURL will display some short information about the request you are making:

$ curl http://www.example.com/ > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 30865  100 30865    0     0   4793      0  0:00:06  0:00:06 --:--:-- 10199

If you wan...

Make an HTTP request to a machine but fake the hostname

Consider you have a website vhost listening to www.example.com, redirecting all incoming requests that do not talk about the configured hostname (this is often used to redirect users to http://www.example.com when entering only http://example.com/).

If you want to make a request to that site's web server without actually talking to www.example.com (e.g. because this is a load balancer's address but you want to access one specific machine), you cannot just request machine1.example.com or localhost as the above vhost will redirect...

Fixing "Lock obtain timed out: SimpleFSLock" for Solr

If your application raises an error like ...
Couldn't connect to the Solr server at http://127.0.0.1:8983/solr. 500 "Lock_obtain_timed_out_SimpleFSLock(...)"

... and if your jetty.request.log contains entries such as ...
127.0.0.1 - - [13/09/2011:12:42:23 +0000] "POST /solr/update HTTP/1.1" 500 4412
127.0.0.1 - - [13/09/2011:13:37:03 +0000] "POST /solr/update HTTP/1.1" 500 4309

... you probably have a lucene-...-write.lock file lying around in your Solr data directory that is not being cleaned up properly. This causes...

Dragging a file into your terminal pastes the file path

When you drag a file from a Nautilus window into a terminal window, the file's path will be pasted into the terminal. This also works with multiple files.

How to install a frozen version of Firefox for your Selenium tests

Whenever Firefox updates, all your Cucumber features that use Selenium break. This is annoying.

In order to remedy this, version 0.5.0 of our geordi gem comes with a script that helps you create an unchanging version of Firefox for your Selenium tests. In particular, this new copy of Firefox will have the following properties:

  • It won't update itself with a newer version
  • It can co-exist with your regular Firefox installation (which you can update at will)
  • It will use a profile separate from the one...

Turn off SSL for scenarios with Selenium

Selenium does not speak SSL because it uses WEBrick that doesn't. When you use Selenium for Cucumber scenarios that visit pages with SSL, they will fail.

To turn off SSL only for scenarios that are executed on WEBrick, put this method into your application controller.

def ensure_proper_protocol
  request.headers['SERVER_SOFTWARE'].andand.include?('WEBrick') || super
end

Let a Rails 3 application make a request to itself

Ever wondered how Rails talks to itself in a Cucumber feature? In Rails 3 you can do it like this:

def rack_env(path)
  { "rack.input" => {},
    "PATH_INFO"=>"#{path}",
    "REQUEST_METHOD"=>"GET" }
end

request = rack_env('/users/new')
response = Rails.application.call(request)
status, headers, body = response

puts status # e.g. 200
puts headers.inspect # hash of headers
puts body.body # html of response body

Instead of Rails.application you can also call any Rack application.

How to use Rails URL helpers in any Ruby class

If you have any class which requires access to some path methods generated by your routes. Even though you could technically include Rails.application.routes.url_helpers, this may include way too many methods and even overwrite some class methods in the worst case.

Instead, most of the time the following is advised to only make the desired methods available:

class Project
  delegate :url_helpers, to: 'Rails.application.routes'

  def project_path
    url_helpers.project_path(self)
  end
end

Let Webrat make a POST request

Just add the parameter :post to the visit method:

visit publish_entry_path, :post

Webrat doesn't follow redirect because it considers the url external

Rails doesn't know which host it is running on. For generating links, it strips the hostname off the request URL, which can lead to errors when you have absolute URLs in your Cucumber tests.

If you really need to use absolute URLs somewhere, say in an email you send, either throw away the host when parsing it (e.g. body.scan(/http:\/\/[^\/]+\/([^\s"<]+)/)) or tell Webrat you're back on your site.

Ruby GetText will eval scripts containing ActiveRecord classes

When the Ruby parser module of Ruby-GetText comes across a file in one of its search directories (e.g. lib/scripts/) and finds out that you are defining ActiveRecord classes inside it, it evaluates the whole file. Here is how to avoid that.

What's happening?

Let's say you have the following script which is only run once, manually, via script/runner:

# lib/scripts/doomsday.rb
class User < ActiveRecord::Base; end
User.destroy_all

In that case we ...

Capybara - The missing API

The Capybara API is somewhat hard for parse for a list of methods you can call on a Capybara node. Below you can find such a list. It's all copied from the Capybara docs, so all credit goes to the Capybara committers.

When you talk to Capybara from a Cucumber step definition, you always have page as the document root node, or whatever you scoped to by saying within(selector) { ... }. You can select child notes by calling page.find(selector) or page.all(selector). You can call the same ...

Always store your Paperclip attachments in a separate folder per environment

tl;dr: Always have your attachment path start with :rails_root/storage/#{Rails.env}#{ENV['RAILS_TEST_NUMBER']}/.


The directory where you save your Paperclip attachments should not look like this:

storage/photos/1/...
storage/photos/2/...
storage/photos/3/...
storage/attachments/1/...
storage/attachments/2/...

The problem with this is that multiple environments (at least development and test) will share the same directory structure. This will cause you pain eventually. Files will get overwritten and...

Nginx Error "413 Request Entity Too Large"

If you get the error "413 Request Entity Too Large" from Nginx client_max_body_size is too low (default is client_max_body_size=1m).

This can happen for example during file upload.

Check whether a Paperclip attachment exists

Don't simply test for the presence of the magic Paperclip attribute, it will return a paperclip Attachment object and thus always be true:

- if user.photo.present? # always true
  = image_tag(user.photo.url)

Use #exists? instead:

- if user.photo.exists?
  = image_tag(user.photo.url)

JSONP - Wikipedia

Under the same origin policy, a web page served from server1.example.com cannot normally connect to or communicate with a server other than server1.example.com. An exception is the HTML <script> element. Taking advantage of the open policy for <script> elements, some pages use them to retrieve Javascript code that operates on dynamically-generated JSON-formatted data from other origins. This usage pattern is known as JSONP. Requests for JSONP retrieve not JSON, but arbitrary JavaScript code.

Cross-Origin Resource Sharing - Wikipedia

Cross-Origin Resource Sharing (CORS) is a browser technology specification, which defines ways for a web service to provide interfaces for sandboxed scripts coming from a different domain under same origin policy. CORS is a modern alternative to the JSONP pattern. While JSONP supports only the GET request method, CORS also supports other types of HTTP requests. Using CORS enables a web programmer to use regular XMLHttpRequest which supports better error handling than JSONP. On the other hand, JSONP works on legacy browsers that do not have C...

Responding to the OPTIONS HTTP method request in Rails: Getting around the Same Origin Policy

Code example for implementing Cross-Origin Resource Sharing (CORS) in Rails.

Zip files with Ruby

When you need to zip up files in Ruby, use zipruby.

sudo gem install zipruby

You can add existing files, add files from strings and even add directories.
Example usage:

require 'zipruby'
cars = %w[audi bmw mercedes]

zipfile = Tempfile.new('my.zip', 'tmp')
Zip::Archive.open(zipfile.path, Zip::CREATE) do |zip|
  zip.add_file '/tmp/me.txt'
  zip.add_dir 'cars' 

  cars.each do |car|
    zip.add_buffer "cars/#{car}.txt", "This #{car} is mine!" 
  end
end

Credits go to winebarrel for the Ruby bin...

Downloading files from Ruby on Rails

To offer files for download, use send_file.

def download(file)
  send_file file.path, :disposition => 'attachment'
end

Note that a send_file replaces the default :render action.