Reverse-proxying web applications with nginx
While you can use Apache as a reverse proxy, it tries to be too smart. Try nginx instead, it's much simpler to set up.
After struggling with Apache for quite a while, since I simply could not make it pass through the Digest Authentication of my target host (that I proxied to), I switched to nginx. Here is what I did.
-
Have nginx
sudo apt-get install nginx
-
Define your nginx config, e.g. at
/etc/nginx/conf.d/reverse-proxy.conf
:server { listen 127.0.0.1; location /...
How to show an ordered crontab
Crontabs are often unordered, especially when generated for an application where you usually group tasks by their domain/scope.
An example crontab might look like this:
# Begin Whenever generated tasks for: project100
MAILTO="log@example.com"
MAILFROM="cron@example.com"
# When server is booting up, ensure Sidekiq is running
@reboot start_sidekiq
23 8 * * * baz
30 * * * * plop
5 8 * * * bar
1 0 * * * foo
# End Whenever generated tasks for: project100
While you can human-parse this one easily, crontabs with several lines are hard ...
RSpec: Stubbing a method that takes a block
If you stub
a method or set expectations with should_receive
these stubbed methods may also yield blocks. This is handy if the returning object is receiving a block call.
Consider this, where you cannot say and_return []
because of the block:
def crawl_messages
Message.find_in_batches do |messages|
messages.each(&:crawl)
end
end
It works similar to and_return
-- just use and_yield
:
describe '#crawl_messages' do
it 'should proc...
How to drop all tables in PostgreSQL
To remove all tables from a database (but keep the database itself), you have two options.
Option 1: Drop the entire schema
You will need to re-create the schema and its permissions. This is usually good enough for development machines only.
DROP SCHEMA public CASCADE;
CREATE SCHEMA public;
GRANT ALL ON SCHEMA public TO postgres;
GRANT ALL ON SCHEMA public TO public;
Applications usually use the "public" schema. You may encounter other schema names when working with a (legacy) application's database.
Note that f...
OpenStack instance not configuring network (DHCP) correctly
We ran into trouble when adding additional compute units to our railscomplete Hosting environment lately.
VM-instances on the new compute units where booting and requesting private IP addresses via DHCP correctly (DHCPDiscover
), but after the answer of the dnsmasq dhcp server (DHCPOffer
) we did not see any further traffic on the host machine. FYI: The instance should request the IP via DHCPRequest
which in turn should be acknowledged by a DHCPAcknowledgment
packet.
We assumed this DHCP UDP traffic did not...
Protip: Clone large projects multiple times
Large projects usually have large test suites that can run for a long time.
This can be annoying as running tests blocks you from picking up the next story -- but it doesn't have to be that way!
Simply clone your project's repo twice (or even more often).
When your work on a feature branch is done, simply push that branch and check it out on your 2nd copy to run tests there.
You can pick up a new story and work on that on your "main" project directory.
If you do it right, you will even be able to run tests in both your 2nd copy and your m...
Angular: Keeping attributes with invalid values in an ngModel
The Angular 1.2 way:
# By default, angular returns undefined for invalid attributes which removes
# the value from the form field's ngModel (which means it's not sent to the
# server, and old values would not be overwritten).
#
# This directive makes sure that form fields with an invalid value return an
# empty string instead of undefined.
for elementType in ['input', 'textarea', 'select']
@app.directive elementType, ->
priority: 1
restrict: 'E'
require: '?ngModel'
link: (scope, element, attributes, ngModelControlle...
Paperclip: Move attachements from local storage to AWS S3
We frequently use the handy Paperclip Gem to manage file attachments.
If you need to move the files from local storage (i.e., your servers' harddisk) to Amazon S3, you can simply change settings for Paperclip to use the S3 storage adapter and use this script to migrate to S3. Put the snippet into a chore if you don't want to run that in the console.
YOUR_LOCAL_STORAGE_MODEL_DIRECTORY should be something like 'storage/your_model'.
Dir.glob(YOUR_LOCAL_STORAGE_MODEL_DIRECTORY**/*).each do |path|...
Solve screen error "Cannot open your terminal '/dev/pts/0' - please check"
When using the screen
tool you may be unable to start a screen session but instead encounter an error:
Cannot open your terminal '/dev/pts/0' - please check.
This is because another user (you) initiated the current terminal -- you probably did a sudo su
into the user you are now trying to run screen as, right?
There are two ways to resolve this:
- Sign out and properly connect / sign in as the user you wish to use.
- Run
script /dev/null
to own the shell (more info over at [Server Fault](http://serverfault.com/questions/116775/...
Accessing Rails config in webpack(er)
It is possible to access Rails config (for example secrets) from within your webpack bundles, thanks to rails-erb-loader. When using webpacker, the setup is like this:
-
Install
rails-erb-loader
:yarn add rails-erb-loader
-
Add this to your
config/webpacker/environment.js
:environment.loaders.prepend('erb', { test: /\.erb$/, enforce: 'pre', use: [{ loader: 'rails-erb-loader', }] })
-
Start using erb. For examp...
How to preview an image before uploading it
When building a form with a file select field, you may want to offer your users a live preview before they upload the file to the server.
HTML5 via jQuery
Luckily, HTML5 has simple support for this. Just create an object URL and set it on an <img>
tag's src
attribute:
$('img').attr('src', URL.createObjectURL(this.files[0]))
Unpoly Compiler
As an Unpoly compiler, it looks like this:
up.compiler '[image_p...
Show MySQL process list without sleeping connections
Usually you don't need to, but when you want to see which queries your MySQL server currently needs to handle (and if there are locks, etc), you could say SHOW PROCESSLIST
in a MySQL shell.
Unfortunately, SHOW PROCESSLIST
does not allow filtering. When you are on MySQL ≥ 5.1.7, do this instead:
SELECT * FROM information_schema.processlist WHERE command != 'Sleep' ORDER BY id;
That also allows you to only show some values or order differently, like so:
SELECT user, time, state, info FROM information_schema.processlist WHERE co...
Fix error: Invalid gemspec / Illformed requirement
When you get an error like this:
Invalid gemspec in [/opt/www/foo-project.makandra.de/shared/bundle/ruby/1.8/specifications/carrierwave-0.6.2.gemspec]: Illformed requirement ["#<YAML::Syck::DefaultKey:0x7fda6f84d2e8> 1.1.4"]
... the machine's Rubygems needs to be updated.
If that happens on your local machine
- Manually remove the offending's gem files and specifications. The paths will be something like
/usr/lib/ruby/gems/1.8/gems/your-broken-gem
and `/usr/lib/ruby/gems/1.8/specificatio...
Rack dies when parsing large forms
- Rack has a limit for how many form parameters it will parse.
- This limit is 65536 by default.
- There is a bug in Rack that will incorrectly count the number of input fields in nested forms. In my case a form with 1326 input fields was enough to break the default limit.
- If Rack thinks your request is too large, the request will fail with a low-level Ruby message like Fix: "undefined method `bytesize' for #" or the standard Rails error box.
- You ...
Fixing "Lock obtain timed out: SimpleFSLock" for Solr
If your application raises an error like ...
Couldn't connect to the Solr server at http://127.0.0.1:8983/solr. 500 "Lock_obtain_timed_out_SimpleFSLock(...)"
... and if your jetty.request.log
contains entries such as ...
127.0.0.1 - - [13/09/2011:12:42:23 +0000] "POST /solr/update HTTP/1.1" 500 4412
127.0.0.1 - - [13/09/2011:13:37:03 +0000] "POST /solr/update HTTP/1.1" 500 4309
... you probably have a lucene-...-write.lock
file lying around in your Solr data directory that is not being cleaned up properly. This causes...
Ubuntu: Share internet connections with other computers
You can configure a Ubuntu system as a gateway in order to share it's internet connection (maybe via WLAN or tethering) with other computers on the network.
On the gateway
- Enable ip traffic forwarding:
-
Open
/etc/sysctl.conf
-
Uncomment the line
net.ipv4.ip_forward=1
-
Reload using
sudo sysctl -p /etc/sysctl.conf
-
- Reconfigure ip_tables to allow NAT:
- Download the attached file
- Replace
online_device
with the name of the network device that provides the internet connection
...
Fix: "undefined method `bytesize' for #<Array>"
I believe that when WEBrick has trouble bringing up your Rails application, the WEBrick component that is supposed to print you a pretty error message has a bug and sometimes fails with this message:
"undefined method `bytesize' for #<Array>"
Starting the application in Passenger gave me a stacktrace in log/development.log
that pointed to the actual problem.
Possible causes discovered by looking at the logs
-----------------------------------------------------...
Howto provide a single page preview for PDF & TXT with carrierwave
Assert rmagick
provision ...
Gemfile
gem 'rmagick', '2.13.2' # at this moment the latest stable version
config/initializer/carrierwave.rb
require 'carrierwave/processing/rmagick'
... and define a custom processor
MyUploader.rb
class MyUploader < CarrierWave::Uploader::Base
include CarrierWave::RMagick
def cover
manipulate! do |frame, index|
frame if index.zero? # take only the first page of the file
end
end
version :preview do
process :cover
process :resize_to_fit => [310,...
Fix slow specs using SOLR
I've recently encountered a weird problem with specs making lots of SOLR queries using the acts_as_solr
plugin: After a certain number of specs, exactly one spec suddenly took over 30 seconds to finish.
It turns out that for some reason, the SOLR server seemed not to close its HTTP connections properly. After the maximum number of connections was reached, the next spec needed to wait for an old connection to time out.
I'm not exactly sure why this happened, why it only seems to be happening in specs and which part of the code is actually...
Speed up Capistrano deployments using a remote cached copy of repository
You can seriously speed up deployments with Capistrano when using a local git repository on the server you are deploying to.
Simply add
set :deploy_via, :remote_cache
set :copy_exclude, [ '.git' ]
to your config/deploy.rb
and Capistrano will create a clone in shared/cached-copy
. This will be updated using git pull
when deploying which transfers less bytes and is usually much faster. If deploy_via
is set to use default settings (being "export
"), Capistrano will do a full clone of the repository from your git host otherwi...
Unicorn: How to force a single threaded boot in development
Unicorn allows you to specify the maximum number of workers. In development this could be useful if you use a debugger, but do not want to overflow the console with other request like from ActionCable. Then just set the maximum number of workers to 1
and the other requests have to wait.
UNICORN_WORKERS=1 rails server
In-depth HTTP traffic analysis using tcpdump & Wireshark
From time to time we're convinced that an error must be very close to the network card, OS IP stack or compiler. In reality this is quite rare, so before continuing, triple-check that the issue is not located between chair and keyboard...
If you're still convinced that a in-depth analysis of network traffic might help you, go on:
-
Find out the IP address the client causing trouble will come from.
-
Replace 147.0.0.123 with the client address, log into your web server and run:
`remote$ sudo tcpdump host 147.0.0.123 and port 80 -s 0 -w...
postgresql create extension without giving the application superuser rights
If you need a postgresql extension for your database it isn't a good idea to give your applications database user superuser rights (like many people on stackoverflow think)
Just login to the database with a superuser account (e.g. postgres) and create the extension with it.
Example:
# with the default configuration of postgresql you normally can login as `postgres` user
# without a password if you use the systems `postgres` user
$ sudo su -l postgres
$ pgsql
postgres=# \c your_database;
psql (9.3.9, server 9.3.5)
You are now connected...
How to silence thin boot messages
Each time thin
boots, it prints a boot message :
Thin web server (v1.6.3 codename Protein Powder)
Maximum connections set to 1024
Listening on localhost:36309, CTRL+C to stop
If you are running parallel tests with thin
, this will clutter you output. Disable thin
logging with these lines:
# e.g. in features/support/thin.rb
require 'thin'
Thin::Logging.silent = true
Note that this disables all logging in tests. Instead, you also might set a different logger with `Thin::Loggi...