Decide whether cronjobs should run on one or all servers

Understanding your type of cronjob

Some cronjobs must only run on a single server. E.g. when you run nightly batch operations on the database, it should probably run on a single server. Running it on multiple servers would likely result in deadlocks or corrupt data.

Some cronjobs must always run on all servers. E.g. starting a sidekiq process on reboot.

Configuring whenever

If not configured otherwise, cronjobs defined in whenever's `s...

Using ngrok for exposing your development server to the internet

Sometimes you need to access a dev server running on localhost from another machine that is not part of the same network. Maybe you want to use your phone to test a web page, but are only in a guest WiFi. In the past, we often used some port forwarding or other techniques to expose the service to the internet.

Enter ngrok, a command line tool that gives you an on-the-fly internet...

Capybara: Preventing server errors from failing your test

When your Rails application server raises error, Capybara will fail your test when it clears the session after the last step. The effect is a test that passes all steps, but fails anyway.

Capybara's behavior will help you to detect and fix errors in your application code. However, sometimes your application will explode with an error outside your control. Two examples:

  • A JavaScript library references a source map in its build, but forgets to package the source map
  • CarrierWave cleans up an upload or cache file after the record was delet...

Debug SAML in development using a local keycloak server

Developing or debugging SAML functionality can be a hassle, especially when you need to go back and forth with someone external who is managing the identity provider (IDP).
But you can setup a local keycloak server to act as your IDP to play around with. This might seam intimidating, but is actually quite simple when using docker and turning off some verification steps.

1. Start a keycloak instance using docker

`mkdir -p keycloak_data && docker run --network=host -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN...

Capistrano 3: Running a command on all servers

This Capistrano task runs a command on all servers.

bundle exec cap production app:run cmd='zgrep -P "..." RAILS_ROOT/log/production.log'

Code

# lib/capistrano/tasks/app.rake

namespace :app do

  # Use e.g. to grep logs on all servers:
  #   b cap production app:run_cmd cmd='zgrep -P "..." RAILS_ROOT/log/production.log' 
  #
  # * Use RAILS_ROOT as a placeholder for the remote Rails root directory.
  # * Append ` || test $? =1;` to grep calls in order to avoid exit code 1 (= "nothing found")
  # * To be able to process ...

How to kill a Rails development server by force

Sometimes, the rails dev server doesn't terminate properly. This can for example happen when the dev server runs in a RubyMine terminal.

When this happens, the old dev server blocks port 3000, so when you try to start a new server, you get the error:

Address already in use - bind(2) for "127.0.0.1" port 3000 (Errno::EADDRINUSE)

You can terminate such a dev server with this command:

lsof -t -i :3000 -s TCP:LISTEN | xargs kill -9

It might be worth it to add this to your bash aliases.

When upgrading/downgrading RubyGems and Bundler on a server, you must clear bundled gems

On application servers, gems are usually bundled into the project directory, at a location shared across deployments.

This is usually shared/bundle inside your project's root directory, e.g. /var/www/your-project/shared/bundle/.
If you can't find that, take a look at current/.bundle/config and look for BUNDLE_PATH.

When you are changing the version of RubyGems or Bundler on a system where gems are installed this way, you must wipe that bundle directory in addition to the user and system gems or gems that are already ins...

Clean up application servers when deploying

Our development process makes us deploy very often. As the number of releases grows, junk clogs up the hard drive of our application servers:

  • Old release code
  • Old tmp folders with compiled view templates etc.
  • Precompiled assets (Javascripts, images...) that no longer exist. When using the asset pipeline, Capistrano will symlink the public/assets directory to shared/assets. This is cool since we can still serve previous assets after a new release, in the window where browser caches might still have references to old assets. But i...

Migrate data between Redis servers

There is a reasonable simple way to move data between Redis servers: Simply temporarily configure the new server as a replica of the old server.

To do this:

  1. Make sure the new Redis server can access the old server. If they are on different networks, a simple SSH tunnel will do.
  2. Connect to the new server using redis-cli.
  3. Tail the log of the redis server (found in /var/logs/redis) in another terminal.
  4. Make sure the server is currently master and not already a replica (check INFO replication)
  5. Enable replication with `REPL...

Using ffmpeg as a HLS streaming server

A practical and detailed walk-through tutorial on using ffmpeg for live-streaming HLS, filled with real-world examples.

  • Using FFmpeg as a HLS streaming server (Part 1) – HLS Basics
  • Using FFmpeg as a HLS streaming server (Part 2) – Enhanced HLS Segmentation
  • Using FFmpeg as a HLS streaming server (Part 3) – Multiple Bitrates
  • Using FFmpeg as a HLS streaming server (Part 4) – Multiple Video Resolutions
  • Using FFmpeg as a HLS streaming server (Part 5) – Folder Structure
  • Using FFmpeg as a HLS streaming server (Part 6) – Independent Seg...

How to run a small web server (one-liner)

Sometimes you just want to have a small web server that serves files to test something.

Serve the current directory

On Ruby 1.9.2+ you can do the following ("." for current directory).

ruby -run -ehttpd . -p8000

Python 2.x offers a similar way.

python -m SimpleHTTPServer 8000 .

This is the same way with Python 3.x

python -m http.server

In both cases your web server is single-threaded and will block when large files are being downloaded from you.

WEBrick also offers [a simple way](https://stackoverflow.com/quest...

Running Rails 2 apps with modern MariaDB SQL server

You might have some trouble running a Rails LTS 2 app with MySQL 5.7.

If you don't want to hack Mysql 5.6 into your modern Ubuntu or use the MySQL sandbox, you might want to try MariaDB 10.x.

MariaDB 10.x should work with both old and new Rails applications.

[Switch to MariaDB](https://makandracards.com/makandra/468343-how-...

How to use the Capistrano 2 shell to execute commands on servers

Capistrano 2 brings the shell command which allows you to run commands on your deployment targets.
There is also invoke to run a command directly from your terminal.

Both commands allow running Capistrano tasks or shell commands, and scope to individual machines or machine roles.

Unfortunately Capistrano 3 does not include these commands any more.

cap shell

Basics

First of all, spawn a Capistrano shell (we're using the multistage extension here):

$ cap staging shell

In your "cap" shell you can now run Capistrano ta...

Migrating IMAP accounts from server to server

Do not investigate in the 2342 tools that exist for migrating IMAP accounts from one server to another. They all suck.

Try your luck with imapsync.
We have no clue why the guys suggest buying the script on their site, maybe they missed removing that part after this April fool's trick.

Go to https://fedorahosted.org/released/imapsync/ and get the sources there.

Hide your Selenium browser window with a VNC server

This is now part of geordi. Please don't follow the instructions below, if you use geordi.

Inspired by the recent headless Selenium note, I found yet another solution for the problem to hide your selenium tests away.

This has the advantages
^

  • not to require a gem (so you do not force this on others)
  • to allow you to take a look at the running webdriver if necessary

Simply make a script th...

MySQL Server and UTF-8 Defaults

Unless all MySQL server defaults are set to UTF-8, mysqldump encodes UTF-8 characters incorrectly and only outputs correct UTF-8 when you switch to Latin 1 (!). Also you need to setup charset and collation manually for each new database.

To prevent this, make sure your /etc/mysql/my.cnf looks like this:

[mysqld]
default-character-set = utf8mb4
collation-server = utf8mb4_unicode_ci

[mysql]
default-character-set=utf8mb4

After that do

sudo /etc/init.d/mysql restart

Capistrano task to tail remote application logs of multiple servers

When your application is running on a multi-server setup, application logs are stored per server (unless you choose a centralized logging solution).
Here is a Capistrano task that connects to all servers and prints logs to your terminal like this:

$ cap production app:logs
00:00 app:logs
      01 tail -n0 -F /var/www/your-application/shared/log/production.log | while read line; do echo "$(hostname): $line"; done
      01 app01-prod: Started GET "/sign_in" for 1.2.3.4 at 2018-04-26 11:28:19 +0200
      01 app01-prod: Proc...

Capistrano 2: How to deploy a single server

When you have a multi-server setup, you'll be adding a new server from time to time. Before doing a full deploy, you might want to test that server in an isolated deploy. There is a single way to do this: the HOSTFILTER env variable.

Commenting out "server" lines in the Capistrano deploy config will raise a Capistrano::NoMatchingServersError with <task> is only run for servers matching {:roles=> <role>}, but no servers matched. Instead, specify the server-under-test like this:

HOSTFILTER=separate-sidekiq.makandra.de cap productio...

Interacting with a Microsoft Exchange server from Ruby

Microsoft Exchange service administrators can enable Exchange Web Services (EWS) which is a rather accessible XML API for interacting with Exchange. This allows you to read and send e-mails, create appointments, invite meeting attendees, track responses, manage to-do tasks, check user availability and all other sorts of things that are usually only accessible from Outlook.

You can implement an EWS by hand-rolling your XML (the [docs](http://msdn.microsoft.com/en-us/...

How to fix webpack-dev-server not found

The bin/webpack-dev-server command is not as smart as e.g. rails server, where it shows the proper fix within the error message.

$ bin/webpack-dev-server                                                                  
yarn run v1.19.1
error Command "webpack-dev-server" not found.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Run yarn install --check-files to fix this error.

Test whether Perfect Forward Secrecy (PFS) is enabled on a server (using OpenSSL)

Use the following command to test if a server (in this example: makandra.com on port 443) uses Perfect Forward Secrecy (PFS):

openssl s_client -connect makandra.com:443 -cipher ECDHE-RSA-RC4-SHA

You should see something like the following:

~ > openssl s_client -connect projecthero.com:443 -cipher ECDHE-RSA-RC4-SHA
CONNECTED(00000003)
depth=1 O = AlphaSSL, CN = AlphaSSL CA - G2
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/C=DE/OU=Domain Control Va...

Install a local Gemfile on a remote server

Call with the server's hostname (and user if you have no SSH agent), e.g.

install-gems-remotely my.server.com
# or without agent:
install-gems-remotely me@my.server.com

When you call it from a rails directory, it uploads your Gemfile, Gemfile.lock as well as the gemspecs of all vendored gems in to a temporary folder on the server and does a bundle install there.

If you need to install gems from anothere Gemfile, just do it like this:
BUNDLE_GEMFILE=Gemfile.something; install-gems-remotely my.server.com


This scri...

Setting up Ubuntu Server as Virtual Machine in VMware

Install Ubuntu Server

  • Download an image from this site
  • Select File > New… to and follow the instructions, choose the .iso file you downloaded as image file
  • Deselect "Simple installation" – you want to configure your system yourself
  • Start the new virtual machine and follow the instructions

Install VMware Tools

  • Choose Virtual Machine > Install VMware Tools from the VMware menu, then:

^
# install required packages
sudo apt-get install build-essential psmisc

# m...

How to: Restart vnc server for geordi

Trying to open a vnc window with geordi geordi vnc ended up with this error:

> VNC viewer could not be opened:
vncviewer: ConnectToTcpAddr: connect: Connection refused

Check if vncserver 17 (default geordi session) is running:

ps -aux|grep vnc

If you see Xvnc4 :17 then the server is already running. Kill this server if it is already running.

vncserver -kill :17

Als...