Decide whether cronjobs should run on one or all servers
Understanding your type of cronjob
Some cronjobs must only run on a single server. E.g. when you run nightly batch operations on the database, it should probably run on a single server. Running it on multiple servers would likely result in deadlocks or corrupt data.
Some cronjobs must always run on all servers. E.g. starting a sidekiq process on reboot.
Configuring whenever
If not configured otherwise, cronjobs defined in whenever's `s...
Using ngrok for exposing your development server to the internet
Sometimes you need to access a dev server running on localhost from another machine that is not part of the same network. Maybe you want to use your phone to test a web page, but are only in a guest WiFi. In the past, we often used some port forwarding or other techniques to expose the service to the internet.
Enter ngrok, a command line tool that gives you an on-the-fly internet...
Capybara: Preventing server errors from failing your test
When your Rails application server raises error, Capybara will fail your test when it clears the session after the last step. The effect is a test that passes all steps, but fails anyway.
Capybara's behavior will help you to detect and fix errors in your application code. However, sometimes your application will explode with an error outside your control. Two examples:
- A JavaScript library references a source map in its build, but forgets to package the source map
- CarrierWave cleans up an upload or cache file after the record was delet...
Debug SAML in development using a local keycloak server
Developing or debugging SAML functionality can be a hassle, especially when you need to go back and forth with someone external who is managing the identity provider (IDP).
But you can setup a local keycloak server to act as your IDP to play around with. This might seam intimidating, but is actually quite simple when using docker and turning off some verification steps.
1. Start a keycloak instance using docker
`mkdir -p keycloak_data && docker run --network=host -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN...
Capistrano 3: Running a command on all servers
This Capistrano task runs a command on all servers.
bundle exec cap production app:run cmd='zgrep -P "..." RAILS_ROOT/log/production.log'
Code
# lib/capistrano/tasks/app.rake
namespace :app do
# Use e.g. to grep logs on all servers:
# b cap production app:run_cmd cmd='zgrep -P "..." RAILS_ROOT/log/production.log'
#
# * Use RAILS_ROOT as a placeholder for the remote Rails root directory.
# * Append ` || test $? =1;` to grep calls in order to avoid exit code 1 (= "nothing found")
# * To be able to process ...
How to kill a Rails development server by force
Sometimes, the rails dev server doesn't terminate properly. This can for example happen when the dev server runs in a RubyMine terminal.
When this happens, the old dev server blocks port 3000, so when you try to start a new server, you get the error:
Address already in use - bind(2) for "127.0.0.1" port 3000 (Errno::EADDRINUSE)
You can terminate such a dev server with this command:
lsof -t -i :3000 -s TCP:LISTEN | xargs kill -9
It might be worth it to add this to your bash aliases.
When upgrading/downgrading RubyGems and Bundler on a server, you must clear bundled gems
On application servers, gems are usually bundled into the project directory, at a location shared across deployments.
This is usually shared/bundle
inside your project's root directory, e.g. /var/www/your-project/shared/bundle/
.
If you can't find that, take a look at current/.bundle/config
and look for BUNDLE_PATH
.
When you are changing the version of RubyGems or Bundler on a system where gems are installed this way, you must wipe that bundle directory in addition to the user and system gems or gems that are already ins...
Clean up application servers when deploying
Our development process makes us deploy very often. As the number of releases grows, junk clogs up the hard drive of our application servers:
- Old release code
- Old
tmp
folders with compiled view templates etc. - Precompiled assets (Javascripts, images...) that no longer exist. When using the asset pipeline, Capistrano will symlink the
public/assets
directory toshared/assets
. This is cool since we can still serve previous assets after a new release, in the window where browser caches might still have references to old assets. But i...
Migrate data between Redis servers
There is a reasonable simple way to move data between Redis servers: Simply temporarily configure the new server as a replica of the old server.
To do this:
- Make sure the new Redis server can access the old server. If they are on different networks, a simple SSH tunnel will do.
- Connect to the new server using
redis-cli
. - Tail the log of the redis server (found in
/var/logs/redis
) in another terminal. - Make sure the server is currently master and not already a replica (check
INFO replication
) - Enable replication with `REPL...
Using ffmpeg as a HLS streaming server
A practical and detailed walk-through tutorial on using ffmpeg for live-streaming HLS, filled with real-world examples.
- Using FFmpeg as a HLS streaming server (Part 1) – HLS Basics
- Using FFmpeg as a HLS streaming server (Part 2) – Enhanced HLS Segmentation
- Using FFmpeg as a HLS streaming server (Part 3) – Multiple Bitrates
- Using FFmpeg as a HLS streaming server (Part 4) – Multiple Video Resolutions
- Using FFmpeg as a HLS streaming server (Part 5) – Folder Structure
- Using FFmpeg as a HLS streaming server (Part 6) – Independent Seg...
How to run a small web server (one-liner)
Sometimes you just want to have a small web server that serves files to test something.
Serve the current directory
On Ruby 1.9.2+ you can do the following (".
" for current directory).
ruby -run -ehttpd . -p8000
Python 2.x offers a similar way.
python -m SimpleHTTPServer 8000 .
This is the same way with Python 3.x
python -m http.server
In both cases your web server is single-threaded and will block when large files are being downloaded from you.
WEBrick also offers [a simple way](https://stackoverflow.com/quest...
Running Rails 2 apps with modern MariaDB SQL server
You might have some trouble running a Rails LTS 2 app with MySQL 5.7.
If you don't want to hack Mysql 5.6 into your modern Ubuntu or use the MySQL sandbox, you might want to try MariaDB 10.x.
MariaDB 10.x should work with both old and new Rails applications.
[Switch to MariaDB](https://makandracards.com/makandra/468343-how-...
How to use the Capistrano 2 shell to execute commands on servers
Capistrano 2 brings the shell
command which allows you to run commands on your deployment targets.
There is also invoke
to run a command directly from your terminal.
Both commands allow running Capistrano tasks or shell commands, and scope to individual machines or machine roles.
Unfortunately Capistrano 3 does not include these commands any more.
cap shell
Basics
First of all, spawn a Capistrano shell (we're using the multistage
extension here):
$ cap staging shell
In your "cap" shell you can now run Capistrano ta...
Migrating IMAP accounts from server to server
Do not investigate in the 2342 tools that exist for migrating IMAP accounts from one server to another. They all suck.
Try your luck with imapsync.
We have no clue why the guys suggest buying the script on their site, maybe they missed removing that part after this April fool's trick.
Go to https://fedorahosted.org/released/imapsync/ and get the sources there.
Hide your Selenium browser window with a VNC server
This is now part of geordi. Please don't follow the instructions below, if you use geordi.
Inspired by the recent headless Selenium note, I found yet another solution for the problem to hide your selenium tests away.
This has the advantages
^
- not to require a gem (so you do not force this on others)
- to allow you to take a look at the running webdriver if necessary
Simply make a script th...
MySQL Server and UTF-8 Defaults
Unless all MySQL server defaults are set to UTF-8, mysqldump encodes UTF-8 characters incorrectly and only outputs correct UTF-8 when you switch to Latin 1 (!). Also you need to setup charset and collation manually for each new database.
To prevent this, make sure your /etc/mysql/my.cnf
looks like this:
[mysqld]
default-character-set = utf8mb4
collation-server = utf8mb4_unicode_ci
[mysql]
default-character-set=utf8mb4
After that do
sudo /etc/init.d/mysql restart
Capistrano task to tail remote application logs of multiple servers
When your application is running on a multi-server setup, application logs are stored per server (unless you choose a centralized logging solution).
Here is a Capistrano task that connects to all servers and prints logs to your terminal like this:
$ cap production app:logs
00:00 app:logs
01 tail -n0 -F /var/www/your-application/shared/log/production.log | while read line; do echo "$(hostname): $line"; done
01 app01-prod: Started GET "/sign_in" for 1.2.3.4 at 2018-04-26 11:28:19 +0200
01 app01-prod: Proc...
Capistrano 2: How to deploy a single server
When you have a multi-server setup, you'll be adding a new server from time to time. Before doing a full deploy, you might want to test that server in an isolated deploy. There is a single way to do this: the HOSTFILTER
env variable.
Commenting out "server" lines in the Capistrano deploy config will raise a Capistrano::NoMatchingServersError
with <task> is only run for servers matching {:roles=> <role>}, but no servers matched
. Instead, specify the server-under-test like this:
HOSTFILTER=separate-sidekiq.makandra.de cap productio...
Interacting with a Microsoft Exchange server from Ruby
Microsoft Exchange service administrators can enable Exchange Web Services (EWS) which is a rather accessible XML API for interacting with Exchange. This allows you to read and send e-mails, create appointments, invite meeting attendees, track responses, manage to-do tasks, check user availability and all other sorts of things that are usually only accessible from Outlook.
You can implement an EWS by hand-rolling your XML (the [docs](http://msdn.microsoft.com/en-us/...
How to fix webpack-dev-server not found
The bin/webpack-dev-server
command is not as smart as e.g. rails server
, where it shows the proper fix within the error message.
$ bin/webpack-dev-server
yarn run v1.19.1
error Command "webpack-dev-server" not found.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Run yarn install --check-files
to fix this error.
Test whether Perfect Forward Secrecy (PFS) is enabled on a server (using OpenSSL)
Use the following command to test if a server (in this example: makandra.com on port 443) uses Perfect Forward Secrecy (PFS):
openssl s_client -connect makandra.com:443 -cipher ECDHE-RSA-RC4-SHA
You should see something like the following:
~ > openssl s_client -connect projecthero.com:443 -cipher ECDHE-RSA-RC4-SHA
CONNECTED(00000003)
depth=1 O = AlphaSSL, CN = AlphaSSL CA - G2
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
0 s:/C=DE/OU=Domain Control Va...
Install a local Gemfile on a remote server
Call with the server's hostname (and user if you have no SSH agent), e.g.
install-gems-remotely my.server.com
# or without agent:
install-gems-remotely me@my.server.com
When you call it from a rails directory, it uploads your Gemfile
, Gemfile.lock
as well as the gemspecs of all vendored gems in to a temporary folder on the server and does a bundle install
there.
If you need to install gems from anothere Gemfile, just do it like this:
BUNDLE_GEMFILE=Gemfile.something; install-gems-remotely my.server.com
This scri...
Setting up Ubuntu Server as Virtual Machine in VMware
Install Ubuntu Server
- Download an image from this site
- Select
File > New…
to and follow the instructions, choose the.iso
file you downloaded as image file - Deselect "Simple installation" – you want to configure your system yourself
- Start the new virtual machine and follow the instructions
Install VMware Tools
- Choose
Virtual Machine > Install VMware Tools
from the VMware menu, then:
^
# install required packages
sudo apt-get install build-essential psmisc
# m...
How to: Restart vnc server for geordi
Trying to open a vnc window with geordi geordi vnc
ended up with this error:
> VNC viewer could not be opened:
vncviewer: ConnectToTcpAddr: connect: Connection refused
Check if vncserver 17 (default geordi session) is running:
ps -aux|grep vnc
If you see Xvnc4 :17
then the server is already running. Kill this server if it is already running.
vncserver -kill :17
Als...