Using the Truemail gem to validate e-mail addresses
The Truemail gem (not to be confused with truemail.io) allows validating email addresses, e.g. when users enter them into a sign-up form. It runs inside your application and does not depend on an external SaaS service.
Truemail supports different validation "layers":
- Regex validation: if the given address is syntactically valid
- DNS validation (called MX validation): if the given domain exists and can receive email
- SMTP validation: connects to the host received from DNS and starts a test d...
Transporting blank values in URL queries
URLs can transport key/value pairs ("parameters") using this syntax:
/path?foo=bar
If the value is blank, mind these subtle differences:
URL | Meaning |
---|---|
/path?foo= |
Parameters have a key foo . Its value is an empty string. |
/path?foo |
Parameters have a key foo . Its value is null . |
/path |
Parameters have no key foo . |
Configuring Webpacker deployments with Capistrano
When deploying a Rails application that is using Webpacker and Capistrano, there are a few configuration tweaks that optimize the experience.
Using capistrano-rails
capistrano-rails is a Gem that adds Rails specifics to Capistrano, i.e. support for Bundler, assets, and migrations. While it is designed for Asset Pipeline (Sprockets) assets, it can easily be configured for Webpacker. This brings these features to the Webpacker world:
- Automatic removal of expired assets
- Manifest backups
Minidusen: Filtering associated records
Minidusen lets you find text in associated records.
Assume the following model where a Contact
record may be associated with a Group
record:
class Contact < ApplicationRecord
belongs_to :group
validates_presence_of :name, :street, :city, :email
end
class Group < ApplicationRecord
has_many :contacts
validates_presence_of :name
end
We can filter contacts by their group name by joining the groups
table and filtering on a joined column.
Note how the joined column is qualified as groups.name
(rather than just `na...
Passive event listeners may speed up your scroll and touch events
Scroll and touch event listeners tend to be computationally expensive as they are triggered very often. Every time the event is fired, the browser needs to wait for the event to be processed before continuing - the event could prevent the default behavior. Luckily there is a concept called passive event listeners which is supported by all modern browsers.
Below are the key parts quoted from WICG's explainer on passive event listeners. See [this demo video](https://www.youtube.com/watch?v=NPM6172...
How to negate scope conditions in Rails
Sometimes you want to find the inverse of an ActiveRecord scope. Depending on what you want to achieve, this is quite easy with Rails 7, and a bit more complicated with Rails 6 and below, or when the inverse scope may contain NULL values. [1]
There are two different ways of "inverting a scope":
As an example, consider the following model.
class User < ApplicationRecord
scope :admins, -> { where(role: ['admin', 'superuser']) }
# ...
end
Mathematical NOT
You know this one from basic set theory. It proces the "complementa...
Manage Linux services on the command line (Ubuntu)
Ubuntu 18.04 uses systemd
to manage services.
There are basically two commands for listing all services and manipulating the state of a certain service: service
and systemctl
:
-
service
manages System V init scripts -
systemctl
controls the state of the systemd system and service manager. It is backwards compatible to System V and includes the System V services
Therefore I prefer to use systemctl
.
See which services are there
>systemctl list-units -a --type=service
UNIT LOAD ...
Migrate gem tests from Travis CI to Github Actions with gemika
We currently test most of our gems on Travis CI, but want to migrate those tests to Github Actions. This is a step-by-step guide on how to do this.
Note that this guide requires the gem to use gemika.
- Go to a new "ci" branch:
git checkout -b ci
- Update gemika to version >= 0.5.0 in all your Gemfiles.
- Have gemika generate a Github Actions workflow definition by running
mkdir -p .github/workflows; bundle exec rake gemika:generate_github_actions_workflow > .github/workf...
Debugging SPF records
While debugging a SPF record I found spf-record.de to be very helpful.
- it lists all IPs that are covered by the SPF record
- shows syntax errors
- helps you debugging errors like DNS lookup limit reached
- it also lets you test a new SPF strings before applying it. This can save you time as you don't have to loop with operations
Also the advanced check at vamsoft.com has a very good interface to test new SPF policies.
PostgreSQL: Importing dumps created with newer versions
When loading a database dump created with pg_dump
into your database, you might run into an error like
pg_restore: error: unsupported version (1.15) in file header
This is because your local pg_restore
version is too old to match the format created by pg_dump
. The version of the PostgreSQL server doesn't matter here.
For example, the official Ubuntu 20.04 sources include only PostgreSQL 12, so your pg_restore
version will also be v12. Ubuntu 22.04 includes version 14 in its sources.
Both seem to be incompatible with dumps ...
Rails: How to restore a postgres dump from the past
It sometimes happen that a database dump, that would want to insert into your development database, does not match the current schema of the database. This often happens when you have an old dump, but your current setup is up to date with the the master.
Hint: In most cases it is sufficient to delete and recreate the local database in order to import the dump. If any problems occur, proceed as follows:
1. Figure out the original migration status of the dumpfile
- Convert your dump to plaintext: `pg_restore -f some.dump > some.dump....
How to implement simple queue limiting/throttling for Sidekiq
The sidekiq-rate-limiter gem allows rate-limiting Sidekiq jobs and works like a charm. However, it needs to be integrated on a per-worker basis.
If you want to limit a whole queue instead, and if your requirements are simple enough, you can do it via a Sidekiq middleware yourself.
Here is an example that limits concurrency of the "mailers" queue to 1. It uses a database mutex via the [with_advisory_lock](https://github.com/ClosureTree/wit...
Rails: How to list all validations on a model or an attribute
If a model inherits from others or uses many concerns / traits, it might be hard to see in the code which validators it has.
But fortunately there's a method for that:
irb(main):002:0> pp UserGroup.validators
[#<ActiveModel::Validations::InclusionValidator:0x00007f55efff97a8
@attributes=[:deleted],
@delimiter=[true, false],
@options={:in=>[true, false], :allow_nil=>false}>,
#<ActiveModel::Validations::InclusionValidator:0x00007f55f15748d0
@attributes=[:cancelled],
@delimiter=[true, false],
@options={:in=>[true, false], ...
How to include Sidekiq job IDs in Rails logs
When logging in Rails, you can use the log_tags
configuration option to add extra information to each line, like :request_id
or :subdomain
. However, those are only valid inside a request context and have no effect when your application is logging from inside a Sidekiq process.
This includes custom as well as any framework logs, like query logging from ActiveRecord.
Since Sidekiq Workers run inside threads of a single process, running multiple jobs in...
How to generate GIDs from an ActiveRecord scope
ActiveRecord provides the ids
method to pluck ids from a scope, but what if you need to pluck Global IDs?
While you could just call map(&:to_global_id)
on your scope, this approach would instantiate each record just to do that. When you have many records, this will at the very least be slow.
Here is a method that does it for you efficiently. It respects Single Table Inheritance (STI).
Put it in your project's ApplicationRecord
to make it available on all models.
class ApplicationRecord
...
Workflow: How to use a key management service to encrypt passwords in the database
This is an extract from the linked article. It shows an approach on how to implement encrypted passwords with the AWS Key Management Service (KMS).
For most applications it's enough to use a hashed password with a salt (e.g. the gem devise defaults to this).
Upon password creation
-
Generate hash as hash of password + salt.
-
Encrypt the hash with a public key from KMS (you can store the public key in your server code).
-
In your database sto...
Carrierwave: How to migrate to another folder structure
A flat folder structure can be cool if you have only a few folders but can be painful for huge amounts. We recently had this issue in a project with more than 100.000 attachments, where we used a structure like this /attachments/123456789/file.pdf
.
Even the ls
command lasted several minutes to show us the content of the attachments folder.
So we decided to use a more hierarchical structure with a limited maximum of folder per layer. Here are a few tips how to migrate your files to their new...
How to fix: Rails query logs always show lib/active_record/log_subscriber.rb as source
Rails 5.2+ supports "verbose query logs" where it shows the source of a query in the application log.
Normally, it looks like this:
User Load (0.5ms) SELECT "users".* FROM "users" WHERE ...
↳ app/controllers/users_controller.rb:42:in `load_users'
However, you may encounter ActiveRecord's LogSubscriber as the source for all/most queries which is not helpful at all:
User Load (0.5ms) SELECT "users".* FROM "users" WHERE ...
↳ activerecord (6.0.3.3) lib/active_record/log_subscriber.rb:100:in `debug'
While th...
Service Worker series by GoMakeThings
Learn how to create offline applications with service workers.
- The amazing power of service workers
- Writing your first service worker with vanilla JS
- Saving recently viewed pages offline with service workers and vanilla JS
- Offline first with service workers and vanilla JS
- Improving web font performance with service workers
- How to set an expiration date for items in a service worker cache
- How to update a service worker
- How to trigger a service worker function from the front end with vanilla JS
- How to immediately ...
Controlling issue grouping in Sentry
When you use Sentry to monitor exceptions, an important feature is Sentry's error grouping mechanism. It will aggregate similar error "events" into one issue, so you can track and monitor it more easily. Grouping is especially important when you try to silence certain errors.
It is worth understanding how Sentry's grouping mechanism works.
The default grouping mechanism
The exact algorithm has changed over time, and Sentry will keep using the algorithm t...
Chrome Lighthouse
Chrome has a built-in utility to check performance and accessibility (and more) of your web app: Lighthouse.
Open the Developer Tools and go to the lighthouse tab:
Then you'll see some suggestions on how to improve your site.
This is cool, because you can even use it with non-public pages or your development environment (but be aware that some settings we're using for development, like not minifying JS and CSS files, might ruin your stats)...
Rails: How to get the ordered list of used middlewares
Rails middlewares are small code pieces that wrap requests to the application. The first middleware gets passed the request, invokes the next, and so on. Finally, the application is invoked, builds a response and passes it back to the last middleware. Each middleware now returns the response until the request is answered. Think of it like Russian Dolls, where each middleware is a doll and the application is the innermost item.
You can run rake middleware
to get the ordered list of used middlewares in a Rails application:
$> rake midd...
PostgreSQL: How to use with_advisory_lock to prevent race conditions
If you want to prevent that two processes run some code at the same time you can use the gem with_advisory_lock.
What happens
- The thread will wait indefinitely until the lock is acquired.
- While inside the block, you will exclusively own the advisory lock.
- The lock will be released after your block ends, even if an exception is raised in the block.
This is usually required if there is no suitable database row to lock on.
Example
You want to generate a...
How to cycle through grep results with vim
grep
is the go-to CLI tool to accomplish tasks like filtering large files for arbitrary keywords. When additional context is needed for search results, you might find yourself adding flags like -B5 -A10
to your query. Now, every search result covers 16 lines of your bash.
There is another way: You can easily pipe your search results to the VIM editor and cycle through them.
Example: Searching for local occurrences of "User"
vim -q <(grep -Hn -r "User" .)
# vim -q starts vim in the "quickfix" mode. See ":help quickfix"
# grep...