Rails: How to restore a postgres dump from the past
It sometimes happen that a database dump, that would want to insert into your development database, does not match the current schema of the database. This often happens when you have an old dump, but your current setup is up to date with the the master.
Hint: In most cases it is sufficient to delete and recreate the local database in order to import the dump. If any problems occur, proceed as follows:
1. Figure out the original migration status of the dumpfile
- Convert your dump to plaintext: `pg_restore -f some.dump > some.dump....
How to implement simple queue limiting/throttling for Sidekiq
The sidekiq-rate-limiter gem allows rate-limiting Sidekiq jobs and works like a charm. However, it needs to be integrated on a per-worker basis.
If you want to limit a whole queue instead, and if your requirements are simple enough, you can do it via a Sidekiq middleware yourself.
Here is an example that limits concurrency of the "mailers" queue to 1. It uses a database mutex via the [with_advisory_lock](https://github.com/ClosureTree/wit...
Rails: How to list all validations on a model or an attribute
If a model inherits from others or uses many concerns / traits, it might be hard to see in the code which validators it has.
But fortunately there's a method for that:
irb(main):002:0> pp UserGroup.validators
[#<ActiveModel::Validations::InclusionValidator:0x00007f55efff97a8
@attributes=[:deleted],
@delimiter=[true, false],
@options={:in=>[true, false], :allow_nil=>false}>,
#<ActiveModel::Validations::InclusionValidator:0x00007f55f15748d0
@attributes=[:cancelled],
@delimiter=[true, false],
@options={:in=>[true, false], ...
How to include Sidekiq job IDs in Rails logs
When logging in Rails, you can use the log_tags
configuration option to add extra information to each line, like :request_id
or :subdomain
. However, those are only valid inside a request context and have no effect when your application is logging from inside a Sidekiq process.
This includes custom as well as any framework logs, like query logging from ActiveRecord.
Since Sidekiq Workers run inside threads of a single process, running multiple jobs in...
How to generate GIDs from an ActiveRecord scope
ActiveRecord provides the ids
method to pluck ids from a scope, but what if you need to pluck Global IDs?
While you could just call map(&:to_global_id)
on your scope, this approach would instantiate each record just to do that. When you have many records, this will at the very least be slow.
Here is a method that does it for you efficiently. It respects Single Table Inheritance (STI).
Put it in your project's ApplicationRecord
to make it available on all models.
class ApplicationRecord
...
Workflow: How to use a key management service to encrypt passwords in the database
This is an extract from the linked article. It shows an approach on how to implement encrypted passwords with the AWS Key Management Service (KMS).
For most applications it's enough to use a hashed password with a salt (e.g. the gem devise defaults to this).
Upon password creation
-
Generate hash as hash of password + salt.
-
Encrypt the hash with a public key from KMS (you can store the public key in your server code).
-
In your database sto...
Carrierwave: How to migrate to another folder structure
A flat folder structure can be cool if you have only a few folders but can be painful for huge amounts. We recently had this issue in a project with more than 100.000 attachments, where we used a structure like this /attachments/123456789/file.pdf
.
Even the ls
command lasted several minutes to show us the content of the attachments folder.
So we decided to use a more hierarchical structure with a limited maximum of folder per layer. Here are a few tips how to migrate your files to their new...
How to fix: Rails query logs always show lib/active_record/log_subscriber.rb as source
Rails 5.2+ supports "verbose query logs" where it shows the source of a query in the application log.
Normally, it looks like this:
User Load (0.5ms) SELECT "users".* FROM "users" WHERE ...
↳ app/controllers/users_controller.rb:42:in `load_users'
However, you may encounter ActiveRecord's LogSubscriber as the source for all/most queries which is not helpful at all:
User Load (0.5ms) SELECT "users".* FROM "users" WHERE ...
↳ activerecord (6.0.3.3) lib/active_record/log_subscriber.rb:100:in `debug'
While th...
Service Worker series by GoMakeThings
Learn how to create offline applications with service workers.
- The amazing power of service workers
- Writing your first service worker with vanilla JS
- Saving recently viewed pages offline with service workers and vanilla JS
- Offline first with service workers and vanilla JS
- Improving web font performance with service workers
- How to set an expiration date for items in a service worker cache
- How to update a service worker
- How to trigger a service worker function from the front end with vanilla JS
- How to immediately ...
Controlling issue grouping in Sentry
When you use Sentry to monitor exceptions, an important feature is Sentry's error grouping mechanism. It will aggregate similar error "events" into one issue, so you can track and monitor it more easily. Grouping is especially important when you try to silence certain errors.
It is worth understanding how Sentry's grouping mechanism works.
The default grouping mechanism
The exact algorithm has changed over time, and Sentry will keep using the algorithm t...
Chrome Lighthouse
Chrome has a built-in utility to check performance and accessibility (and more) of your web app: Lighthouse.
Open the Developer Tools and go to the lighthouse tab:
Then you'll see some suggestions on how to improve your site.
This is cool, because you can even use it with non-public pages or your development environment (but be aware that some settings we're using for development, like not minifying JS and CSS files, might ruin your stats)...
Rails: How to get the ordered list of used middlewares
Rails middlewares are small code pieces that wrap requests to the application. The first middleware gets passed the request, invokes the next, and so on. Finally, the application is invoked, builds a response and passes it back to the last middleware. Each middleware now returns the response until the request is answered. Think of it like Russian Dolls, where each middleware is a doll and the application is the innermost item.
You can run rake middleware
to get the ordered list of used middlewares in a Rails application:
$> rake midd...
PostgreSQL: How to use with_advisory_lock to prevent race conditions
If you want to prevent that two processes run some code at the same time you can use the gem with_advisory_lock.
What happens
- The thread will wait indefinitely until the lock is acquired.
- While inside the block, you will exclusively own the advisory lock.
- The lock will be released after your block ends, even if an exception is raised in the block.
This is usually required if there is no suitable database row to lock on.
Example
You want to generate a...
How to cycle through grep results with vim
grep
is the go-to CLI tool to accomplish tasks like filtering large files for arbitrary keywords. When additional context is needed for search results, you might find yourself adding flags like -B5 -A10
to your query. Now, every search result covers 16 lines of your bash.
There is another way: You can easily pipe your search results to the VIM editor and cycle through them.
Example: Searching for local occurrences of "User"
vim -q <(grep -Hn -r "User" .)
# vim -q starts vim in the "quickfix" mode. See ":help quickfix"
# grep...
How to migrate CoffeeScript files from Sprockets to Webpack(er)
If you migrate a Rails application from Sprockets to Webpack(er), you can either transpile your CoffeeScript files to JavaScript or integrate a CoffeeScript compiler to your new process. This checklist can be used to achieve the latter.
- If you need to continue exposing your CoffeeScript classes to the global namespace, define them on
window
directly:
-class @User
+class window.User
- Replace Sprocket's
require
statement with Webpacker's...
Vortrag: Content Security Policy: Eine Einführung
Grundidee
CSP hat zum Ziel einen Browser-seitigen Mechanismus zu schaffen um einige Angriffe auf Webseiten zu verhindern, hauptsächlich XSS-Angriffe.
Einschub: Was ist XSS?
XSS = Cross Site Scripting. Passiert wenn ein User ungefiltertes HTML in die Webseite einfügen kann.
<div class="comment">
Danke für den interessanten Beitrag! <script>alert('you have been hacked')</script>
</div>
Rails löst das Problem weitgehend, aber
- Programmierfehler weiter möglich
- manchmal Sicherheitslücken in Gems oder Rails
Lösungsid...
Introducing GoodJob 1.0, a new Postgres-based, multithreaded, ActiveJob backend for Ruby on Rails
GoodJob is a new background worker gem. It's compatible with ActiveJob.
We're huge fans of Sidekiq for its stability and features. One advantage of GoodJob over Sidekiq is that GoodJob doesn't require Redis. So in cases where you don't have Redis or don't want to pay for a Redis HA quorum node, this might be an alternative worth checking out.
Automatically validating dependency licenses with License Finder
"Open-source software (OSS) is great. Anyone can use virtually any open-source code in their projects."
Well, it depends. Licenses can make things difficult, especially when you are developing closed-source software. Since some OSS licenses even require the employing application to be open-sourced as well (looking at you, GPL), you cannot use such software in a closed-source project.
To be sure on this, we have developed a project-level integration of Pivotal's excellent [license_finder](https:/...
ActiveRecord: String and text fields should always validate their length
If you have a :string
or :text
field, you should pair it with a model validation that restricts its length.
There are two motivations for this:
- In modern Rails, database types
:string
and:text
no longer have a relevant size limit. Without a validation a malicious user can quickly exhaust the hard drive of your database server. - In legacy Rails (or database schemas migrated from legacy Rails), database types
:string
and:text
had a database-side length constraint. When the user enters a longer string, the ActiveRecord valida...
FactoryBot: Traits for enums
FactoryBot allows to create traits from Enums since version 6.0.0
The automatic definition of traits for Active Record enum attributes is enabled by default, for non-Active Record enums you can use the traits_for_enum
method.
Example
factory :user do
traits_for_enum :role, %w[admin contact] # you can use User::ROLES here, of course
end
is equivalent to
factory :user do
trait :admin do
role { 'admin' }
end
trait :contact do
role { 'c...
PostgreSQL: Difference between text and varchar columns
TL;DR PostgreSQL handles Rails 4+ text
and string
columns the same. Some libraries may still reflect on the column type and e.g. render differently sized text fields.
PostgreSQL offers three character types for your columns:
-
character varying(n)
(also calledvarchar
or juststring
): Contents are limited to n characters, smaller contents are allowed. -
character(n)
: All contents are padded with spaces to allocate exactly n characters. -
text
: There is no upper or lower character limit (except for the absolute...
How to use Active Job to decouple your background processing from a gem
In a web application you sometimes have tasks that can not be processed during a request but need to go to the background.
There are several gems that help to you do that, like Sidekiq or Resque.
With newer Rails you can also use ActiveJob as interface for a background processing library. See here for a list of supported queueing adapters.
For ...
Howto: Select2 with AJAX
Select2 comes with AJAX support built in, using jQuery's AJAX methods.
...
For remote data sources only, Select2 does not create a new element until the item has been selected for the first time. This is done for performance reasons. Once an has been created, it will remain in the DOM even if the selection is later changed.
If you have a huge collection of records for your select2 input, you can populate it via AJAX in order to not pollute your HTML with lots of <option>
elements.
All you have to do is to provide...
The JavaScript Object Model: Prototypes and properties
Speaker today is Henning Koch, Head of Development at makandra.
This talk will be in German with English slides.
Introduction
As web developers we work with JavaScript every day, even when our backend code uses another language. While we've become quite adept with JavaScript at a basic level, I think many of us lack a deep understanding of the JavaScript object model and its capabilities.
Some of the questions we will answer in this talk:
- How does the
new
keyword construct an object? - What is the differen...