When your Rails application offers downloading a bunch of files as ZIP archive, you basically have two options:
This card is about option 2, and it is actually fairly easy to set up.
We are using this to generate ZIP archives with lots of files (500k+) on the fly, and it works like a charm.
Offering downloads of large archives can be cumbersome:
When you generate them on the fly, downloads start immediately, and you can add contents while the user is downloading.
The excellent
zip_tricks
Show archive.org snapshot
gem helps you out, and is not too difficult to integrate.
If you want to send only files from ActiveStorage or Carrierwave attachments, it's even simpler when using
zipline
Show archive.org snapshot
.
You want to use ZipTricks if you need some level of control, or if you are building some of the ZIP contents on the fly (like some CSV).
Add zip_tricks
to your Gemfile
and bundle install
.
In your controller, include ZipTricks::RailsStreaming
.
You can then use the zip_tricks_stream
method in controller actions to generated your contents, as described in the documentation.
Make sure you disable Rails middlewares that generate ETag headers for you (like Rails' Rack::ETag Show archive.org snapshot or Rack::SteadyETag Show archive.org snapshot ).
Last-Modified
or ETag
header to achieve that.Example controller:
class DocumentsController < ApplicationController
include ZipTricks::RailsStreaming
def download
documents = Document.all
fresh_when Time.current # Sets `Last-Modified` header (see above). Or say "fresh_when documents" to use the scope.
send_file_headers! filename: 'all_documents.zip' # Sets `Content-Disposition` and `Content-Type` headers.
zip_tricks_stream do |zip|
documents.find_each do |document|
filename = document.filename
path = document.file.path
zip.write_stored_file(filename) do |stream|
File.open(path, 'rb') do |source|
IO.copy_stream(source, stream)
end
end
end
end
end
end
What happens here is:
zip_tricks_stream
generates a buffer object (zip
) can be written to from inside the block.zip_tricks_stream
runs and writes to the buffer object.About compression:
write_stored_file
for files that are large or unlikely to compress significantly (like PNG, JPEG, MP4, ...)write_deflated_file
when adding files that compress well, like CSV, XML, or other text documents.Zipline uses ZipTricks under the hood and is specifically intended for streaming existing file attachments (from ActiveStorage, Carrierwave, etc.).
zipline
to your Gemfile
and bundle install
.include Zipline
.zipline
method in controller actions as described in the gem's documentation.Note that Zipline will set a Last-Modified
header already, disabling any etagging middlewares and allowing streaming.
Example controller:
class DocumentsController < ApplicationController
include Zipline
def download
documents = Document.all
files_and_filenames = documents.find_each.lazy.map do |document|
[document.file, document.filename]
end
zipline(files_and_filenames, 'all_documents.zip')
end
end
Take care to use lazy.map
instead of map
, or your controller action has to iterate the entire collection before it can start streaming.