Read more

Legacy CarrierWave: How to generate versions with different file extensions

Michael Leimstädtner
June 22, 2020Software engineer at makandra GmbH

Relevant CarrierWave's internas have changed in version 3. See this card for updated instructions:
https://makandracards.com/makandra/611988-how-to-upgrade-carrierwave-to-3-x

We use CarrierWave in many of our projects to store and serve files of various formats - mostly images. A common use case of CarrierWave's DSL is to "process" the original file in order to create multiple "versions", for example different resolutions of the same image.

Illustration money motivation

Opscomplete powered by makandra brand

Save money by migrating from AWS to our fully managed hosting in Germany.

  • Trusted by over 100 customers
  • Ready to use with Ruby, Node.js, PHP
  • Proactive management by operations experts
Read more Show archive.org snapshot

Now we could go one step further: What if we want to create versions that have a different file extension than the original file? For example, let's assume we'd like to create a version "mp4" that is always an MPEG-4 video, no matter if the original file was of the MOV or AVI format..

Forcing different formats is not supported "out of the box" by CarrierWave, but it takes only a few tweaks to your Uploader to make it happen.

Example Uploader

Given an original file is named "file.mp4" or "file.avi", this uploader will always create a version called "file_1080p.mp4".
Please note that you still have to change the contents of the file to match its extension. In my case the transcode_video processor is in charge of this (not part of the example).

The set_file_specs method is provided by the trait below. Use it in your own uploader to force a certain file name and extension for a given version.

Uploader

class VideoUploader < CarrierWave::Uploader::Base

  include DoesCarrierwaveFormatPatches

  version :mp4 do
    set_file_specs file_type: 'video/mp4', extension: :mp4, suffix: '1080p'
    process transcode_video: [format: :mp4, resolution: 1080]
  end

end

Trait

This trait uses our gem Modularity Show archive.org snapshot . Just include the code in your uploader if you don't want to use the gem.

module DoesCarrierwaveFormatPatches
  as_trait do
  
    after :store, :remove_stale_original
  
    def self.set_file_specs(file_type:, extension:, suffix: nil)
      define_method :content_type do
        file_type
      end
  
      # This filename is used when storing the version file to the public directory.
      define_method :full_filename do |*_args|
        suffix ||= version_name
        _full_filename(extension: extension, suffix: suffix)
      end
  
      # CarrierWave infers the cache filename for versions from
      # the filename of the *original* version (see retrieve_from_cache!).
      #
      # To ensure that the original filename does not collide with any version
      # filename, we have to make sure that each version uses a unique suffix.
      #
      # This filename is used when caching the version file to the uploads directory.
      define_method :full_original_filename do
        suffix ||= version_name
        _full_filename(extension: extension, suffix: suffix)
      end
    end
  
    # CarrierWave usually handles the removal of files that are no longer referenced.
    # Unfortunately their business logic conflicts with our excessive filename overwriting
    # and would cause all version files to be removed when the original file is updated.
    # We now do the cleanup ourselfs, see #remove_stale_original.
    def remove_previously_stored_files_after_update
      false
    end
  
    private
  
    # This filename is used when storing the original file to the public directory
    def full_filename(*)
      if original?
        extension = File.extname(super)
        _full_filename(extension: extension)
      else
        raise NotImplementedError, 'Please overwrite this method in each version to support the correct format and MIME type.'
      end
    end
  
    def _full_filename(extension:, suffix: nil)
      sanitized_extension = extension.to_s.downcase.delete('.')
      suffix = "_#{suffix}" if suffix
      basename = File.basename(filename, File.extname(filename))
  
      "#{basename}#{suffix}.#{sanitized_extension}"
    end
  
    # If a video clip is replaced with a new upload, CarrierWave would normally tidy up all
    # previously generated version files after save. However, this special uploader creates
    # versions with the same name and extension disregarding the input format. Hence, a new
    # upload overwrites all existing files, and there is nothing for CarrierWave to tidy up.
    #
    # *But* if the new upload is of a different file extension, the previous and new original
    # files differ (in their extension). CarrierWave attempts to tidy up previous versions,
    # but since the the previous versions were overwritten, it would delete the newly created
    # versions. So we disabled that functionality and tidy up the previous original ourselves.
    def remove_stale_original(*)
      return unless original?
  
      current_original = path
      stale_originals = Dir[path.chomp(File.extname(current_original)) + '.*'] # Same filename, any extension
      stale_originals.delete(current_original) do
        raise StandardError, 'Could not find the current original file, something is wrong!'
      end
  
      if stale_originals.one?
        file_to_remove = stale_originals.first
        log("Removing old original video version: #{file_to_remove}")
        FileUtils.rm(file_to_remove)
      elsif stale_originals.none?
        # Nothing to do, the original file extension stayed the same.
      else
        raise StandardError, "Did not expect to remove multiple files, aborting: #{stale_originals}"
      end
    end
  
    def original?
      version_name.blank?
    end

  end
end

A few notes

  • The version method content_type has to be overwritten for the file to be delivered with the correct headers.
  • The version method full_original_filename controls the filename inside the "cache_dir".
  • The version method full_filename controls the filename inside the "store_dir".
  • A nasty edge case is solved with remove_stale_original. It occurs for example if you replace an existing file "foo.avi" with "foo.mov".
Michael Leimstädtner
June 22, 2020Software engineer at makandra GmbH
Posted by Michael Leimstädtner to makandra dev (2020-06-22 11:32)