Posted over 2 years ago. Visible to the public. Repeats.

Running external commands with Open3

There are various ways to run external commands from within Ruby, but the most powerful ones are Open3.capture3 and Open3.popen3. Since those can do almost everything you would possibly need in a clean way, I prefer to simply always use them.

Behind the scenes, Open3 actually just uses Ruby's spawn command, but gives you a much better API.

Open3.capture3

Basic usage is

Copy
require 'open3' stdout_str, error_str, status = Open3.capture3('/some/binary', 'with', 'some', 'args') if status.success? # okay else raise "did not work" end

Open3 will raise an error if the binary cannot run at all, but won't raise an error if the binary returns an exit code > 0. This will instead be reflected in status.

stdout_str, error_str are strings containing the command's output and error output.

Shell expansion

If you have to run the command through the user's shell, pass the command as a string instead of an array:

Copy
Open3.capture3('/some/binary with some args')

This is not recommended though, due to additional overhead and the potential for shell injections.

Stdin

If the command reads from stdin and you want to feed it some data, you can use

Copy
Open3.capture3('/some/binary', stdin_data: 'this is read from stdin')

ENV variables

To pass additional ENV variables to the command, pass them as a hash in the first argument:

Copy
Open3.capture3({'VAR' => 'value'}, '/some/binary')

You can use unsetenv_others: true to clear all other ENV variables.

Working directory

Run the command with a different working directory, by using chdir:

Copy
Open3.capture3('/some/binary', chdir: '/some/directory')

File descriptors

Ruby's spawn command gives you full control over file descriptors (and lets you point stdin to a file, or merge stdout and stderr) etc. Open3 however does not allow you to do this. If you need it, you have to use spawn directly.

Open3.popen3

This version allows your code to interact with the external command while it is running. Always use it in its block form:

Copy
Open3.popen3('/some/command') do |stdin, stdout, stderr, wait_thr| stdin.puts "This is sent to the command" stdin.close # we're done stdout_str = stdout.read # read stdout to string. note that this will block until the command is done! stderr_str = stderr.read # read stderr to string status = wait_thr.value # will block until the command finishes; returns status that responds to .success? etc end

This form is useful for long running commands you have to interact with, or if you want to pipe large amounts of data without keeping everything in memory.

However, it is tricky to use this correctly in all circumstances. If the command produces lots of output, for example, you need to make sure the stdout and stderr streams are continuously read, otherwise the command will block. You would need to use threads (or IO.select) to achieve this.

By refactoring problematic code and creating automated tests, makandra can vastly improve the maintainability of your Rails application.

Owner of this card:

Avatar
Tobias Kraze
Last edit:
over 2 years ago
by Henning Koch
Keywords:
ruby, shell, code
About this deck:
We are makandra and do test-driven, agile Ruby on Rails software development.
License for source code
Posted by Tobias Kraze to makandra dev
This website uses cookies to improve usability and analyze traffic.
Accept or learn more