Installing Atexit Handlers On Module Load Considered Harmful

A common idiom in Ruby testing frameworks is the use of at_exit as a way to schedule interesting work. I am not a fan — it is an idiom that is ripe for abuse. The C library’s atexit(3) function that inspired Ruby’s at_exit function was originally intended to allow the registration of handlers to tear down and clean up after a program during normal termination. It isn’t the right place for significant computation. Cleanup handlers don’t alter the result of the program, they just tidy up after it. Running the majority of the work of a script out of the atexit handler violates this principle.

As a concrete example, take the Ruby test/unit framework. When the test-unit module is loaded, it defines a variety of classes including test cases, test suites, and the runner class. The runner is responsible for coordinating the actual execution of the test suite. So far, so good. However, the last line of the module is a bare function call that ultimately installs an at_exit handler that invokes the runner an has it run all of the discovered test cases. The problem is that this takes the control of when the runner is invoked out of the hands of the user. If test/unit gets loaded as a side-effect somewhere deep in a dependent library, you are screwed. There is no simple way to disable just the one atexit handler and test/unit does not expose an API for disabling the runner. Thus, the obvious monkey patch is dependent on the version of test/unit that is installed which means you have to chase internal details of test/unit.

In practice, the result is that your test suite starts running even when it shouldn’t. For instance, we recently saw one customer that upgraded to Shoulda 3.0 and suddenly had the test suite run whenever he ran migrations! Others have had their test suites double on them.

To be fair, this is not a problem unique to test/unit, nor is it the fault of Shoulda. The specific case was easily resolved by adding the test-unit gem to the Gemfile or switching to just requiring shoulda-matchers rather than shoulda itself. Still, it shoulda never have happened and if test/unit didn’t install atexit handlers unbidden, it wouldn’t have.

The moral of the story: installing atexit handlers automatically leads to nasty surprises.

Post a Comment