Object
Autotest continuously scans the files in your project for changes and runs the appropriate tests. Test failures are run until they have all passed. Then the full test suite is run to ensure that nothing else was inadvertantly broken.
If you want Autotest to start over from the top, hit ^C once. If you want Autotest to quit, hit ^C twice.
Rails:
The autotest command will automatically discover a Rails directory by looking for config/environment.rb. When Rails is discovered, autotest uses RailsAutotest to perform file mappings and other work. See RailsAutotest for details.
Plugins:
Plugins are available by creating a .autotest file either in your project root or in your home directory. You can then write event handlers in the form of:
Autotest.add_hook hook_name { |autotest| ... }
The available hooks are listed in ALL_HOOKS.
See example_dot_autotest.rb for more details.
If a hook returns a true value, it signals to autotest that the hook was handled and should not continue executing hooks.
Naming:
Autotest uses a simple naming scheme to figure out how to map implementation files to test files following the Test::Unit naming scheme.
Test files must be stored in test/
Test files names must start with test_
Test class names must start with Test
Implementation files must be stored in lib/
Implementation files must match up with a test file named
test_.*
Strategy:
Find all files and associate them from impl <-> test.
Run all tests.
Scan for failures.
Detect changes in ANY (ruby?. file, rerun all failures + changed files.
Until 0 defects, goto 3.
When 0 defects, goto 2.
Add a proc to the collection of discovery procs. See autodiscover.
# File lib/autotest.rb, line 174 174: def self.add_discovery &proc 175: @@discoveries << proc 176: end
Add the supplied block to the available hooks, with the given name.
# File lib/autotest.rb, line 844 844: def self.add_hook name, &block 845: HOOKS[name] << block 846: end
Automatically find all potential autotest runner styles by searching your loadpath, vendor/plugins, and rubygems for “autotest/discover.rb“. If found, that file is loaded and it should register discovery procs with autotest using add_discovery. That proc should return one or more strings describing the user’s current environment. Those styles are then combined to dynamically invoke an autotest plugin to suite your environment. That plugin should define a subclass of Autotest with a corresponding name.
All autotest/discover.rb files loaded.
Those procs determine your styles (eg [“rails”, “rspec”]).
Require file by sorting styles and joining (eg ‘autotest/rails_rspec’).
Invoke run method on appropriate class (eg Autotest::RailsRspec.run).
Autotest.add_discovery do "rails" if File.exist? 'config/environment.rb' end
# File lib/autotest.rb, line 203 203: def self.autodiscover 204: require 'rubygems' 205: 206: # *sigh* 207: # 208: # This is needed for rspec's hacky discovery mechanism. For some 209: # reason rspec2 added generators that create 210: # "autotest/discover.rb" right in the project directory instead of 211: # keeping it in the rspec gem and properly deciding that the 212: # project is an rspec based project or not. See the url for more 213: # details: 214: # 215: # http://rubyforge.org/tracker/?func=detail&atid=1678&aid=28775&group_id=419 216: # 217: # For the record, the sane way to do it is the bacon way: 218: # 219: # "Since version 1.0, there is autotest support. You need to tag 220: # your test directories (test/ or spec/) by creating an .bacon 221: # file there. Autotest then will find it." 222: # 223: # I'm submitting a counter-patch to rspec to fix stuff properly, 224: # but for now I'm stuck with this because their brokenness is 225: # documented in multiple books. 226: # 227: # I'm removing this code once a sane rspec goes out. 228: 229: hacky_discovery = Gem::Specification.any? { |s| s.name =~ /^rspec/ } 230: $: << '.' if hacky_discovery 231: 232: Gem.find_files("autotest/discover").each do |f| 233: load f 234: end 235: 236: # call all discovery procs and determine the style to use 237: @@discoveries.map{ |proc| proc.call }.flatten.compact.sort.uniq 238: end
Initialize the instance and then load the user’s .autotest file, if any.
# File lib/autotest.rb, line 274 274: def initialize 275: # these two are set directly because they're wrapped with 276: # add/remove/clear accessor methods 277: @exception_list = [] 278: @test_mappings = [] 279: @child = nil 280: 281: self.completed_re = 282: /\d+ tests, \d+ assertions, \d+ failures, \d+ errors(, \d+ skips)?/ 283: self.extra_class_map = {} 284: self.extra_files = [] 285: self.failed_results_re = /^\s+\d+\) (?:Failure|Error):\n(.*?)\((.*?)\)/ 286: self.files_to_test = new_hash_of_arrays 287: self.find_order = [] 288: self.known_files = nil 289: self.libs = ]. lib test].join(File::PATH_SEPARATOR) 290: self.order = :random 291: self.output = $stderr 292: self.prefix = nil 293: self.sleep = 1 294: self.testlib = "test/unit" 295: specified_directories = ARGV.reject { |arg| arg.start_with?("-") } # options are not directories 296: self.find_directories = specified_directories.empty? ? ['.'] : specified_directories 297: self.unit_diff = nil 298: self.latest_results = nil 299: 300: # file in /lib -> run test in /test 301: self.add_mapping(/^lib\/.*\.rb$/) do |filename, _| 302: possible = File.basename(filename).gsub '_', '_?' # ' stupid emacs 303: files_matching %^test/.*#{possible}$% 304: end 305: 306: # file in /test -> run it 307: self.add_mapping(/^test.*\/test_.*rb$/) do |filename, _| 308: filename 309: end 310: 311: default_configs = [File.expand_path('~/.autotest'), './.autotest'] 312: configs = options[:rc] || default_configs 313: 314: configs.each do |f| 315: load f if File.exist? f 316: end 317: end
# File lib/autotest.rb, line 69 69: def self.options 70: @@options ||= {} 71: end
# File lib/autotest.rb, line 85 85: def self.parse_options args = ARGV 86: require 'optparse' 87: options = { 88: :args => args.dup 89: } 90: 91: OptionParser.new do |opts| 92: opts.banner = Continuous testing for your ruby app. Autotest automatically tests code that has changed. It assumes the code is in lib, and tests are in tests. Autotest uses plugins to control what happens. You configure plugins with require statements in the .autotest file in your project base directory, and a default configuration for all your projects in the .autotest file in your home directory. Usage: autotest [options].gsub(/^ /, '') 93: 94: opts.on "-f", "--fast-start", "Do not run full tests at start" do 95: options[:no_full_after_start] = true 96: end 97: 98: opts.on("-c", "--no-full-after-failed", 99: "Do not run all tests on red->green") do 100: options[:no_full_after_failed] = true 101: end 102: 103: opts.on "-v", "--verbose", "Be annoyingly verbose (debugs .autotest)." do 104: options[:verbose] = true 105: end 106: 107: opts.on "-q", "--quiet", "Be quiet." do 108: options[:quiet] = true 109: end 110: 111: opts.on("-r", "--rc CONF", String, "Override path to config file") do |o| 112: options[:rc] = Array(o) 113: end 114: 115: opts.on("-s", "--style STYLE", String, 116: "Manually specify test style. (default: autodiscover)") do |style| 117: options[:style] = Array(style) 118: end 119: 120: opts.on("-w", "--warnings", "Turn on ruby warnings") do 121: $-w = true 122: end 123: 124: opts.on "-h", "--help", "Show this." do 125: puts opts 126: exit 1 127: end 128: end.parse! args 129: 130: Autotest.options.merge! options 131: 132: options 133: end
Initialize and run the system.
# File lib/autotest.rb, line 243 243: def self.run 244: new.run 245: end
Calculates the autotest runner to use to run the tests.
Can be overridden with —style, otherwise uses ::autodiscover.
# File lib/autotest.rb, line 152 152: def self.runner 153: style = options[:style] || Autotest.autodiscover 154: target = Autotest 155: 156: unless style.empty? then 157: mod = "autotest/#{style.join "_"}" 158: puts "loading #{mod}" 159: begin 160: require mod 161: rescue LoadError 162: abort "Autotest style #{mod} doesn't seem to exist. Aborting." 163: end 164: target = Autotest.const_get(style.map {|s| s.capitalize}.join) 165: end 166: 167: target 168: end
Adds regexp to the list of exceptions for find_file. This must be called before the exceptions are compiled.
# File lib/autotest.rb, line 770 770: def add_exception regexp 771: raise "exceptions already compiled" if defined? @exceptions 772: 773: @exception_list << regexp 774: nil 775: end
Adds a file mapping, optionally prepending the mapping to the front of the list if prepend is true. regexp should match a file path in the codebase. proc is passed a matched filename and Regexp.last_match. proc should return an array of tests to run.
For example, if test_helper.rb is modified, rerun all tests:
at.add_mapping(/test_helper.rb/) do |f, _| at.files_matching(/^test.*rb$/) end
# File lib/autotest.rb, line 734 734: def add_mapping regexp, prepend = false, &proc 735: if prepend then 736: @test_mappings.unshift [regexp, proc] 737: else 738: @test_mappings.push [regexp, proc] 739: end 740: nil 741: end
Installs a sigint handler.
# File lib/autotest.rb, line 415 415: def add_sigint_handler 416: trap 'INT' do 417: Process.kill "KILL", @child if @child 418: 419: if self.interrupted then 420: self.wants_to_quit = true 421: else 422: unless hook :interrupt then 423: puts "Interrupt a second time to quit" 424: self.interrupted = true 425: Kernel.sleep 1.5 426: end 427: raise Interrupt, nil # let the run loop catch it 428: end 429: end 430: end
Installs a sigquit handler
# File lib/autotest.rb, line 435 435: def add_sigquit_handler 436: trap 'QUIT' do 437: restart 438: end 439: end
If there are no files left to test (because they’ve all passed), then all is good.
# File lib/autotest.rb, line 462 462: def all_good 463: files_to_test.empty? 464: end
Clears the list of exceptions for find_file. This must be called before the exceptions are compiled.
# File lib/autotest.rb, line 791 791: def clear_exceptions 792: raise "exceptions already compiled" if defined? @exceptions 793: @exception_list.clear 794: nil 795: end
Clears all file mappings. This is DANGEROUS as it entirely disables autotest. You must add at least one file mapping that does a good job of rerunning appropriate tests.
# File lib/autotest.rb, line 758 758: def clear_mappings 759: @test_mappings.clear 760: nil 761: end
Returns a hash mapping a file name to the known failures for that file.
# File lib/autotest.rb, line 483 483: def consolidate_failures failed 484: filters = new_hash_of_arrays 485: 486: class_map = Hash[*self.find_order.grep(/^test/).map { |f| # TODO: ugly 487: [path_to_classname(f), f] 488: }.flatten] 489: class_map.merge! self.extra_class_map 490: 491: failed.each do |method, klass| 492: if class_map.has_key? klass then 493: filters[class_map[klass]] << method 494: else 495: output.puts "Unable to map class #{klass} to a file" 496: end 497: end 498: 499: filters 500: end
Return a compiled regexp of exceptions for find_files or nil if no filtering should take place. This regexp is generated from exception_list.
# File lib/autotest.rb, line 802 802: def exceptions 803: unless defined? @exceptions then 804: @exceptions = if @exception_list.empty? then 805: nil 806: else 807: Regexp.union(*@exception_list) 808: end 809: end 810: 811: @exceptions 812: end
Returns all known files in the codebase matching regexp.
# File lib/autotest.rb, line 718 718: def files_matching regexp 719: self.find_order.select { |k| k =~ regexp } 720: end
Find the files to process, ignoring temporary files, source configuration management files, etc., and return a Hash mapping filename to modification time.
# File lib/autotest.rb, line 507 507: def find_files 508: result = {} 509: targets = self.find_directories + self.extra_files 510: self.find_order.clear 511: 512: targets.each do |target| 513: order = [] 514: Find.find target do |f| 515: Find.prune if f =~ self.exceptions 516: Find.prune if f =~ /^\.\/tmp/ # temp dir, used by isolate 517: 518: next unless File.file? f 519: next if f =~ /(swp|~|rej|orig)$/ # temporary/patch files 520: next if f =~ /(,v)$/ # RCS files 521: next if f =~ /\/\.?#/ # Emacs autosave/cvs merge files 522: 523: filename = f.sub(/^\.\//, '') 524: 525: result[filename] = File.stat(filename).mtime rescue next 526: order << filename 527: end 528: self.find_order.push(*order.sort) 529: end 530: 531: result 532: end
Find the files which have been modified, update the recorded timestamps, and use this to update the files to test. Returns the latest mtime of the files modified or nil when nothing was modified.
# File lib/autotest.rb, line 540 540: def find_files_to_test files = find_files 541: updated = files.select { |filename, mtime| self.last_mtime < mtime } 542: 543: # nothing to update or initially run 544: unless updated.empty? || self.last_mtime.to_i == 0 then 545: p updated if options[:verbose] 546: 547: hook :updated, updated 548: end 549: 550: updated.map { |f,m| test_files_for f }.flatten.uniq.each do |filename| 551: self.files_to_test[filename] # creates key with default value 552: end 553: 554: if updated.empty? then 555: nil 556: else 557: files.values.max 558: end 559: end
Keep running the tests after a change, until all pass.
# File lib/autotest.rb, line 354 354: def get_to_green 355: begin 356: run_tests 357: wait_for_changes unless all_good 358: end until all_good 359: end
Check results for failures, set the “bar” to red or green, and if there are failures record this.
# File lib/autotest.rb, line 565 565: def handle_results results 566: results = results.gsub(/\e\[\d+m/, '') # strip ascii color 567: failed = results.scan self.failed_results_re 568: completed = results[self.completed_re] 569: 570: if completed then 571: completed = completed.scan(/(\d+) (\w+)/).map { |v, k| [k, v.to_i] } 572: 573: self.latest_results = Hash[*completed.flatten] 574: self.files_to_test = consolidate_failures failed 575: 576: color = self.files_to_test.empty? ? :green : :red 577: hook color unless $TESTING 578: else 579: self.latest_results = nil 580: end 581: 582: self.tainted = true unless self.files_to_test.empty? 583: end
Call the event hook named name, passing in optional args depending on the hook itself.
Returns false if no hook handled the event.
This executes all registered hooks until one returns truthy. Pay attention to the return value of your block!
# File lib/autotest.rb, line 828 828: def hook name, *args 829: deprecated = { 830: # none currently 831: } 832: 833: if deprecated[name] and not HOOKS[name].empty? then 834: warn "hook #{name} has been deprecated, use #{deprecated[name]}" 835: end 836: 837: HOOKS[name].any? { |plugin| plugin[self, *args] } 838: end
Lazy accessor for the known_files hash.
# File lib/autotest.rb, line 588 588: def known_files 589: unless @known_files then 590: @known_files = Hash[*find_order.map { |f| [f, true] }.flatten] 591: end 592: @known_files 593: end
Generate the commands to test the supplied files
# File lib/autotest.rb, line 605 605: def make_test_cmd files_to_test 606: cmds = [] 607: full, partial = reorder(files_to_test).partition { |k,v| v.empty? } 608: diff = self.unit_diff 609: diff = " | #{diff}" if diff and diff !~ /^\|/ 610: 611: unless full.empty? then 612: classes = full.map {|k,v| k}.flatten.uniq 613: classes.unshift testlib 614: classes = classes.join " " 615: cmds << "#{ruby_cmd} -e \"%w[#{classes}].each { |f| require f }\"#{diff}" 616: end 617: 618: partial.each do |klass, methods| 619: regexp = Regexp.union(*methods).source 620: cmds << "#{ruby_cmd} #{klass} -n \"/^(#{regexp})$/\"#{diff}" 621: end 622: 623: cmds.join "#{SEP} " 624: end
# File lib/autotest.rb, line 626 626: def new_hash_of_arrays 627: Hash.new { |h,k| h[k] = [] } 628: end
# File lib/autotest.rb, line 73 73: def options 74: self.class.options 75: end
Convert a path in a string, s, into a class name, changing underscores to CamelCase, etc.
# File lib/autotest.rb, line 470 470: def path_to_classname s 471: sep = File::SEPARATOR 472: f = s.sub(/^test#{sep}/, '').sub(/\.rb$/, '').split sep 473: f = f.map { |path| path.split(/_|(\d+)/).map { |seg| seg.capitalize }.join } 474: f = f.map { |path| path =~ /^Test/ ? path : "Test#{path}" } 475: 476: f.join '::' 477: end
Removes regexp to the list of exceptions for find_file. This must be called before the exceptions are compiled.
# File lib/autotest.rb, line 781 781: def remove_exception regexp 782: raise "exceptions already compiled" if defined? @exceptions 783: @exception_list.delete regexp 784: nil 785: end
Removed a file mapping matching regexp.
# File lib/autotest.rb, line 746 746: def remove_mapping regexp 747: @test_mappings.delete_if do |k,v| 748: k == regexp 749: end 750: nil 751: end
# File lib/autotest.rb, line 630 630: def reorder files_to_test 631: case self.order 632: when :alpha then 633: files_to_test.sort_by { |k,v| k } 634: when :reverse then 635: files_to_test.sort_by { |k,v| k }.reverse 636: when :random then 637: max = files_to_test.size 638: files_to_test.sort_by { |k,v| rand max } 639: when :natural then 640: (self.find_order & files_to_test.keys).map { |f| [f, files_to_test[f]] } 641: else 642: raise "unknown order type: #{self.order.inspect}" 643: end 644: end
Rerun the tests from cold (reset state)
# File lib/autotest.rb, line 649 649: def rerun_all_tests 650: reset 651: run_tests 652: 653: hook :all_good if all_good 654: end
Clear all state information about test failures and whether interrupts will kill autotest.
# File lib/autotest.rb, line 660 660: def reset 661: self.files_to_test.clear 662: self.find_order.clear 663: 664: self.interrupted = false 665: self.known_files = nil 666: self.last_mtime = T0 667: self.tainted = false 668: self.wants_to_quit = false 669: 670: hook :reset 671: end
# File lib/autotest.rb, line 441 441: def restart 442: Process.kill "KILL", @child if @child 443: 444: cmd = [$0, *options[:args]] 445: 446: index = $LOAD_PATH.index RbConfig::CONFIG["sitelibdir"] 447: 448: if index then 449: extra = $LOAD_PATH[0...index] 450: cmd = [Gem.ruby, "-I", extra.join(":")] + cmd 451: end 452: 453: puts cmd.join(" ") if options[:verbose] 454: 455: exec(*cmd) 456: end
Determine and return the path of the ruby executable.
# File lib/autotest.rb, line 676 676: def ruby 677: ruby = ENV['RUBY'] 678: ruby ||= File.join(RbConfig::CONFIG['bindir'], 679: RbConfig::CONFIG['ruby_install_name']) 680: 681: ruby.gsub! File::SEPARATOR, File::ALT_SEPARATOR if File::ALT_SEPARATOR 682: 683: return ruby 684: end
Returns the base of the ruby command.
# File lib/autotest.rb, line 598 598: def ruby_cmd 599: "#{prefix}#{ruby} -I#{libs} -rubygems" 600: end
Repeatedly run failed tests, then all tests, then wait for changes and carry on until killed.
# File lib/autotest.rb, line 323 323: def run 324: hook :initialize 325: hook :post_initialize 326: 327: reset 328: add_sigint_handler 329: 330: self.last_mtime = Time.now if options[:no_full_after_start] 331: 332: loop do 333: begin # ^c handler 334: get_to_green 335: if tainted? and not options[:no_full_after_failed] then 336: rerun_all_tests 337: else 338: hook :all_good 339: end 340: wait_for_changes 341: rescue Interrupt 342: break if wants_to_quit 343: reset 344: end 345: end 346: hook :quit 347: rescue Exception => err 348: hook(:died, err) or raise err 349: end
# File lib/autotest/preload.rb, line 29 29: def run_tests 30: hook :run_command 31: 32: new_mtime = self.find_files_to_test 33: return unless new_mtime 34: self.last_mtime = new_mtime 35: 36: begin 37: # TODO: deal with unit_diff and partial test runs later 38: original_argv = ARGV.dup 39: ARGV.clear 40: 41: @child = fork do 42: trap "QUIT", "DEFAULT" 43: trap "INT", "DEFAULT" 44: files_to_test.keys.each do |file| 45: load file 46: end 47: end 48: Process.wait 49: ensure 50: @child = nil 51: ARGV.replace original_argv 52: end 53: 54: hook :ran_command 55: end
Look for files to test then run the tests and handle the results.
# File lib/autotest.rb, line 364 364: def run_tests 365: new_mtime = self.find_files_to_test 366: return unless new_mtime 367: self.last_mtime = new_mtime 368: 369: cmd = self.make_test_cmd self.files_to_test 370: return if cmd.empty? 371: 372: hook :run_command, cmd 373: 374: puts cmd unless options[:quiet] 375: 376: old_sync = $stdout.sync 377: $stdout.sync = true 378: self.results = [] 379: line = [] 380: begin 381: open "| #{cmd}", "r" do |f| 382: until f.eof? do 383: c = f.getc or break 384: if RUBY19 then 385: print c 386: else 387: putc c 388: end 389: line << c 390: if c == \n\ then 391: self.results << if RUBY19 then 392: line.join 393: else 394: line.pack "c*" 395: end 396: line.clear 397: end 398: end 399: end 400: ensure 401: $stdout.sync = old_sync 402: end 403: hook :ran_command 404: self.results = self.results.join 405: 406: handle_results self.results 407: end
Return the name of the file with the tests for filename by finding a test_mapping that matches the file and executing the mapping’s proc.
# File lib/autotest.rb, line 691 691: def test_files_for filename 692: result = @test_mappings.find { |file_re, ignored| filename =~ file_re } 693: 694: p :test_file_for => [filename, result.first] if result and $DEBUG 695: 696: result = result.nil? ? [] : [result.last.call(filename, $~)].flatten 697: 698: output.puts "No tests matched #{filename}" if 699: (options[:verbose] or $TESTING) and result.empty? 700: 701: result.sort.uniq.select { |f| known_files[f] } 702: end
Disabled; run with --debug to generate this.
Generated with the Darkfish Rdoc Generator 1.1.6.