[前][次][番号順一覧][スレッド一覧]

ruby-changes:6606

From: drbrain <ko1@a...>
Date: Fri, 18 Jul 2008 09:46:49 +0900 (JST)
Subject: [ruby-changes:6606] Ruby:r18121 (trunk): Import RDoc r101.

drbrain	2008-07-18 09:46:16 +0900 (Fri, 18 Jul 2008)

  New Revision: 18121

  http://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=rev&revision=18121

  Log:
    Import RDoc r101.

  Added directories:
    trunk/lib/rdoc/generator/texinfo/
    trunk/lib/rdoc/parser/
  Added files:
    trunk/lib/rdoc/generator/html/frameless.rb
    trunk/lib/rdoc/generator/texinfo/class.texinfo.erb
    trunk/lib/rdoc/generator/texinfo/file.texinfo.erb
    trunk/lib/rdoc/generator/texinfo/method.texinfo.erb
    trunk/lib/rdoc/generator/texinfo/texinfo.erb
    trunk/lib/rdoc/generator/texinfo.rb
    trunk/lib/rdoc/known_classes.rb
    trunk/lib/rdoc/markup/to_texinfo.rb
    trunk/lib/rdoc/parser/c.rb
    trunk/lib/rdoc/parser/f95.rb
    trunk/lib/rdoc/parser/ruby.rb
    trunk/lib/rdoc/parser/simple.rb
    trunk/lib/rdoc/parser.rb
    trunk/test/rdoc/test_rdoc_info_formatting.rb
    trunk/test/rdoc/test_rdoc_info_sections.rb
    trunk/test/rdoc/test_rdoc_markup_to_html.rb
    trunk/test/rdoc/test_rdoc_markup_to_html_crossref.rb
    trunk/test/rdoc/test_rdoc_parser_c.rb
    trunk/test/rdoc/test_rdoc_parser_ruby.rb
    trunk/test/rdoc/test_rdoc_ri_driver.rb
  Removed directories:
    trunk/lib/rdoc/parsers/
  Removed files:
    trunk/test/rdoc/test_rdoc_c_parser.rb
  Modified files:
    trunk/ChangeLog
    trunk/lib/rdoc/code_objects.rb
    trunk/lib/rdoc/generator/html/hefss.rb
    trunk/lib/rdoc/generator/html/html.rb
    trunk/lib/rdoc/generator/html/kilmer.rb
    trunk/lib/rdoc/generator/html/one_page_html.rb
    trunk/lib/rdoc/generator/html.rb
    trunk/lib/rdoc/generator/ri.rb
    trunk/lib/rdoc/generator.rb
    trunk/lib/rdoc/markup/attribute_manager.rb
    trunk/lib/rdoc/markup/fragments.rb
    trunk/lib/rdoc/markup/preprocess.rb
    trunk/lib/rdoc/markup/to_html.rb
    trunk/lib/rdoc/markup/to_html_crossref.rb
    trunk/lib/rdoc/options.rb
    trunk/lib/rdoc/rdoc.rb
    trunk/lib/rdoc/ri/descriptions.rb
    trunk/lib/rdoc/ri/driver.rb
    trunk/lib/rdoc/ri.rb
    trunk/lib/rdoc.rb
    trunk/test/rdoc/test_rdoc_ri_default_display.rb

Index: ChangeLog
===================================================================
--- ChangeLog	(revision 18120)
+++ ChangeLog	(revision 18121)
@@ -1,3 +1,7 @@
+Fri Jul 18 09:44:30 2008
+
+	* lib/rdoc/*: Import RDoc r101.
+
 Thu Jul 17 23:45:55 2008  Yusuke Endoh  <mame@t...>
 
 	* test/rdoc/test_rdoc_c_parser.rb (teardown): close tempfile.
Index: lib/rdoc.rb
===================================================================
--- lib/rdoc.rb	(revision 18120)
+++ lib/rdoc.rb	(revision 18121)
@@ -1,8 +1,8 @@
 $DEBUG_RDOC = nil
 
 ##
-# = RDOC - Ruby Documentation System
-# 
+# RDoc - Ruby Documentation System
+#
 # This package contains RDoc and RDoc::Markup.  RDoc is an application that
 # produces documentation for one or more Ruby source files.  We work similarly
 # to JavaDoc, parsing the source, and extracting the definition for classes,
@@ -12,12 +12,12 @@
 # RDoc::Markup is a library that converts plain text into various output
 # formats.  The markup library is used to interpret the comment blocks that
 # RDoc uses to document methods, classes, and so on.
-# 
+#
 # == Roadmap
-# 
+#
 # * If you want to use RDoc to create documentation for your Ruby source files,
 #   read on.
-# * If you want to include extensions written in C, see RDoc::C_Parser
+# * If you want to include extensions written in C, see RDoc::Parser::C
 # * For information on the various markups available in comment blocks, see
 #   RDoc::Markup.
 # * If you want to drive RDoc programmatically, see RDoc::RDoc.
@@ -25,63 +25,63 @@
 #   at RDoc::Markup.
 # * If you want to try writing your own HTML output template, see
 #   RDoc::Generator::HTML
-# 
+#
 # == Summary
-# 
+#
 # Once installed, you can create documentation using the 'rdoc' command
 # (the command is 'rdoc.bat' under Windows)
-# 
+#
 #   % rdoc [options] [names...]
-# 
+#
 # Type "rdoc --help" for an up-to-date option summary.
-# 
+#
 # A typical use might be to generate documentation for a package of Ruby
-# source (such as rdoc itself). 
-# 
+# source (such as rdoc itself).
+#
 #   % rdoc
-# 
+#
 # This command generates documentation for all the Ruby and C source
 # files in and below the current directory.  These will be stored in a
 # documentation tree starting in the subdirectory 'doc'.
-# 
+#
 # You can make this slightly more useful for your readers by having the
 # index page contain the documentation for the primary file.  In our
 # case, we could type
-# 
+#
 #   % rdoc --main rdoc.rb
-# 
+#
 # You'll find information on the various formatting tricks you can use
 # in comment blocks in the documentation this generates.
-# 
+#
 # RDoc uses file extensions to determine how to process each file.  File names
 # ending +.rb+ and <tt>.rbw</tt> are assumed to be Ruby source.  Files
 # ending +.c+ are parsed as C files.  All other files are assumed to
 # contain just Markup-style markup (with or without leading '#' comment
 # markers).  If directory names are passed to RDoc, they are scanned
 # recursively for C and Ruby source files only.
-# 
+#
 # = Markup
-# 
+#
 # For information on how to make lists, hyperlinks, etc. with RDoc, see
 # RDoc::Markup.
-# 
+#
 # Comment blocks can be written fairly naturally, either using '#' on
 # successive lines of the comment, or by including the comment in
 # an =begin/=end block.  If you use the latter form, the =begin line must be
 # flagged with an RDoc tag:
-# 
+#
 #   =begin rdoc
 #   Documentation to be processed by RDoc.
 #   
 #   ...
 #   =end
-# 
+#
 # RDoc stops processing comments if it finds a comment line containing
 # a <tt>--</tt>.  This can be used to separate external from internal
 # comments, or to stop a comment being associated with a method, class, or
 # module.  Commenting can be turned back on with a line that starts with a
 # <tt>++</tt>.
-# 
+#
 #   ##
 #   # Extract the age and calculate the date-of-birth.
 #   #--
@@ -92,40 +92,40 @@
 #   def get_dob(person)
 #     # ...
 #   end
-# 
+#
 # Names of classes, source files, and any method names containing an
 # underscore or preceded by a hash character are automatically hyperlinked
-# from comment text to their description. 
-# 
+# from comment text to their description.
+#
 # Method parameter lists are extracted and displayed with the method
 # description.  If a method calls +yield+, then the parameters passed to yield
 # will also be displayed:
-# 
+#
 #   def fred
 #     ...
 #     yield line, address
-# 
+#
 # This will get documented as:
-# 
+#
 #   fred() { |line, address| ... }
-# 
+#
 # You can override this using a comment containing ':yields: ...' immediately
 # after the method definition
-# 
+#
 #   def fred # :yields: index, position
 #     # ...
 #   
 #     yield line, address
-# 
+#
 # which will get documented as
-# 
+#
 #    fred() { |index, position| ... }
-# 
+#
 # +:yields:+ is an example of a documentation directive.  These appear
 # immediately after the start of the document element they are modifying.
-# 
+#
 # == Directives
-# 
+#
 # [+:nodoc:+ / +:nodoc:+ all]
 #   Don't include this element in the documentation.  For classes
 #   and modules, the methods, aliases, constants, and attributes
@@ -143,27 +143,27 @@
 #       class Output
 #       end
 #     end
-#   
-#   In the above code, only class +MyModule::Input+ will be documented.
-#   :nodoc: is global across all files the class or module appears in, so use
-#   :stopdoc:/:startdoc: to only omit documentation for a particular set of
-#   methods, etc.
-# 
+#
+#   In the above code, only class +MyModule::Input+ will be documented.The
+#   The :nodoc: directive is global across all files the class or module
+#   appears in, so use :stopdoc:/:startdoc: to only omit documentation for a
+#   particular set of methods, etc.
+#
 # [+:doc:+]
 #   Force a method or attribute to be documented even if it wouldn't otherwise
 #   be.  Useful if, for example, you want to include documentation of a
 #   particular private method.
-# 
+#
 # [+:notnew:+]
 #   Only applicable to the +initialize+ instance method.  Normally RDoc
-#   assumes   that the documentation and parameters for #initialize are
+#   assumes that the documentation and parameters for #initialize are
 #   actually for the ::new method, and so fakes out a ::new for the class.
 #   The :notnew: modifier stops this.  Remember that #initialize is protected,
 #   so you won't see the documentation unless you use the -a command line
 #   option.
-# 
+#
 # Comment blocks can contain other directives:
-# 
+#
 # [<tt>:section: title</tt>]
 #   Starts a new section in the output.  The title following +:section:+ is
 #   used as the section heading, and the remainder of the comment containing
@@ -178,66 +178,66 @@
 #     # This is the section that I wrote.
 #     # See it glisten in the noon-day sun.
 #     # ----------------------------------------
-# 
+#
 # [+:call-seq:+]
 #   Lines up to the next blank line in the comment are treated as the method's
 #   calling sequence, overriding the default parsing of method parameters and
 #   yield arguments.
-# 
+#
 # [+:include:+ _filename_]
 #   \Include the contents of the named file at this point.  The file will be
 #   searched for in the directories listed by the +--include+ option, or in
 #   the current directory by default.  The contents of the file will be
-#   shifted to have the same indentation as the ':' at the start of the
-#   :include: directive.
-# 
+#   shifted to have the same indentation as the ':' at the start of
+#   the :include: directive.
+#
 # [+:title:+ _text_]
 #   Sets the title for the document.  Equivalent to the <tt>--title</tt>
 #   command line parameter.  (The command line parameter overrides any :title:
 #   directive in the source).
-# 
+#
 # [+:enddoc:+]
 #   Document nothing further at the current level.
-# 
+#
 # [+:main:+ _name_]
 #   Equivalent to the <tt>--main</tt> command line parameter.
-# 
+#
 # [+:stopdoc:+ / +:startdoc:+]
 #   Stop and start adding new documentation elements to the current container.
 #   For example, if a class has a number of constants that you don't want to
 #   document, put a +:stopdoc:+ before the first, and a +:startdoc:+ after the
 #   last.  If you don't specify a +:startdoc:+ by the end of the container,
 #   disables documentation for the entire class or module.
-# 
+#
 # = Other stuff
-# 
+#
 # RDoc is currently being maintained by Eric Hodel <drbrain@s...>
 #
 # Dave Thomas <dave@p...> is the original author of RDoc.
-# 
+#
 # == Credits
-# 
+#
 # * The Ruby parser in rdoc/parse.rb is based heavily on the outstanding
 #   work of Keiju ISHITSUKA of Nippon Rational Inc, who produced the Ruby
 #   parser for irb and the rtags package.
-# 
+#
 # * Code to diagram classes and modules was written by Sergey A Yanovitsky
 #   (Jah) of Enticla.
-# 
+#
 # * Charset patch from MoonWolf.
-# 
+#
 # * Rich Kilmer wrote the kilmer.rb output template.
-# 
+#
 # * Dan Brickley led the design of the RDF format.
-# 
+#
 # == License
-# 
+#
 # RDoc is Copyright (c) 2001-2003 Dave Thomas, The Pragmatic Programmers.  It
 # is free software, and may be redistributed under the terms specified
 # in the README file of the Ruby distribution.
-# 
+#
 # == Warranty
-# 
+#
 # This software is provided "as is" and without any express or implied
 # warranties, including, without limitation, the implied warranties of
 # merchantibility and fitness for a particular purpose.
@@ -254,7 +254,7 @@
   ##
   # RDoc version you are using
 
-  VERSION = "2.0.0"
+  VERSION = "2.1.0"
 
   ##
   # Name of the dotfile that contains the description of files to be processed
Index: lib/rdoc/code_objects.rb
===================================================================
--- lib/rdoc/code_objects.rb	(revision 18120)
+++ lib/rdoc/code_objects.rb	(revision 18121)
@@ -6,8 +6,8 @@
 module RDoc
 
   ##
-  # We contain the common stuff for contexts (which are containers)
-  # and other elements (methods, attributes and so on)
+  # We contain the common stuff for contexts (which are containers) and other
+  # elements (methods, attributes and so on)
 
   class CodeObject
 
@@ -31,6 +31,13 @@
 
     attr_reader :document_self
 
+    def initialize
+      @document_self = true
+      @document_children = true
+      @force_documentation = false
+      @done_documenting = false
+    end
+
     def document_self=(val)
       @document_self = val
       if !val
@@ -72,13 +79,6 @@
     def remove_methods_etc
     end
 
-    def initialize
-      @document_self = true
-      @document_children = true
-      @force_documentation = false
-      @done_documenting = false
-    end
-
     # Access the code object's comment
     attr_reader :comment
 
@@ -106,15 +106,24 @@
 
   end
 
-  # A Context is something that can hold modules, classes, methods, 
-  # attributes, aliases, requires, and includes. Classes, modules, and
-  # files are all Contexts.
+  ##
+  # A Context is something that can hold modules, classes, methods,
+  # attributes, aliases, requires, and includes. Classes, modules, and files
+  # are all Contexts.
 
   class Context < CodeObject
-    attr_reader   :name, :method_list, :attributes, :aliases, :constants
-    attr_reader   :requires, :includes, :in_files, :visibility
 
-    attr_reader   :sections
+    attr_reader :aliases
+    attr_reader :attributes
+    attr_reader :constants
+    attr_reader :current_section
+    attr_reader :in_files
+    attr_reader :includes
+    attr_reader :method_list
+    attr_reader :name
+    attr_reader :requires
+    attr_reader :sections
+    attr_reader :visibility
 
     class Section
       attr_reader :title, :comment, :sequence
@@ -129,12 +138,22 @@
         set_comment(comment)
       end
 
-      private
+      def ==(other)
+        self.class === other and @sequence == other.sequence
+      end
 
-      # Set the comment for this section from the original comment block
-      # If the first line contains :section:, strip it and use the rest. Otherwise
-      # remove lines up to the line containing :section:, and look for 
-      # those lines again at the end and remove them. This lets us write
+      def inspect
+        "#<%s:0x%x %s %p>" % [
+          self.class, object_id,
+          @sequence, title
+        ]
+      end
+
+      ##
+      # Set the comment for this section from the original comment block If
+      # the first line contains :section:, strip it and use the rest.
+      # Otherwise remove lines up to the line containing :section:, and look
+      # for those lines again at the end and remove them. This lets us write
       #
       #   # ---------------------
       #   # :SECTION: The title
@@ -144,9 +163,10 @@
       def set_comment(comment)
         return unless comment
 
-        if comment =~ /^.*?:section:.*$/
+        if comment =~ /^#[ \t]*:section:.*\n/
           start = $`
           rest = $'
+
           if start.empty?
             @comment = rest
           else
@@ -157,13 +177,13 @@
         end
         @comment = nil if @comment.empty?
       end
+
     end
 
-
     def initialize
-      super()
+      super
 
-      @in_files    = []
+      @in_files = []
 
       @name    ||= "unknown"
       @comment ||= ""
@@ -177,29 +197,37 @@
       initialize_classes_and_modules
     end
 
+    ##
     # map the class hash to an array externally
+
     def classes
       @classes.values
     end
 
+    ##
     # map the module hash to an array externally
+
     def modules
       @modules.values
     end
 
+    ##
     # Change the default visibility for new methods
+
     def ongoing_visibility=(vis)
       @visibility = vis
     end
 
-    # Given an array +methods+ of method names, set the
-    # visibility of the corresponding AnyMethod object
+    ##
+    # Yields Method and Attr entries matching the list of names in +methods+.
+    # Attributes are only returned when +singleton+ is false.
 
-    def set_visibility_for(methods, vis, singleton=false)
+    def methods_matching(methods, singleton = false)
       count = 0
+
       @method_list.each do |m|
-        if methods.include?(m.name) && m.singleton == singleton
-          m.visibility = vis
+        if methods.include? m.name and m.singleton == singleton then
+          yield m
           count += 1
         end
       end
@@ -209,14 +237,23 @@
       # perhaps we need to look at attributes
 
       @attributes.each do |a|
-        if methods.include?(a.name)
-          a.visibility = vis
-          count += 1
-        end
+        yield a if methods.include? a.name
       end
     end
 
+    ##
+    # Given an array +methods+ of method names, set the visibility of the
+    # corresponding AnyMethod object
+
+    def set_visibility_for(methods, vis, singleton = false)
+      methods_matching methods, singleton do |m|
+        m.visibility = vis
+      end
+    end
+
+    ##
     # Record the file that we happen to find it in
+
     def record_location(toplevel)
       @in_files << toplevel unless @in_files.include?(toplevel)
     end
@@ -269,10 +306,10 @@
 
     # Requires always get added to the top-level (file) context
     def add_require(a_require)
-      if self.kind_of? TopLevel
-        add_to(@requires, a_require)
+      if TopLevel === self then
+        add_to @requires, a_require
       else
-        parent.add_require(a_require)
+        parent.add_require a_require
       end
     end
 
@@ -292,7 +329,7 @@
     end
 
     def add_to(array, thing)
-      array <<  thing if @document_self  && !@done_documenting
+      array << thing if @document_self and not @done_documenting
       thing.parent = self
       thing.section = @current_section
     end
@@ -371,26 +408,30 @@
       name <=> other.name
     end
 
-    # Look up the given symbol. If method is non-nil, then
-    # we assume the symbol references a module that
-    # contains that method
-    def find_symbol(symbol, method=nil)
+    ##
+    # Look up +symbol+.  If +method+ is non-nil, then we assume the symbol
+    # references a module that contains that method.
+
+    def find_symbol(symbol, method = nil)
       result = nil
+
       case symbol
-      when /^::(.*)/
+      when /^::(.*)/ then
         result = toplevel.find_symbol($1)
-      when /::/
+      when /::/ then
         modules = symbol.split(/::/)
-        unless modules.empty?
+
+        unless modules.empty? then
           module_name = modules.shift
           result = find_module_named(module_name)
-          if result
+          if result then
             modules.each do |name|
               result = result.find_module_named(name)
               break unless result
             end
           end
         end
+
       else
         # if a method is specified, then we're definitely looking for
         # a module, otherwise it could be any symbol
@@ -408,22 +449,21 @@
           end
         end
       end
-      if result && method
-        if !result.respond_to?(:find_local_symbol)
-          #p result.name
-          #p method
-          fail
-        end
+
+      if result and method then
+        fail unless result.respond_to? :find_local_symbol
         result = result.find_local_symbol(method)
       end
+
       result
     end
-           
+
     def find_local_symbol(symbol)
       res = find_method_named(symbol) ||
             find_constant_named(symbol) ||
             find_attribute_named(symbol) ||
-            find_module_named(symbol) 
+            find_module_named(symbol) ||
+            find_file_named(symbol)
     end
 
     # Handle sections
@@ -454,7 +494,14 @@
     def find_attribute_named(name)
       @attributes.find {|m| m.name == name}
     end
-    
+
+    ##
+    # Find a named file, or return nil
+
+    def find_file_named(name)
+      toplevel.class.find_file_named(name)
+    end
+
   end
 
   ##
@@ -465,24 +512,31 @@
     attr_accessor :file_relative_name
     attr_accessor :file_absolute_name
     attr_accessor :diagram
-    
+
     @@all_classes = {}
     @@all_modules = {}
+    @@all_files   = {}
 
     def self.reset
       @@all_classes = {}
       @@all_modules = {}
+      @@all_files   = {}
     end
 
     def initialize(file_name)
       super()
       @name = "TopLevel"
-      @file_relative_name = file_name
-      @file_absolute_name = file_name
-      @file_stat          = File.stat(file_name)
-      @diagram            = nil
+      @file_relative_name    = file_name
+      @file_absolute_name    = file_name
+      @file_stat             = File.stat(file_name)
+      @diagram               = nil
+      @@all_files[file_name] = self
     end
 
+    def file_base_name
+      File.basename @file_absolute_name
+    end
+
     def full_name
       nil
     end
@@ -497,7 +551,7 @@
       cls = collection[name]
 
       if cls
-        puts "Reusing class/module #{name}" if $DEBUG_RDOC
+        puts "Reusing class/module #{name}" #if $DEBUG_RDOC
       else
         if class_type == NormalModule
           all = @@all_modules
@@ -534,6 +588,10 @@
       nil
     end
 
+    def self.find_file_named(name)
+      @@all_files[name]
+    end
+
     def find_local_symbol(symbol)
       find_class_or_module_named(symbol) || super
     end
@@ -553,8 +611,9 @@
 
   end
 
-  # ClassModule is the base class for objects representing either a
-  # class or a module.
+  ##
+  # ClassModule is the base class for objects representing either a class or a
+  # module.
 
   class ClassModule < Context
 
@@ -603,29 +662,63 @@
     end
   end
 
+  ##
   # Anonymous classes
+
   class AnonClass < ClassModule
   end
 
+  ##
   # Normal classes
+
   class NormalClass < ClassModule
+
+    def inspect
+      superclass = @superclass ? " < #{@superclass}" : nil
+      "<%s:0x%x class %s%s includes: %p attributes: %p methods: %p aliases: %p>" % [
+        self.class, object_id,
+        @name, superclass, @includes, @attributes, @method_list, @aliases
+      ]
+    end
+
   end
 
+  ##
   # Singleton classes
+
   class SingleClass < ClassModule
   end
 
+  ##
   # Module
+
   class NormalModule < ClassModule
+
+    def comment=(comment)
+      return if comment.empty?
+      comment = @comment << "# ---\n" << comment unless @comment.empty?
+
+      super
+    end
+
+    def inspect
+      "#<%s:0x%x module %s includes: %p attributes: %p methods: %p aliases: %p>" % [
+        self.class, object_id,
+        @name, @includes, @attributes, @method_list, @aliases
+      ]
+    end
+
     def is_module?
       true
     end
+
   end
 
   ##
   # AnyMethod is the base class for objects representing methods
 
   class AnyMethod < CodeObject
+
     attr_accessor :name
     attr_accessor :visibility
     attr_accessor :block_params
@@ -663,12 +756,22 @@
       @name <=> other.name
     end
 
-    def to_s
-      res = self.class.name + ": " + @name + " (" + @text + ")\n"
-      res << @comment.to_s
-      res
+    def add_alias(method)
+      @aliases << method
     end
 
+    def inspect
+      alias_for = @is_alias_for ? " (alias for #{@is_alias_for.name})" : nil
+      "#<%s:0x%x %s%s%s (%s)%s>" % [
+        self.class, object_id,
+        @parent.name,
+        singleton ? '::' : '#',
+        name,
+        visibility,
+        alias_for,
+      ]
+    end
+
     def param_seq
       p = params.gsub(/\s*\#.*/, '')
       p = p.tr("\n", " ").squeeze(" ")
@@ -691,16 +794,34 @@
       p
     end
 
-    def add_alias(method)
-      @aliases << method
+    def to_s
+      res = self.class.name + ": " + @name + " (" + @text + ")\n"
+      res << @comment.to_s
+      res
     end
+
   end
 
-  # Represent an alias, which is an old_name/ new_name pair associated
-  # with a particular context
+  ##
+  # GhostMethod represents a method referenced only by a comment
+
+  class GhostMethod < AnyMethod
+  end
+
+  ##
+  # MetaMethod represents a meta-programmed method
+
+  class MetaMethod < AnyMethod
+  end
+
+  ##
+  # Represent an alias, which is an old_name/ new_name pair associated with a
+  # particular context
+
   class Alias < CodeObject
+
     attr_accessor :text, :old_name, :new_name, :comment
-    
+
     def initialize(text, old_name, new_name, comment)
       super()
       @text = text
@@ -709,12 +830,22 @@
       self.comment = comment
     end
 
+    def inspect
+      "#<%s:0x%x %s.alias_method %s, %s>" % [
+        self.class, object_id,
+        parent.name, @old_name, @new_name,
+      ]
+    end
+
     def to_s
       "alias: #{self.old_name} ->  #{self.new_name}\n#{self.comment}"
     end
+
   end
 
+  ##
   # Represent a constant
+
   class Constant < CodeObject
     attr_accessor :name, :value
 
@@ -726,7 +857,9 @@
     end
   end
 
+  ##
   # Represent attributes
+
   class Attr < CodeObject
     attr_accessor :text, :name, :rw, :visibility
 
@@ -739,16 +872,33 @@
       self.comment = comment
     end
 
+    def <=>(other)
+      self.name <=> other.name
+    end
+
+    def inspect
+      attr = case rw
+             when 'RW' then :attr_accessor
+             when 'R'  then :attr_reader
+             when 'W'  then :attr_writer
+             else
+               " (#{rw})"
+             end
+
+      "#<%s:0x%x %s.%s :%s>" % [
+        self.class, object_id,
+        @parent.name, attr, @name,
+      ]
+    end
+
     def to_s
       "attr: #{self.name} #{self.rw}\n#{self.comment}"
     end
 
-    def <=>(other)
-      self.name <=> other.name
-    end
   end
 
-  # a required file
+  ##
+  # A required file
 
   class Require < CodeObject
     attr_accessor :name
@@ -759,18 +909,40 @@
       self.comment = comment
     end
 
+    def inspect
+      "#<%s:0x%x require '%s' in %s>" % [
+        self.class,
+        object_id,
+        @name,
+        @parent.file_base_name,
+      ]
+    end
+
   end
 
-  # an included module
+  ##
+  # An included module
+
   class Include < CodeObject
+
     attr_accessor :name
 
     def initialize(name, comment)
       super()
       @name = name
       self.comment = comment
+
     end
 
+    def inspect
+      "#<%s:0x%x %s.include %s>" % [
+        self.class,
+        object_id,
+        @parent.name,
+        @name,
+      ]
+    end
+
   end
 
 end
Index: lib/rdoc/generator/html/frameless.rb
===================================================================
--- lib/rdoc/generator/html/frameless.rb	(revision 0)
+++ lib/rdoc/generator/html/frameless.rb	(revision 18121)
@@ -0,0 +1,795 @@
+require 'rdoc/generator/html'
+require 'rdoc/generator/html/one_page_html'
+
+##
+# = CSS2 RDoc HTML template
+#
+# This is a template for RDoc that uses XHTML 1.0 Transitional and dictates a
+# bit more of the appearance of the output to cascading stylesheets than the
+# default. It was designed for clean inline code display, and uses DHTMl to
+# toggle the visbility of each method's source with each click on the '[source]'
+# link.
+#
+# == Authors
+#
+# * Michael Granger <ged@F...>
+#
+# Copyright (c) 2002, 2003 The FaerieMUD Consortium. Some rights reserved.
+#
+# This work is licensed under the Creative Commons Attribution License. To view
+# a copy of this license, visit http://creativecommons.org/licenses/by/1.0/ or
+# send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California
+# 94305, USA.
+
+module RDoc::Generator::HTML::FRAMELESS
+
+  FRAMELESS = true
+
+  FONTS = "Verdana,Arial,Helvetica,sans-serif"
+
+  STYLE = <<-EOF
+body {
+  font-family: #{FONTS};
+  font-size: 90%;
+  margin: 0;
+  margin-left: 40px;
+  padding: 0;
+  background: white;
+}
+
+h1, h2, h3, h4 {
+  margin: 0;
+  color: #efefef;
+  background: transparent;
+}
+
+h1 {
+  font-size: 150%;
+}
+
+h2,h3,h4 {
+  margin-top: 1em;
+}
+
+:link, :visited {
+  background: #eef;
+  color: #039;
+  text-decoration: none;
+}
+
+:link:hover, :visited:hover {
+  background: #039;
+  color: #eef;
+}
+
+/* Override the base stylesheet's Anchor inside a table cell */
+td > :link, td > :visited {
+  background: transparent;
+  color: #039;
+  text-decoration: none;
+}
+
+/* and inside a section title */
+.section-title > :link, .section-title > :visited {
+  background: transparent;
+  color: #eee;
+  text-decoration: none;
+}
+
+/* === Structural elements =================================== */
+
+.index {
+  margin: 0;
+  margin-left: -40px;
+  padding: 0;
+  font-size: 90%;
+}
+
+.index :link, .index :visited {
+  margin-left: 0.7em;
+}
+
+.index .section-bar {
+  margin-left: 0px;
+  padding-left: 0.7em;
+  background: #ccc;
+  font-size: small;
+}
+
+#classHeader, #fileHeader {
+  width: auto;
+  color: white;
+  padding: 0.5em 1.5em 0.5em 1.5em;
+  margin: 0;
+  margin-left: -40px;
+  border-bottom: 3px solid #006;
+}
+
+#classHeader :link, #fileHeader :link,
+#classHeader :visited, #fileHeader :visited {
+  background: inherit;
+  color: white;
+}
+
+#classHeader td, #fileHeader td {
+  background: inherit;
+  color: white;
+}
+
+#fileHeader {
+  background: #057;
+}
+
+#classHeader {
+  background: #048;
+}
+
+.class-name-in-header {
+  font-size:  180%;
+  font-weight: bold;
+}
+
+#bodyContent {
+  padding: 0 1.5em 0 1.5em;
+}
+
+#description {
+  padding: 0.5em 1.5em;
+  background: #efefef;
+  border: 1px dotted #999;
+}
+
+#description h1, #description h2, #description h3,
+#description h4, #description h5, #description h6 {
+  color: #125;
+  background: transparent;
+}
+
+#copyright {
+  color: #333;
+  background: #efefef;
+  font: 0.75em sans-serif;
+  margin-top: 5em;
+  margin-bottom: 0;
+  padding: 0.5em 2em;
+}
+
+/* === Classes =================================== */
+
+table.header-table {
+  color: white;
+  font-size: small;
+}
+
+.type-note {
+  font-size: small;
+  color: #dedede;
+}
+
+.xxsection-bar {
+  background: #eee;
+  color: #333;
+  padding: 3px;
+}
+
+.section-bar {
+  color: #333;
+  border-bottom: 1px solid #999;
+  margin-left: -20px;
+}
+
+.section-title {
+  background: #79a;
+  color: #eee;
+  padding: 3px;
+  margin-top: 2em;
+  margin-left: -30px;
+  border: 1px solid #999;
+}
+
+.top-aligned-row {
+  vertical-align: top
+}
+
+.bottom-aligned-row {
+  vertical-align: bottom
+}
+
+/* --- Context section classes ----------------------- */
+
+.context-row { }
+
+.context-item-name {
+  font-family: monospace;
+  font-weight: bold;
+  color: black;
+}
+
+.context-item-value {
+  font-size: small;
+  color: #448;
+}
+
+.context-item-desc {
+  color: #333;
+  padding-left: 2em;
+}
+
+/* --- Method classes -------------------------- */
+
+.method-detail {
+  background: #efefef;
+  padding: 0;
+  margin-top: 0.5em;
+  margin-bottom: 1em;
+  border: 1px dotted #ccc;
+}
+
+.method-heading {
+  color: black;
+  background: #ccc;
+  border-bottom: 1px solid #666;
+  padding: 0.2em 0.5em 0 0.5em;
+}
+
+.method-signature {
+  color: black;
+  background: inherit;
+}
+
+.method-name {
+  font-weight: bold;
+}
+
+.method-args {
+  font-style: italic;
+}
+
+.method-description {
+  padding: 0 0.5em 0 0.5em;
+}
+
+/* --- Source code sections -------------------- */
+
+:link.source-toggle, :visited.source-toggle {
+  font-size: 90%;
+}
+
+div.method-source-code {
+  background: #262626;
+  color: #ffdead;
+  margin: 1em;
+  padding: 0.5em;
+  border: 1px dashed #999;
+  overflow: hidden;
+}
+
+div.method-source-code pre {
+  color: #ffdead;
+  overflow: hidden;
+}
+
+/* --- Ruby keyword styles --------------------- */
+
+.standalone-code {
+  background: #221111;
+  color: #ffdead;
+  overflow: hidden;
+}
+
+.ruby-constant {
+  color: #7fffd4;
+  background: transparent;
+}
+
+.ruby-keyword {
+  color: #00ffff;
+  background: transparent;
+}
+
+.ruby-ivar {
+  color: #eedd82;
+  background: transparent;
+}
+
+.ruby-operator {
+  color: #00ffee;
+  background: transparent;
+}
+
+.ruby-identifier {
+  color: #ffdead;
+  background: transparent;
+}
+
+.ruby-node {
+  color: #ffa07a;
+  background: transparent;
+}
+
+.ruby-comment {
+  color: #b22222;
+  font-weight: bold;
+  background: transparent;
+}
+
+.ruby-regexp {
+  color: #ffa07a;
+  background: transparent;
+}
+
+.ruby-value {
+  color: #7fffd4;
+  background: transparent;
+}
+
+EOF
+
+  ##
+  # Header template
+
+  XHTML_PREAMBLE = <<-EOF
+<?xml version="1.0" encoding="<%= values["charset"] %>"?>
+<!DOCTYPE html
+     PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+     "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+  EOF
+
+  HEADER = XHTML_PREAMBLE + <<-EOF
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head>
+  <title><%= values["title"] %></title>
+  <meta http-equiv="Content-Type" content="text/html; charset=<%= values["charset"] %>" />
+  <meta http-equiv="Content-Script-Type" content="text/javascript" />
+  <link rel="stylesheet" href="<%= values["style_url"] %>" type="text/css" media="screen" />
+  <script type="text/javascript">
+  // <![CDATA[
+
+  function popupCode( url ) {
+    window.open(url, "Code", "resizable=yes,scrollbars=yes,toolbar=no,status=no,height=150,width=400")
+  }
+
+  function toggleCode( id ) {
+    if ( document.getElementById )
+      elem = document.getElementById( id );
+    else if ( document.all )
+      elem = eval( "document.all." + id );
+    else
+      return false;
+
+    elemStyle = elem.style;
+
+    if ( elemStyle.display != "block" ) {
+      elemStyle.display = "block"
+    } else {
+      elemStyle.display = "none"
+    }
+
+    return true;
+  }
+
+  // Make codeblocks hidden by default
+  document.writeln( "<style type=\\"text/css\\">div.method-source-code { display: none }</style>" )
+
+  // ]]>
+  </script>
+
+</head>
+<body>
+EOF
+
+  ##
+  # Context content template
+
+  CONTEXT_CONTENT = %{
+}
+
+  ##
+  # Footer template
+
+  FOOTER = <<-EOF
+  <div id="popupmenu" class="index">
+    <ul>
+    <li class="index-entries section-bar">Classes
+      <ul>
+<% values["class_list"].each do |klass| %>
+        <li><a href="<%= klass["href"] %>"><%= klass["name"] %></a>
+<% end %>
+      </ul>
+    </li>
+
+    <li class="index-entries section-bar">Methods
+      <ul>
+<% values["method_list"].each do |file| %>
+        <li><a href="<%= file["href"] %>"><%= file["name"] %></a>
+<% end %>
+      </ul>
+    </li>
+
+    <li class="index-entries section-bar">Files
+      <ul>
+<% values["file_list"].each do |file| %>
+        <li><a href="<%= file["href"] %>"><%= file["name"] %></a>
+<% end %>
+      </ul>
+    </li>
+    </ul>
+  </li>
+
+</body>
+</html>
+  EOF
+
+  ##
+  # File page header template
+
+  FILE_PAGE = <<-EOF
+  <div id="fileHeader">
+    <h1><%= values["short_name"] %></h1>
+
+    <table class="header-table">
+    <tr class="top-aligned-row">
+      <td><strong>Path:</strong></td>
+      <td><%= values["full_path"] %>
+<% if values["cvsurl"] then %>
+        &nbsp;(<a href="<%= values["cvsurl"] %>"><acronym title="Concurrent Versioning System">CVS</acronym></a>)
+<% end %>
+      </td>
+    </tr>
+
+    <tr class="top-aligned-row">
+      <td><strong>Last Update:</strong></td>
+      <td><%= values["dtm_modified"] %></td>
+    </tr>
+    </table>
+  </div>
+  EOF
+
+  ##
+  # Class page header template
+
+  CLASS_PAGE = <<-EOF
+    <div id="classHeader">
+      <table class="header-table">
+      <tr class="top-aligned-row">
+        <td><strong><%= values["classmod"] %></strong></td>
+        <td class="class-name-in-header"><%= values["full_name"] %></td>
+      </tr>
+
+      <tr class="top-aligned-row">
+        <td><strong>In:</strong></td>
+        <td>
+<% values["infiles"].each do |infiles| %>
+<% if infiles["full_path_url"] then %>
+          <a href="<%= infiles["full_path_url"] %>">
+<% end %>
+            <%= infiles["full_path"] %>
+<% if infiles["full_path_url"] then %>
+          </a>
+<% end %>
+<% if infiles["cvsurl"] then %>
+          &nbsp;(<a href="<%= infiles["cvsurl"] %>"><acronym title="Concurrent Versioning System">CVS</acronym></a>)
+<% end %>
+          <br />
+<% end %><%# values["infiles"] %>
+        </td>
+      </tr>
+
+<% if values["parent"] then %>
+      <tr class="top-aligned-row">
+        <td><strong>Parent:</strong></td>
+        <td>
+<% if values["par_url"] then %>
+          <a href="<%= values["par_url"] %>">
+<% end %>
+            <%= values["parent"] %>
+<% if values["par_url"] then %>
+          </a>
+<% end %>
+        </td>
+      </tr>
+<% end %>
+    </table>
+  </div>
+  EOF
+
+  ##
+  # Method list template
+
+  METHOD_LIST = <<-EOF
+
+  <div id="contextContent">
+<% if values["diagram"] then %>
+    <div id="diagram">
+      <%= values["diagram"] %>
+    </div>
+<% end %>
+
+<% if values["description"] then %>
+    <div id="description">
+      <%= values["description"] %>
+    </div>
+<% end %>
+
+<% if values["requires"] then %>
+    <div id="requires-list">
+      <h3 class="section-bar">Required files</h3>
+
+      <div class="name-list">
+<% values["requires"].each do |requires| %>
+        <%= href requires["aref"], requires["name"] %>&nbsp;&nbsp;
+<% end %><%# values["requires"] %>
+      </div>
+    </div>
+<% end %>
+
+<% if values["toc"] then %>
+    <div id="contents-list">
+      <h3 class="section-bar">Contents</h3>
+      <ul>
+<% values["toc"].each do |toc| %>
+      <li><a href="#<%= values["href"] %>"><%= values["secname"] %></a></li>
+<% end %><%# values["toc"] %>
+     </ul>
+<% end %>
+   </div>
+
+<% if values["methods"] then %>
+    <div id="method-list">
+      <h3 class="section-bar">Methods</h3>
+
+      <div class="name-list">
+<% values["methods"].each do |methods| %>
+        <%= href methods["aref"], methods["name"] %>&nbsp;&nbsp;
+<% end %><%# values["methods"] %>
+      </div>
+    </div>
+<% end %>
+
+  </div>
+
+
+    <!-- if includes -->
+<% if values["includes"] then %>
+    <div id="includes">
+      <h3 class="section-bar">Included Modules</h3>
+
+      <div id="includes-list">
+<% values["includes"].each do |includes| %>
+        <span class="include-name"><%= href includes["aref"], includes["name"] %></span>
+<% end %><%# values["includes"] %>
+      </div>
+    </div>
+<% end %>
+
+<% values["sections"].each do |sections| %>
+    <div id="section">
+<% if sections["sectitle"] then %>
+      <h2 class="section-title"><a name="<%= sections["secsequence"] %>"><%= sections["sectitle"] %></a></h2>
+<% if sections["seccomment"] then %>
+      <div class="section-comment">
+        <%= sections["seccomment"] %>
+      </div>
+<% end %>
+<% end %>
+
+<% if values["classlist"] then %>
+    <div id="class-list">
+      <h3 class="section-bar">Classes and Modules</h3>
+
+      <%= values["classlist"] %>
+    </div>
+<% end %>
+
+<% if values["constants"] then %>
+    <div id="constants-list">
+      <h3 class="section-bar">Constants</h3>
+
+      <div class="name-list">
+        <table summary="Constants">
+<% values["constants"].each do |constants| %>
+        <tr class="top-aligned-row context-row">
+          <td class="context-item-name"><%= constants["name"] %></td>
+          <td>=</td>
+          <td class="context-item-value"><%= constants["value"] %></td>
+<% if values["desc"] then %>
+          <td width="3em">&nbsp;</td>
+          <td class="context-item-desc"><%= constants["desc"] %></td>
+<% end %>
+        </tr>
+<% end %><%# values["constants"] %>
+        </table>
+      </div>
+    </div>
+<% end %>
+
+<% if values["aliases"] then %>
+    <div id="aliases-list">
+      <h3 class="section-bar">External Aliases</h3>
+
+      <div class="name-list">
+                        <table summary="aliases">
+<% values["aliases"].each do |aliases| $stderr.puts({ :aliases => aliases }.inspect) %>
+        <tr class="top-aligned-row context-row">
+          <td class="context-item-name"><%= values["old_name"] %></td>
+          <td>-&gt;</td>
+          <td class="context-item-value"><%= values["new_name"] %></td>
+        </tr>
+<% if values["desc"] then %>
+      <tr class="top-aligned-row context-row">
+        <td>&nbsp;</td>
+        <td colspan="2" class="context-item-desc"><%= values["desc"] %></td>
+      </tr>
+<% end %>
+<% end %><%# values["aliases"] %>
+        </table>
+      </div>
+    </div>
+<% end %>
+
+
+<% if values["attributes"] then %>
+    <div id="attribute-list">
+      <h3 class="section-bar">Attributes</h3>
+
+      <div class="name-list">
+        <table>
+<% values["attributes"].each do |attributes| $stderr.puts({ :attributes => attributes }.inspect) %>
+        <tr class="top-aligned-row context-row">
+          <td class="context-item-name"><%= values["name"] %></td>
+<% if values["rw"] then %>
+          <td class="context-item-value">&nbsp;[<%= values["rw"] %>]&nbsp;</td>
+<% end %>
+<% unless values["rw"] then %>
+          <td class="context-item-value">&nbsp;&nbsp;</td>
+<% end %>
+          <td class="context-item-desc"><%= values["a_desc"] %></td>
+        </tr>
+<% end %><%# values["attributes"] %>
+        </table>
+      </div>
+    </div>
+<% end %>
+
+    <!-- if method_list -->
+<% if sections["method_list"] then %>
+    <div id="methods">
+<% sections["method_list"].each do |method_list| %>
+<% if method_list["methods"] then %>
+      <h3 class="section-bar"><%= method_list["type"] %> <%= method_list["category"] %> methods</h3>
+
+<% method_list["methods"].each do |methods| %>
+      <div id="method-<%= methods["aref"] %>" class="method-detail">
+        <a name="<%= methods["aref"] %>"></a>
+
+        <div class="method-heading">
+<% if methods["codeurl"] then %>
+          <a href="<%= methods["codeurl"] %>" target="Code" class="method-signature"
+            onclick="popupCode('<%= methods["codeurl"] %>');return false;">
+<% end %>
+<% if methods["sourcecode"] then %>
+          <a href="#<%= methods["aref"] %>" class="method-signature">
+<% end %>
+<% if methods["callseq"] then %>
+          <span class="method-name"><%= methods["callseq"] %></span>
+<% end %>
+<% unless methods["callseq"] then %>
+          <span class="method-name"><%= methods["name"] %></span><span class="method-args"><%= methods["params"] %></span>
+<% end %>
+<% if methods["codeurl"] then %>
+          </a>
+<% end %>
+<% if methods["sourcecode"] then %>
+          </a>
+<% end %>
+        </div>
+
+        <div class="method-description">
+<% if methods["m_desc"] then %>
+          <%= methods["m_desc"] %>
+<% end %>
+<% if methods["sourcecode"] then %>
+          <p><a class="source-toggle" href="#"
+            onclick="toggleCode('<%= methods["aref"] %>-source');return false;">[Source]</a></p>
+          <div class="method-source-code" id="<%= methods["aref"] %>-source">
+<pre>
+<%= methods["sourcecode"] %>
+</pre>
+          </div>
+<% end %>
+        </div>
+      </div>
+
+<% end %><%# method_list["methods"] %>
+<% end %>
+<% end %><%# sections["method_list"] %>
+
+    </div>
+<% end %>
+<% end %><%# values["sections"] %>
+  EOF
+
+  ##
+  # Body template
+
+  BODY = HEADER + %{
+
+<%= template_include %>  <!-- banner header -->
+
+  <div id="bodyContent">
+
+} +  METHOD_LIST + %{
+
+  </div>
+
+} + FOOTER
+
+  ##
+  # Source code template
+
+  SRC_PAGE = XHTML_PREAMBLE + <<-EOF
+<html>
+<head>
+  <title><%= values["title"] %></title>
+  <meta http-equiv="Content-Type" content="text/html; charset=<%= values["charset"] %>" />
+  <link rel="stylesheet" href="<%= values["style_url"] %>" type="text/css" media="screen" />
+</head>
+<body class="standalone-code">
+  <pre><%= values["code"] %></pre>
+</body>
+</html>
+  EOF
+
+  ##
+  # Index file templates
+
+  FR_INDEX_BODY = %{
+<%= template_include %>
+}
+
+  FILE_INDEX = XHTML_PREAMBLE + <<-EOF
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head>
+  <title><%= values["list_title"] %></title>
+  <meta http-equiv="Content-Type" content="text/html; charset=<%= values["charset"] %>" />
+  <link rel="stylesheet" href="<%= values["style_url"] %>" type="text/css" />
+  <base target="docwin" />
+</head>
+<body>
+<div class="index">
+  <h1 class="section-bar"><%= values["list_title"] %></h1>
+  <div class="index-entries">
+<% values["entries"].each do |entries| %>
+    <a href="<%= entries["href"] %>"><%= entries["name"] %></a><br />
+<% end %><%# values["entries"] %>
+  </div>
+</div>
+</body>
+</html>
+  EOF
+
+  CLASS_INDEX = FILE_INDEX
+  METHOD_INDEX = FILE_INDEX
+
+  INDEX = <<-EOF
+<?xml version="1.0" encoding="<%= values["charset"] %>"?>
+<!DOCTYPE html
+     PUBLIC "-//W3C//DTD XHTML 1.0 Frameset//EN"
+     "http://www.w3.org/TR/xhtml1/DTD/xhtml1-frameset.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head>
+  <title><%= values["title"] %></title>
+  <meta http-equiv="Content-Type" content="text/html; charset=<%= values["charset"] %>" />
+</head>
+<frameset rows="20%, 80%">
+  <frameset cols="45%,55%">
+    <frame src="fr_class_index.html"  name="Classes" />
+    <frame src="fr_method_index.html" name="Methods" />
+  </frameset>
+  <frame src="<%= values["initial_page"] %>" name="docwin" />
+</frameset>
+</html>
+  EOF
+
+end
+
Index: lib/rdoc/generator/html/one_page_html.rb
===================================================================
--- lib/rdoc/generator/html/one_page_html.rb	(revision 18120)
+++ lib/rdoc/generator/html/one_page_html.rb	(revision 18121)
@@ -17,7 +17,7 @@
 <% unless requires["aref"] then %>
 <li><%= requires["name"] %></li>
 <% end %>
-<% end # files["requires"] %>
+<% end %><%# files["requires"] %>
 </ul>
 <% end %>
 
@@ -31,7 +31,7 @@
 <% unless includes["aref"] then %>
 <li><%= includes["name"] %></li>
 <% end %>
-<% end # classes["includes"] %>
+<% end %><%# classes["includes"] %>
 </ul>
 <% end %>
 
@@ -42,7 +42,7 @@
 <table>
 <% sections["attributes"].each do |attributes| %>
 <tr><td><%= attributes["name"] %></td><td><%= attributes["rw"] %></td><td><%= attributes["a_desc"] %></td></tr>
-<% end # sections["attributes"] %>
+<% end %><%# sections["attributes"] %>
 </table>
 <% end %>
 
@@ -68,11 +68,11 @@
 <%= methods["sourcecode"] %>
 </pre></blockquote>
 <% end %>
-<% end # method_list["methods"] %>
+<% end %><%# method_list["methods"] %>
 <% end %>
-<% end # sections["method_list"] %>
+<% end %><%# sections["method_list"] %>
 <% end %>
-<% end # classes["sections"] %>
+<% end %><%# classes["sections"] %>
 <% end %>
   EOF
 
@@ -91,7 +91,7 @@
   <tr><td>Modified:</td><td><%= files["dtm_modified"] %></td></tr>
 </table>
 } + CONTENTS_XML + %{
-<% end # values["files"] %>
+<% end %><%# values["files"] %>
 
 <% if values["classes"] then %>
 <h2>Classes</h2>
@@ -107,11 +107,11 @@
 (in files
 <% classes["infiles"].each do |infiles| %>
 <%= href infiles["full_path_url"], infiles["full_path"] %>
-<% end # classes["infiles"] %>
+<% end %><%# classes["infiles"] %>
 )
 <% end %>
 } + CONTENTS_XML + %{
-<% end # values["classes"] %>
+<% end %><%# values["classes"] %>
 <% end %>
 </body>
 </html>
Index: lib/rdoc/generator/html/kilmer.rb
===================================================================
--- lib/rdoc/generator/html/kilmer.rb	(revision 18120)
+++ lib/rdoc/generator/html/kilmer.rb	(revision 18121)
@@ -119,7 +119,7 @@
 <div class="name-list">
 <% values["requires"].each do |requires| %>
 <%= href requires["aref"], requires["name"] %>
-<% end # values["requires"] %>
+<% end %><%# values["requires"] %>
 <% end %>
 </div>
 
@@ -130,7 +130,7 @@
 <div class="name-list">
 <% values["methods"].each do |methods| %>
 <%= href methods["aref"], methods["name"] %>,
-<% end # values["methods"] %>
+<% end %><%# values["methods"] %>
 </div>
 <% end %>
 
@@ -162,7 +162,7 @@
        <td class="attr-name"><%= attributes["name"] %></td>
        <td><%= attributes["a_desc"] %></td>
      </tr>
-<% end # sections["attributes"] %>
+<% end %><%# sections["attributes"] %>
 </table>
 <% end %>
 
@@ -175,7 +175,7 @@
 
   <%= template_include %>  <!-- method descriptions -->
 
-<% end # values["sections"] %>
+<% end %><%# values["sections"] %>
 
 </body>
 </html>
@@ -221,7 +221,7 @@
 <% if infiles["cvsurl"] then %>
 &nbsp;(<a href="<%= infiles["cvsurl"] %>"><acronym title="Concurrent Versioning System">CVS</acronym></a>)
 <% end %>
-<% end # values["infiles"] %>
+<% end %><%# values["infiles"] %>
       </td>
      </tr>
 <% if values["parent"] then %>
@@ -250,7 +250,7 @@
 <div class="name-list">
 <% values["includes"].each do |includes| %>
     <span class="method-name"><%= href includes["aref"], includes["name"] %></span>
-<% end # values["includes"] %>
+<% end %><%# values["includes"] %>
 </div>
 <% end %>
 
@@ -285,7 +285,7 @@
 This method is also aliased as
 <% values["aka"].each do |aka| $stderr.puts({ :aka => aka }.inspect) %>
 <a href="<%= values["aref"] %>"><%= values["name"] %></a>
-<% end # values["aka"] %>
+<% end %><%# values["aka"] %>
 </div>
 <% end %>
 <% if values["sourcecode"] then %>
@@ -293,9 +293,9 @@
 <%= values["sourcecode"] %>
 </pre>
 <% end %>
-<% end # values["methods"] %>
+<% end %><%# values["methods"] %>
 <% end %>
-<% end # values["method_list"] %>
+<% end %><%# values["method_list"] %>
 <% end %>
   EOF
 
@@ -364,7 +364,7 @@
 <div class="banner"><%= values["list_title"] %></div>
 <% values["entries"].each do |entries| %>
 <a href="<%= entries["href"] %>"><%= entries["name"] %></a><br />
-<% end # values["entries"] %>
+<% end %><%# values["entries"] %>
 </body></html>
   EOF
 
Index: lib/rdoc/generator/html/html.rb
===================================================================
--- lib/rdoc/generator/html/html.rb	(revision 18120)
+++ lib/rdoc/generator/html/html.rb	(revision 18121)
@@ -7,8 +7,8 @@
 # This is a template for RDoc that uses XHTML 1.0 Transitional and dictates a
 # bit more of the appearance of the output to cascading stylesheets than the
 # default. It was designed for clean inline code display, and uses DHTMl to
-# toggle the visibility of each method's source with each click on the '[source]'
-# link.
+# toggle the visibility of each method's source with each click on the
+# '[source]' link.
 #
 # == Authors
 #
@@ -16,10 +16,10 @@
 #
 # Copyright (c) 2002, 2003 The FaerieMUD Consortium. Some rights reserved.
 #
-# This work is licensed under the Creative Commons Attribution License. To view
-# a copy of this license, visit http://creativecommons.org/licenses/by/1.0/ or
-# send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California
-# 94305, USA.
+# This work is licensed under the Creative Commons Attribution License. To
+# view a copy of this license, visit
+# http://creativecommons.org/licenses/by/1.0/ or send a letter to Creative
+# Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.
 
 module RDoc::Generator::HTML::HTML
 
@@ -361,7 +361,7 @@
         &nbsp;(<a href="<%= infiles["cvsurl"] %>"><acronym title="Concurrent Versioning System">CVS</acronym></a>)
 <% end %>
         <br />
-<% end # values["infiles"] %>
+<% end %><%# values["infiles"] %>
             </td>
         </tr>
 
@@ -388,39 +388,38 @@
 #####################################################################
 
   METHOD_LIST = <<-EOF
-
   <div id="contextContent">
 <% if values["diagram"] then %>
     <div id="diagram">
       <%= values["diagram"] %>
     </div>
-<% end %>
+<% end
 
-<% if values["description"] then %>
+   if values["description"] then %>
     <div id="description">
       <%= values["description"] %>
     </div>
-<% end %>
+<% end
 
-<% if values["requires"] then %>
+   if values["requires"] then %>
     <div id="requires-list">
       <h3 class="section-bar">Required files</h3>
 
       <div class="name-list">
 <% values["requires"].each do |requires| %>
         <%= href requires["aref"], requires["name"] %>&nbsp;&nbsp;
-<% end # values["requires"] %>
+<% end %><%# values["requires"] %>
       </div>
     </div>
-<% end %>
+<% end
 
-<% if values["toc"] then %>
+   if values["toc"] then %>
     <div id="contents-list">
       <h3 class="section-bar">Contents</h3>
       <ul>
 <% values["toc"].each do |toc| %>
-      <li><a href="#<%= values["href"] %>"><%= values["secname"] %></a></li>
-<% end # values["toc"] %>
+      <li><a href="#<%= toc["href"] %>"><%= toc["secname"] %></a></li>
+<% end %><%# values["toc"] %>
      </ul>
 <% end %>
    </div>
@@ -430,16 +429,14 @@
       <h3 class="section-bar">Methods</h3>
 
       <div class="name-list">
-<% values["methods"].each do |methods| %>
+<%   values["methods"].each do |methods| %>
         <%= href methods["aref"], methods["name"] %>&nbsp;&nbsp;
-<% end # values["methods"] %>
+<%   end %><%# values["methods"] %>
       </div>
     </div>
 <% end %>
-
   </div>
 
-
     <!-- if includes -->
 <% if values["includes"] then %>
     <div id="includes">
@@ -448,140 +445,137 @@
       <div id="includes-list">
 <% values["includes"].each do |includes| %>
         <span class="include-name"><%= href includes["aref"], includes["name"] %></span>
-<% end # values["includes"] %>
+<% end %><%# values["includes"] %>
       </div>
     </div>
-<% end %>
+<% end
 
-<% values["sections"].each do |sections| %>
+   values["sections"].each do |sections| %>
     <div id="section">
-<% if sections["sectitle"] then %>
+<%   if sections["sectitle"] then %>
       <h2 class="section-title"><a name="<%= sections["secsequence"] %>"><%= sections["sectitle"] %></a></h2>
-<% if sections["seccomment"] then %>
+<%     if sections["seccomment"] then %>
       <div class="section-comment">
         <%= sections["seccomment"] %>
       </div>
-<% end %>
-<% end %>
+<%     end
+     end
 
-<% if values["classlist"] then %>
+     if sections["classlist"] then %>
     <div id="class-list">
       <h3 class="section-bar">Classes and Modules</h3>
 
-      <%= values["classlist"] %>
+      <%= sections["classlist"] %>
     </div>
-<% end %>
+<%   end
 
-<% if values["constants"] then %>
+     if sections["constants"] then %>
     <div id="constants-list">
       <h3 class="section-bar">Constants</h3>
 
       <div class="name-list">
         <table summary="Constants">
-<% values["constants"].each do |constants| %>
+<%     sections["constants"].each do |constants| %>
         <tr class="top-aligned-row context-row">
           <td class="context-item-name"><%= constants["name"] %></td>
           <td>=</td>
           <td class="context-item-value"><%= constants["value"] %></td>
-<% if values["desc"] then %>
+<%       if sections["desc"] then %>
           <td width="3em">&nbsp;</td>
           <td class="context-item-desc"><%= constants["desc"] %></td>
-<% end %>
+<%       end %>
         </tr>
-<% end # values["constants"] %>
+<%     end %><%# sections["constants"] %>
         </table>
       </div>
     </div>
-<% end %>
+<%   end
 
-<% if values["aliases"] then %>
+     if sections["aliases"] then %>
     <div id="aliases-list">
       <h3 class="section-bar">External Aliases</h3>
 
       <div class="name-list">
-                        <table summary="aliases">
-<% values["aliases"].each do |aliases| $stderr.puts({ :aliases => aliases }.inspect) %>
+      <table summary="aliases">
+<%     sections["aliases"].each do |aliases| %>
         <tr class="top-aligned-row context-row">
-          <td class="context-item-name"><%= values["old_name"] %></td>
+          <td class="context-item-name"><%= aliases["old_name"] %></td>
           <td>-&gt;</td>
-          <td class="context-item-value"><%= values["new_name"] %></td>
+          <td class="context-item-value"><%= aliases["new_name"] %></td>
         </tr>
-<% if values["desc"] then %>
+<%       if aliases["desc"] then %>
       <tr class="top-aligned-row context-row">
         <td>&nbsp;</td>
-        <td colspan="2" class="context-item-desc"><%= values["desc"] %></td>
+        <td colspan="2" class="context-item-desc"><%= aliases["desc"] %></td>
       </tr>
-<% end %>
-<% end # values["aliases"] %>
+<%       end
+       end %><%# sections["aliases"] %>
         </table>
       </div>
     </div>
-<% end %>
+<%   end %>
 
-
-<% if values["attributes"] then %>
+<%   if sections["attributes"] then %>
     <div id="attribute-list">
       <h3 class="section-bar">Attributes</h3>
 
       <div class="name-list">
         <table>
-<% values["attributes"].each do |attributes| $stderr.puts({ :attributes => attributes }.inspect) %>
+<%     sections["attributes"].each do |attribute| %>
         <tr class="top-aligned-row context-row">
-          <td class="context-item-name"><%= values["name"] %></td>
-<% if values["rw"] then %>
-          <td class="context-item-value">&nbsp;[<%= values["rw"] %>]&nbsp;</td>
-<% end %>
-<% unless values["rw"] then %>
+          <td class="context-item-name"><%= attribute["name"] %></td>
+<%       if attribute["rw"] then %>
+          <td class="context-item-value">&nbsp;[<%= attribute["rw"] %>]&nbsp;</td>
+<%       end
+         unless attribute["rw"] then %>
           <td class="context-item-value">&nbsp;&nbsp;</td>
-<% end %>
-          <td class="context-item-desc"><%= values["a_desc"] %></td>
+<%       end %>
+          <td class="context-item-desc"><%= attribute["a_desc"] %></td>
         </tr>
-<% end # values["attributes"] %>
+<%     end %><%# sections["attributes"] %>
         </table>
       </div>
     </div>
-<% end %>
-      
+<%   end %>
 
-
     <!-- if method_list -->
-<% if sections["method_list"] then %>
+<%   if sections["method_list"] then %>
     <div id="methods">
-<% sections["method_list"].each do |method_list| %>
-<% if method_list["methods"] then %>
+<%     sections["method_list"].each do |method_list|
+         if method_list["methods"] then %>
       <h3 class="section-bar"><%= method_list["type"] %> <%= method_list["category"] %> methods</h3>
 
-<% method_list["methods"].each do |methods| %>
+<%         method_list["methods"].each do |methods| %>
       <div id="method-<%= methods["aref"] %>" class="method-detail">
         <a name="<%= methods["aref"] %>"></a>
 
         <div class="method-heading">
-<% if methods["codeurl"] then %>
+<%           if methods["codeurl"] then %>
           <a href="<%= methods["codeurl"] %>" target="Code" class="method-signature"
             onclick="popupCode('<%= methods["codeurl"] %>');return false;">
-<% end %>
-<% if methods["sourcecode"] then %>
+<%           end
+             if methods["sourcecode"] then %>
           <a href="#<%= methods["aref"] %>" class="method-signature">
-<% end %>
-<% if methods["callseq"] then %>
+<%           end
+             if methods["callseq"] then %>
           <span class="method-name"><%= methods["callseq"] %></span>
-<% end %>
-<% unless methods["callseq"] then %>
+<%           end
+             unless methods["callseq"] then %>
           <span class="method-name"><%= methods["name"] %></span><span class="method-args"><%= methods["params"] %></span>
-<% end %>
-<% if methods["codeurl"] then %>
+<%           end
+             if methods["codeurl"] then %>
           </a>
-<% end %>
-<% if methods["sourcecode"] then %>
+<%           end
+             if methods["sourcecode"] then %>
           </a>
-<% end %>
+<%           end %>
         </div>
 
         <div class="method-description">
-<% if methods["m_desc"] then %>
+<%           if methods["m_desc"] then %>
           <%= methods["m_desc"] %>
-<% end %>
-<% if methods["sourcecode"] then %>
+<%           end
+               if methods["sourcecode"] then %>
           <p><a class="source-toggle" href="#"
             onclick="toggleCode('<%= methods["aref"] %>-source');return false;">[Source]</a></p>
           <div class="method-source-code" id="<%= methods["aref"] %>-source">
@@ -589,17 +583,17 @@
 <%= methods["sourcecode"] %>
 </pre>
           </div>
-<% end %>
+<%           end %>
         </div>
       </div>
 
-<% end # method_list["methods"] %>
-<% end %>
-<% end # sections["method_list"] %>
+<%         end %><%# method_list["methods"] %><%
+         end
+       end %><%# sections["method_list"] %>
 
     </div>
-<% end %>
-<% end # values["sections"] %>
+<%   end %>
+<% end %><%# values["sections"] %>
   EOF
 
 #####################################################################
@@ -663,7 +657,7 @@
   <div id="index-entries">
 <% values["entries"].each do |entries| %>
     <a href="<%= entries["href"] %>"><%= entries["name"] %></a><br />
-<% end # values["entries"] %>
+<% end %><%#  values["entries"] %>
   </div>
 </div>
 </body>
Index: lib/rdoc/generator/html/hefss.rb
===================================================================
--- lib/rdoc/generator/html/hefss.rb	(revision 18120)
+++ lib/rdoc/generator/html/hefss.rb	(revision 18121)
@@ -141,7 +141,7 @@
 <div class="name-list">
 <% values["requires"].each do |requires| %>
 <%= href requires["aref"], requires["name"] %>
-<% end # values["requires"] %>
+<% end %><%# values["requires"] %>
 <% end %>
 </div>
 
@@ -156,10 +156,10 @@
 <div class="name-list">
 <% method_list["methods"].each do |methods| %>
 <a href="<%= methods["codeurl"] %>" target="source"><%= methods["name"] %></a>
-<% end # values["methods"] %>
+<% end %><%# values["methods"] %>
 </div>
 <% end %>
-<% end # values["method_list"] %>
+<% end %><%# values["method_list"] %>
 <% end %>
 
 <% if sections["attributes"] then %>
@@ -178,10 +178,10 @@
        <td class="attr-name"><%= attributes["name"] %></td>
        <td><%= attributes["a_desc"] %></td>
      </tr>
-<% end # values["attributes"] %>
+<% end %><%# values["attributes"] %>
 </table>
 <% end %>
-<% end # values["sections"] %>
+<% end %><%# values["sections"] %>
 <% end %>
 
 <% if values["classlist"] then %>
@@ -237,7 +237,7 @@
 <% if infiles["cvsurl"] then %>
 &nbsp;(<a href="<%= infiles["cvsurl"] %>"><acronym title="Concurrent Versioning System">CVS</acronym></a>)
 <% end %>
-<% end # values["infiles"] %>
+<% end %><%# values["infiles"] %>
       </td>
      </tr>
 <% if values["parent"] then %>
@@ -266,7 +266,7 @@
 <div class="name-list">
 <% values["includes"].each do |includes| %>
     <span class="method-name"><%= href includes["aref"], includes["name"] %></span>
-<% end # values["includes"] %>
+<% end %><%# values["includes"] %>
 </div>
 <% end %>
 
@@ -293,11 +293,11 @@
 <%= method_list["m_desc"] %>
 </div>
 <% end %>
-<% end # method_list["methods"] %>
+<% end %><%# method_list["methods"] %>
 <% end %>
-<% end # sections["method_list"] %>
+<% end %><%# sections["method_list"] %>
 <% end %>
-<% end # values["sections"] %>
+<% end %><%# values["sections"] %>
 <% end %>
   EOF
 
@@ -365,7 +365,7 @@
 <div class="banner"><%= values["list_title"] %></div>
 <% values["entries"].each do |entries| %>
 <a href="<%= entries["href"] %>"><%= entries["name"] %></a><br />
-<% end # values["entries"] %>
+<% end %><%# values["entries"] %>
 </body></html>
   EOF
 
Index: lib/rdoc/generator/html.rb
===================================================================
--- lib/rdoc/generator/html.rb	(revision 18120)
+++ lib/rdoc/generator/html.rb	(revision 18121)
@@ -82,7 +82,7 @@
     @classes    = []
 
     write_style_sheet
-    gen_sub_directories()
+    gen_sub_directories
     build_indices
     generate_html
   end
@@ -157,6 +157,7 @@
     # the individual descriptions for files and classes
     gen_into(@files)
     gen_into(@classes)
+
     # and the index files
     gen_file_index
     gen_class_index
@@ -168,14 +169,21 @@
   end
 
   def gen_into(list)
+    @file_list ||= index_to_links @files
+    @class_list ||= index_to_links @classes
+    @method_list ||= index_to_links RDoc::Generator::Method.all_methods
+
     list.each do |item|
-      if item.document_self
-        op_file = item.path
-        FileUtils.mkdir_p(File.dirname(op_file))
-        open(op_file, "w") { |file| item.write_on(file) }
+      next unless item.document_self
+
+      op_file = item.path
+
+      FileUtils.mkdir_p File.dirname(op_file)
+
+      open op_file, 'w' do |io|
+        item.write_on io, @file_list, @class_list, @method_list
       end
     end
-
   end
 
   def gen_file_index
@@ -221,9 +229,23 @@
   # line.
 
   def gen_main_index
-    template = RDoc::TemplatePage.new @template::INDEX
+    if @template.const_defined? :FRAMELESS then
+      main = @files.find do |file|
+        @main_page == file.name
+      end
 
+      if main.nil? then
+        main = @classes.find do |klass|
+          main_page == klass.context.full_name
+        end
+      end
+    else
+      main = RDoc::TemplatePage.new @template::INDEX
+    end
+
     open 'index.html', 'w'  do |f|
+      style_url = style_url '', @options.css
+
       classes = @classes.sort.map { |klass| klass.value_hash }
 
       values = {
@@ -237,18 +259,31 @@
 
       values['inline_source'] = @options.inline_source
 
-      template.write_html_on f, values
+      if main.respond_to? :write_on then
+        main.write_on f, @file_list, @class_list, @method_list, values
+      else
+        main.write_html_on f, values
+      end
     end
   end
 
+  def index_to_links(collection)
+    collection.sort.map do |f|
+      next unless f.document_self
+      { "href" => f.path, "name" => f.index_name }
+    end.compact
+  end
+
   ##
   # Returns the url of the main page
 
   def main_url
     @main_page = @options.main_page
     @main_page_ref = nil
-    if @main_page
+
+    if @main_page then
       @main_page_ref = RDoc::Generator::AllReferences[@main_page]
+
       if @main_page_ref then
         @main_page_path = @main_page_ref.path
       else
@@ -351,15 +386,8 @@
   end
 
   def gen_an_index(collection, title)
-    res = []
-    collection.sort.each do |f|
-      if f.document_self
-        res << { "href" => f.path, "name" => f.index_name }
-      end
-    end
-
     return {
-      "entries" => res,
+      "entries" => index_to_links(collection),
       'list_title' => title,
       'index_url'  => main_url,
     }
@@ -367,4 +395,3 @@
 
 end
 
-
Index: lib/rdoc/generator/ri.rb
===================================================================
--- lib/rdoc/generator/ri.rb	(revision 18120)
+++ lib/rdoc/generator/ri.rb	(revision 18121)
@@ -45,7 +45,7 @@
   def process_class(from_class)
     generate_class_info(from_class)
 
-    # now recurse into this classes constituent classes
+    # now recurse into this class' constituent classes
     from_class.each_classmodule do |mod|
       process_class(mod)
     end
Index: lib/rdoc/generator/texinfo/file.texinfo.erb
===================================================================
--- lib/rdoc/generator/texinfo/file.texinfo.erb	(revision 0)
+++ lib/rdoc/generator/texinfo/file.texinfo.erb	(revision 18121)
@@ -0,0 +1,6 @@
+<% if false %>
+<h2>File: <%= @v['file']["short_name"] %></h2>
+Path: <%= @v['file']["full_path"] %>
+
+<%= TexinfoTemplate.new(@v, 'content.texinfo.erb').render %>
+<% end %>
Index: lib/rdoc/generator/texinfo/method.texinfo.erb
===================================================================
--- lib/rdoc/generator/texinfo/method.texinfo.erb	(revision 0)
+++ lib/rdoc/generator/texinfo/method.texinfo.erb	(revision 18121)
@@ -0,0 +1,6 @@
+@node <%= @v['class']['full_name'].gsub(/::/, '-') %><%= method_prefix @v['list'] %><%= @v['method']['name'] %>
+@section <%= @v['class']["classmod"] %> <%= @v['class']['full_name'] %><%= method_prefix @v['list'] %><%= @v['method']['name'] %>
+<%= @v['method']["type"] %> <%= @v['method']["category"] %> method:
+<%= target @v['method']["aref"], @v['method']['callseq'] ||
+                                 @v['method']["name"] + @v['method']["params"] %>
+<%= @v['method']["m_desc"] %>
Index: lib/rdoc/generator/texinfo/texinfo.erb
===================================================================
--- lib/rdoc/generator/texinfo/texinfo.erb	(revision 0)
+++ lib/rdoc/generator/texinfo/texinfo.erb	(revision 18121)
@@ -0,0 +1,28 @@
+\input texinfo   @c -*-texinfo-*-
+@c %**start of header
+@setfilename <%= @v['filename'] %>
+@settitle <%= @v['title'] %>
+@c %**end of header
+
+@contents @c TODO: whitespace is a mess... =\
+
+@ifnottex
+@node Top
+
+@top <%= @v['title'] %>
+@end ifnottex
+
+<% if @f = @v['files'].detect { |f| f.name =~ /Readme/i }  %>
+<%= @f.values['description'] %><% end %>
+
+@menu
+<% @v['classes'].each do |klass| %>
+* <%= klass.name.gsub(/::/, '-') %>::<% end %>
+@c TODO: add files
+@end menu
+
+<% (@v['classes'] || []).each_with_index do |klass, i| %>
+<%= TexinfoTemplate.new(@v.merge('class' => klass.values), 
+                        'class.texinfo.erb').render %><% end %>
+
+@bye
Index: lib/rdoc/generator/texinfo/class.texinfo.erb
===================================================================
--- lib/rdoc/generator/texinfo/class.texinfo.erb	(revision 0)
+++ lib/rdoc/generator/texinfo/class.texinfo.erb	(revision 18121)
@@ -0,0 +1,44 @@
+@node <%= @v['class']['full_name'].gsub(/::/, '-') %>
+@chapter <%= @v['class']["classmod"] %> <%= @v['class']['full_name'] %>
+
+<% if @v['class']["parent"] and @v['class']['par_url'] %>
+Inherits <%= href @v['class']["par_url"], @v['class']["parent"] %><% end %>
+
+<%= @v['class']["description"] %>
+
+<% if @v['class']["includes"] %>
+Includes
+<%   @v['class']["includes"].each do |include| %>
+* <%=  href include["aref"], include["name"] %>
+<%   end # @v['class']["includes"] %>
+<% end %>
+
+<% if @v['class']["sections"] %>
+<%   @v['class']["sections"].each do |section| %>
+<%     if section["attributes"] %>
+Attributes
+<%       section["attributes"].each do |attributes| %>
+* <%=      attributes["name"] %> <%= attributes["rw"] %> <%= attributes["a_desc"] %>
+<%       end # section["attributes"] %>
+<%     end %>
+<%   end %>
+
+<%   @v['class']["sections"].each do |section| %>
+<%     if section["method_list"] %>
+Methods
+@menu
+<%     section["method_list"].each_with_index do |method_list, i| %>
+<%= i %>
+<%       (method_list["methods"] || []).each do |method| %>
+* <%= @v['class']['full_name'].gsub(/::/, '-') %><%= method_prefix method_list %><%= method['name'] %>::<% end %>
+<% end %>
+@end menu
+
+<%     section["method_list"].each do |method_list| %>
+<%       (method_list["methods"] || []).uniq.each do |method| %>
+<%=        TexinfoTemplate.new(@v.merge({'method' => method, 'list' => method_list}),
+                               'method.texinfo.erb').render %><% end %>
+<%     end # section["method_list"] %>
+<%   end %>
+<% end # @v['class']["sections"] %>
+<% end %>
Index: lib/rdoc/generator/texinfo.rb
===================================================================
--- lib/rdoc/generator/texinfo.rb	(revision 0)
+++ lib/rdoc/generator/texinfo.rb	(revision 18121)
@@ -0,0 +1,84 @@
+require 'rdoc/rdoc'
+require 'rdoc/generator'
+require 'rdoc/markup/to_texinfo'
+
+module RDoc
+  RDoc::GENERATORS['texinfo'] = RDoc::Generator.new("rdoc/generator/texinfo",
+                                                    :Texinfo,
+                                                    'texinfo')
+  module Generator
+    # This generates Texinfo files for viewing with GNU Info or Emacs
+    # from RDoc extracted from Ruby source files.
+    class Texinfo
+      # What should the .info file be named by default?
+      DEFAULT_INFO_FILENAME = 'rdoc.info'
+
+      include Generator::MarkUp
+
+      # Accept some options
+      def initialize(options)
+        @options = options
+        @options.inline_source = true
+        @options.op_name ||= 'rdoc.texinfo'
+        @options.formatter = ::RDoc::Markup::ToTexInfo.new
+      end
+
+      # Generate the +texinfo+ files
+      def generate(toplevels)
+        @toplevels = toplevels
+        @files, @classes = ::RDoc::Generator::Context.build_indicies(@toplevels,
+                                                                     @options)
+
+        (@files + @classes).each { |x| x.value_hash }
+
+        open(@options.op_name, 'w') do |f|
+          f.puts TexinfoTemplate.new('files' => @files,
+                                     'classes' => @classes,
+                                     'filename' => @options.op_name.gsub(/texinfo/, 'info'),
+                                     'title' => @options.title).render
+        end
+        # TODO: create info files and install?
+      end
+
+      class << self
+        # Factory? We don't need no stinkin' factory!
+        alias_method :for, :new
+      end
+    end
+
+    # Basically just a wrapper around ERB.
+    # Should probably use RDoc::TemplatePage instead
+    class TexinfoTemplate
+      BASE_DIR = ::File.expand_path(::File.dirname(__FILE__)) # have to calculate this when the file's loaded.
+
+      def initialize(values, file = 'texinfo.erb')
+        @v, @file = [values, file]
+      end
+     
+      def template
+        ::File.read(::File.join(BASE_DIR, 'texinfo', @file))
+      end
+
+      # Go!
+      def render
+        ERB.new(template).result binding
+      end
+
+      def href(location, text)
+        text # TODO: how does texinfo do hyperlinks?
+      end
+
+      def target(name, text)
+        text # TODO: how do hyperlink targets work?
+      end
+
+      # TODO: this is probably implemented elsewhere?
+      def method_prefix(section)
+        { 'Class' => '.',
+          'Module' => '::',
+          'Instance' => '#',
+        }[section['category']]
+      end
+    end
+  end
+end
Index: lib/rdoc/generator.rb
===================================================================
--- lib/rdoc/generator.rb	(revision 18120)
+++ lib/rdoc/generator.rb	(revision 18121)
@@ -22,27 +22,6 @@
   CSS_NAME  = "rdoc-style.css"
 
   ##
-  # Converts a target url to one that is relative to a given path
-
-  def self.gen_url(path, target)
-    from          = ::File.dirname path
-    to, to_file   = ::File.split target
-
-    from = from.split "/"
-    to   = to.split "/"
-
-    while from.size > 0 and to.size > 0 and from[0] == to[0] do
-      from.shift
-      to.shift
-    end
-
-    from.fill ".."
-    from.concat to
-    from << to_file
-    ::File.join(*from)
-  end
-
-  ##
   # Build a hash of all items that can be cross-referenced.  This is used when
   # we output required and included names: if the names appear in this hash,
   # we can generate an html cross reference to the appropriate description.
@@ -80,11 +59,6 @@
     def markup(str, remove_para = false)
       return '' unless str
 
-      unless defined? @formatter then
-        @formatter = RDoc::Markup::ToHtmlCrossref.new(path, self,
-                                                       @options.show_hash)
-      end
-
       # Convert leading comment markers to spaces, but only if all non-blank
       # lines have them
       if str =~ /^(?>\s*)[^\#]/ then
@@ -93,7 +67,7 @@
         content = str.gsub(/^\s*(#+)/) { $1.tr '#', ' ' }
       end
 
-      res = @formatter.convert content
+      res = formatter.convert content
 
       if remove_para then
         res.sub!(/^<p>/, '')
@@ -114,7 +88,7 @@
       if %r{^(https?:/)?/} =~ css_name
         css_name
       else
-        RDoc::Generator.gen_url path, css_name
+        RDoc::Markup::ToHtml.gen_relative_url path, css_name
       end
     end
 
@@ -186,6 +160,11 @@
       @template = options.template_class
     end
 
+    def formatter
+      @formatter ||= @options.formatter ||
+        RDoc::Markup::ToHtmlCrossref.new(path, self, @options.show_hash)
+    end
+
     ##
     # convenience method to build a hyperlink
 
@@ -201,7 +180,7 @@
       if @options.all_one_file
         "#" + path
       else
-        RDoc::Generator.gen_url from_path, path
+        RDoc::Markup::ToHtml.gen_relative_url from_path, path
       end
     end
 
@@ -215,7 +194,7 @@
       list = @context.method_list
 
       unless @options.show_all then
-        list = list.find_all do |m|
+        list = list.select do |m|
           m.visibility == :public or
             m.visibility == :protected or
             m.force_documentation
@@ -230,17 +209,15 @@
     ##
     # Build a summary list of all the methods in this context
 
-    def build_method_summary_list(path_prefix="")
+    def build_method_summary_list(path_prefix = "")
       collect_methods unless @methods
-      meths = @methods.sort
-      res = []
-      meths.each do |meth|
-        res << {
+
+      @methods.sort.map do |meth|
+        {
           "name" => CGI.escapeHTML(meth.name),
           "aref" => "#{path_prefix}\##{meth.aref}"
         }
       end
-      res
     end
 
     ##
@@ -248,36 +225,40 @@
     # corresponding method
 
     def build_alias_summary_list(section)
-      values = []
-      @context.aliases.each do |al|
+      @context.aliases.map do |al|
         next unless al.section == section
+
         res = {
           'old_name' => al.old_name,
           'new_name' => al.new_name,
         }
-        if al.comment && !al.comment.empty?
-          res['desc'] = markup(al.comment, true)
+
+        if al.comment and not al.comment.empty? then
+          res['desc'] = markup al.comment, true
         end
-        values << res
-      end
-      values
+
+        res
+      end.compact
     end
 
     ##
     # Build a list of constants
 
     def build_constants_summary_list(section)
-      values = []
-      @context.constants.each do |co|
+      @context.constants.map do |co|
         next unless co.section == section
+
         res = {
           'name'  => co.name,
           'value' => CGI.escapeHTML(co.value)
         }
-        res['desc'] = markup(co.comment, true) if co.comment && !co.comment.empty?
-        values << res
-      end
-      values
+
+        if co.comment and not co.comment.empty? then
+          res['desc'] = markup co.comment, true
+        end
+
+        res
+      end.compact
     end
 
     def build_requires_list(context)
@@ -339,54 +320,58 @@
     def build_method_detail_list(section)
       outer = []
 
-      methods = @methods.sort
+      methods = @methods.sort.select do |m|
+        m.document_self and m.section == section
+      end
+
       for singleton in [true, false]
         for vis in [ :public, :protected, :private ]
           res = []
           methods.each do |m|
-            if m.section == section and
-                m.document_self and
-                m.visibility == vis and
-                m.singleton == singleton
-              row = {}
-              if m.call_seq
-                row["callseq"] = m.call_seq.gsub(/->/, '&rarr;')
-              else
-                row["name"]        = CGI.escapeHTML(m.name)
-                row["params"]      = m.params
-              end
-              desc = m.description.strip
-              row["m_desc"]      = desc unless desc.empty?
-              row["aref"]        = m.aref
-              row["visibility"]  = m.visibility.to_s
+            next unless m.visibility == vis and m.singleton == singleton
 
-              alias_names = []
-              m.aliases.each do |other|
-                if other.viewer   # won't be if the alias is private
-                  alias_names << {
-                    'name' => other.name,
-                    'aref'  => other.viewer.as_href(path)
-                  }
-                end
+            row = {}
+
+            if m.call_seq then
+              row["callseq"] = m.call_seq.gsub(/->/, '&rarr;')
+            else
+              row["name"]        = CGI.escapeHTML(m.name)
+              row["params"]      = m.params
+            end
+
+            desc = m.description.strip
+            row["m_desc"]      = desc unless desc.empty?
+            row["aref"]        = m.aref
+            row["visibility"]  = m.visibility.to_s
+
+            alias_names = []
+
+            m.aliases.each do |other|
+              if other.viewer then # won't be if the alias is private
+                alias_names << {
+                  'name' => other.name,
+                  'aref'  => other.viewer.as_href(path)
+                }
               end
-              unless alias_names.empty?
-                row["aka"] = alias_names
-              end
+            end
 
-              if @options.inline_source
-                code = m.source_code
-                row["sourcecode"] = code if code
-              else
-                code = m.src_url
-                if code
-                  row["codeurl"] = code
-                  row["imgurl"]  = m.img_url
-                end
+            row["aka"] = alias_names unless alias_names.empty?
+
+            if @options.inline_source then
+              code = m.source_code
+              row["sourcecode"] = code if code
+            else
+              code = m.src_url
+              if code then
+                row["codeurl"] = code
+                row["imgurl"]  = m.img_url
               end
-              res << row
             end
+
+            res << row
           end
-          if res.size > 0
+
+          if res.size > 0 then
             outer << {
               "type"     => vis.to_s.capitalize,
               "category" => singleton ? "Class" : "Instance",
@@ -395,6 +380,7 @@
           end
         end
       end
+
       outer
     end
 
@@ -403,8 +389,8 @@
     # in this context.
 
     def build_class_list(level, from, section, infile=nil)
-      res = ""
-      prefix = "&nbsp;&nbsp;::" * level;
+      prefix = '&nbsp;&nbsp;::' * level;
+      res = ''
 
       from.modules.sort.each do |mod|
         next unless mod.section == section
@@ -412,8 +398,8 @@
         if mod.document_self
           res <<
             prefix <<
-            "Module " <<
-            href(url(mod.viewer.path), "link", mod.full_name) <<
+            'Module ' <<
+            href(url(mod.viewer.path), 'link', mod.full_name) <<
             "<br />\n" <<
             build_class_list(level + 1, mod, section, infile)
         end
@@ -421,12 +407,13 @@
 
       from.classes.sort.each do |cls|
         next unless cls.section == section
-        next if infile && !cls.defined_in?(infile)
+        next if infile and not cls.defined_in?(infile)
+
         if cls.document_self
-          res      <<
+          res <<
             prefix <<
-            "Class " <<
-            href(url(cls.viewer.path), "link", cls.full_name) <<
+            'Class ' <<
+            href(url(cls.viewer.path), 'link', cls.full_name) <<
             "<br />\n" <<
             build_class_list(level + 1, cls, section, infile)
         end
@@ -436,7 +423,7 @@
     end
 
     def url(target)
-      RDoc::Generator.gen_url path, target
+      RDoc::Markup::ToHtml.gen_relative_url path, target
     end
 
     def aref_to(target)
@@ -475,7 +462,7 @@
     def add_table_of_sections
       toc = []
       @context.sections.each do |section|
-        if section.title
+        if section.title then
           toc << {
             'secname' => section.title,
             'href'    => section.sequence
@@ -495,11 +482,13 @@
 
     attr_reader :methods
     attr_reader :path
+    attr_reader :values
 
     def initialize(context, html_file, prefix, options)
-      super(context, options)
+      super context, options
 
       @html_file = html_file
+      @html_class = self
       @is_module = context.is_module?
       @values    = {}
 
@@ -540,11 +529,19 @@
       name
     end
 
-    def write_on(f)
+    def write_on(f, file_list, class_list, method_list, overrides = {})
       value_hash
+
+      @values['file_list'] = file_list
+      @values['class_list'] = class_list
+      @values['method_list'] = method_list
+
+      @values.update overrides
+
       template = RDoc::TemplatePage.new(@template::BODY,
                                         @template::CLASS_PAGE,
                                         @template::METHOD_LIST)
+
       template.write_html_on(f, @values)
     end
 
@@ -561,30 +558,29 @@
       ml = build_method_summary_list @path
       @values["methods"] = ml unless ml.empty?
 
-      il = build_include_list(@context)
+      il = build_include_list @context
       @values["includes"] = il unless il.empty?
 
       @values["sections"] = @context.sections.map do |section|
-
         secdata = {
           "sectitle" => section.title,
           "secsequence" => section.sequence,
-          "seccomment" => markup(section.comment)
+          "seccomment" => markup(section.comment),
         }
 
-        al = build_alias_summary_list(section)
+        al = build_alias_summary_list section
         secdata["aliases"] = al unless al.empty?
 
-        co = build_constants_summary_list(section)
+        co = build_constants_summary_list section
         secdata["constants"] = co unless co.empty?
 
-        al = build_attribute_list(section)
+        al = build_attribute_list section
         secdata["attributes"] = al unless al.empty?
 
-        cl = build_class_list(0, @context, section)
+        cl = build_class_list 0, @context, section
         secdata["classlist"] = cl unless cl.empty?
 
-        mdl = build_method_detail_list(section)
+        mdl = build_method_detail_list section
         secdata["method_list"] = mdl unless mdl.empty?
 
         secdata
@@ -594,23 +590,25 @@
     end
 
     def build_attribute_list(section)
-      atts = @context.attributes.sort
-      res = []
-      atts.each do |att|
+      @context.attributes.sort.map do |att|
         next unless att.section == section
-        if att.visibility == :public || att.visibility == :protected || @options.show_all
+
+        if att.visibility == :public or att.visibility == :protected or
+           @options.show_all then
+
           entry = {
             "name"   => CGI.escapeHTML(att.name),
             "rw"     => att.rw,
             "a_desc" => markup(att.comment, true)
           }
-          unless att.visibility == :public || att.visibility == :protected
+
+          unless att.visibility == :public or att.visibility == :protected then
             entry["rw"] << "-"
           end
-          res << entry
+
+          entry
         end
-      end
-      res
+      end.compact
     end
 
     def class_attribute_values
@@ -680,9 +678,10 @@
 
     attr_reader :path
     attr_reader :name
+    attr_reader :values
 
     def initialize(context, options, file_dir)
-      super(context, options)
+      super context, options
 
       @values = {}
 
@@ -755,7 +754,7 @@
         }
 
         cl = build_class_list(0, @context, section, file_context)
-        @values["classlist"] = cl unless cl.empty?
+        secdata["classlist"] = cl unless cl.empty?
 
         mdl = build_method_detail_list(section)
         secdata["method_list"] = mdl unless mdl.empty?
@@ -764,7 +763,7 @@
         secdata["aliases"] = al unless al.empty?
 
         co = build_constants_summary_list(section)
-        @values["constants"] = co unless co.empty?
+        secdata["constants"] = co unless co.empty?
 
         secdata
       end
@@ -772,9 +771,15 @@
       @values
     end
 
-    def write_on(f)
+    def write_on(f, file_list, class_list, method_list, overrides = {})
       value_hash
 
+      @values['file_list'] = file_list
+      @values['class_list'] = class_list
+      @values['method_list'] = method_list
+
+      @values.update overrides
+
       template = RDoc::TemplatePage.new(@template::BODY,
                                         @template::FILE_PAGE,
                                         @template::METHOD_LIST)
@@ -829,15 +834,17 @@
     end
 
     def initialize(context, html_class, options)
+      # TODO: rethink the class hierarchy here...
       @context    = context
       @html_class = html_class
       @options    = options
 
+      @@seq       = @@seq.succ
+      @seq        = @@seq
+
       # HACK ugly
       @template = options.template_class
 
-      @@seq       = @@seq.succ
-      @seq        = @@seq
       @@all_methods << self
 
       context.viewer = self
@@ -846,7 +853,7 @@
         @source_code = markup_code(ts)
         unless @options.inline_source
           @src_url = create_source_code_file(@source_code)
-          @img_url = RDoc::Generator.gen_url path, 'source.png'
+          @img_url = RDoc::Markup::ToHtml.gen_relative_url path, 'source.png'
         end
       end
 
@@ -861,10 +868,32 @@
       if @options.all_one_file
         "#" + path
       else
-        RDoc::Generator.gen_url from_path, path
+        RDoc::Markup::ToHtml.gen_relative_url from_path, path
       end
     end
 
+    def formatter
+      @formatter ||= @options.formatter ||
+        RDoc::Markup::ToHtmlCrossref.new(path, self, @options.show_hash)
+    end
+
+    def inspect
+      alias_for = if @context.is_alias_for then
+                    " (alias_for #{@context.is_alias_for})"
+                  else
+                    nil
+                  end
+
+      "#<%s:0x%x %s%s%s (%s)%s>" % [
+        self.class, object_id,
+        @context.parent.name,
+        @context.singleton ? '::' : '#',
+        name,
+        @context.visibility,
+        alias_for
+      ]
+    end
+
     def name
       @context.name
     end
@@ -961,7 +990,7 @@
         template.write_html_on(f, values)
       end
 
-      RDoc::Generator.gen_url path, file_path
+      RDoc::Markup::ToHtml.gen_relative_url path, file_path
     end
 
     def <=>(other)
@@ -976,19 +1005,18 @@
       src = ""
       tokens.each do |t|
         next unless t
-        #    p t.class
 #        style = STYLE_MAP[t.class]
         style = case t
-                when RubyToken::TkCONSTANT then "ruby-constant"
-                when RubyToken::TkKW       then "ruby-keyword kw"
-                when RubyToken::TkIVAR     then "ruby-ivar"
-                when RubyToken::TkOp       then "ruby-operator"
-                when RubyToken::TkId       then "ruby-identifier"
-                when RubyToken::TkNode     then "ruby-node"
-                when RubyToken::TkCOMMENT  then "ruby-comment cmt"
-                when RubyToken::TkREGEXP   then "ruby-regexp re"
-                when RubyToken::TkSTRING   then "ruby-value str"
-                when RubyToken::TkVal      then "ruby-value"
+                when RDoc::RubyToken::TkCONSTANT then "ruby-constant"
+                when RDoc::RubyToken::TkKW       then "ruby-keyword kw"
+                when RDoc::RubyToken::TkIVAR     then "ruby-ivar"
+                when RDoc::RubyToken::TkOp       then "ruby-operator"
+                when RDoc::RubyToken::TkId       then "ruby-identifier"
+                when RDoc::RubyToken::TkNode     then "ruby-node"
+                when RDoc::RubyToken::TkCOMMENT  then "ruby-comment cmt"
+                when RDoc::RubyToken::TkREGEXP   then "ruby-regexp re"
+                when RDoc::RubyToken::TkSTRING   then "ruby-value str"
+                when RDoc::RubyToken::TkVal      then "ruby-value"
                 else
                     nil
                 end
Index: lib/rdoc/rdoc.rb
===================================================================
--- lib/rdoc/rdoc.rb	(revision 18120)
+++ lib/rdoc/rdoc.rb	(revision 18121)
@@ -1,10 +1,13 @@
 require 'rdoc'
 
-require 'rdoc/parsers/parse_rb.rb'
-require 'rdoc/parsers/parse_c.rb'
-require 'rdoc/parsers/parse_f95.rb'
-require 'rdoc/parsers/parse_simple.rb'
+require 'rdoc/parser'
 
+# Simple must come first
+require 'rdoc/parser/simple'
+require 'rdoc/parser/ruby'
+require 'rdoc/parser/c'
+require 'rdoc/parser/f95'
+
 require 'rdoc/stats'
 require 'rdoc/options'
 
@@ -146,7 +149,10 @@
         case type = stat.ftype
         when "file"
           next if @last_created and stat.mtime < @last_created
-          file_list << rel_file_name.sub(/^\.\//, '') if force_doc || ParserFactory.can_parse(rel_file_name)
+
+          if force_doc or ::RDoc::Parser.can_parse(rel_file_name) then
+            file_list << rel_file_name.sub(/^\.\//, '')
+          end
         when "directory"
           next if rel_file_name == "CVS" || rel_file_name == ".svn"
           dot_doc = File.join(rel_file_name, DOT_DOC_FILENAME)
@@ -198,14 +204,18 @@
                     File.read fn
                   end
 
-        if /coding:\s*(\S+)/ =~ content[/\A(?:.*\n){0,2}/]
-          if enc = Encoding.find($1)
-            content.force_encoding(enc)
+        if defined? Encoding then
+          if /coding:\s*(\S+)/ =~ content[/\A(?:.*\n){0,2}/]
+            if enc = ::Encoding.find($1)
+              content.force_encoding(enc)
+            end
           end
         end
 
-        top_level = TopLevel.new(fn)
-        parser = ParserFactory.parser_for(top_level, fn, content, options, @stats)
+        top_level = ::RDoc::TopLevel.new fn
+
+        parser = ::RDoc::Parser.for top_level, fn, content, options, @stats
+
         file_info << parser.scan
         @stats.num_files += 1
       end
@@ -241,17 +251,19 @@
 
       file_info = parse_files @options
 
+      @options.title = "RDoc Documentation"
+
       if file_info.empty?
         $stderr.puts "\nNo newer files." unless @options.quiet
       else
-        gen = @options.generator
+        @gen = @options.generator
 
-        $stderr.puts "\nGenerating #{gen.key.upcase}..." unless @options.quiet
+        $stderr.puts "\nGenerating #{@gen.key.upcase}..." unless @options.quiet
 
-        require gen.file_name
+        require @gen.file_name
 
-        gen_class = ::RDoc::Generator.const_get gen.class_name
-        gen = gen_class.for @options
+        gen_class = ::RDoc::Generator.const_get @gen.class_name
+        @gen = gen_class.for @options
 
         pwd = Dir.pwd
 
@@ -259,7 +271,7 @@
 
         begin
           Diagram.new(file_info, @options).draw if @options.diagram
-          gen.generate(file_info)
+          @gen.generate(file_info)
           update_output_dir(".", start_time)
         ensure
           Dir.chdir(pwd)
Index: lib/rdoc/parser/ruby.rb
===================================================================
--- lib/rdoc/parser/ruby.rb	(revision 0)
+++ lib/rdoc/parser/ruby.rb	(revision 18121)
@@ -0,0 +1,2853 @@
+##
+# This file contains stuff stolen outright from:
+#
+#   rtags.rb -
+#   ruby-lex.rb - ruby lexcal analyzer
+#   ruby-token.rb - ruby tokens
+#       by Keiju ISHITSUKA (Nippon Rational Inc.)
+#
+
+require 'e2mmap'
+require 'irb/slex'
+
+require 'rdoc/code_objects'
+require 'rdoc/tokenstream'
+require 'rdoc/markup/preprocess'
+require 'rdoc/parser'
+
+$TOKEN_DEBUG ||= nil
+#$TOKEN_DEBUG = $DEBUG_RDOC
+
+##
+# Definitions of all tokens involved in the lexical analysis
+
+module RDoc::RubyToken
+
+  EXPR_BEG   = :EXPR_BEG
+  EXPR_MID   = :EXPR_MID
+  EXPR_END   = :EXPR_END
+  EXPR_ARG   = :EXPR_ARG
+  EXPR_FNAME = :EXPR_FNAME
+  EXPR_DOT   = :EXPR_DOT
+  EXPR_CLASS = :EXPR_CLASS
+
+  class Token
+    NO_TEXT = "??".freeze
+
+    attr_accessor :text
+    attr_reader :line_no
+    attr_reader :char_no
+
+    def initialize(line_no, char_no)
+      @line_no = line_no
+      @char_no = char_no
+      @text    = NO_TEXT
+    end
+
+    def ==(other)
+      self.class == other.class and
+        other.line_no == @line_no and
+        other.char_no == @char_no and
+        other.text == @text
+    end
+
+    ##
+    # Because we're used in contexts that expect to return a token, we set the
+    # text string and then return ourselves
+
+    def set_text(text)
+      @text = text
+      self
+    end
+
+  end
+
+  class TkNode < Token
+    attr :node
+  end
+
+  class TkId < Token
+    def initialize(line_no, char_no, name)
+      super(line_no, char_no)
+      @name = name
+    end
+    attr :name
+  end
+
+  class TkKW < TkId
+  end
+
+  class TkVal < Token
+    def initialize(line_no, char_no, value = nil)
+      super(line_no, char_no)
+      set_text(value)
+    end
+  end
+
+  class TkOp < Token
+    def name
+      self.class.op_name
+    end
+  end
+
+  class TkOPASGN < TkOp
+    def initialize(line_no, char_no, op)
+      super(line_no, char_no)
+      op = TkReading2Token[op] unless Symbol === op
+      @op = op
+    end
+    attr :op
+  end
+
+  class TkUnknownChar < Token
+    def initialize(line_no, char_no, id)
+      super(line_no, char_no)
+      @name = char_no.chr
+    end
+    attr :name
+  end
+
+  class TkError < Token
+  end
+
+  def set_token_position(line, char)
+    @prev_line_no = line
+    @prev_char_no = char
+  end
+
+  def Token(token, value = nil)
+    tk = nil
+    case token
+    when String, Symbol
+      source = String === token ? TkReading2Token : TkSymbol2Token
+      raise TkReading2TokenNoKey, token if (tk = source[token]).nil?
+      tk = Token(tk[0], value)
+    else
+      tk = if (token.ancestors & [TkId, TkVal, TkOPASGN, TkUnknownChar]).empty?
+             token.new(@prev_line_no, @prev_char_no)
+           else
+             token.new(@prev_line_no, @prev_char_no, value)
+           end
+    end
+    tk
+  end
+
+  TokenDefinitions = [
+    [:TkCLASS,      TkKW,  "class",  EXPR_CLASS],
+    [:TkMODULE,     TkKW,  "module", EXPR_BEG],
+    [:TkDEF,        TkKW,  "def",    EXPR_FNAME],
+    [:TkUNDEF,      TkKW,  "undef",  EXPR_FNAME],
+    [:TkBEGIN,      TkKW,  "begin",  EXPR_BEG],
+    [:TkRESCUE,     TkKW,  "rescue", EXPR_MID],
+    [:TkENSURE,     TkKW,  "ensure", EXPR_BEG],
+    [:TkEND,        TkKW,  "end",    EXPR_END],
+    [:TkIF,         TkKW,  "if",     EXPR_BEG, :TkIF_MOD],
+    [:TkUNLESS,     TkKW,  "unless", EXPR_BEG, :TkUNLESS_MOD],
+    [:TkTHEN,       TkKW,  "then",   EXPR_BEG],
+    [:TkELSIF,      TkKW,  "elsif",  EXPR_BEG],
+    [:TkELSE,       TkKW,  "else",   EXPR_BEG],
+    [:TkCASE,       TkKW,  "case",   EXPR_BEG],
+    [:TkWHEN,       TkKW,  "when",   EXPR_BEG],
+    [:TkWHILE,      TkKW,  "while",  EXPR_BEG, :TkWHILE_MOD],
+    [:TkUNTIL,      TkKW,  "until",  EXPR_BEG, :TkUNTIL_MOD],
+    [:TkFOR,        TkKW,  "for",    EXPR_BEG],
+    [:TkBREAK,      TkKW,  "break",  EXPR_END],
+    [:TkNEXT,       TkKW,  "next",   EXPR_END],
+    [:TkREDO,       TkKW,  "redo",   EXPR_END],
+    [:TkRETRY,      TkKW,  "retry",  EXPR_END],
+    [:TkIN,         TkKW,  "in",     EXPR_BEG],
+    [:TkDO,         TkKW,  "do",     EXPR_BEG],
+    [:TkRETURN,     TkKW,  "return", EXPR_MID],
+    [:TkYIELD,      TkKW,  "yield",  EXPR_END],
+    [:TkSUPER,      TkKW,  "super",  EXPR_END],
+    [:TkSELF,       TkKW,  "self",   EXPR_END],
+    [:TkNIL,        TkKW,  "nil",    EXPR_END],
+    [:TkTRUE,       TkKW,  "true",   EXPR_END],
+    [:TkFALSE,      TkKW,  "false",  EXPR_END],
+    [:TkAND,        TkKW,  "and",    EXPR_BEG],
+    [:TkOR,         TkKW,  "or",     EXPR_BEG],
+    [:TkNOT,        TkKW,  "not",    EXPR_BEG],
+    [:TkIF_MOD,     TkKW],
+    [:TkUNLESS_MOD, TkKW],
+    [:TkWHILE_MOD,  TkKW],
+    [:TkUNTIL_MOD,  TkKW],
+    [:TkALIAS,      TkKW,  "alias",    EXPR_FNAME],
+    [:TkDEFINED,    TkKW,  "defined?", EXPR_END],
+    [:TklBEGIN,     TkKW,  "BEGIN",    EXPR_END],
+    [:TklEND,       TkKW,  "END",      EXPR_END],
+    [:Tk__LINE__,   TkKW,  "__LINE__", EXPR_END],
+    [:Tk__FILE__,   TkKW,  "__FILE__", EXPR_END],
+
+    [:TkIDENTIFIER, TkId],
+    [:TkFID,        TkId],
+    [:TkGVAR,       TkId],
+    [:TkIVAR,       TkId],
+    [:TkCONSTANT,   TkId],
+
+    [:TkINTEGER,    TkVal],
+    [:TkFLOAT,      TkVal],
+    [:TkSTRING,     TkVal],
+    [:TkXSTRING,    TkVal],
+    [:TkREGEXP,     TkVal],
+    [:TkCOMMENT,    TkVal],
+
+    [:TkDSTRING,    TkNode],
+    [:TkDXSTRING,   TkNode],
+    [:TkDREGEXP,    TkNode],
+    [:TkNTH_REF,    TkId],
+    [:TkBACK_REF,   TkId],
+
+    [:TkUPLUS,      TkOp,   "+@"],
+    [:TkUMINUS,     TkOp,   "-@"],
+    [:TkPOW,        TkOp,   "**"],
+    [:TkCMP,        TkOp,   "<=>"],
+    [:TkEQ,         TkOp,   "=="],
+    [:TkEQQ,        TkOp,   "==="],
+    [:TkNEQ,        TkOp,   "!="],
+    [:TkGEQ,        TkOp,   ">="],
+    [:TkLEQ,        TkOp,   "<="],
+    [:TkANDOP,      TkOp,   "&&"],
+    [:TkOROP,       TkOp,   "||"],
+    [:TkMATCH,      TkOp,   "=~"],
+    [:TkNMATCH,     TkOp,   "!~"],
+    [:TkDOT2,       TkOp,   ".."],
+    [:TkDOT3,       TkOp,   "..."],
+    [:TkAREF,       TkOp,   "[]"],
+    [:TkASET,       TkOp,   "[]="],
+    [:TkLSHFT,      TkOp,   "<<"],
+    [:TkRSHFT,      TkOp,   ">>"],
+    [:TkCOLON2,     TkOp],
+    [:TkCOLON3,     TkOp],
+#   [:OPASGN,       TkOp],               # +=, -=  etc. #
+    [:TkASSOC,      TkOp,   "=>"],
+    [:TkQUESTION,   TkOp,   "?"],        #?
+    [:TkCOLON,      TkOp,   ":"],        #:
+
+    [:TkfLPAREN],         # func( #
+    [:TkfLBRACK],         # func[ #
+    [:TkfLBRACE],         # func{ #
+    [:TkSTAR],            # *arg
+    [:TkAMPER],           # &arg #
+    [:TkSYMBOL,     TkId],          # :SYMBOL
+    [:TkSYMBEG,     TkId],
+    [:TkGT,         TkOp,   ">"],
+    [:TkLT,         TkOp,   "<"],
+    [:TkPLUS,       TkOp,   "+"],
+    [:TkMINUS,      TkOp,   "-"],
+    [:TkMULT,       TkOp,   "*"],
+    [:TkDIV,        TkOp,   "/"],
+    [:TkMOD,        TkOp,   "%"],
+    [:TkBITOR,      TkOp,   "|"],
+    [:TkBITXOR,     TkOp,   "^"],
+    [:TkBITAND,     TkOp,   "&"],
+    [:TkBITNOT,     TkOp,   "~"],
+    [:TkNOTOP,      TkOp,   "!"],
+
+    [:TkBACKQUOTE,  TkOp,   "`"],
+
+    [:TkASSIGN,     Token,  "="],
+    [:TkDOT,        Token,  "."],
+    [:TkLPAREN,     Token,  "("],  #(exp)
+    [:TkLBRACK,     Token,  "["],  #[arry]
+    [:TkLBRACE,     Token,  "{"],  #{hash}
+    [:TkRPAREN,     Token,  ")"],
+    [:TkRBRACK,     Token,  "]"],
+    [:TkRBRACE,     Token,  "}"],
+    [:TkCOMMA,      Token,  ","],
+    [:TkSEMICOLON,  Token,  ";"],
+
+    [:TkRD_COMMENT],
+    [:TkSPACE],
+    [:TkNL],
+    [:TkEND_OF_SCRIPT],
+
+    [:TkBACKSLASH,  TkUnknownChar,  "\\"],
+    [:TkAT,         TkUnknownChar,  "@"],
+    [:TkDOLLAR,     TkUnknownChar,  "\$"], #"
+  ]
+
+  # {reading => token_class}
+  # {reading => [token_class, *opt]}
+  TkReading2Token = {}
+  TkSymbol2Token = {}
+
+  def self.def_token(token_n, super_token = Token, reading = nil, *opts)
+    token_n = token_n.id2name unless String === token_n
+
+    fail AlreadyDefinedToken, token_n if const_defined?(token_n)
+
+    token_c =  Class.new super_token
+    const_set token_n, token_c
+#    token_c.inspect
+
+    if reading
+      if TkReading2Token[reading]
+        fail TkReading2TokenDuplicateError, token_n, reading
+      end
+      if opts.empty?
+        TkReading2Token[reading] = [token_c]
+      else
+        TkReading2Token[reading] = [token_c].concat(opts)
+      end
+    end
+    TkSymbol2Token[token_n.intern] = token_c
+
+    if token_c <= TkOp
+      token_c.class_eval %{
+        def self.op_name; "#{reading}"; end
+      }
+    end
+  end
+
+  for defs in TokenDefinitions
+    def_token(*defs)
+  end
+
+  NEWLINE_TOKEN = TkNL.new(0,0)
+  NEWLINE_TOKEN.set_text("\n")
+
+end
+
+##
+# Lexical analyzer for Ruby source
+
+class RDoc::RubyLex
+
+  ##
+  # Read an input stream character by character. We allow for unlimited
+  # ungetting of characters just read.
+  #
+  # We simplify the implementation greatly by reading the entire input
+  # into a buffer initially, and then simply traversing it using
+  # pointers.
+  #
+  # We also have to allow for the <i>here document diversion</i>. This
+  # little gem comes about when the lexer encounters a here
+  # document. At this point we effectively need to split the input
+  # stream into two parts: one to read the body of the here document,
+  # the other to read the rest of the input line where the here
+  # document was initially encountered. For example, we might have
+  #
+  #   do_something(<<-A, <<-B)
+  #     stuff
+  #     for
+  #   A
+  #     stuff
+  #     for
+  #   B
+  #
+  # When the lexer encounters the <<A, it reads until the end of the
+  # line, and keeps it around for later. It then reads the body of the
+  # here document.  Once complete, it needs to read the rest of the
+  # original line, but then skip the here document body.
+  #
+
+  class BufferedReader
+
+    attr_reader :line_num
+
+    def initialize(content, options)
+      @options = options
+
+      if /\t/ =~ content
+        tab_width = @options.tab_width
+        content = content.split(/\n/).map do |line|
+          1 while line.gsub!(/\t+/) { ' ' * (tab_width*$&.length - $`.length % tab_width)}  && $~ #`
+          line
+        end .join("\n")
+      end
+      @content   = content
+      @content << "\n" unless @content[-1,1] == "\n"
+      @size      = @content.size
+      @offset    = 0
+      @hwm       = 0
+      @line_num  = 1
+      @read_back_offset = 0
+      @last_newline = 0
+      @newline_pending = false
+    end
+
+    def column
+      @offset - @last_newline
+    end
+
+    def getc
+      return nil if @offset >= @size
+      ch = @content[@offset, 1]
+
+      @offset += 1
+      @hwm = @offset if @hwm < @offset
+
+      if @newline_pending
+        @line_num += 1
+        @last_newline = @offset - 1
+        @newline_pending = false
+      end
+
+      if ch == "\n"
+        @newline_pending = true
+      end
+      ch
+    end
+
+    def getc_already_read
+      getc
+    end
+
+    def ungetc(ch)
+      raise "unget past beginning of file" if @offset <= 0
+      @offset -= 1
+      if @content[@offset] == ?\n
+        @newline_pending = false
+      end
+    end
+
+    def get_read
+      res = @content[@read_back_offset...@offset]
+      @read_back_offset = @offset
+      res
+    end
+
+    def peek(at)
+      pos = @offset + at
+      if pos >= @size
+        nil
+      else
+        @content[pos, 1]
+      end
+    end
+
+    def peek_equal(str)
+      @content[@offset, str.length] == str
+    end
+
+    def divert_read_from(reserve)
+      @content[@offset, 0] = reserve
+      @size      = @content.size
+    end
+  end
+
+  # end of nested class BufferedReader
+
+  extend Exception2MessageMapper
+  def_exception(:AlreadyDefinedToken, "Already defined token(%s)")
+  def_exception(:TkReading2TokenNoKey, "key nothing(key='%s')")
+  def_exception(:TkSymbol2TokenNoKey, "key nothing(key='%s')")
+  def_exception(:TkReading2TokenDuplicateError,
+                "key duplicate(token_n='%s', key='%s')")
+  def_exception(:SyntaxError, "%s")
+
+  include RDoc::RubyToken
+  include IRB
+
+  attr_reader :continue
+  attr_reader :lex_state
+
+  def self.debug?
+    false
+  end
+
+  def initialize(content, options)
+    lex_init
+
+    @options = options
+
+    @reader = BufferedReader.new content, @options
+
+    @exp_line_no = @line_no = 1
+    @base_char_no = 0
+    @indent = 0
+
+    @ltype = nil
+    @quoted = nil
+    @lex_state = EXPR_BEG
+    @space_seen = false
+
+    @continue = false
+    @line = ""
+
+    @skip_space = false
+    @read_auto_clean_up = false
+    @exception_on_syntax_error = true
+  end
+
+  attr_accessor :skip_space
+  attr_accessor :read_auto_clean_up
+  attr_accessor :exception_on_syntax_error
+  attr_reader :indent
+
+  # io functions
+  def line_no
+    @reader.line_num
+  end
+
+  def char_no
+    @reader.column
+  end
+
+  def get_read
+    @reader.get_read
+  end
+
+  def getc
+    @reader.getc
+  end
+
+  def getc_of_rests
+    @reader.getc_already_read
+  end
+
+  def gets
+    c = getc or return
+    l = ""
+    begin
+      l.concat c unless c == "\r"
+      break if c == "\n"
+    end while c = getc
+    l
+  end
+
+
+  def ungetc(c = nil)
+    @reader.ungetc(c)
+  end
+
+  def peek_equal?(str)
+    @reader.peek_equal(str)
+  end
+
+  def peek(i = 0)
+    @reader.peek(i)
+  end
+
+  def lex
+    until (TkNL === (tk = token) or TkEND_OF_SCRIPT === tk) and
+           not @continue or tk.nil?
+    end
+
+    line = get_read
+
+    if line == "" and TkEND_OF_SCRIPT === tk or tk.nil? then
+      nil
+    else
+      line
+    end
+  end
+
+  def token
+    set_token_position(line_no, char_no)
+    begin
+      begin
+        tk = @OP.match(self)
+        @space_seen = TkSPACE === tk
+      rescue SyntaxError
+        abort if @exception_on_syntax_error
+        tk = TkError.new(line_no, char_no)
+      end
+    end while @skip_space and TkSPACE === tk
+    if @read_auto_clean_up
+      get_read
+    end
+#   throw :eof unless tk
+    tk
+  end
+
+  ENINDENT_CLAUSE = [
+    "case", "class", "def", "do", "for", "if",
+    "module", "unless", "until", "while", "begin" #, "when"
+  ]
+  DEINDENT_CLAUSE = ["end" #, "when"
+  ]
+
+  PERCENT_LTYPE = {
+    "q" => "\'",
+    "Q" => "\"",
+    "x" => "\`",
+    "r" => "/",
+    "w" => "]"
+  }
+
+  PERCENT_PAREN = {
+    "{" => "}",
+    "[" => "]",
+    "<" => ">",
+    "(" => ")"
+  }
+
+  Ltype2Token = {
+    "\'" => TkSTRING,
+    "\"" => TkSTRING,
+    "\`" => TkXSTRING,
+    "/" => TkREGEXP,
+    "]" => TkDSTRING
+  }
+  Ltype2Token.default = TkSTRING
+
+  DLtype2Token = {
+    "\"" => TkDSTRING,
+    "\`" => TkDXSTRING,
+    "/" => TkDREGEXP,
+  }
+
+  def lex_init()
+    @OP = IRB::SLex.new
+    @OP.def_rules("\0", "\004", "\032") do |chars, io|
+      Token(TkEND_OF_SCRIPT).set_text(chars)
+    end
+
+    @OP.def_rules(" ", "\t", "\f", "\r", "\13") do |chars, io|
+      @space_seen = TRUE
+      while (ch = getc) =~ /[ \t\f\r\13]/
+        chars << ch
+      end
+      ungetc
+      Token(TkSPACE).set_text(chars)
+    end
+
+    @OP.def_rule("#") do
+      |op, io|
+      identify_comment
+    end
+
+    @OP.def_rule("=begin", proc{@prev_char_no == 0 && peek(0) =~ /\s/}) do
+      |op, io|
+      str = op
+      @ltype = "="
+
+
+      begin
+        line = ""
+        begin
+          ch = getc
+          line << ch
+        end until ch == "\n"
+        str << line
+      end until line =~ /^=end/
+
+      ungetc
+
+      @ltype = nil
+
+      if str =~ /\A=begin\s+rdoc/i
+        str.sub!(/\A=begin.*\n/, '')
+        str.sub!(/^=end.*/m, '')
+        Token(TkCOMMENT).set_text(str)
+      else
+        Token(TkRD_COMMENT)#.set_text(str)
+      end
+    end
+
+    @OP.def_rule("\n") do
+      print "\\n\n" if RDoc::RubyLex.debug?
+      case @lex_state
+      when EXPR_BEG, EXPR_FNAME, EXPR_DOT
+        @continue = TRUE
+      else
+        @continue = FALSE
+        @lex_state = EXPR_BEG
+      end
+      Token(TkNL).set_text("\n")
+    end
+
+    @OP.def_rules("*", "**",
+                  "!", "!=", "!~",
+                  "=", "==", "===",
+                  "=~", "<=>",
+                  "<", "<=",
+                  ">", ">=", ">>") do
+      |op, io|
+      @lex_state = EXPR_BEG
+      Token(op).set_text(op)
+    end
+
+    @OP.def_rules("<<") do
+      |op, io|
+      tk = nil
+      if @lex_state != EXPR_END && @lex_state != EXPR_CLASS &&
+        (@lex_state != EXPR_ARG || @space_seen)
+        c = peek(0)
+        if /[-\w_\"\'\`]/ =~ c
+          tk = identify_here_document
+        end
+      end
+      if !tk
+        @lex_state = EXPR_BEG
+        tk = Token(op).set_text(op)
+      end
+      tk
+    end
+
+    @OP.def_rules("'", '"') do
+      |op, io|
+      identify_string(op)
+    end
+
+    @OP.def_rules("`") do
+      |op, io|
+      if @lex_state == EXPR_FNAME
+        Token(op).set_text(op)
+      else
+        identify_string(op)
+      end
+    end
+
+    @OP.def_rules('?') do
+      |op, io|
+      if @lex_state == EXPR_END
+        @lex_state = EXPR_BEG
+        Token(TkQUESTION).set_text(op)
+      else
+        ch = getc
+        if @lex_state == EXPR_ARG && ch !~ /\s/
+          ungetc
+          @lex_state = EXPR_BEG
+          Token(TkQUESTION).set_text(op)
+        else
+          str = op
+          str << ch
+          if (ch == '\\') #'
+            str << read_escape
+          end
+          @lex_state = EXPR_END
+          Token(TkINTEGER).set_text(str)
+        end
+      end
+    end
+
+    @OP.def_rules("&", "&&", "|", "||") do
+      |op, io|
+      @lex_state = EXPR_BEG
+      Token(op).set_text(op)
+    end
+
+    @OP.def_rules("+=", "-=", "*=", "**=",
+                  "&=", "|=", "^=", "<<=", ">>=", "||=", "&&=") do
+      |op, io|
+      @lex_state = EXPR_BEG
+      op =~ /^(.*)=$/
+      Token(TkOPASGN, $1).set_text(op)
+    end
+
+    @OP.def_rule("+@", proc{@lex_state == EXPR_FNAME}) do |op, io|
+      Token(TkUPLUS).set_text(op)
+    end
+
+    @OP.def_rule("-@", proc{@lex_state == EXPR_FNAME}) do |op, io|
+      Token(TkUMINUS).set_text(op)
+    end
+
+    @OP.def_rules("+", "-") do
+      |op, io|
+      catch(:RET) do
+        if @lex_state == EXPR_ARG
+          if @space_seen and peek(0) =~ /[0-9]/
+            throw :RET, identify_number(op)
+          else
+            @lex_state = EXPR_BEG
+          end
+        elsif @lex_state != EXPR_END and peek(0) =~ /[0-9]/
+          throw :RET, identify_number(op)
+        else
+          @lex_state = EXPR_BEG
+        end
+        Token(op).set_text(op)
+      end
+    end
+
+    @OP.def_rule(".") do
+      @lex_state = EXPR_BEG
+      if peek(0) =~ /[0-9]/
+        ungetc
+        identify_number("")
+      else
+        # for obj.if
+        @lex_state = EXPR_DOT
+        Token(TkDOT).set_text(".")
+      end
+    end
+
+    @OP.def_rules("..", "...") do
+      |op, io|
+      @lex_state = EXPR_BEG
+      Token(op).set_text(op)
+    end
+
+    lex_int2
+  end
+
+  def lex_int2
+    @OP.def_rules("]", "}", ")") do
+      |op, io|
+      @lex_state = EXPR_END
+      @indent -= 1
+      Token(op).set_text(op)
+    end
+
+    @OP.def_rule(":") do
+      if @lex_state == EXPR_END || peek(0) =~ /\s/
+        @lex_state = EXPR_BEG
+        tk = Token(TkCOLON)
+      else
+        @lex_state = EXPR_FNAME
+        tk = Token(TkSYMBEG)
+      end
+      tk.set_text(":")
+    end
+
+    @OP.def_rule("::") do
+      if @lex_state == EXPR_BEG or @lex_state == EXPR_ARG && @space_seen
+        @lex_state = EXPR_BEG
+        tk = Token(TkCOLON3)
+      else
+        @lex_state = EXPR_DOT
+        tk = Token(TkCOLON2)
+      end
+      tk.set_text("::")
+    end
+
+    @OP.def_rule("/") do
+      |op, io|
+      if @lex_state == EXPR_BEG || @lex_state == EXPR_MID
+        identify_string(op)
+      elsif peek(0) == '='
+        getc
+        @lex_state = EXPR_BEG
+        Token(TkOPASGN, :/).set_text("/=") #")
+      elsif @lex_state == EXPR_ARG and @space_seen and peek(0) !~ /\s/
+        identify_string(op)
+      else
+        @lex_state = EXPR_BEG
+        Token("/").set_text(op)
+      end
+    end
+
+    @OP.def_rules("^") do
+      @lex_state = EXPR_BEG
+      Token("^").set_text("^")
+    end
+
+    @OP.def_rules(",", ";") do
+      |op, io|
+      @lex_state = EXPR_BEG
+      Token(op).set_text(op)
+    end
+
+    @OP.def_rule("~") do
+      @lex_state = EXPR_BEG
+      Token("~").set_text("~")
+    end
+
+    @OP.def_rule("~@", proc{@lex_state = EXPR_FNAME}) do
+      @lex_state = EXPR_BEG
+      Token("~").set_text("~@")
+    end
+
+    @OP.def_rule("(") do
+      @indent += 1
+      if @lex_state == EXPR_BEG || @lex_state == EXPR_MID
+        @lex_state = EXPR_BEG
+        tk = Token(TkfLPAREN)
+      else
+        @lex_state = EXPR_BEG
+        tk = Token(TkLPAREN)
+      end
+      tk.set_text("(")
+    end
+
+    @OP.def_rule("[]", proc{@lex_state == EXPR_FNAME}) do
+      Token("[]").set_text("[]")
+    end
+
+    @OP.def_rule("[]=", proc{@lex_state == EXPR_FNAME}) do
+      Token("[]=").set_text("[]=")
+    end
+
+    @OP.def_rule("[") do
+      @indent += 1
+      if @lex_state == EXPR_FNAME
+        t = Token(TkfLBRACK)
+      else
+        if @lex_state == EXPR_BEG || @lex_state == EXPR_MID
+          t = Token(TkLBRACK)
+        elsif @lex_state == EXPR_ARG && @space_seen
+          t = Token(TkLBRACK)
+        else
+          t = Token(TkfLBRACK)
+        end
+        @lex_state = EXPR_BEG
+      end
+      t.set_text("[")
+    end
+
+    @OP.def_rule("{") do
+      @indent += 1
+      if @lex_state != EXPR_END && @lex_state != EXPR_ARG
+        t = Token(TkLBRACE)
+      else
+        t = Token(TkfLBRACE)
+      end
+      @lex_state = EXPR_BEG
+      t.set_text("{")
+    end
+
+    @OP.def_rule('\\') do   #'
+      if getc == "\n"
+        @space_seen = true
+        @continue = true
+        Token(TkSPACE).set_text("\\\n")
+      else
+        ungetc
+        Token("\\").set_text("\\")  #"
+      end
+    end
+
+    @OP.def_rule('%') do
+      |op, io|
+      if @lex_state == EXPR_BEG || @lex_state == EXPR_MID
+        identify_quotation('%')
+      elsif peek(0) == '='
+        getc
+        Token(TkOPASGN, "%").set_text("%=")
+      elsif @lex_state == EXPR_ARG and @space_seen and peek(0) !~ /\s/
+        identify_quotation('%')
+      else
+        @lex_state = EXPR_BEG
+        Token("%").set_text("%")
+      end
+    end
+
+    @OP.def_rule('$') do  #'
+      identify_gvar
+    end
+
+    @OP.def_rule('@') do
+      if peek(0) =~ /[@\w_]/
+        ungetc
+        identify_identifier
+      else
+        Token("@").set_text("@")
+      end
+    end
+
+    @OP.def_rule("__END__", proc{@prev_char_no == 0 && peek(0) =~ /[\r\n]/}) do
+      throw :eof
+    end
+
+    @OP.def_rule("") do
+      |op, io|
+      printf "MATCH: start %s: %s\n", op, io.inspect if RDoc::RubyLex.debug?
+      if peek(0) =~ /[0-9]/
+        t = identify_number("")
+      elsif peek(0) =~ /[\w_]/
+        t = identify_identifier
+      end
+      printf "MATCH: end %s: %s\n", op, io.inspect if RDoc::RubyLex.debug?
+      t
+    end
+  end
+
+  def identify_gvar
+    @lex_state = EXPR_END
+    str = "$"
+
+    tk = case ch = getc
+         when /[~_*$?!@\/\\;,=:<>".]/   #"
+           str << ch
+           Token(TkGVAR, str)
+
+         when "-"
+           str << "-" << getc
+           Token(TkGVAR, str)
+
+         when "&", "`", "'", "+"
+           str << ch
+           Token(TkBACK_REF, str)
+
+         when /[1-9]/
+           str << ch
+           while (ch = getc) =~ /[0-9]/
+             str << ch
+           end
+           ungetc
+           Token(TkNTH_REF)
+         when /\w/
+           ungetc
+           ungetc
+           return identify_identifier
+         else
+           ungetc
+           Token("$")
+         end
+    tk.set_text(str)
+  end
+
+  def identify_identifier
+    token = ""
+    token.concat getc if peek(0) =~ /[$@]/
+    token.concat getc if peek(0) == "@"
+
+    while (ch = getc) =~ /\w|_/
+      print ":", ch, ":" if RDoc::RubyLex.debug?
+      token.concat ch
+    end
+    ungetc
+
+    if ch == "!" or ch == "?"
+      token.concat getc
+    end
+    # fix token
+
+    # $stderr.puts "identifier - #{token}, state = #@lex_state"
+
+    case token
+    when /^\$/
+      return Token(TkGVAR, token).set_text(token)
+    when /^\@/
+      @lex_state = EXPR_END
+      return Token(TkIVAR, token).set_text(token)
+    end
+
+    if @lex_state != EXPR_DOT
+      print token, "\n" if RDoc::RubyLex.debug?
+
+      token_c, *trans = TkReading2Token[token]
+      if token_c
+        # reserved word?
+
+        if (@lex_state != EXPR_BEG &&
+            @lex_state != EXPR_FNAME &&
+            trans[1])
+          # modifiers
+          token_c = TkSymbol2Token[trans[1]]
+          @lex_state = trans[0]
+        else
+          if @lex_state != EXPR_FNAME
+            if ENINDENT_CLAUSE.include?(token)
+              @indent += 1
+            elsif DEINDENT_CLAUSE.include?(token)
+              @indent -= 1
+            end
+            @lex_state = trans[0]
+          else
+            @lex_state = EXPR_END
+          end
+        end
+        return Token(token_c, token).set_text(token)
+      end
+    end
+
+    if @lex_state == EXPR_FNAME
+      @lex_state = EXPR_END
+      if peek(0) == '='
+        token.concat getc
+      end
+    elsif @lex_state == EXPR_BEG || @lex_state == EXPR_DOT
+      @lex_state = EXPR_ARG
+    else
+      @lex_state = EXPR_END
+    end
+
+    if token[0, 1] =~ /[A-Z]/
+      return Token(TkCONSTANT, token).set_text(token)
+    elsif token[token.size - 1, 1] =~ /[!?]/
+      return Token(TkFID, token).set_text(token)
+    else
+      return Token(TkIDENTIFIER, token).set_text(token)
+    end
+  end
+
+  def identify_here_document
+    ch = getc
+    if ch == "-"
+      ch = getc
+      indent = true
+    end
+    if /['"`]/ =~ ch            # '
+      lt = ch
+      quoted = ""
+      while (c = getc) && c != lt
+        quoted.concat c
+      end
+    else
+      lt = '"'
+      quoted = ch.dup
+      while (c = getc) && c =~ /\w/
+        quoted.concat c
+      end
+      ungetc
+    end
+
+    ltback, @ltype = @ltype, lt
+    reserve = ""
+
+    while ch = getc
+      reserve << ch
+      if ch == "\\"    #"
+        ch = getc
+        reserve << ch
+      elsif ch == "\n"
+        break
+      end
+    end
+
+    str = ""
+    while (l = gets)
+      l.chomp!
+      l.strip! if indent
+      break if l == quoted
+      str << l.chomp << "\n"
+    end
+
+    @reader.divert_read_from(reserve)
+
+    @ltype = ltback
+    @lex_state = EXPR_END
+    Token(Ltype2Token[lt], str).set_text(str.dump)
+  end
+
+  def identify_quotation(initial_char)
+    ch = getc
+    if lt = PERCENT_LTYPE[ch]
+      initial_char += ch
+      ch = getc
+    elsif ch =~ /\W/
+      lt = "\""
+    else
+      fail SyntaxError, "unknown type of %string ('#{ch}')"
+    end
+#     if ch !~ /\W/
+#       ungetc
+#       next
+#     end
+    #@ltype = lt
+    @quoted = ch unless @quoted = PERCENT_PAREN[ch]
+    identify_string(lt, @quoted, ch, initial_char)
+  end
+
+  def identify_number(start)
+    str = start.dup
+
+    if start == "+" or start == "-" or start == ""
+      start = getc
+      str << start
+    end
+
+    @lex_state = EXPR_END
+
+    if start == "0"
+      if peek(0) == "x"
+        ch = getc
+        str << ch
+        match = /[0-9a-f_]/
+      else
+        match = /[0-7_]/
+      end
+      while ch = getc
+        if ch !~ match
+          ungetc
+          break
+        else
+          str << ch
+        end
+      end
+      return Token(TkINTEGER).set_text(str)
+    end
+
+    type = TkINTEGER
+    allow_point = TRUE
+    allow_e = TRUE
+    while ch = getc
+      case ch
+      when /[0-9_]/
+        str << ch
+
+      when allow_point && "."
+        type = TkFLOAT
+        if peek(0) !~ /[0-9]/
+          ungetc
+          break
+        end
+        str << ch
+        allow_point = false
+
+      when allow_e && "e", allow_e && "E"
+        str << ch
+        type = TkFLOAT
+        if peek(0) =~ /[+-]/
+          str << getc
+        end
+        allow_e = false
+        allow_point = false
+      else
+        ungetc
+        break
+      end
+    end
+    Token(type).set_text(str)
+  end
+
+  def identify_string(ltype, quoted = ltype, opener=nil, initial_char = nil)
+    @ltype = ltype
+    @quoted = quoted
+    subtype = nil
+
+    str = ""
+    str << initial_char if initial_char
+    str << (opener||quoted)
+
+    nest = 0
+    begin
+      while ch = getc
+        str << ch
+        if @quoted == ch
+          if nest == 0
+            break
+          else
+            nest -= 1
+          end
+        elsif opener == ch
+          nest += 1
+        elsif @ltype != "'" && @ltype != "]" and ch == "#"
+          ch = getc
+          if ch == "{"
+            subtype = true
+            str << ch << skip_inner_expression
+          else
+            ungetc(ch)
+          end
+        elsif ch == '\\' #'
+          str << read_escape
+        end
+      end
+      if @ltype == "/"
+        if peek(0) =~ /i|o|n|e|s/
+          str << getc
+        end
+      end
+      if subtype
+        Token(DLtype2Token[ltype], str)
+      else
+        Token(Ltype2Token[ltype], str)
+      end.set_text(str)
+    ensure
+      @ltype = nil
+      @quoted = nil
+      @lex_state = EXPR_END
+    end
+  end
+
+  def skip_inner_expression
+    res = ""
+    nest = 0
+    while (ch = getc)
+      res << ch
+      if ch == '}'
+        break if nest.zero?
+        nest -= 1
+      elsif ch == '{'
+        nest += 1
+      end
+    end
+    res
+  end
+
+  def identify_comment
+    @ltype = "#"
+    comment = "#"
+    while ch = getc
+      if ch == "\\"
+        ch = getc
+        if ch == "\n"
+          ch = " "
+        else
+          comment << "\\"
+        end
+      else
+        if ch == "\n"
+          @ltype = nil
+          ungetc
+          break
+        end
+      end
+      comment << ch
+    end
+    return Token(TkCOMMENT).set_text(comment)
+  end
+
+  def read_escape
+    res = ""
+    case ch = getc
+    when /[0-7]/
+      ungetc ch
+      3.times do
+        case ch = getc
+        when /[0-7]/
+        when nil
+          break
+        else
+          ungetc
+          break
+        end
+        res << ch
+      end
+
+    when "x"
+      res << ch
+      2.times do
+        case ch = getc
+        when /[0-9a-fA-F]/
+        when nil
+          break
+        else
+          ungetc
+          break
+        end
+        res << ch
+      end
+
+    when "M"
+      res << ch
+      if (ch = getc) != '-'
+        ungetc
+      else
+        res << ch
+        if (ch = getc) == "\\" #"
+          res << ch
+          res << read_escape
+        else
+          res << ch
+        end
+      end
+
+    when "C", "c" #, "^"
+      res << ch
+      if ch == "C" and (ch = getc) != "-"
+        ungetc
+      else
+        res << ch
+        if (ch = getc) == "\\" #"
+          res << ch
+          res << read_escape
+        else
+          res << ch
+        end
+      end
+    else
+      res << ch
+    end
+    res
+  end
+end
+
+##
+# Extracts code elements from a source file returning a TopLevel object
+# containing the constituent file elements.
+#
+# This file is based on rtags
+#
+# RubyParser understands how to document:
+# * classes
+# * modules
+# * methods
+# * constants
+# * aliases
+# * private, public, protected
+# * private_class_function, public_class_function
+# * module_function
+# * attr, attr_reader, attr_writer, attr_accessor
+# * extra accessors given on the command line
+# * metaprogrammed methods
+# * require
+# * include
+#
+# == Method Arguments
+#
+#--
+# NOTE: I don't think this works, needs tests, remove the paragraph following
+# this block when known to work
+#
+# The parser extracts the arguments from the method definition.  You can
+# override this with a custom argument definition using the :args: directive:
+#
+#   ##
+#   # This method tries over and over until it is tired
+#   
+#   def go_go_go(thing_to_try, tries = 10) # :args: thing_to_try
+#     puts thing_to_try
+#     go_go_go thing_to_try, tries - 1
+#   end
+#
+# If you have a more-complex set of overrides you can use the :call-seq:
+# directive:
+#++
+#
+# The parser extracts the arguments from the method definition.  You can
+# override this with a custom argument definition using the :call-seq:
+# directive:
+#
+#   ##
+#   # This method can be called with a range or an offset and length
+#   #
+#   # :call-seq:
+#   #   my_method(Range)
+#   #   my_method(offset, length)
+#   
+#   def my_method(*args)
+#   end
+#
+# The parser extracts +yield+ expressions from method bodies to gather the
+# yielded argument names.  If your method manually calls a block instead of
+# yielding or you want to override the discovered argument names use
+# the :yields: directive:
+#
+#   ##
+#   # My method is awesome
+#   
+#   def my_method(&block) # :yields: happy, times
+#     block.call 1, 2
+#   end
+#
+# == Metaprogrammed Methods
+#
+# To pick up a metaprogrammed method, the parser looks for a comment starting
+# with '##' before an identifier:
+#
+#   ##
+#   # This is a meta-programmed method!
+#   
+#   add_my_method :meta_method, :arg1, :arg2
+#
+# The parser looks at the token after the identifier to determine the name, in
+# this example, :meta_method.  If a name cannot be found, a warning is printed
+# and 'unknown is used.
+#
+# You can force the name of a method using the :method: directive:
+#
+#   ##
+#   # :method: woo_hoo!
+#
+# By default, meta-methods are instance methods.  To indicate that a method is
+# a singleton method instead use the :singleton-method: directive:
+#
+#   ##
+#   # :singleton-method:
+#
+# You can also use the :singleton-method: directive with a name:
+#
+#   ##
+#   # :singleton-method: woo_hoo!
+#
+# == Hidden methods
+#
+# You can provide documentation for methods that don't appear using
+# the :method: and :singleton-method: directives:
+#
+#   ##
+#   # :method: ghost_method
+#   # There is a method here, but you can't see it!
+#   
+#   ##
+#   # this is a comment for a regular method
+#   
+#   def regular_method() end
+#
+# Note that by default, the :method: directive will be ignored if there is a
+# standard rdocable item following it.
+
+class RDoc::Parser::Ruby < RDoc::Parser
+
+  parse_files_matching(/\.rbw?$/)
+
+  include RDoc::RubyToken
+  include RDoc::TokenStream
+
+  NORMAL = "::"
+  SINGLE = "<<"
+
+  def initialize(top_level, file_name, content, options, stats)
+    super
+
+    @size = 0
+    @token_listeners = nil
+    @scanner = RDoc::RubyLex.new content, @options
+    @scanner.exception_on_syntax_error = false
+
+    reset
+  end
+
+  def add_token_listener(obj)
+    @token_listeners ||= []
+    @token_listeners << obj
+  end
+
+  ##
+  # Look for the first comment in a file that isn't a shebang line.
+
+  def collect_first_comment
+    skip_tkspace
+    res = ''
+    first_line = true
+
+    tk = get_tk
+
+    while TkCOMMENT === tk
+      if first_line and tk.text =~ /\A#!/ then
+        skip_tkspace
+        tk = get_tk
+      elsif first_line and tk.text =~ /\A#\s*-\*-/ then
+        first_line = false
+        skip_tkspace
+        tk = get_tk
+      else
+        first_line = false
+        res << tk.text << "\n"
+        tk = get_tk
+
+        if TkNL === tk then
+          skip_tkspace false
+          tk = get_tk
+        end
+      end
+    end
+
+    unget_tk tk
+
+    res
+  end
+
+  def error(msg)
+    msg = make_message msg
+    $stderr.puts msg
+    exit(1)
+  end
+
+  ##
+  # Look for a 'call-seq' in the comment, and override the normal parameter
+  # stuff
+
+  def extract_call_seq(comment, meth)
+    if comment.sub!(/:?call-seq:(.*?)^\s*\#?\s*$/m, '') then
+      seq = $1
+      seq.gsub!(/^\s*\#\s*/, '')
+      meth.call_seq = seq
+    end
+
+    meth
+  end
+
+  def get_bool
+    skip_tkspace
+    tk = get_tk
+    case tk
+    when TkTRUE
+      true
+    when TkFALSE, TkNIL
+      false
+    else
+      unget_tk tk
+      true
+    end
+  end
+
+  ##
+  # Look for the name of a class of module (optionally with a leading :: or
+  # with :: separated named) and return the ultimate name and container
+
+  def get_class_or_module(container)
+    skip_tkspace
+    name_t = get_tk
+
+    # class ::A -> A is in the top level
+    if TkCOLON2 === name_t then
+      name_t = get_tk
+      container = @top_level
+    end
+
+    skip_tkspace(false)
+
+    while TkCOLON2 === peek_tk do
+      prev_container = container
+      container = container.find_module_named(name_t.name)
+      if !container
+#          warn("Couldn't find module #{name_t.name}")
+        container = prev_container.add_module RDoc::NormalModule, name_t.name
+      end
+      get_tk
+      name_t = get_tk
+    end
+    skip_tkspace(false)
+    return [container, name_t]
+  end
+
+  ##
+  # Return a superclass, which can be either a constant of an expression
+
+  def get_class_specification
+    tk = get_tk
+    return "self" if TkSELF === tk
+
+    res = ""
+    while TkCOLON2 === tk or TkCOLON3 === tk or TkCONSTANT === tk do
+      res += tk.text
+      tk = get_tk
+    end
+
+    unget_tk(tk)
+    skip_tkspace(false)
+
+    get_tkread # empty out read buffer
+
+    tk = get_tk
+
+    case tk
+    when TkNL, TkCOMMENT, TkSEMICOLON then
+      unget_tk(tk)
+      return res
+    end
+
+    res += parse_call_parameters(tk)
+    res
+  end
+
+  ##
+  # Parse a constant, which might be qualified by one or more class or module
+  # names
+
+  def get_constant
+    res = ""
+    skip_tkspace(false)
+    tk = get_tk
+
+    while TkCOLON2 === tk or TkCOLON3 === tk or TkCONSTANT === tk do
+      res += tk.text
+      tk = get_tk
+    end
+
+#      if res.empty?
+#        warn("Unexpected token #{tk} in constant")
+#      end
+    unget_tk(tk)
+    res
+  end
+
+  ##
+  # Get a constant that may be surrounded by parens
+
+  def get_constant_with_optional_parens
+    skip_tkspace(false)
+    nest = 0
+    while TkLPAREN === (tk = peek_tk) or TkfLPAREN === tk do
+      get_tk
+      skip_tkspace(true)
+      nest += 1
+    end
+
+    name = get_constant
+
+    while nest > 0
+      skip_tkspace(true)
+      tk = get_tk
+      nest -= 1 if TkRPAREN === tk
+    end
+    name
+  end
+
+  def get_symbol_or_name
+    tk = get_tk
+    case tk
+    when  TkSYMBOL
+      tk.text.sub(/^:/, '')
+    when TkId, TkOp
+      tk.name
+    when TkSTRING
+      tk.text
+    else
+      raise "Name or symbol expected (got #{tk})"
+    end
+  end
+
+  def get_tk
+    tk = nil
+    if @tokens.empty?
+      tk = @scanner.token
+      @read.push @scanner.get_read
+      puts "get_tk1 => #{tk.inspect}" if $TOKEN_DEBUG
+    else
+      @read.push @unget_read.shift
+      tk = @tokens.shift
+      puts "get_tk2 => #{tk.inspect}" if $TOKEN_DEBUG
+    end
+
+    if TkSYMBEG === tk then
+      set_token_position(tk.line_no, tk.char_no)
+      tk1 = get_tk
+      if TkId === tk1 or TkOp === tk1 or TkSTRING === tk1 then
+        if tk1.respond_to?(:name)
+          tk = Token(TkSYMBOL).set_text(":" + tk1.name)
+        else
+          tk = Token(TkSYMBOL).set_text(":" + tk1.text)
+        end
+        # remove the identifier we just read (we're about to
+        # replace it with a symbol)
+        @token_listeners.each do |obj|
+          obj.pop_token
+        end if @token_listeners
+      else
+        warn("':' not followed by identifier or operator")
+        tk = tk1
+      end
+    end
+
+    # inform any listeners of our shiny new token
+    @token_listeners.each do |obj|
+      obj.add_token(tk)
+    end if @token_listeners
+
+    tk
+  end
+
+  def get_tkread
+    read = @read.join("")
+    @read = []
+    read
+  end
+
+  ##
+  # Look for directives in a normal comment block:
+  #
+  #   #-- - don't display comment from this point forward
+  #
+  # This routine modifies it's parameter
+
+  def look_for_directives_in(context, comment)
+    preprocess = RDoc::Markup::PreProcess.new(@file_name,
+                                              @options.rdoc_include)
+
+    preprocess.handle(comment) do |directive, param|
+      case directive
+      when 'enddoc' then
+        throw :enddoc
+      when 'main' then
+        @options.main_page = param
+        ''
+      when 'method', 'singleton-method' then
+        false # ignore
+      when 'section' then
+        context.set_current_section(param, comment)
+        comment.replace ''
+        break
+      when 'startdoc' then
+        context.start_doc
+        context.force_documentation = true
+        ''
+      when 'stopdoc' then
+        context.stop_doc
+        ''
+      when 'title' then
+        @options.title = param
+        ''
+      else
+        warn "Unrecognized directive '#{directive}'"
+        false
+      end
+    end
+
+    remove_private_comments(comment)
+  end
+
+  def make_message(msg)
+    prefix = "\n" + @file_name + ":"
+    if @scanner
+      prefix << "#{@scanner.line_no}:#{@scanner.char_no}: "
+    end
+    return prefix + msg
+  end
+
+  def parse_attr(context, single, tk, comment)
+    args = parse_symbol_arg(1)
+    if args.size > 0
+      name = args[0]
+      rw = "R"
+      skip_tkspace(false)
+      tk = get_tk
+      if TkCOMMA === tk then
+        rw = "RW" if get_bool
+      else
+        unget_tk tk
+      end
+      att = RDoc::Attr.new get_tkread, name, rw, comment
+      read_documentation_modifiers att, RDoc::ATTR_MODIFIERS
+      if att.document_self
+        context.add_attribute(att)
+      end
+    else
+      warn("'attr' ignored - looks like a variable")
+    end
+  end
+
+  def parse_attr_accessor(context, single, tk, comment)
+    args = parse_symbol_arg
+    read = get_tkread
+    rw = "?"
+
+    # If nodoc is given, don't document any of them
+
+    tmp = RDoc::CodeObject.new
+    read_documentation_modifiers tmp, RDoc::ATTR_MODIFIERS
+    return unless tmp.document_self
+
+    case tk.name
+    when "attr_reader"   then rw = "R"
+    when "attr_writer"   then rw = "W"
+    when "attr_accessor" then rw = "RW"
+    else
+      rw = @options.extra_accessor_flags[tk.name]
+      rw = '?' if rw.nil?
+    end
+
+    for name in args
+      att = RDoc::Attr.new get_tkread, name, rw, comment
+      context.add_attribute att
+    end
+  end
+
+  def parse_alias(context, single, tk, comment)
+    skip_tkspace
+    if TkLPAREN === peek_tk then
+      get_tk
+      skip_tkspace
+    end
+    new_name = get_symbol_or_name
+    @scanner.instance_eval{@lex_state = EXPR_FNAME}
+    skip_tkspace
+    if TkCOMMA === peek_tk then
+      get_tk
+      skip_tkspace
+    end
+    old_name = get_symbol_or_name
+
+    al = RDoc::Alias.new get_tkread, old_name, new_name, comment
+    read_documentation_modifiers al, RDoc::ATTR_MODIFIERS
+    if al.document_self
+      context.add_alias(al)
+    end
+  end
+
+  def parse_call_parameters(tk)
+    end_token = case tk
+                when TkLPAREN, TkfLPAREN
+                  TkRPAREN
+                when TkRPAREN
+                  return ""
+                else
+                  TkNL
+                end
+    nest = 0
+
+    loop do
+      puts("Call param: #{tk}, #{@scanner.continue} " +
+        "#{@scanner.lex_state} #{nest}") if $DEBUG_RDOC
+        case tk
+        when TkSEMICOLON
+          break
+        when TkLPAREN, TkfLPAREN
+          nest += 1
+        when end_token
+          if end_token == TkRPAREN
+            nest -= 1
+            break if @scanner.lex_state == EXPR_END and nest <= 0
+          else
+            break unless @scanner.continue
+          end
+        when TkCOMMENT
+          unget_tk(tk)
+          break
+        end
+        tk = get_tk
+    end
+    res = get_tkread.tr("\n", " ").strip
+    res = "" if res == ";"
+    res
+  end
+
+  def parse_class(container, single, tk, comment, &block)
+    progress("c")
+
+    @stats.num_classes += 1
+
+    container, name_t = get_class_or_module(container)
+
+    case name_t
+    when TkCONSTANT
+      name = name_t.name
+      superclass = "Object"
+
+      if TkLT === peek_tk then
+        get_tk
+        skip_tkspace(true)
+        superclass = get_class_specification
+        superclass = "<unknown>" if superclass.empty?
+      end
+
+      if single == SINGLE
+        cls_type = RDoc::SingleClass
+      else
+        cls_type = RDoc::NormalClass
+      end
+
+      cls = container.add_class cls_type, name, superclass
+      read_documentation_modifiers cls, RDoc::CLASS_MODIFIERS
+      cls.record_location(@top_level)
+      parse_statements(cls)
+      cls.comment = comment
+
+    when TkLSHFT
+      case name = get_class_specification
+      when "self", container.name
+        parse_statements(container, SINGLE, &block)
+      else
+        other = RDoc::TopLevel.find_class_named(name)
+        unless other
+          #            other = @top_level.add_class(NormalClass, name, nil)
+          #            other.record_location(@top_level)
+          #            other.comment = comment
+          other = RDoc::NormalClass.new "Dummy", nil
+        end
+        read_documentation_modifiers other, RDoc::CLASS_MODIFIERS
+        parse_statements(other, SINGLE, &block)
+      end
+
+    else
+      warn("Expected class name or '<<'. Got #{name_t.class}: #{name_t.text.inspect}")
+    end
+  end
+
+  def parse_constant(container, single, tk, comment)
+    name = tk.name
+    skip_tkspace(false)
+    eq_tk = get_tk
+
+    unless TkASSIGN === eq_tk then
+      unget_tk(eq_tk)
+      return
+    end
+
+
+    nest = 0
+    get_tkread
+
+    tk = get_tk
+    if TkGT === tk then
+      unget_tk(tk)
+      unget_tk(eq_tk)
+      return
+    end
+
+    loop do
+      puts "Param: %p, %s %s %s" %
+        [tk.text, @scanner.continue, @scanner.lex_state, nest] if $DEBUG_RDOC
+
+        case tk
+        when TkSEMICOLON
+          break
+        when TkLPAREN, TkfLPAREN
+          nest += 1
+        when TkRPAREN
+          nest -= 1
+        when TkCOMMENT
+          if nest <= 0 && @scanner.lex_state == EXPR_END
+            unget_tk(tk)
+            break
+          end
+        when TkNL
+          if (@scanner.lex_state == EXPR_END and nest <= 0) || !@scanner.continue
+            unget_tk(tk)
+            break
+          end
+        end
+        tk = get_tk
+    end
+
+    res = get_tkread.tr("\n", " ").strip
+    res = "" if res == ";"
+
+    con = RDoc::Constant.new name, res, comment
+    read_documentation_modifiers con, RDoc::CONSTANT_MODIFIERS
+
+    if con.document_self
+      container.add_constant(con)
+    end
+  end
+
+  def parse_comment(container, tk, comment)
+    progress(".")
+    @stats.num_methods += 1
+    line_no = tk.line_no
+    column  = tk.char_no
+
+    singleton = !!comment.sub!(/(^# +:?)(singleton-)(method:)/, '\1\3')
+
+    if comment.sub!(/^# +:?method: *(\S*).*?\n/i, '') then
+      name = $1 unless $1.empty?
+    else
+      return nil
+    end
+
+    meth = RDoc::GhostMethod.new get_tkread, name
+    meth.singleton = singleton
+
+    meth.start_collecting_tokens
+    indent = TkSPACE.new 1, 1
+    indent.set_text " " * column
+
+    position_comment = TkCOMMENT.new(line_no, 1, "# File #{@top_level.file_absolute_name}, line #{line_no}")
+    meth.add_tokens [position_comment, NEWLINE_TOKEN, indent]
+
+    meth.params = ''
+
+    extract_call_seq comment, meth
+
+    container.add_method meth if meth.document_self
+
+    meth.comment = comment
+  end
+
+  def parse_include(context, comment)
+    loop do
+      skip_tkspace_comment
+
+      name = get_constant_with_optional_parens
+      context.add_include RDoc::Include.new(name, comment) unless name.empty?
+
+      return unless TkCOMMA === peek_tk
+      get_tk
+    end
+  end
+
+  ##
+  # Parses a meta-programmed method
+
+  def parse_meta_method(container, single, tk, comment)
+    progress(".")
+    @stats.num_methods += 1
+    line_no = tk.line_no
+    column  = tk.char_no
+
+    start_collecting_tokens
+    add_token tk
+    add_token_listener self
+
+    skip_tkspace false
+
+    singleton = !!comment.sub!(/(^# +:?)(singleton-)(method:)/, '\1\3')
+
+    if comment.sub!(/^# +:?method: *(\S*).*?\n/i, '') then
+      name = $1 unless $1.empty?
+    end
+
+    if name.nil? then
+      name_t = get_tk
+      case name_t
+      when TkSYMBOL then
+        name = name_t.text[1..-1]
+      when TkSTRING then
+        name = name_t.text[1..-2]
+      else
+        warn "#{container.top_level.file_relative_name}:#{name_t.line_no} unknown name token #{name_t.inspect} for meta-method"
+        name = 'unknown'
+      end
+    end
+
+    meth = RDoc::MetaMethod.new get_tkread, name
+    meth.singleton = singleton
+
+    remove_token_listener self
+
+    meth.start_collecting_tokens
+    indent = TkSPACE.new 1, 1
+    indent.set_text " " * column
+
+    position_comment = TkCOMMENT.new(line_no, 1, "# File #{@top_level.file_absolute_name}, line #{line_no}")
+    meth.add_tokens [position_comment, NEWLINE_TOKEN, indent]
+    meth.add_tokens @token_stream
+
+    add_token_listener meth
+
+    meth.params = ''
+
+    extract_call_seq comment, meth
+
+    container.add_method meth if meth.document_self
+
+    last_tk = tk
+
+    while tk = get_tk do
+      case tk
+      when TkSEMICOLON then
+        break
+      when TkNL then
+        break unless last_tk and TkCOMMA === last_tk
+      when TkSPACE then
+        # expression continues
+      else
+        last_tk = tk
+      end
+    end
+
+    remove_token_listener meth
+
+    meth.comment = comment
+  end
+
+  ##
+  # Parses a method
+
+  def parse_method(container, single, tk, comment)
+    progress(".")
+    @stats.num_methods += 1
+    line_no = tk.line_no
+    column  = tk.char_no
+
+    start_collecting_tokens
+    add_token(tk)
+    add_token_listener(self)
+
+    @scanner.instance_eval do @lex_state = EXPR_FNAME end
+
+    skip_tkspace(false)
+    name_t = get_tk
+    back_tk = skip_tkspace
+    meth = nil
+    added_container = false
+
+    dot = get_tk
+    if TkDOT === dot or TkCOLON2 === dot then
+      @scanner.instance_eval do @lex_state = EXPR_FNAME end
+      skip_tkspace
+      name_t2 = get_tk
+
+      case name_t
+      when TkSELF then
+        name = name_t2.name
+      when TkCONSTANT then
+        name = name_t2.name
+        prev_container = container
+        container = container.find_module_named(name_t.name)
+        unless container then
+          added_container = true
+          obj = name_t.name.split("::").inject(Object) do |state, item|
+            state.const_get(item)
+          end rescue nil
+
+          type = obj.class == Class ? RDoc::NormalClass : RDoc::NormalModule
+
+          unless [Class, Module].include?(obj.class) then
+            warn("Couldn't find #{name_t.name}. Assuming it's a module")
+          end
+
+          if type == RDoc::NormalClass then
+            container = prev_container.add_class(type, name_t.name, obj.superclass.name)
+          else
+            container = prev_container.add_module(type, name_t.name)
+          end
+
+          container.record_location @top_level
+        end
+      else
+        # warn("Unexpected token '#{name_t2.inspect}'")
+        # break
+        skip_method(container)
+        return
+      end
+
+      meth = RDoc::AnyMethod.new(get_tkread, name)
+      meth.singleton = true
+    else
+      unget_tk dot
+      back_tk.reverse_each do |token|
+        unget_tk token
+      end
+      name = name_t.name
+
+      meth = RDoc::AnyMethod.new get_tkread, name
+      meth.singleton = (single == SINGLE)
+    end
+
+    remove_token_listener self
+
+    meth.start_collecting_tokens
+    indent = TkSPACE.new 1, 1
+    indent.set_text " " * column
+
+    token = TkCOMMENT.new(line_no, 1, "# File #{@top_level.file_absolute_name}, line #{line_no}")
+    meth.add_tokens [token, NEWLINE_TOKEN, indent]
+    meth.add_tokens @token_stream
+
+    add_token_listener meth
+
+    @scanner.instance_eval do @continue = false end
+    parse_method_parameters meth
+
+    if meth.document_self then
+      container.add_method meth
+    elsif added_container then
+      container.document_self = false
+    end
+
+    # Having now read the method parameters and documentation modifiers, we
+    # now know whether we have to rename #initialize to ::new
+
+    if name == "initialize" && !meth.singleton then
+      if meth.dont_rename_initialize then
+        meth.visibility = :protected
+      else
+        meth.singleton = true
+        meth.name = "new"
+        meth.visibility = :public
+      end
+    end
+
+    parse_statements(container, single, meth)
+
+    remove_token_listener(meth)
+
+    extract_call_seq comment, meth
+
+    meth.comment = comment
+  end
+
+  def parse_method_or_yield_parameters(method = nil,
+                                       modifiers = RDoc::METHOD_MODIFIERS)
+    skip_tkspace(false)
+    tk = get_tk
+
+    # Little hack going on here. In the statement
+    #  f = 2*(1+yield)
+    # We see the RPAREN as the next token, so we need
+    # to exit early. This still won't catch all cases
+    # (such as "a = yield + 1"
+    end_token = case tk
+                when TkLPAREN, TkfLPAREN
+                  TkRPAREN
+                when TkRPAREN
+                  return ""
+                else
+                  TkNL
+                end
+    nest = 0
+
+    loop do
+      puts "Param: %p, %s %s %s" %
+        [tk.text, @scanner.continue, @scanner.lex_state, nest] if $DEBUG_RDOC
+        case tk
+        when TkSEMICOLON
+          break
+        when TkLBRACE
+          nest += 1
+        when TkRBRACE
+          # we might have a.each {|i| yield i }
+          unget_tk(tk) if nest.zero?
+          nest -= 1
+          break if nest <= 0
+        when TkLPAREN, TkfLPAREN
+          nest += 1
+        when end_token
+          if end_token == TkRPAREN
+            nest -= 1
+            break if @scanner.lex_state == EXPR_END and nest <= 0
+          else
+            break unless @scanner.continue
+          end
+        when method && method.block_params.nil? && TkCOMMENT
+          unget_tk(tk)
+          read_documentation_modifiers(method, modifiers)
+        end
+      tk = get_tk
+    end
+    res = get_tkread.tr("\n", " ").strip
+    res = "" if res == ";"
+    res
+  end
+
+  ##
+  # Capture the method's parameters. Along the way, look for a comment
+  # containing:
+  #
+  #    # yields: ....
+  #
+  # and add this as the block_params for the method
+
+  def parse_method_parameters(method)
+    res = parse_method_or_yield_parameters(method)
+    res = "(" + res + ")" unless res[0] == ?(
+    method.params = res unless method.params
+    if method.block_params.nil?
+      skip_tkspace(false)
+      read_documentation_modifiers method, RDoc::METHOD_MODIFIERS
+    end
+  end
+
+  def parse_module(container, single, tk, comment)
+    progress("m")
+    @stats.num_modules += 1
+    container, name_t = get_class_or_module(container)
+#      skip_tkspace
+    name = name_t.name
+
+    mod = container.add_module RDoc::NormalModule, name
+    mod.record_location @top_level
+    read_documentation_modifiers mod, RDoc::CLASS_MODIFIERS
+    parse_statements(mod)
+    mod.comment = comment
+  end
+
+  def parse_require(context, comment)
+    skip_tkspace_comment
+    tk = get_tk
+    if TkLPAREN === tk then
+      skip_tkspace_comment
+      tk = get_tk
+    end
+
+    name = nil
+    case tk
+    when TkSTRING
+      name = tk.text
+      #    when TkCONSTANT, TkIDENTIFIER, TkIVAR, TkGVAR
+      #      name = tk.name
+    when TkDSTRING
+      warn "Skipping require of dynamic string: #{tk.text}"
+      #   else
+      #     warn "'require' used as variable"
+    end
+    if name
+      context.add_require RDoc::Require.new(name, comment)
+    else
+      unget_tk(tk)
+    end
+  end
+
+  def parse_statements(container, single = NORMAL, current_method = nil,
+                       comment = '')
+    nest = 1
+    save_visibility = container.visibility
+
+    non_comment_seen = true
+
+    while tk = get_tk do
+      keep_comment = false
+
+      non_comment_seen = true unless TkCOMMENT === tk
+
+      case tk
+      when TkNL then
+        skip_tkspace true # Skip blanks and newlines
+        tk = get_tk
+
+        if TkCOMMENT === tk then
+          if non_comment_seen then
+            # Look for RDoc in a comment about to be thrown away
+            parse_comment container, tk, comment unless comment.empty?
+
+            comment = ''
+            non_comment_seen = false
+          end
+
+          while TkCOMMENT === tk do
+            comment << tk.text << "\n"
+            tk = get_tk          # this is the newline
+            skip_tkspace(false)  # leading spaces
+            tk = get_tk
+          end
+
+          unless comment.empty? then
+            look_for_directives_in container, comment
+
+            if container.done_documenting then
+              container.ongoing_visibility = save_visibility
+            end
+          end
+
+          keep_comment = true
+        else
+          non_comment_seen = true
+        end
+
+        unget_tk tk
+        keep_comment = true
+
+      when TkCLASS then
+        if container.document_children then
+          parse_class container, single, tk, comment
+        else
+          nest += 1
+        end
+
+      when TkMODULE then
+        if container.document_children then
+          parse_module container, single, tk, comment
+        else
+          nest += 1
+        end
+
+      when TkDEF then
+        if container.document_self then
+          parse_method container, single, tk, comment
+        else
+          nest += 1
+        end
+
+      when TkCONSTANT then
+        if container.document_self then
+          parse_constant container, single, tk, comment
+        end
+
+      when TkALIAS then
+        if container.document_self then
+          parse_alias container, single, tk, comment
+        end
+
+      when TkYIELD then
+        if current_method.nil? then
+          warn "Warning: yield outside of method" if container.document_self
+        else
+          parse_yield container, single, tk, current_method
+        end
+
+      # Until and While can have a 'do', which shouldn't increase the nesting.
+      # We can't solve the general case, but we can handle most occurrences by
+      # ignoring a do at the end of a line.
+
+      when  TkUNTIL, TkWHILE then
+        nest += 1
+        puts "Found #{tk.class} in #{container.name}, nest = #{nest}, " +
+             "line #{tk.line_no}" if $DEBUG_RDOC
+        skip_optional_do_after_expression
+
+      # 'for' is trickier
+      when TkFOR then
+        nest += 1
+        puts "Found #{tk.class} in #{container.name}, nest = #{nest}, " +
+             "line #{tk.line_no}" if $DEBUG_RDOC
+        skip_for_variable
+        skip_optional_do_after_expression
+
+      when TkCASE, TkDO, TkIF, TkUNLESS, TkBEGIN then
+        nest += 1
+        puts "Found #{tk.class} in #{container.name}, nest = #{nest}, " +
+             "line #{tk.line_no}" if $DEBUG_RDOC
+
+      when TkIDENTIFIER then
+        if nest == 1 and current_method.nil? then
+          case tk.name
+          when 'private', 'protected', 'public', 'private_class_method',
+               'public_class_method', 'module_function' then
+            parse_visibility container, single, tk
+            keep_comment = true
+          when 'attr' then
+            parse_attr container, single, tk, comment
+          when /^attr_(reader|writer|accessor)$/, @options.extra_accessors then
+            parse_attr_accessor container, single, tk, comment
+          when 'alias_method' then
+            if container.document_self then
+              parse_alias container, single, tk, comment
+            end
+          else
+            if container.document_self and comment =~ /\A#\#$/ then
+              parse_meta_method container, single, tk, comment
+            end
+          end
+        end
+
+        case tk.name
+        when "require" then
+          parse_require container, comment
+        when "include" then
+          parse_include container, comment
+        end
+
+      when TkEND then
+        nest -= 1
+        puts "Found 'end' in #{container.name}, nest = #{nest}, line #{tk.line_no}" if $DEBUG_RDOC
+        puts "Method = #{current_method.name}" if $DEBUG_RDOC and current_method
+        if nest == 0 then
+          read_documentation_modifiers container, RDoc::CLASS_MODIFIERS
+          container.ongoing_visibility = save_visibility
+          return
+        end
+
+      end
+
+      comment = '' unless keep_comment
+
+      begin
+        get_tkread
+        skip_tkspace(false)
+      end while peek_tk == TkNL
+    end
+  end
+
+  def parse_symbol_arg(no = nil)
+    args = []
+    skip_tkspace_comment
+    case tk = get_tk
+    when TkLPAREN
+      loop do
+        skip_tkspace_comment
+        if tk1 = parse_symbol_in_arg
+          args.push tk1
+          break if no and args.size >= no
+        end
+
+        skip_tkspace_comment
+        case tk2 = get_tk
+        when TkRPAREN
+          break
+        when TkCOMMA
+        else
+          warn("unexpected token: '#{tk2.inspect}'") if $DEBUG_RDOC
+          break
+        end
+      end
+    else
+      unget_tk tk
+      if tk = parse_symbol_in_arg
+        args.push tk
+        return args if no and args.size >= no
+      end
+
+      loop do
+        skip_tkspace(false)
+
+        tk1 = get_tk
+        unless TkCOMMA === tk1 then
+          unget_tk tk1
+          break
+        end
+
+        skip_tkspace_comment
+        if tk = parse_symbol_in_arg
+          args.push tk
+          break if no and args.size >= no
+        end
+      end
+    end
+    args
+  end
+
+  def parse_symbol_in_arg
+    case tk = get_tk
+    when TkSYMBOL
+      tk.text.sub(/^:/, '')
+    when TkSTRING
+      eval @read[-1]
+    else
+      warn("Expected symbol or string, got #{tk.inspect}") if $DEBUG_RDOC
+      nil
+    end
+  end
+
+  def parse_toplevel_statements(container)
+    comment = collect_first_comment
+    look_for_directives_in(container, comment)
+    container.comment = comment unless comment.empty?
+    parse_statements container, NORMAL, nil, comment
+  end
+
+  def parse_visibility(container, single, tk)
+    singleton = (single == SINGLE)
+
+    vis_type = tk.name
+
+    vis = case vis_type
+          when 'private'   then :private
+          when 'protected' then :protected
+          when 'public'    then :public
+          when 'private_class_method' then
+            singleton = true
+            :private
+          when 'public_class_method' then
+            singleton = true
+            :public
+          when 'module_function' then
+            singleton = true
+            :public
+          else
+            raise "Invalid visibility: #{tk.name}"
+          end
+
+    skip_tkspace_comment false
+
+    case peek_tk
+      # Ryan Davis suggested the extension to ignore modifiers, because he
+      # often writes
+      #
+      #   protected unless $TESTING
+      #
+    when TkNL, TkUNLESS_MOD, TkIF_MOD, TkSEMICOLON then
+      container.ongoing_visibility = vis
+    else
+      if vis_type == 'module_function' then
+        args = parse_symbol_arg
+        container.set_visibility_for args, :private, false
+
+        module_functions = []
+
+        container.methods_matching args do |m|
+          s_m = m.dup
+          s_m.singleton = true if RDoc::AnyMethod === s_m
+          s_m.visibility = :public
+          module_functions << s_m
+        end
+
+        module_functions.each do |s_m|
+          case s_m
+          when RDoc::AnyMethod then
+            container.add_method s_m
+          when RDoc::Attr then
+            container.add_attribute s_m
+          end
+        end
+      else
+        args = parse_symbol_arg
+        container.set_visibility_for args, vis, singleton
+      end
+    end
+  end
+
+  def parse_yield_parameters
+    parse_method_or_yield_parameters
+  end
+
+  def parse_yield(context, single, tk, method)
+    if method.block_params.nil?
+      get_tkread
+      @scanner.instance_eval{@continue = false}
+      method.block_params = parse_yield_parameters
+    end
+  end
+
+  def peek_read
+    @read.join('')
+  end
+
+  ##
+  # Peek at the next token, but don't remove it from the stream
+
+  def peek_tk
+    unget_tk(tk = get_tk)
+    tk
+  end
+
+  def progress(char)
+    unless @options.quiet
+      @progress.print(char)
+      @progress.flush
+    end
+  end
+
+  ##
+  # Directives are modifier comments that can appear after class, module, or
+  # method names. For example:
+  #
+  #   def fred # :yields: a, b
+  #
+  # or:
+  #
+  #   class MyClass # :nodoc:
+  #
+  # We return the directive name and any parameters as a two element array
+
+  def read_directive(allowed)
+    tk = get_tk
+    puts "directive: #{tk.text.inspect}" if $DEBUG_RDOC
+    result = nil
+    if TkCOMMENT === tk
+      if tk.text =~ /\s*:?(\w+):\s*(.*)/
+        directive = $1.downcase
+        if allowed.include?(directive)
+          result = [directive, $2]
+        end
+      end
+    else
+      unget_tk(tk)
+    end
+    result
+  end
+
+  def read_documentation_modifiers(context, allow)
+    dir = read_directive(allow)
+
+    case dir[0]
+    when "notnew", "not_new", "not-new" then
+      context.dont_rename_initialize = true
+
+    when "nodoc" then
+      context.document_self = false
+      if dir[1].downcase == "all"
+        context.document_children = false
+      end
+
+    when "doc" then
+      context.document_self = true
+      context.force_documentation = true
+
+    when "yield", "yields" then
+      unless context.params.nil?
+        context.params.sub!(/(,|)\s*&\w+/,'') # remove parameter &proc
+      end
+
+      context.block_params = dir[1]
+
+    when "arg", "args" then
+      context.params = dir[1]
+    end if dir
+  end
+
+  def remove_private_comments(comment)
+    comment.gsub!(/^#--.*?^#\+\+/m, '')
+    comment.sub!(/^#--.*/m, '')
+  end
+
+  def remove_token_listener(obj)
+    @token_listeners.delete(obj)
+  end
+
+  def reset
+    @tokens = []
+    @unget_read = []
+    @read = []
+  end
+
+  def scan
+    reset
+
+    catch(:eof) do
+      catch(:enddoc) do
+        begin
+          parse_toplevel_statements(@top_level)
+        rescue Exception => e
+          $stderr.puts <<-EOF
+
+
+RDoc failure in #{@file_name} at or around line #{@scanner.line_no} column
+#{@scanner.char_no}
+
+Before reporting this, could you check that the file you're documenting
+compiles cleanly--RDoc is not a full Ruby parser, and gets confused easily if
+fed invalid programs.
+
+The internal error was:
+
+          EOF
+
+          e.set_backtrace(e.backtrace[0,4])
+          raise
+        end
+      end
+    end
+
+    @top_level
+  end
+
+  ##
+  # while, until, and for have an optional do
+
+  def skip_optional_do_after_expression
+    skip_tkspace(false)
+    tk = get_tk
+    case tk
+    when TkLPAREN, TkfLPAREN
+      end_token = TkRPAREN
+    else
+      end_token = TkNL
+    end
+
+    nest = 0
+    @scanner.instance_eval{@continue = false}
+
+    loop do
+      puts("\nWhile: #{tk.text.inspect}, #{@scanner.continue} " \
+           "#{@scanner.lex_state} #{nest}") if $DEBUG_RDOC
+      case tk
+      when TkSEMICOLON
+        break
+      when TkLPAREN, TkfLPAREN
+        nest += 1
+      when TkDO
+        break if nest.zero?
+      when end_token
+        if end_token == TkRPAREN
+          nest -= 1
+          break if @scanner.lex_state == EXPR_END and nest.zero?
+        else
+          break unless @scanner.continue
+        end
+      end
+      tk = get_tk
+    end
+    skip_tkspace(false)
+
+    get_tk if TkDO === peek_tk
+  end
+
+  ##
+  # skip the var [in] part of a 'for' statement
+
+  def skip_for_variable
+    skip_tkspace(false)
+    tk = get_tk
+    skip_tkspace(false)
+    tk = get_tk
+    unget_tk(tk) unless TkIN === tk
+  end
+
+  def skip_method(container)
+    meth = RDoc::AnyMethod.new "", "anon"
+    parse_method_parameters(meth)
+    parse_statements(container, false, meth)
+  end
+
+  ##
+  # Skip spaces
+
+  def skip_tkspace(skip_nl = true)
+    tokens = []
+
+    while TkSPACE === (tk = get_tk) or (skip_nl and TkNL === tk) do
+      tokens.push tk
+    end
+
+    unget_tk(tk)
+    tokens
+  end
+
+  ##
+  # Skip spaces until a comment is found
+
+  def skip_tkspace_comment(skip_nl = true)
+    loop do
+      skip_tkspace(skip_nl)
+      return unless TkCOMMENT === peek_tk
+      get_tk
+    end
+  end
+
+  def unget_tk(tk)
+    @tokens.unshift tk
+    @unget_read.unshift @read.pop
+
+    # Remove this token from any listeners
+    @token_listeners.each do |obj|
+      obj.pop_token
+    end if @token_listeners
+  end
+
+  def warn(msg)
+    return if @options.quiet
+    msg = make_message msg
+    $stderr.puts msg
+  end
+
+end
+
Index: lib/rdoc/parser/c.rb
===================================================================
--- lib/rdoc/parser/c.rb	(revision 0)
+++ lib/rdoc/parser/c.rb	(revision 18121)
@@ -0,0 +1,666 @@
+require 'rdoc/parser'
+require 'rdoc/known_classes'
+
+##
+# We attempt to parse C extension files. Basically we look for
+# the standard patterns that you find in extensions: <tt>rb_define_class,
+# rb_define_method</tt> and so on. We also try to find the corresponding
+# C source for the methods and extract comments, but if we fail
+# we don't worry too much.
+#
+# The comments associated with a Ruby method are extracted from the C
+# comment block associated with the routine that _implements_ that
+# method, that is to say the method whose name is given in the
+# <tt>rb_define_method</tt> call. For example, you might write:
+#
+#  /*
+#   * Returns a new array that is a one-dimensional flattening of this
+#   * array (recursively). That is, for every element that is an array,
+#   * extract its elements into the new array.
+#   *
+#   *    s = [ 1, 2, 3 ]           #=> [1, 2, 3]
+#   *    t = [ 4, 5, 6, [7, 8] ]   #=> [4, 5, 6, [7, 8]]
+#   *    a = [ s, t, 9, 10 ]       #=> [[1, 2, 3], [4, 5, 6, [7, 8]], 9, 10]
+#   *    a.flatten                 #=> [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
+#   */
+#   static VALUE
+#   rb_ary_flatten(ary)
+#       VALUE ary;
+#   {
+#       ary = rb_obj_dup(ary);
+#       rb_ary_flatten_bang(ary);
+#       return ary;
+#   }
+#
+#   ...
+#
+#   void
+#   Init_Array()
+#   {
+#     ...
+#     rb_define_method(rb_cArray, "flatten", rb_ary_flatten, 0);
+#
+# Here RDoc will determine from the rb_define_method line that there's a
+# method called "flatten" in class Array, and will look for the implementation
+# in the method rb_ary_flatten. It will then use the comment from that
+# method in the HTML output. This method must be in the same source file
+# as the rb_define_method.
+#
+# C classes can be diagrammed (see /tc/dl/ruby/ruby/error.c), and RDoc
+# integrates C and Ruby source into one tree
+#
+# The comment blocks may include special directives:
+#
+# [Document-class: <i>name</i>]
+#   This comment block is documentation for the given class. Use this
+#   when the <tt>Init_xxx</tt> method is not named after the class.
+#
+# [Document-method: <i>name</i>]
+#   This comment documents the named method. Use when RDoc cannot
+#   automatically find the method from it's declaration
+#
+# [call-seq:  <i>text up to an empty line</i>]
+#   Because C source doesn't give descripive names to Ruby-level parameters,
+#   you need to document the calling sequence explicitly
+#
+# In addition, RDoc assumes by default that the C method implementing a
+# Ruby function is in the same source file as the rb_define_method call.
+# If this isn't the case, add the comment:
+#
+#    rb_define_method(....);  // in: filename
+#
+# As an example, we might have an extension that defines multiple classes
+# in its Init_xxx method. We could document them using
+#
+#   /*
+#    * Document-class:  MyClass
+#    *
+#    * Encapsulate the writing and reading of the configuration
+#    * file. ...
+#    */
+#   
+#   /*
+#    * Document-method: read_value
+#    *
+#    * call-seq:
+#    *   cfg.read_value(key)            -> value
+#    *   cfg.read_value(key} { |key| }  -> value
+#    *
+#    * Return the value corresponding to +key+ from the configuration.
+#    * In the second form, if the key isn't found, invoke the
+#    * block and return its value.
+#    */
+
+class RDoc::Parser::C < RDoc::Parser
+
+  parse_files_matching(/\.(?:([CcHh])\1?|c([+xp])\2|y)\z/)
+
+  attr_writer :progress
+
+  @@enclosure_classes = {}
+  @@known_bodies = {}
+
+  ##
+  # Prepare to parse a C file
+
+  def initialize(top_level, file_name, content, options, stats)
+    super
+
+    @known_classes = RDoc::KNOWN_CLASSES.dup
+    @content = handle_tab_width handle_ifdefs_in(@content)
+    @classes = Hash.new
+    @file_dir = File.dirname(@file_name)
+  end
+
+  def do_aliases
+    @content.scan(%r{rb_define_alias\s*\(\s*(\w+),\s*"([^"]+)",\s*"([^"]+)"\s*\)}m) do
+      |var_name, new_name, old_name|
+      @stats.num_methods += 1
+      class_name = @known_classes[var_name] || var_name
+      class_obj  = find_class(var_name, class_name)
+
+      class_obj.add_alias RDoc::Alias.new("", old_name, new_name, "")
+    end
+ end
+
+  def do_classes
+    @content.scan(/(\w+)\s* = \s*rb_define_module\s*\(\s*"(\w+)"\s*\)/mx) do 
+      |var_name, class_name|
+      handle_class_module(var_name, "module", class_name, nil, nil)
+    end
+
+    # The '.' lets us handle SWIG-generated files
+    @content.scan(/([\w\.]+)\s* = \s*rb_define_class\s*
+              \(
+                 \s*"(\w+)",
+                 \s*(\w+)\s*
+              \)/mx) do |var_name, class_name, parent|
+      handle_class_module(var_name, "class", class_name, parent, nil)
+    end
+
+    @content.scan(/(\w+)\s*=\s*boot_defclass\s*\(\s*"(\w+?)",\s*(\w+?)\s*\)/) do
+      |var_name, class_name, parent|
+      parent = nil if parent == "0"
+      handle_class_module(var_name, "class", class_name, parent, nil)
+    end
+
+    @content.scan(/(\w+)\s* = \s*rb_define_module_under\s*
+              \(
+                 \s*(\w+),
+                 \s*"(\w+)"
+              \s*\)/mx) do |var_name, in_module, class_name|
+      handle_class_module(var_name, "module", class_name, nil, in_module)
+    end
+
+    @content.scan(/([\w\.]+)\s* = \s*rb_define_class_under\s*
+              \(
+                 \s*(\w+),
+                 \s*"(\w+)",
+                 \s*(\w+)\s*
+              \s*\)/mx) do |var_name, in_module, class_name, parent|
+      handle_class_module(var_name, "class", class_name, parent, in_module)
+    end
+  end
+
+  def do_constants
+    @content.scan(%r{\Wrb_define_
+                   (
+                      variable |
+                      readonly_variable |
+                      const |
+                      global_const |
+                    )
+               \s*\(
+                 (?:\s*(\w+),)?
+                 \s*"(\w+)",
+                 \s*(.*?)\s*\)\s*;
+                 }xm) do |type, var_name, const_name, definition|
+      var_name = "rb_cObject" if !var_name or var_name == "rb_mKernel"
+      handle_constants(type, var_name, const_name, definition)
+    end
+  end
+
+  ##
+  # Look for includes of the form:
+  #
+  #   rb_include_module(rb_cArray, rb_mEnumerable);
+
+  def do_includes
+    @content.scan(/rb_include_module\s*\(\s*(\w+?),\s*(\w+?)\s*\)/) do |c,m|
+      if cls = @classes[c]
+        m = @known_classes[m] || m
+        cls.add_include RDoc::Include.new(m, "")
+      end
+    end
+  end
+
+  def do_methods
+    @content.scan(%r{rb_define_
+                   (
+                      singleton_method |
+                      method           |
+                      module_function  |
+                      private_method
+                   )
+                   \s*\(\s*([\w\.]+),
+                     \s*"([^"]+)",
+                     \s*(?:RUBY_METHOD_FUNC\(|VALUEFUNC\()?(\w+)\)?,
+                     \s*(-?\w+)\s*\)
+                   (?:;\s*/[*/]\s+in\s+(\w+?\.[cy]))?
+                 }xm) do
+      |type, var_name, meth_name, meth_body, param_count, source_file|
+
+      # Ignore top-object and weird struct.c dynamic stuff
+      next if var_name == "ruby_top_self"
+      next if var_name == "nstr"
+      next if var_name == "envtbl"
+      next if var_name == "argf"   # it'd be nice to handle this one
+
+      var_name = "rb_cObject" if var_name == "rb_mKernel"
+      handle_method(type, var_name, meth_name,
+                    meth_body, param_count, source_file)
+    end
+
+    @content.scan(%r{rb_define_attr\(
+                             \s*([\w\.]+),
+                             \s*"([^"]+)",
+                             \s*(\d+),
+                             \s*(\d+)\s*\);
+                }xm) do |var_name, attr_name, attr_reader, attr_writer|
+      #var_name = "rb_cObject" if var_name == "rb_mKernel"
+      handle_attr(var_name, attr_name,
+                  attr_reader.to_i != 0,
+                  attr_writer.to_i != 0)
+    end
+
+    @content.scan(%r{rb_define_global_function\s*\(
+                             \s*"([^"]+)",
+                             \s*(?:RUBY_METHOD_FUNC\(|VALUEFUNC\()?(\w+)\)?,
+                             \s*(-?\w+)\s*\)
+                (?:;\s*/[*/]\s+in\s+(\w+?\.[cy]))?
+                }xm) do |meth_name, meth_body, param_count, source_file|
+      handle_method("method", "rb_mKernel", meth_name,
+                    meth_body, param_count, source_file)
+    end
+
+    @content.scan(/define_filetest_function\s*\(
+                             \s*"([^"]+)",
+                             \s*(?:RUBY_METHOD_FUNC\(|VALUEFUNC\()?(\w+)\)?,
+                             \s*(-?\w+)\s*\)/xm) do
+      |meth_name, meth_body, param_count|
+
+      handle_method("method", "rb_mFileTest", meth_name, meth_body, param_count)
+      handle_method("singleton_method", "rb_cFile", meth_name, meth_body, param_count)
+    end
+ end
+
+  def find_attr_comment(attr_name)
+    if @content =~ %r{((?>/\*.*?\*/\s+))
+                   rb_define_attr\((?:\s*(\w+),)?\s*"#{attr_name}"\s*,.*?\)\s*;}xmi
+      $1
+    elsif @content =~ %r{Document-attr:\s#{attr_name}\s*?\n((?>.*?\*/))}m
+      $1
+    else
+      ''
+    end
+  end
+
+  ##
+  # Find the C code corresponding to a Ruby method
+
+  def find_body(meth_name, meth_obj, body, quiet = false)
+    case body
+    when %r"((?>/\*.*?\*/\s*))(?:static\s+)?VALUE\s+#{meth_name}
+            \s*(\([^)]*\))\s*\{.*?^\}"xm
+      comment, params = $1, $2
+      body_text = $&
+
+      remove_private_comments(comment) if comment
+
+      # see if we can find the whole body
+
+      re = Regexp.escape(body_text) + '[^(]*^\{.*?^\}'
+      if Regexp.new(re, Regexp::MULTILINE).match(body)
+        body_text = $&
+      end
+
+      # The comment block may have been overridden with a 'Document-method'
+      # block. This happens in the interpreter when multiple methods are
+      # vectored through to the same C method but those methods are logically
+      # distinct (for example Kernel.hash and Kernel.object_id share the same
+      # implementation
+
+      override_comment = find_override_comment(meth_obj.name)
+      comment = override_comment if override_comment
+
+      find_modifiers(comment, meth_obj) if comment
+
+#        meth_obj.params = params
+      meth_obj.start_collecting_tokens
+      meth_obj.add_token(RDoc::RubyToken::Token.new(1,1).set_text(body_text))
+      meth_obj.comment = mangle_comment(comment)
+    when %r{((?>/\*.*?\*/\s*))^\s*\#\s*define\s+#{meth_name}\s+(\w+)}m
+      comment = $1
+      find_body($2, meth_obj, body, true)
+      find_modifiers(comment, meth_obj)
+      meth_obj.comment = mangle_comment(comment) + meth_obj.comment
+    when %r{^\s*\#\s*define\s+#{meth_name}\s+(\w+)}m
+      unless find_body($1, meth_obj, body, true)
+        warn "No definition for #{meth_name}" unless quiet
+        return false
+      end
+    else
+
+      # No body, but might still have an override comment
+      comment = find_override_comment(meth_obj.name)
+
+      if comment
+        find_modifiers(comment, meth_obj)
+        meth_obj.comment = mangle_comment(comment)
+      else
+        warn "No definition for #{meth_name}" unless quiet
+        return false
+      end
+    end
+    true
+  end
+
+  def find_class(raw_name, name)
+    unless @classes[raw_name]
+      if raw_name =~ /^rb_m/
+        container = @top_level.add_module RDoc::NormalModule, name
+      else
+        container = @top_level.add_class RDoc::NormalClass, name, nil
+      end
+
+      container.record_location @top_level
+      @classes[raw_name] = container
+    end
+    @classes[raw_name]
+  end
+
+  ##
+  # Look for class or module documentation above Init_+class_name+(void),
+  # in a Document-class +class_name+ (or module) comment or above an
+  # rb_define_class (or module).  If a comment is supplied above a matching
+  # Init_ and a rb_define_class the Init_ comment is used.
+  #
+  #   /*
+  #    * This is a comment for Foo
+  #    */
+  #   Init_Foo(void) {
+  #       VALUE cFoo = rb_define_class("Foo", rb_cObject);
+  #   }
+  #
+  #   /*
+  #    * Document-class: Foo
+  #    * This is a comment for Foo
+  #    */
+  #   Init_foo(void) {
+  #       VALUE cFoo = rb_define_class("Foo", rb_cObject);
+  #   }
+  #
+  #   /*
+  #    * This is a comment for Foo
+  #    */
+  #   VALUE cFoo = rb_define_class("Foo", rb_cObject);
+
+  def find_class_comment(class_name, class_meth)
+    comment = nil
+    if @content =~ %r{((?>/\*.*?\*/\s+))
+                   (static\s+)?void\s+Init_#{class_name}\s*(?:_\(\s*)?\(\s*(?:void\s*)\)}xmi
+      comment = $1
+    elsif @content =~ %r{Document-(class|module):\s#{class_name}\s*?\n((?>.*?\*/))}m
+      comment = $2
+    else
+      if @content =~ /rb_define_(class|module)/m then
+        class_name = class_name.split("::").last
+        comments = []
+        @content.split(/(\/\*.*?\*\/)\s*?\n/m).each_with_index do |chunk, index|
+          comments[index] = chunk
+          if chunk =~ /rb_define_(class|module).*?"(#{class_name})"/m then
+            comment = comments[index-1]
+            break
+          end
+        end
+      end
+    end
+    class_meth.comment = mangle_comment(comment) if comment
+  end
+
+  ##
+  # Finds a comment matching +type+ and +const_name+ either above the
+  # comment or in the matching Document- section.
+
+  def find_const_comment(type, const_name)
+    if @content =~ %r{((?>^\s*/\*.*?\*/\s+))
+                   rb_define_#{type}\((?:\s*(\w+),)?\s*"#{const_name}"\s*,.*?\)\s*;}xmi
+      $1
+    elsif @content =~ %r{Document-(?:const|global|variable):\s#{const_name}\s*?\n((?>.*?\*/))}m
+      $1
+    else
+      ''
+    end
+  end
+
+  ##
+  # If the comment block contains a section that looks like:
+  #
+  #    call-seq:
+  #        Array.new
+  #        Array.new(10)
+  #
+  # use it for the parameters.
+
+  def find_modifiers(comment, meth_obj)
+    if comment.sub!(/:nodoc:\s*^\s*\*?\s*$/m, '') or
+       comment.sub!(/\A\/\*\s*:nodoc:\s*\*\/\Z/, '')
+      meth_obj.document_self = false
+    end
+    if comment.sub!(/call-seq:(.*?)^\s*\*?\s*$/m, '') or
+       comment.sub!(/\A\/\*\s*call-seq:(.*?)\*\/\Z/, '')
+      seq = $1
+      seq.gsub!(/^\s*\*\s*/, '')
+      meth_obj.call_seq = seq
+    end
+  end
+
+  def find_override_comment(meth_name)
+    name = Regexp.escape(meth_name)
+    if @content =~ %r{Document-method:\s#{name}\s*?\n((?>.*?\*/))}m
+      $1
+    end
+  end
+
+  def handle_attr(var_name, attr_name, reader, writer)
+    rw = ''
+    if reader
+      #@stats.num_methods += 1
+      rw << 'R'
+    end
+    if writer
+      #@stats.num_methods += 1
+      rw << 'W'
+    end
+
+    class_name = @known_classes[var_name]
+
+    return unless class_name
+
+    class_obj  = find_class(var_name, class_name)
+
+    if class_obj
+      comment = find_attr_comment(attr_name)
+      unless comment.empty?
+        comment = mangle_comment(comment)
+      end
+      att = RDoc::Attr.new '', attr_name, rw, comment
+      class_obj.add_attribute(att)
+    end
+  end
+
+  def handle_class_module(var_name, class_mod, class_name, parent, in_module)
+    progress(class_mod[0, 1])
+
+    parent_name = @known_classes[parent] || parent
+
+    if in_module
+      enclosure = @classes[in_module] || @@enclosure_classes[in_module]
+      unless enclosure
+        if enclosure = @known_classes[in_module]
+          handle_class_module(in_module, (/^rb_m/ =~ in_module ? "module" : "class"),
+                              enclosure, nil, nil)
+          enclosure = @classes[in_module]
+        end
+      end
+      unless enclosure
+        warn("Enclosing class/module '#{in_module}' for " +
+              "#{class_mod} #{class_name} not known")
+        return
+      end
+    else
+      enclosure = @top_level
+    end
+
+    if class_mod == "class"
+      cm = enclosure.add_class RDoc::NormalClass, class_name, parent_name
+      @stats.num_classes += 1
+    else
+      cm = enclosure.add_module RDoc::NormalModule, class_name
+      @stats.num_modules += 1
+    end
+    cm.record_location(enclosure.toplevel)
+
+    find_class_comment(cm.full_name, cm)
+    @classes[var_name] = cm
+    @@enclosure_classes[var_name] = cm
+    @known_classes[var_name] = cm.full_name
+  end
+
+  ##
+  # Adds constant comments.  By providing some_value: at the start ofthe
+  # comment you can override the C value of the comment to give a friendly
+  # definition.
+  #
+  #   /* 300: The perfect score in bowling */
+  #   rb_define_const(cFoo, "PERFECT", INT2FIX(300);
+  #
+  # Will override +INT2FIX(300)+ with the value +300+ in the output RDoc.
+  # Values may include quotes and escaped colons (\:).
+
+  def handle_constants(type, var_name, const_name, definition)
+    #@stats.num_constants += 1
+    class_name = @known_classes[var_name]
+
+    return unless class_name
+
+    class_obj  = find_class(var_name, class_name)
+
+    unless class_obj
+      warn("Enclosing class/module '#{const_name}' for not known")
+      return
+    end
+
+    comment = find_const_comment(type, const_name)
+
+    # In the case of rb_define_const, the definition and comment are in
+    # "/* definition: comment */" form.  The literal ':' and '\' characters
+    # can be escaped with a backslash.
+    if type.downcase == 'const' then
+       elements = mangle_comment(comment).split(':')
+       if elements.nil? or elements.empty? then
+          con = RDoc::Constant.new(const_name, definition,
+                                   mangle_comment(comment))
+       else
+          new_definition = elements[0..-2].join(':')
+          if new_definition.empty? then # Default to literal C definition
+             new_definition = definition
+          else
+             new_definition.gsub!("\:", ":")
+             new_definition.gsub!("\\", '\\')
+          end
+          new_definition.sub!(/\A(\s+)/, '')
+          new_comment = $1.nil? ? elements.last : "#{$1}#{elements.last.lstrip}"
+          con = RDoc::Constant.new(const_name, new_definition,
+                                   mangle_comment(new_comment))
+       end
+    else
+       con = RDoc::Constant.new const_name, definition, mangle_comment(comment)
+    end
+
+    class_obj.add_constant(con)
+  end
+
+  ##
+  # Removes #ifdefs that would otherwise confuse us
+
+  def handle_ifdefs_in(body)
+    body.gsub(/^#ifdef HAVE_PROTOTYPES.*?#else.*?\n(.*?)#endif.*?\n/m, '\1')
+  end
+
+  def handle_method(type, var_name, meth_name, meth_body, param_count,
+                    source_file = nil)
+    progress(".")
+
+    @stats.num_methods += 1
+    class_name = @known_classes[var_name]
+
+    return unless class_name
+
+    class_obj  = find_class(var_name, class_name)
+
+    if class_obj
+      if meth_name == "initialize"
+        meth_name = "new"
+        type = "singleton_method"
+      end
+      meth_obj = RDoc::AnyMethod.new("", meth_name)
+      meth_obj.singleton =
+  %w{singleton_method module_function}.include?(type)
+
+      p_count = (Integer(param_count) rescue -1)
+
+      if p_count < 0
+        meth_obj.params = "(...)"
+      elsif p_count == 0
+        meth_obj.params = "()"
+      else
+        meth_obj.params = "(" + (1..p_count).map{|i| "p#{i}"}.join(", ") + ")"
+      end
+
+      if source_file
+        file_name = File.join(@file_dir, source_file)
+        body = (@@known_bodies[source_file] ||= File.read(file_name))
+      else
+        body = @content
+      end
+      if find_body(meth_body, meth_obj, body) and meth_obj.document_self
+        class_obj.add_method(meth_obj)
+      end
+    end
+  end
+
+  def handle_tab_width(body)
+    if /\t/ =~ body
+      tab_width = @options.tab_width
+      body.split(/\n/).map do |line|
+        1 while line.gsub!(/\t+/) { ' ' * (tab_width*$&.length - $`.length % tab_width)}  && $~ #`
+        line
+      end .join("\n")
+    else
+      body
+    end
+  end
+
+  ##
+  # Remove the /*'s and leading asterisks from C comments
+
+  def mangle_comment(comment)
+    comment.sub!(%r{/\*+}) { " " * $&.length }
+    comment.sub!(%r{\*+/}) { " " * $&.length }
+    comment.gsub!(/^[ \t]*\*/m) { " " * $&.length }
+    comment
+  end
+
+  def progress(char)
+    unless @options.quiet
+      @progress.print(char)
+      @progress.flush
+    end
+  end
+
+  ##
+  # Removes lines that are commented out that might otherwise get picked up
+  # when scanning for classes and methods
+
+  def remove_commented_out_lines
+    @content.gsub!(%r{//.*rb_define_}, '//')
+  end
+
+  def remove_private_comments(comment)
+     comment.gsub!(/\/?\*--(.*?)\/?\*\+\+/m, '')
+     comment.sub!(/\/?\*--.*/m, '')
+  end
+
+  ##
+  # Extract the classes/modules and methods from a C file and return the
+  # corresponding top-level object
+
+  def scan
+    remove_commented_out_lines
+    do_classes
+    do_constants
+    do_methods
+    do_includes
+    do_aliases
+    @top_level
+  end
+
+  def warn(msg)
+    $stderr.puts
+    $stderr.puts msg
+    $stderr.flush
+  end
+
+end
+
Index: lib/rdoc/parser/f95.rb
===================================================================
--- lib/rdoc/parser/f95.rb	(revision 0)
+++ lib/rdoc/parser/f95.rb	(revision 18121)
@@ -0,0 +1,1837 @@
+require 'rdoc/parser'
+
+##
+# = Fortran95 RDoc Parser
+#
+# == Overview
+#
+# This parser parses Fortran95 files with suffixes "f90", "F90", "f95" and
+# "F95". Fortran95 files are expected to be conformed to Fortran95 standards.
+#
+# == Rules
+#
+# Fundamental rules are same as that of the Ruby parser.  But comment markers
+# are '!' not '#'.
+#
+# === Correspondence between RDoc documentation and Fortran95 programs
+#
+# F95 parses main programs, modules, subroutines, functions, derived-types,
+# public variables, public constants, defined operators and defined
+# assignments.  These components are described in items of RDoc documentation,
+# as follows.
+#
+# Files :: Files (same as Ruby)
+# Classes:: Modules
+# Methods:: Subroutines, functions, variables, constants, derived-types,
+#           defined operators, defined assignments
+# Required files:: Files in which imported modules, external subroutines and
+#                  external functions are defined.
+# Included Modules:: List of imported modules
+# Attributes:: List of derived-types, List of imported modules all of whose
+#              components are published again
+#
+# Components listed in 'Methods' (subroutines, functions, ...) defined in
+# modules are described in the item of 'Classes'.  On the other hand,
+# components defined in main programs or as external procedures are described
+# in the item of 'Files'.
+#
+# === Components parsed by default
+#
+# By default, documentation on public components (subroutines, functions,
+# variables, constants, derived-types, defined operators, defined assignments)
+# are generated.
+#
+# With "--all" option, documentation on all components are generated (almost
+# same as the Ruby parser).
+#
+# === Information parsed automatically
+#
+# The following information is automatically parsed.
+#
+# * Types of arguments
+# * Types of variables and constants
+# * Types of variables in the derived types, and initial values
+# * NAMELISTs and types of variables in them, and initial values
+#
+# Aliases by interface statement are described in the item of 'Methods'.
+#
+# Components which are imported from other modules and published again are
+# described in the item of 'Methods'.
+#
+# === Format of comment blocks
+#
+# Comment blocks should be written as follows.
+#
+# Comment blocks are considered to be ended when the line without '!' appears.
+#
+# The indentation is not necessary.
+#
+#   ! (Top of file)
+#   !
+#   ! Comment blocks for the files.
+#   !
+#   !--
+#   ! The comment described in the part enclosed by
+#   ! "!--" and "!++" is ignored.
+#   !++
+#   !
+#   module hogehoge
+#     !
+#     ! Comment blocks for the modules (or the programs).
+#     !
+#   
+#     private
+#   
+#     logical            :: a     ! a private variable
+#     real, public       :: b     ! a public variable
+#     integer, parameter :: c = 0 ! a public constant
+#   
+#     public :: c
+#     public :: MULTI_ARRAY
+#     public :: hoge, foo
+#   
+#     type MULTI_ARRAY
+#       !
+#       ! Comment blocks for the derived-types.
+#       !
+#       real, pointer :: var(:) =>null() ! Comments block for the variables.
+#       integer       :: num = 0
+#     end type MULTI_ARRAY
+#   
+#   contains
+#   
+#     subroutine hoge( in,   &   ! Comment blocks between continuation lines are ignored.
+#         &            out )
+#       !
+#       ! Comment blocks for the subroutines or functions
+#       !
+#       character(*),intent(in):: in ! Comment blocks for the arguments.
+#       character(*),intent(out),allocatable,target  :: in
+#                                    ! Comment blocks can be
+#                                    ! written under Fortran statements.
+#   
+#       character(32) :: file ! This comment parsed as a variable in below NAMELIST.
+#       integer       :: id
+#   
+#       namelist /varinfo_nml/ file, id
+#               !
+#               ! Comment blocks for the NAMELISTs.
+#               ! Information about variables are described above.
+#               !
+#   
+#     ....
+#   
+#     end subroutine hoge
+#   
+#     integer function foo( in )
+#       !
+#       ! This part is considered as comment block.
+#   
+#       ! Comment blocks under blank lines are ignored.
+#       !
+#       integer, intent(in):: inA ! This part is considered as comment block.
+#   
+#                                 ! This part is ignored.
+#   
+#     end function foo
+#   
+#     subroutine hide( in,   &
+#       &              out )      !:nodoc:
+#       !
+#       ! If "!:nodoc:" is described at end-of-line in subroutine
+#       ! statement as above, the subroutine is ignored.
+#       ! This assignment can be used to modules, subroutines,
+#       ! functions, variables, constants, derived-types,
+#       ! defined operators, defined assignments,
+#       ! list of imported modules ("use" statement).
+#       !
+#   
+#     ....
+#   
+#     end subroutine hide
+#   
+#   end module hogehoge
+
+class RDoc::Parser::F95 < RDoc::Parser
+
+  parse_files_matching(/\.((f|F)9(0|5)|F)$/)
+
+  class Token
+
+    NO_TEXT = "??".freeze
+
+    def initialize(line_no, char_no)
+      @line_no = line_no
+      @char_no = char_no
+      @text    = NO_TEXT
+    end
+    # Because we're used in contexts that expect to return a token,
+    # we set the text string and then return ourselves
+    def set_text(text)
+      @text = text
+      self
+    end
+
+    attr_reader :line_no, :char_no, :text
+
+  end
+
+  @@external_aliases = []
+  @@public_methods   = []
+
+  ##
+  # "false":: Comments are below source code
+  # "true" :: Comments are upper source code
+
+  COMMENTS_ARE_UPPER  = false
+
+  ##
+  # Internal alias message
+
+  INTERNAL_ALIAS_MES = "Alias for"
+
+  ##
+  # External alias message
+
+  EXTERNAL_ALIAS_MES = "The entity is"
+
+  ##
+  # Define code constructs
+
+  def scan
+    # remove private comment
+    remaining_code = remove_private_comments(@content)
+
+    # continuation lines are united to one line
+    remaining_code = united_to_one_line(remaining_code)
+
+    # semicolons are replaced to line feed
+    remaining_code = semicolon_to_linefeed(remaining_code)
+
+    # collect comment for file entity
+    whole_comment, remaining_code = collect_first_comment(remaining_code)
+    @top_level.comment = whole_comment
+
+    # String "remaining_code" is converted to Array "remaining_lines"
+    remaining_lines = remaining_code.split("\n")
+
+    # "module" or "program" parts are parsed (new)
+    #
+    level_depth = 0
+    block_searching_flag = nil
+    block_searching_lines = []
+    pre_comment = []
+    module_program_trailing = ""
+    module_program_name = ""
+    other_block_level_depth = 0
+    other_block_searching_flag = nil
+    remaining_lines.collect!{|line|
+      if !block_searching_flag && !other_block_searching_flag
+        if line =~ /^\s*?module\s+(\w+)\s*?(!.*?)?$/i
+          block_searching_flag = :module
+          block_searching_lines << line
+          module_program_name = $1
+          module_program_trailing = find_comments($2)
+          next false
+        elsif line =~ /^\s*?program\s+(\w+)\s*?(!.*?)?$/i ||
+               line =~ /^\s*?\w/ && !block_start?(line)
+          block_searching_flag = :program
+          block_searching_lines << line
+          module_program_name = $1 || ""
+          module_program_trailing = find_comments($2)
+          next false
+
+        elsif block_start?(line)
+          other_block_searching_flag = true
+          next line
+
+        elsif line =~ /^\s*?!\s?(.*)/
+          pre_comment << line
+          next line
+        else
+          pre_comment = []
+          next line
+        end
+      elsif other_block_searching_flag
+        other_block_level_depth += 1 if block_start?(line)
+        other_block_level_depth -= 1 if block_end?(line)
+        if other_block_level_depth < 0
+          other_block_level_depth = 0
+          other_block_searching_flag = nil
+        end
+        next line
+      end
+
+      block_searching_lines << line
+      level_depth += 1 if block_start?(line)
+      level_depth -= 1 if block_end?(line)
+      if level_depth >= 0
+        next false
+      end
+
+      # "module_program_code" is formatted.
+      # ":nodoc:" flag is checked.
+      #
+      module_program_code = block_searching_lines.join("\n")
+      module_program_code = remove_empty_head_lines(module_program_code)
+      if module_program_trailing =~ /^:nodoc:/
+        # next loop to search next block
+        level_depth = 0
+        block_searching_flag = false
+        block_searching_lines = []
+        pre_comment = []
+        next false
+      end
+
+      # NormalClass is created, and added to @top_level
+      #
+      if block_searching_flag == :module
+        module_name = module_program_name
+        module_code = module_program_code
+        module_trailing = module_program_trailing
+        progress "m"
+        @stats.num_modules += 1
+        f9x_module = @top_level.add_module NormalClass, module_name
+        f9x_module.record_location @top_level
+
+        f9x_comment = COMMENTS_ARE_UPPER ? 
+          find_comments(pre_comment.join("\n"))  + "\n" + module_trailing :
+            module_trailing + "\n" + find_comments(module_code.sub(/^.*$\n/i, ''))
+        f9x_module.comment = f9x_comment
+        parse_program_or_module(f9x_module, module_code)
+
+        TopLevel.all_files.each do |name, toplevel|
+          if toplevel.include_includes?(module_name, @options.ignore_case)
+            if !toplevel.include_requires?(@file_name, @options.ignore_case)
+              toplevel.add_require(Require.new(@file_name, ""))
+            end
+          end
+          toplevel.each_classmodule{|m|
+            if m.include_includes?(module_name, @options.ignore_case)
+              if !m.include_requires?(@file_name, @options.ignore_case)
+                m.add_require(Require.new(@file_name, ""))
+              end
+            end
+          }
+        end
+      elsif block_searching_flag == :program
+        program_name = module_program_name
+        program_code = module_program_code
+        program_trailing = module_program_trailing
+        progress "p"
+        program_comment = COMMENTS_ARE_UPPER ? 
+          find_comments(pre_comment.join("\n")) + "\n" + program_trailing : 
+            program_trailing + "\n" + find_comments(program_code.sub(/^.*$\n/i, ''))
+        program_comment = "\n\n= <i>Program</i> <tt>#{program_name}</tt>\n\n" \
+                          + program_comment
+        @top_level.comment << program_comment
+        parse_program_or_module(@top_level, program_code, :private)
+      end
+
+      # next loop to search next block
+      level_depth = 0
+      block_searching_flag = false
+      block_searching_lines = []
+      pre_comment = []
+      next false
+    }
+
+    remaining_lines.delete_if{ |line|
+      line == false
+    }
+
+    # External subprograms and functions are parsed
+    #
+    parse_program_or_module(@top_level, remaining_lines.join("\n"),
+                            :public, true)
+
+    @top_level
+  end  # End of scan
+
+  private
+
+  def parse_program_or_module(container, code,
+                              visibility=:public, external=nil)
+    return unless container
+    return unless code
+    remaining_lines = code.split("\n")
+    remaining_code = "#{code}"
+
+    #
+    # Parse variables before "contains" in module
+    #
+    level_depth = 0
+    before_contains_lines = []
+    before_contains_code = nil
+    before_contains_flag = nil
+    remaining_lines.each{ |line|
+      if !before_contains_flag
+        if line =~ /^\s*?module\s+\w+\s*?(!.*?)?$/i
+          before_contains_flag = true
+        end
+      else
+        break if line =~ /^\s*?contains\s*?(!.*?)?$/i
+        level_depth += 1 if block_start?(line)
+        level_depth -= 1 if block_end?(line)
+        break if level_depth < 0
+        before_contains_lines << line
+      end
+    }
+    before_contains_code = before_contains_lines.join("\n")
+    if before_contains_code
+      before_contains_code.gsub!(/^\s*?interface\s+.*?\s+end\s+interface.*?$/im, "")
+      before_contains_code.gsub!(/^\s*?type[\s\,]+.*?\s+end\s+type.*?$/im, "")
+    end
+
+    #
+    # Parse global "use"
+    #
+    use_check_code = "#{before_contains_code}"
+    cascaded_modules_list = []
+    while use_check_code =~ /^\s*?use\s+(\w+)(.*?)(!.*?)?$/i
+      use_check_code = $~.pre_match
+      use_check_code << $~.post_match
+      used_mod_name = $1.strip.chomp
+      used_list = $2 || ""
+      used_trailing = $3 || ""
+      next if used_trailing =~ /!:nodoc:/
+      if !container.include_includes?(used_mod_name, @options.ignore_case)
+        progress "."
+        container.add_include Include.new(used_mod_name, "")
+      end
+      if ! (used_list =~ /\,\s*?only\s*?:/i )
+        cascaded_modules_list << "\#" + used_mod_name
+      end
+    end
+
+    #
+    # Parse public and private, and store information.
+    # This information is used when "add_method" and
+    # "set_visibility_for" are called.
+    #
+    visibility_default, visibility_info = 
+              parse_visibility(remaining_lines.join("\n"), visibility, container)
+    @@public_methods.concat visibility_info
+    if visibility_default == :public
+      if !cascaded_modules_list.empty?
+        cascaded_modules = 
+          Attr.new("Cascaded Modules",
+                   "Imported modules all of whose components are published again",
+                   "",
+                   cascaded_modules_list.join(", "))
+        container.add_attribute(cascaded_modules)
+      end
+    end
+
+    #
+    # Check rename elements
+    #
+    use_check_code = "#{before_contains_code}"
+    while use_check_code =~ /^\s*?use\s+(\w+)\s*?\,(.+)$/i
+      use_check_code = $~.pre_match
+      use_check_code << $~.post_match
+      used_mod_name = $1.strip.chomp
+      used_elements = $2.sub(/\s*?only\s*?:\s*?/i, '')
+      used_elements.split(",").each{ |used|
+        if /\s*?(\w+)\s*?=>\s*?(\w+)\s*?/ =~ used
+          local = $1
+          org = $2
+          @@public_methods.collect!{ |pub_meth|
+            if local == pub_meth["name"] ||
+                local.upcase == pub_meth["name"].upcase &&
+                @options.ignore_case
+              pub_meth["name"] = org
+              pub_meth["local_name"] = local
+            end
+            pub_meth
+          }
+        end
+      }
+    end
+
+    #
+    # Parse private "use"
+    #
+    use_check_code = remaining_lines.join("\n")
+    while use_check_code =~ /^\s*?use\s+(\w+)(.*?)(!.*?)?$/i
+      use_check_code = $~.pre_match
+      use_check_code << $~.post_match
+      used_mod_name = $1.strip.chomp
+      used_trailing = $3 || ""
+      next if used_trailing =~ /!:nodoc:/
+      if !container.include_includes?(used_mod_name, @options.ignore_case)
+        progress "."
+        container.add_include Include.new(used_mod_name, "")
+      end
+    end
+
+    container.each_includes{ |inc|
+      TopLevel.all_files.each do |name, toplevel|
+        indicated_mod = toplevel.find_symbol(inc.name,
+                                             nil, @options.ignore_case)
+        if indicated_mod
+          indicated_name = indicated_mod.parent.file_relative_name
+          if !container.include_requires?(indicated_name, @options.ignore_case)
+            container.add_require(Require.new(indicated_name, ""))
+          end
+          break
+        end
+      end
+    }
+
+    #
+    # Parse derived-types definitions
+    #
+    derived_types_comment = ""
+    remaining_code = remaining_lines.join("\n")
+    while remaining_code =~ /^\s*?
+                                  type[\s\,]+(public|private)?\s*?(::)?\s*?
+                                  (\w+)\s*?(!.*?)?$
+                                  (.*?)
+                                  ^\s*?end\s+type.*?$
+                            /imx
+      remaining_code = $~.pre_match
+      remaining_code << $~.post_match
+      typename = $3.chomp.strip
+      type_elements = $5 || ""
+      type_code = remove_empty_head_lines($&)
+      type_trailing = find_comments($4)
+      next if type_trailing =~ /^:nodoc:/
+      type_visibility = $1
+      type_comment = COMMENTS_ARE_UPPER ? 
+        find_comments($~.pre_match) + "\n" + type_trailing :
+          type_trailing + "\n" + find_comments(type_code.sub(/^.*$\n/i, ''))
+      type_element_visibility_public = true
+      type_code.split("\n").each{ |line|
+        if /^\s*?private\s*?$/ =~ line
+          type_element_visibility_public = nil
+          break
+        end
+      } if type_code
+
+      args_comment = ""
+      type_args_info = nil
+
+      if @options.show_all
+        args_comment = find_arguments(nil, type_code, true)
+      else
+        type_public_args_list = []
+        type_args_info = definition_info(type_code)
+        type_args_info.each{ |arg|
+          arg_is_public = type_element_visibility_public
+          arg_is_public = true if arg.include_attr?("public")
+          arg_is_public = nil if arg.include_attr?("private")
+          type_public_args_list << arg.varname if arg_is_public
+        }
+        args_comment = find_arguments(type_public_args_list, type_code)
+      end
+
+      type = AnyMethod.new("type #{typename}", typename)
+      type.singleton = false
+      type.params = ""
+      type.comment = "<b><em> Derived Type </em></b> :: <tt></tt>\n"
+      type.comment << args_comment if args_comment
+      type.comment << type_comment if type_comment
+      progress "t"
+      @stats.num_methods += 1
+      container.add_method type
+
+      set_visibility(container, typename, visibility_default, @@public_methods)
+
+      if type_visibility
+        type_visibility.gsub!(/\s/,'')
+        type_visibility.gsub!(/\,/,'')
+        type_visibility.gsub!(/:/,'')
+        type_visibility.downcase!
+        if type_visibility == "public"
+          container.set_visibility_for([typename], :public)
+        elsif type_visibility == "private"
+          container.set_visibility_for([typename], :private)
+        end
+      end
+
+      check_public_methods(type, container.name)
+
+      if @options.show_all
+        derived_types_comment << ", " unless derived_types_comment.empty?
+        derived_types_comment << typename
+      else
+        if type.visibility == :public
+        derived_types_comment << ", " unless derived_types_comment.empty?
+        derived_types_comment << typename
+        end
+      end
+
+    end
+
+    if !derived_types_comment.empty?
+      derived_types_table = 
+        Attr.new("Derived Types", "Derived_Types", "", 
+                 derived_types_comment)
+      container.add_attribute(derived_types_table)
+    end
+
+    #
+    # move interface scope
+    #
+    interface_code = ""
+    while remaining_code =~ /^\s*?
+                                 interface(
+                                            \s+\w+                      |
+                                            \s+operator\s*?\(.*?\)       |
+                                            \s+assignment\s*?\(\s*?=\s*?\)
+                                          )?\s*?$
+                                 (.*?)
+                                 ^\s*?end\s+interface.*?$
+                            /imx
+      interface_code << remove_empty_head_lines($&) + "\n"
+      remaining_code = $~.pre_match
+      remaining_code << $~.post_match
+    end
+
+    #
+    # Parse global constants or variables in modules
+    #
+    const_var_defs = definition_info(before_contains_code)
+    const_var_defs.each{|defitem|
+      next if defitem.nodoc
+      const_or_var_type = "Variable"
+      const_or_var_progress = "v"
+      if defitem.include_attr?("parameter")
+        const_or_var_type = "Constant"
+        const_or_var_progress = "c"
+      end
+      const_or_var = AnyMethod.new(const_or_var_type, defitem.varname)
+      const_or_var.singleton = false
+      const_or_var.params = ""
+      self_comment = find_arguments([defitem.varname], before_contains_code)
+      const_or_var.comment = "<b><em>" + const_or_var_type + "</em></b> :: <tt></tt>\n"
+      const_or_var.comment << self_comment if self_comment
+      progress const_or_var_progress
+      @stats.num_methods += 1
+      container.add_method const_or_var
+
+      set_visibility(container, defitem.varname, visibility_default, @@public_methods)
+
+      if defitem.include_attr?("public")
+        container.set_visibility_for([defitem.varname], :public)
+      elsif defitem.include_attr?("private")
+        container.set_visibility_for([defitem.varname], :private)
+      end
+
+      check_public_methods(const_or_var, container.name)
+
+    } if const_var_defs
+
+    remaining_lines = remaining_code.split("\n")
+
+    # "subroutine" or "function" parts are parsed (new)
+    #
+    level_depth = 0
+    block_searching_flag = nil
+    block_searching_lines = []
+    pre_comment = []
+    procedure_trailing = ""
+    procedure_name = ""
+    procedure_params = ""
+    procedure_prefix = ""
+    procedure_result_arg = ""
+    procedure_type = ""
+    contains_lines = []
+    contains_flag = nil
+    remaining_lines.collect!{|line|
+      if !block_searching_flag
+        # subroutine
+        if line =~ /^\s*?
+                         (recursive|pure|elemental)?\s*?
+                         subroutine\s+(\w+)\s*?(\(.*?\))?\s*?(!.*?)?$
+                   /ix
+          block_searching_flag = :subroutine
+          block_searching_lines << line
+
+          procedure_name = $2.chomp.strip
+          procedure_params = $3 || ""
+          procedure_prefix = $1 || ""
+          procedure_trailing = $4 || "!"
+          next false
+
+        # function
+        elsif line =~ /^\s*?
+                       (recursive|pure|elemental)?\s*?
+                       (
+                           character\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                         | type\s*?\([\w\s]+?\)\s+
+                         | integer\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                         | real\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                         | double\s+precision\s+
+                         | logical\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                         | complex\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                       )?
+                       function\s+(\w+)\s*?
+                       (\(.*?\))?(\s+result\((.*?)\))?\s*?(!.*?)?$
+                      /ix
+          block_searching_flag = :function
+          block_searching_lines << line
+
+          procedure_prefix = $1 || ""
+          procedure_type = $2 ? $2.chomp.strip : nil
+          procedure_name = $8.chomp.strip
+          procedure_params = $9 || ""
+          procedure_result_arg = $11 ? $11.chomp.strip : procedure_name
+          procedure_trailing = $12 || "!"
+          next false
+        elsif line =~ /^\s*?!\s?(.*)/
+          pre_comment << line
+          next line
+        else
+          pre_comment = []
+          next line
+        end
+      end
+      contains_flag = true if line =~ /^\s*?contains\s*?(!.*?)?$/
+      block_searching_lines << line
+      contains_lines << line if contains_flag
+
+      level_depth += 1 if block_start?(line)
+      level_depth -= 1 if block_end?(line)
+      if level_depth >= 0
+        next false
+      end
+
+      # "procedure_code" is formatted.
+      # ":nodoc:" flag is checked.
+      #
+      procedure_code = block_searching_lines.join("\n")
+      procedure_code = remove_empty_head_lines(procedure_code)
+      if procedure_trailing =~ /^!:nodoc:/
+        # next loop to search next block
+        level_depth = 0
+        block_searching_flag = nil
+        block_searching_lines = []
+        pre_comment = []
+        procedure_trailing = ""
+        procedure_name = ""
+        procedure_params = ""
+        procedure_prefix = ""
+        procedure_result_arg = ""
+        procedure_type = ""
+        contains_lines = []
+        contains_flag = nil
+        next false
+      end
+
+      # AnyMethod is created, and added to container
+      #
+      subroutine_function = nil
+      if block_searching_flag == :subroutine
+        subroutine_prefix   = procedure_prefix
+        subroutine_name     = procedure_name
+        subroutine_params   = procedure_params
+        subroutine_trailing = procedure_trailing
+        subroutine_code     = procedure_code
+
+        subroutine_comment = COMMENTS_ARE_UPPER ? 
+          pre_comment.join("\n") + "\n" + subroutine_trailing : 
+            subroutine_trailing + "\n" + subroutine_code.sub(/^.*$\n/i, '')
+        subroutine = AnyMethod.new("subroutine", subroutine_name)
+        parse_subprogram(subroutine, subroutine_params,
+                         subroutine_comment, subroutine_code,
+                         before_contains_code, nil, subroutine_prefix)
+        progress "s"
+        @stats.num_methods += 1
+        container.add_method subroutine
+        subroutine_function = subroutine
+
+      elsif block_searching_flag == :function
+        function_prefix     = procedure_prefix
+        function_type       = procedure_type
+        function_name       = procedure_name
+        function_params_org = procedure_params
+        function_result_arg = procedure_result_arg
+        function_trailing   = procedure_trailing
+        function_code_org   = procedure_code
+
+        function_comment = COMMENTS_ARE_UPPER ?
+          pre_comment.join("\n") + "\n" + function_trailing :
+            function_trailing + "\n " + function_code_org.sub(/^.*$\n/i, '')
+
+        function_code = "#{function_code_org}"
+        if function_type
+          function_code << "\n" + function_type + " :: " + function_result_arg
+        end
+
+        function_params =
+          function_params_org.sub(/^\(/, "\(#{function_result_arg}, ")
+
+        function = AnyMethod.new("function", function_name)
+        parse_subprogram(function, function_params,
+                         function_comment, function_code,
+                         before_contains_code, true, function_prefix)
+
+        # Specific modification due to function
+        function.params.sub!(/\(\s*?#{function_result_arg}\s*?,\s*?/, "\( ")
+        function.params << " result(" + function_result_arg + ")"
+        function.start_collecting_tokens
+        function.add_token Token.new(1,1).set_text(function_code_org)
+
+        progress "f"
+        @stats.num_methods += 1
+        container.add_method function
+        subroutine_function = function
+
+      end
+
+      # The visibility of procedure is specified
+      #
+      set_visibility(container, procedure_name, 
+                     visibility_default, @@public_methods)
+
+      # The alias for this procedure from external modules
+      #
+      check_external_aliases(procedure_name,
+                             subroutine_function.params,
+                             subroutine_function.comment, subroutine_function) if external
+      check_public_methods(subroutine_function, container.name)
+
+
+      # contains_lines are parsed as private procedures
+      if contains_flag
+        parse_program_or_module(container,
+                                contains_lines.join("\n"), :private)
+      end
+
+      # next loop to search next block
+      level_depth = 0
+      block_searching_flag = nil
+      block_searching_lines = []
+      pre_comment = []
+      procedure_trailing = ""
+      procedure_name = ""
+      procedure_params = ""
+      procedure_prefix = ""
+      procedure_result_arg = ""
+      contains_lines = []
+      contains_flag = nil
+      next false
+    } # End of remaining_lines.collect!{|line|
+
+    # Array remains_lines is converted to String remains_code again
+    #
+    remaining_code = remaining_lines.join("\n")
+
+    #
+    # Parse interface
+    #
+    interface_scope = false
+    generic_name = ""
+    interface_code.split("\n").each{ |line|
+      if /^\s*?
+               interface(
+                          \s+\w+|
+                          \s+operator\s*?\(.*?\)|
+                          \s+assignment\s*?\(\s*?=\s*?\)
+                        )?
+               \s*?(!.*?)?$
+         /ix =~ line
+        generic_name = $1 ? $1.strip.chomp : nil
+        interface_trailing = $2 || "!"
+        interface_scope = true
+        interface_scope = false if interface_trailing =~ /!:nodoc:/
+#        if generic_name =~ /operator\s*?\((.*?)\)/i
+#          operator_name = $1
+#          if operator_name && !operator_name.empty?
+#            generic_name = "#{operator_name}"
+#          end
+#        end
+#        if generic_name =~ /assignment\s*?\((.*?)\)/i
+#          assignment_name = $1
+#          if assignment_name && !assignment_name.empty?
+#            generic_name = "#{assignment_name}"
+#          end
+#        end
+      end
+      if /^\s*?end\s+interface/i =~ line
+        interface_scope = false
+        generic_name = nil
+      end
+      # internal alias
+      if interface_scope && /^\s*?module\s+procedure\s+(.*?)(!.*?)?$/i =~ line
+        procedures = $1.strip.chomp
+        procedures_trailing = $2 || "!"
+        next if procedures_trailing =~ /!:nodoc:/
+        procedures.split(",").each{ |proc|
+          proc.strip!
+          proc.chomp!
+          next if generic_name == proc || !generic_name
+          old_meth = container.find_symbol(proc, nil, @options.ignore_case)
+          next if !old_meth
+          nolink = old_meth.visibility == :private ? true : nil
+          nolink = nil if @options.show_all
+          new_meth = 
+             initialize_external_method(generic_name, proc, 
+                                        old_meth.params, nil, 
+                                        old_meth.comment, 
+                                        old_meth.clone.token_stream[0].text, 
+                                        true, nolink)
+          new_meth.singleton = old_meth.singleton
+
+          progress "i"
+          @stats.num_methods += 1
+          container.add_method new_meth
+
+          set_visibility(container, generic_name, visibility_default, @@public_methods)
+
+          check_public_methods(new_meth, container.name)
+
+        }
+      end
+
+      # external aliases
+      if interface_scope
+        # subroutine
+        proc = nil
+        params = nil
+        procedures_trailing = nil
+        if line =~ /^\s*?
+                         (recursive|pure|elemental)?\s*?
+                         subroutine\s+(\w+)\s*?(\(.*?\))?\s*?(!.*?)?$
+                   /ix
+          proc = $2.chomp.strip
+          generic_name = proc unless generic_name
+          params = $3 || ""
+          procedures_trailing = $4 || "!"
+
+        # function
+        elsif line =~ /^\s*?
+                       (recursive|pure|elemental)?\s*?
+                       (
+                           character\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                         | type\s*?\([\w\s]+?\)\s+
+                         | integer\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                         | real\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                         | double\s+precision\s+
+                         | logical\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                         | complex\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                       )?
+                       function\s+(\w+)\s*?
+                       (\(.*?\))?(\s+result\((.*?)\))?\s*?(!.*?)?$
+                      /ix
+          proc = $8.chomp.strip
+          generic_name = proc unless generic_name
+          params = $9 || ""
+          procedures_trailing = $12 || "!"
+        else
+          next
+        end
+        next if procedures_trailing =~ /!:nodoc:/
+        indicated_method = nil
+        indicated_file   = nil
+        TopLevel.all_files.each do |name, toplevel|
+          indicated_method = toplevel.find_local_symbol(proc, @options.ignore_case)
+          indicated_file = name
+          break if indicated_method
+        end
+
+        if indicated_method
+          external_method = 
+            initialize_external_method(generic_name, proc, 
+                                       indicated_method.params, 
+                                       indicated_file, 
+                                       indicated_method.comment)
+
+          progress "e"
+          @stats.num_methods += 1
+          container.add_method external_method
+          set_visibility(container, generic_name, visibility_default, @@public_methods)
+          if !container.include_requires?(indicated_file, @options.ignore_case)
+            container.add_require(Require.new(indicated_file, ""))
+          end
+          check_public_methods(external_method, container.name)
+
+        else
+          @@external_aliases << {
+            "new_name"  => generic_name,
+            "old_name"  => proc,
+            "file_or_module" => container,
+            "visibility" => find_visibility(container, generic_name, @@public_methods) || visibility_default
+          }
+        end
+      end
+
+    } if interface_code # End of interface_code.split("\n").each ...
+
+    #
+    # Already imported methods are removed from @@public_methods.
+    # Remainders are assumed to be imported from other modules.
+    #
+    @@public_methods.delete_if{ |method| method["entity_is_discovered"]}
+
+    @@public_methods.each{ |pub_meth|
+      next unless pub_meth["file_or_module"].name == container.name
+      pub_meth["used_modules"].each{ |used_mod|
+        TopLevel.all_classes_and_modules.each{ |modules|
+          if modules.name == used_mod ||
+              modules.name.upcase == used_mod.upcase &&
+              @options.ignore_case
+            modules.method_list.each{ |meth|
+              if meth.name == pub_meth["name"] ||
+                  meth.name.upcase == pub_meth["name"].upcase &&
+                  @options.ignore_case
+                new_meth = initialize_public_method(meth,
+                                                    modules.name)
+                if pub_meth["local_name"]
+                  new_meth.name = pub_meth["local_name"]
+                end
+                progress "e"
+                @stats.num_methods += 1
+                container.add_method new_meth
+              end
+            }
+          end
+        }
+      }
+    }
+
+    container
+  end  # End of parse_program_or_module
+
+  ##
+  # Parse arguments, comment, code of subroutine and function.  Return
+  # AnyMethod object.
+
+  def parse_subprogram(subprogram, params, comment, code, 
+                       before_contains=nil, function=nil, prefix=nil)
+    subprogram.singleton = false
+    prefix = "" if !prefix
+    arguments = params.sub(/\(/, "").sub(/\)/, "").split(",") if params
+    args_comment, params_opt = 
+      find_arguments(arguments, code.sub(/^s*?contains\s*?(!.*?)?$.*/im, ""),
+                     nil, nil, true)
+    params_opt = "( " + params_opt + " ) " if params_opt
+    subprogram.params = params_opt || ""
+    namelist_comment = find_namelists(code, before_contains)
+
+    block_comment = find_comments comment
+    if function
+      subprogram.comment = "<b><em> Function </em></b> :: <em>#{prefix}</em>\n"
+    else
+      subprogram.comment = "<b><em> Subroutine </em></b> :: <em>#{prefix}</em>\n"
+    end
+    subprogram.comment << args_comment if args_comment
+    subprogram.comment << block_comment if block_comment
+    subprogram.comment << namelist_comment if namelist_comment
+
+    # For output source code
+    subprogram.start_collecting_tokens
+    subprogram.add_token Token.new(1,1).set_text(code)
+
+    subprogram
+  end
+
+  ##
+  # Collect comment for file entity
+
+  def collect_first_comment(body)
+    comment = ""
+    not_comment = ""
+    comment_start = false
+    comment_end   = false
+    body.split("\n").each{ |line|
+      if comment_end
+        not_comment << line
+        not_comment << "\n"
+      elsif /^\s*?!\s?(.*)$/i =~ line
+        comment_start = true
+        comment << $1
+        comment << "\n"
+      elsif /^\s*?$/i =~ line
+        comment_end = true if comment_start && COMMENTS_ARE_UPPER
+      else
+        comment_end = true
+        not_comment << line
+        not_comment << "\n"
+      end
+    }
+    return comment, not_comment
+  end
+
+
+  ##
+  # Return comments of definitions of arguments
+  #
+  # If "all" argument is true, information of all arguments are returned.
+  #
+  # If "modified_params" is true, list of arguments are decorated, for
+  # example, optional arguments are parenthetic as "[arg]".
+
+  def find_arguments(args, text, all=nil, indent=nil, modified_params=nil)
+    return unless args || all
+    indent = "" unless indent
+    args = ["all"] if all
+    params = "" if modified_params
+    comma = ""
+    return unless text
+    args_rdocforms = "\n"
+    remaining_lines = "#{text}"
+    definitions = definition_info(remaining_lines)
+    args.each{ |arg|
+      arg.strip!
+      arg.chomp!
+      definitions.each { |defitem|
+        if arg == defitem.varname.strip.chomp || all
+          args_rdocforms << <<-"EOF"
+
+#{indent}<tt><b>#{defitem.varname.chomp.strip}#{defitem.arraysuffix}</b> #{defitem.inivalue}</tt> :: 
+#{indent}   <tt>#{defitem.types.chomp.strip}</tt>
+EOF
+          if !defitem.comment.chomp.strip.empty?
+            comment = ""
+            defitem.comment.split("\n").each{ |line|
+              comment << "       " + line + "\n"
+            }
+            args_rdocforms << <<-"EOF"
+
+#{indent}   <tt></tt> :: 
+#{indent}       <tt></tt>
+#{indent}       #{comment.chomp.strip}
+EOF
+          end
+
+          if modified_params
+            if defitem.include_attr?("optional")
+              params << "#{comma}[#{arg}]"
+            else
+              params << "#{comma}#{arg}"
+            end
+            comma = ", "
+          end
+        end
+      }
+    }
+    if modified_params
+      return args_rdocforms, params
+    else
+      return args_rdocforms
+    end
+  end
+
+  ##
+  # Return comments of definitions of namelists
+
+  def find_namelists(text, before_contains=nil)
+    return nil if !text
+    result = ""
+    lines = "#{text}"
+    before_contains = "" if !before_contains
+    while lines =~ /^\s*?namelist\s+\/\s*?(\w+)\s*?\/([\s\w\,]+)$/i
+      lines = $~.post_match
+      nml_comment = COMMENTS_ARE_UPPER ? 
+          find_comments($~.pre_match) : find_comments($~.post_match)
+      nml_name = $1
+      nml_args = $2.split(",")
+      result << "\n\n=== NAMELIST <tt><b>" + nml_name + "</tt></b>\n\n"
+      result << nml_comment + "\n" if nml_comment
+      if lines.split("\n")[0] =~ /^\//i
+        lines = "namelist " + lines
+      end
+      result << find_arguments(nml_args, "#{text}" + "\n" + before_contains)
+    end
+    return result
+  end
+
+  ##
+  # Comments just after module or subprogram, or arguments are returned. If
+  # "COMMENTS_ARE_UPPER" is true, comments just before modules or subprograms
+  # are returnd
+
+  def find_comments text
+    return "" unless text
+    lines = text.split("\n")
+    lines.reverse! if COMMENTS_ARE_UPPER
+    comment_block = Array.new
+    lines.each do |line|
+      break if line =~ /^\s*?\w/ || line =~ /^\s*?$/
+      if COMMENTS_ARE_UPPER
+        comment_block.unshift line.sub(/^\s*?!\s?/,"")
+      else
+        comment_block.push line.sub(/^\s*?!\s?/,"")
+      end
+    end
+    nice_lines = comment_block.join("\n").split "\n\s*?\n"
+    nice_lines[0] ||= ""
+    nice_lines.shift
+  end
+
+  def progress(char)
+    unless @options.quiet
+      @progress.print(char)
+      @progress.flush
+    end
+  end
+
+  ##
+  # Create method for internal alias
+
+  def initialize_public_method(method, parent)
+    return if !method || !parent
+
+    new_meth = AnyMethod.new("External Alias for module", method.name)
+    new_meth.singleton    = method.singleton
+    new_meth.params       = method.params.clone
+    new_meth.comment      = remove_trailing_alias(method.comment.clone)
+    new_meth.comment      << "\n\n#{EXTERNAL_ALIAS_MES} #{parent.strip.chomp}\##{method.name}"
+
+    return new_meth
+  end
+
+  ##
+  # Create method for external alias
+  #
+  # If argument "internal" is true, file is ignored.
+
+  def initialize_external_method(new, old, params, file, comment, token=nil,
+                                 internal=nil, nolink=nil)
+    return nil unless new || old
+
+    if internal
+      external_alias_header = "#{INTERNAL_ALIAS_MES} "
+      external_alias_text   = external_alias_header + old 
+    elsif file
+      external_alias_header = "#{EXTERNAL_ALIAS_MES} "
+      external_alias_text   = external_alias_header + file + "#" + old
+    else
+      return nil
+    end
+    external_meth = AnyMethod.new(external_alias_text, new)
+    external_meth.singleton    = false
+    external_meth.params       = params
+    external_comment = remove_trailing_alias(comment) + "\n\n" if comment
+    external_meth.comment = external_comment || ""
+    if nolink && token
+      external_meth.start_collecting_tokens
+      external_meth.add_token Token.new(1,1).set_text(token)
+    else
+      external_meth.comment << external_alias_text
+    end
+
+    return external_meth
+  end
+
+  ##
+  # Parse visibility
+
+  def parse_visibility(code, default, container)
+    result = []
+    visibility_default = default || :public
+
+    used_modules = []
+    container.includes.each{|i| used_modules << i.name} if container
+
+    remaining_code = code.gsub(/^\s*?type[\s\,]+.*?\s+end\s+type.*?$/im, "")
+    remaining_code.split("\n").each{ |line|
+      if /^\s*?private\s*?$/ =~ line
+        visibility_default = :private
+        break
+      end
+    } if remaining_code
+
+    remaining_code.split("\n").each{ |line|
+      if /^\s*?private\s*?(::)?\s+(.*)\s*?(!.*?)?/i =~ line
+        methods = $2.sub(/!.*$/, '')
+        methods.split(",").each{ |meth|
+          meth.sub!(/!.*$/, '')
+          meth.gsub!(/:/, '')
+          result << {
+            "name" => meth.chomp.strip,
+            "visibility" => :private,
+            "used_modules" => used_modules.clone,
+            "file_or_module" => container,
+            "entity_is_discovered" => nil,
+            "local_name" => nil
+          }
+        }
+      elsif /^\s*?public\s*?(::)?\s+(.*)\s*?(!.*?)?/i =~ line
+        methods = $2.sub(/!.*$/, '')
+        methods.split(",").each{ |meth|
+          meth.sub!(/!.*$/, '')
+          meth.gsub!(/:/, '')
+          result << {
+            "name" => meth.chomp.strip,
+            "visibility" => :public,
+            "used_modules" => used_modules.clone,
+            "file_or_module" => container,
+            "entity_is_discovered" => nil,
+            "local_name" => nil
+          }
+        }
+      end
+    } if remaining_code
+
+    if container
+      result.each{ |vis_info|
+        vis_info["parent"] = container.name
+      }
+    end
+
+    return visibility_default, result
+  end
+
+  ##
+  # Set visibility
+  #
+  # "subname" element of "visibility_info" is deleted.
+
+  def set_visibility(container, subname, visibility_default, visibility_info)
+    return unless container || subname || visibility_default || visibility_info
+    not_found = true
+    visibility_info.collect!{ |info|
+      if info["name"] == subname ||
+          @options.ignore_case && info["name"].upcase == subname.upcase
+        if info["file_or_module"].name == container.name
+          container.set_visibility_for([subname], info["visibility"])
+          info["entity_is_discovered"] = true
+          not_found = false
+        end
+      end
+      info
+    }
+    if not_found
+      return container.set_visibility_for([subname], visibility_default)
+    else
+      return container
+    end
+  end
+
+  ##
+  # Find visibility
+
+  def find_visibility(container, subname, visibility_info)
+    return nil if !subname || !visibility_info
+    visibility_info.each{ |info|
+      if info["name"] == subname ||
+          @options.ignore_case && info["name"].upcase == subname.upcase
+        if info["parent"] == container.name
+          return info["visibility"]
+        end
+      end
+    }
+    return nil
+  end
+
+  ##
+  # Check external aliases
+
+  def check_external_aliases(subname, params, comment, test=nil)
+    @@external_aliases.each{ |alias_item|
+      if subname == alias_item["old_name"] ||
+                  subname.upcase == alias_item["old_name"].upcase &&
+                          @options.ignore_case
+
+        new_meth = initialize_external_method(alias_item["new_name"], 
+                                              subname, params, @file_name, 
+                                              comment)
+        new_meth.visibility = alias_item["visibility"]
+
+        progress "e"
+        @stats.num_methods += 1
+        alias_item["file_or_module"].add_method(new_meth)
+
+        if !alias_item["file_or_module"].include_requires?(@file_name, @options.ignore_case)
+          alias_item["file_or_module"].add_require(Require.new(@file_name, ""))
+        end
+      end
+    }
+  end
+
+  ##
+  # Check public_methods
+
+  def check_public_methods(method, parent)
+    return if !method || !parent
+    @@public_methods.each{ |alias_item|
+      parent_is_used_module = nil
+      alias_item["used_modules"].each{ |used_module|
+        if used_module == parent ||
+            used_module.upcase == parent.upcase &&
+            @options.ignore_case
+          parent_is_used_module = true
+        end
+      }
+      next if !parent_is_used_module
+
+      if method.name == alias_item["name"] ||
+          method.name.upcase == alias_item["name"].upcase &&
+          @options.ignore_case
+
+        new_meth = initialize_public_method(method, parent)
+        if alias_item["local_name"]
+          new_meth.name = alias_item["local_name"]
+        end
+
+        progress "e"
+        @stats.num_methods += 1
+        alias_item["file_or_module"].add_method new_meth
+      end
+    }
+  end
+
+  ##
+  # Continuous lines are united.
+  #
+  # Comments in continuous lines are removed.
+
+  def united_to_one_line(f90src)
+    return "" unless f90src
+    lines = f90src.split("\n")
+    previous_continuing = false
+    now_continuing = false
+    body = ""
+    lines.each{ |line|
+      words = line.split("")
+      next if words.empty? && previous_continuing
+      commentout = false
+      brank_flag = true ; brank_char = ""
+      squote = false    ; dquote = false
+      ignore = false
+      words.collect! { |char|
+        if previous_continuing && brank_flag
+          now_continuing = true
+          ignore         = true
+          case char
+          when "!"                       ; break
+          when " " ; brank_char << char  ; next ""
+          when "&"
+            brank_flag = false
+            now_continuing = false
+            next ""
+          else 
+            brank_flag     = false
+            now_continuing = false
+            ignore         = false
+            next brank_char + char
+          end
+        end
+        ignore = false
+
+        if now_continuing
+          next ""
+        elsif !(squote) && !(dquote) && !(commentout)
+          case char
+          when "!" ; commentout = true     ; next char
+          when "\""; dquote = true         ; next char
+          when "\'"; squote = true         ; next char
+          when "&" ; now_continuing = true ; next ""
+          else next char
+          end
+        elsif commentout
+          next char
+        elsif squote
+          case char
+          when "\'"; squote = false ; next char
+          else next char
+          end
+        elsif dquote
+          case char
+          when "\""; dquote = false ; next char
+          else next char
+          end
+        end
+      }
+      if !ignore && !previous_continuing || !brank_flag
+        if previous_continuing
+          body << words.join("")
+        else
+          body << "\n" + words.join("")
+        end
+      end
+      previous_continuing = now_continuing ? true : nil
+      now_continuing = nil
+    }
+    return body
+  end
+
+
+  ##
+  # Continuous line checker
+
+  def continuous_line?(line)
+    continuous = false
+    if /&\s*?(!.*)?$/ =~ line
+      continuous = true
+      if comment_out?($~.pre_match)
+        continuous = false
+      end
+    end
+    return continuous
+  end
+
+  ##
+  # Comment out checker
+
+  def comment_out?(line)
+    return nil unless line
+    commentout = false
+    squote = false ; dquote = false
+    line.split("").each { |char|
+      if !(squote) && !(dquote)
+        case char
+        when "!" ; commentout = true ; break
+        when "\""; dquote = true
+        when "\'"; squote = true
+        else next
+        end
+      elsif squote
+        case char
+        when "\'"; squote = false
+        else next
+        end
+      elsif dquote
+        case char
+        when "\""; dquote = false
+        else next
+        end
+      end
+    }
+    return commentout
+  end
+
+  ##
+  # Semicolons are replaced to line feed.
+
+  def semicolon_to_linefeed(text)
+    return "" unless text
+    lines = text.split("\n")
+    lines.collect!{ |line|
+      words = line.split("")
+      commentout = false
+      squote = false ; dquote = false
+      words.collect! { |char|
+        if !(squote) && !(dquote) && !(commentout)
+          case char
+          when "!" ; commentout = true ; next char
+          when "\""; dquote = true     ; next char
+          when "\'"; squote = true     ; next char
+          when ";" ;                     "\n"
+          else next char
+          end
+        elsif commentout
+          next char
+        elsif squote
+          case char
+          when "\'"; squote = false ; next char
+          else next char
+          end
+        elsif dquote
+          case char
+          when "\""; dquote = false ; next char
+          else next char
+          end
+        end
+      }
+      words.join("")
+    }
+    return lines.join("\n")
+  end
+
+  ##
+  # Which "line" is start of block (module, program, block data, subroutine,
+  # function) statement ?
+
+  def block_start?(line)
+    return nil if !line
+
+    if line =~ /^\s*?module\s+(\w+)\s*?(!.*?)?$/i    ||
+        line =~ /^\s*?program\s+(\w+)\s*?(!.*?)?$/i  ||
+        line =~ /^\s*?block\s+data(\s+\w+)?\s*?(!.*?)?$/i     ||
+        line =~ \
+                /^\s*?
+                 (recursive|pure|elemental)?\s*?
+                 subroutine\s+(\w+)\s*?(\(.*?\))?\s*?(!.*?)?$
+                /ix ||
+        line =~ \
+                /^\s*?
+                 (recursive|pure|elemental)?\s*?
+                 (
+                     character\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                   | type\s*?\([\w\s]+?\)\s+
+                   | integer\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                   | real\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                   | double\s+precision\s+
+                   | logical\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                   | complex\s*?(\([\w\s\=\(\)\*]+?\))?\s+
+                 )?
+                 function\s+(\w+)\s*?
+                 (\(.*?\))?(\s+result\((.*?)\))?\s*?(!.*?)?$
+                /ix
+      return true
+    end
+
+    return nil
+  end
+
+  ##
+  # Which "line" is end of block (module, program, block data, subroutine,
+  # function) statement ?
+
+  def block_end?(line)
+    return nil if !line
+
+    if line =~ /^\s*?end\s*?(!.*?)?$/i                 ||
+        line =~ /^\s*?end\s+module(\s+\w+)?\s*?(!.*?)?$/i       ||
+        line =~ /^\s*?end\s+program(\s+\w+)?\s*?(!.*?)?$/i      ||
+        line =~ /^\s*?end\s+block\s+data(\s+\w+)?\s*?(!.*?)?$/i  ||
+        line =~ /^\s*?end\s+subroutine(\s+\w+)?\s*?(!.*?)?$/i   ||
+        line =~ /^\s*?end\s+function(\s+\w+)?\s*?(!.*?)?$/i
+      return true
+    end
+
+    return nil
+  end
+
+  ##
+  # Remove "Alias for" in end of comments
+
+  def remove_trailing_alias(text)
+    return "" if !text
+    lines = text.split("\n").reverse
+    comment_block = Array.new
+    checked = false
+    lines.each do |line|
+      if !checked 
+        if /^\s?#{INTERNAL_ALIAS_MES}/ =~ line ||
+            /^\s?#{EXTERNAL_ALIAS_MES}/ =~ line
+          checked = true
+          next
+        end
+      end
+      comment_block.unshift line
+    end
+    nice_lines = comment_block.join("\n")
+    nice_lines ||= ""
+    return nice_lines
+  end
+
+  ##
+  # Empty lines in header are removed
+
+  def remove_empty_head_lines(text)
+    return "" unless text
+    lines = text.split("\n")
+    header = true
+    lines.delete_if{ |line|
+      header = false if /\S/ =~ line
+      header && /^\s*?$/ =~ line
+    }
+    lines.join("\n")
+  end
+
+  ##
+  # header marker "=", "==", ... are removed
+
+  def remove_header_marker(text)
+    return text.gsub(/^\s?(=+)/, '<tt></tt>\1')
+  end
+
+  def remove_private_comments(body)
+    body.gsub!(/^\s*!--\s*?$.*?^\s*!\+\+\s*?$/m, '')
+    return body
+  end
+
+  ##
+  # Information of arguments of subroutines and functions in Fortran95
+
+  class Fortran95Definition
+
+    # Name of variable
+    #
+    attr_reader   :varname
+
+    # Types of variable
+    #
+    attr_reader   :types
+
+    # Initial Value
+    #
+    attr_reader   :inivalue
+
+    # Suffix of array
+    #
+    attr_reader   :arraysuffix
+
+    # Comments
+    #
+    attr_accessor   :comment
+
+    # Flag of non documentation
+    #
+    attr_accessor   :nodoc
+
+    def initialize(varname, types, inivalue, arraysuffix, comment,
+                   nodoc=false)
+      @varname = varname
+      @types = types
+      @inivalue = inivalue
+      @arraysuffix = arraysuffix
+      @comment = comment
+      @nodoc = nodoc
+    end
+
+    def to_s
+      return <<-EOF
+<Fortran95Definition: 
+varname=#{@varname}, types=#{types},
+inivalue=#{@inivalue}, arraysuffix=#{@arraysuffix}, nodoc=#{@nodoc}, 
+comment=
+#{@comment}
+>
+EOF
+    end
+
+    #
+    # If attr is included, true is returned
+    #
+    def include_attr?(attr)
+      return if !attr
+      @types.split(",").each{ |type|
+        return true if type.strip.chomp.upcase == attr.strip.chomp.upcase
+      }
+      return nil
+    end
+
+  end # End of Fortran95Definition
+
+  ##
+  # Parse string argument "text", and Return Array of Fortran95Definition
+  # object
+
+  def definition_info(text)
+    return nil unless text
+    lines = "#{text}"
+    defs = Array.new
+    comment = ""
+    trailing_comment = ""
+    under_comment_valid = false
+    lines.split("\n").each{ |line|
+      if /^\s*?!\s?(.*)/ =~ line
+        if COMMENTS_ARE_UPPER
+          comment << remove_header_marker($1)
+          comment << "\n"
+        elsif defs[-1] && under_comment_valid
+          defs[-1].comment << "\n"
+          defs[-1].comment << remove_header_marker($1)
+        end
+        next
+      elsif /^\s*?$/ =~ line
+        comment = ""
+        under_comment_valid = false
+        next
+      end
+      type = ""
+      characters = ""
+      if line =~ /^\s*?
+                  (
+                      character\s*?(\([\w\s\=\(\)\*]+?\))?[\s\,]*
+                    | type\s*?\([\w\s]+?\)[\s\,]*
+                    | integer\s*?(\([\w\s\=\(\)\*]+?\))?[\s\,]*
+                    | real\s*?(\([\w\s\=\(\)\*]+?\))?[\s\,]*
+                    | double\s+precision[\s\,]*
+                    | logical\s*?(\([\w\s\=\(\)\*]+?\))?[\s\,]*
+                    | complex\s*?(\([\w\s\=\(\)\*]+?\))?[\s\,]*
+                  )
+                  (.*?::)?
+                  (.+)$
+                 /ix
+        characters = $8
+        type = $1
+        type << $7.gsub(/::/, '').gsub(/^\s*?\,/, '') if $7
+      else
+        under_comment_valid = false
+        next
+      end
+      squote = false ; dquote = false ; bracket = 0
+      iniflag = false; commentflag = false
+      varname = "" ; arraysuffix = "" ; inivalue = ""
+      start_pos = defs.size
+      characters.split("").each { |char|
+        if !(squote) && !(dquote) && bracket <= 0 && !(iniflag) && !(commentflag)
+          case char
+          when "!" ; commentflag = true
+          when "(" ; bracket += 1       ; arraysuffix = char
+          when "\""; dquote = true
+          when "\'"; squote = true
+          when "=" ; iniflag = true     ; inivalue << char
+          when ","
+            defs << Fortran95Definition.new(varname, type, inivalue, arraysuffix, comment)
+            varname = "" ; arraysuffix = "" ; inivalue = ""
+            under_comment_valid = true
+          when " " ; next
+          else     ; varname << char
+          end
+        elsif commentflag
+          comment << remove_header_marker(char)
+          trailing_comment << remove_header_marker(char)
+        elsif iniflag
+          if dquote
+            case char
+            when "\"" ; dquote = false ; inivalue << char
+            else      ; inivalue << char
+            end
+          elsif squote
+            case char
+            when "\'" ; squote = false ; inivalue << char
+            else      ; inivalue << char
+            end
+          elsif bracket > 0
+            case char
+            when "(" ; bracket += 1 ; inivalue << char
+            when ")" ; bracket -= 1 ; inivalue << char
+            else     ; inivalue << char
+            end
+          else
+            case char
+            when ","
+              defs << Fortran95Definition.new(varname, type, inivalue, arraysuffix, comment)
+              varname = "" ; arraysuffix = "" ; inivalue = ""
+              iniflag = false
+              under_comment_valid = true
+            when "(" ; bracket += 1 ; inivalue << char
+            when "\""; dquote = true  ; inivalue << char
+            when "\'"; squote = true  ; inivalue << char
+            when "!" ; commentflag = true
+            else     ; inivalue << char
+            end
+          end
+        elsif !(squote) && !(dquote) && bracket > 0
+          case char
+          when "(" ; bracket += 1 ; arraysuffix << char
+          when ")" ; bracket -= 1 ; arraysuffix << char
+          else     ; arraysuffix << char
+          end
+        elsif squote
+          case char
+          when "\'"; squote = false ; inivalue << char
+          else     ; inivalue << char
+          end
+        elsif dquote
+          case char
+          when "\""; dquote = false ; inivalue << char
+          else     ; inivalue << char
+          end
+        end
+      }
+      defs << Fortran95Definition.new(varname, type, inivalue, arraysuffix, comment)
+      if trailing_comment =~ /^:nodoc:/
+        defs[start_pos..-1].collect!{ |defitem|
+          defitem.nodoc = true
+        }
+      end
+      varname = "" ; arraysuffix = "" ; inivalue = ""
+      comment = ""
+      under_comment_valid = true
+      trailing_comment = ""
+    }
+    return defs
+  end
+
+end
+
Index: lib/rdoc/parser/simple.rb
===================================================================
--- lib/rdoc/parser/simple.rb	(revision 0)
+++ lib/rdoc/parser/simple.rb	(revision 18121)
@@ -0,0 +1,38 @@
+require 'rdoc/parser'
+
+##
+# Parse a non-source file. We basically take the whole thing as one big
+# comment. If the first character in the file is '#', we strip leading pound
+# signs.
+
+class RDoc::Parser::Simple < RDoc::Parser
+
+  parse_files_matching(//)
+
+  ##
+  # Prepare to parse a plain file
+
+  def initialize(top_level, file_name, content, options, stats)
+    super
+
+    preprocess = RDoc::Markup::PreProcess.new @file_name, @options.rdoc_include
+
+    preprocess.handle @content do |directive, param|
+      warn "Unrecognized directive '#{directive}' in #{@file_name}"
+    end
+  end
+
+  ##
+  # Extract the file contents and attach them to the toplevel as a comment
+
+  def scan
+    @top_level.comment = remove_private_comments(@content)
+    @top_level
+  end
+
+  def remove_private_comments(comment)
+    comment.gsub(/^--[^-].*?^\+\+/m, '').sub(/^--.*/m, '')
+  end
+
+end
+
Index: lib/rdoc/options.rb
===================================================================
--- lib/rdoc/options.rb	(revision 18120)
+++ lib/rdoc/options.rb	(revision 18121)
@@ -39,7 +39,7 @@
   ##
   # Pattern for additional attr_... style methods
 
-  attr_reader :extra_accessors
+  attr_accessor :extra_accessors
 
   ##
   # Should we draw fileboxes in diagrams
@@ -62,6 +62,11 @@
   attr_accessor :generator
 
   ##
+  # Formatter to mark up text with
+
+  attr_accessor :formatter
+  
+  ##
   # image format for diagrams
 
   attr_reader :image_format
@@ -95,7 +100,7 @@
   ##
   # The name to use for the output
 
-  attr_reader :op_name
+  attr_accessor :op_name
 
   ##
   # Are we promiscuous about showing module contents across multiple files
@@ -105,7 +110,7 @@
   ##
   # Don't display progress as we process the files
 
-  attr_reader :quiet
+  attr_accessor :quiet
 
   ##
   # Array of directories to search for files to satisfy an :include:
@@ -175,7 +180,6 @@
     @extra_accessor_flags = {}
     @promiscuous = false
     @force_update = false
-    @title = "RDoc Documentation"
 
     @css = nil
     @webcvs = nil
@@ -513,6 +517,8 @@
       end
     end
 
+    argv.insert(0, *ENV['RDOCOPT'].split) if ENV['RDOCOPT']
+
     opts.parse! argv
 
     @files = argv.dup
Index: lib/rdoc/parser.rb
===================================================================
--- lib/rdoc/parser.rb	(revision 0)
+++ lib/rdoc/parser.rb	(revision 18121)
@@ -0,0 +1,109 @@
+require 'rdoc'
+require 'rdoc/code_objects'
+require 'rdoc/markup/preprocess'
+require 'rdoc/stats'
+
+##
+# A parser is simple a class that implements
+#
+#   #initialize(file_name, body, options)
+#
+# and
+#
+#   #scan
+#
+# The initialize method takes a file name to be used, the body of the file,
+# and an RDoc::Options object. The scan method is then called to return an
+# appropriately parsed TopLevel code object.
+#
+# The ParseFactory is used to redirect to the correct parser given a
+# filename extension. This magic works because individual parsers have to
+# register themselves with us as they are loaded in. The do this using the
+# following incantation
+#
+#   require "rdoc/parser"
+#   
+#   class RDoc::Parser::Xyz < RDoc::Parser
+#     parse_files_matching /\.xyz$/ # <<<<
+#   
+#     def initialize(file_name, body, options)
+#       ...
+#     end
+#   
+#     def scan
+#       ...
+#     end
+#   end
+#
+# Just to make life interesting, if we suspect a plain text file, we also
+# look for a shebang line just in case it's a potential shell script
+
+class RDoc::Parser
+
+  @parsers = []
+
+  class << self
+    attr_reader :parsers
+  end
+
+  attr_writer :progress
+
+  ##
+  # Alias an extension to another extension. After this call, files ending
+  # "new_ext" will be parsed using the same parser as "old_ext"
+
+  def self.alias_extension(old_ext, new_ext)
+    parser = can_parse "xxx.#{old_ext}"
+    return false unless parser
+
+    RDoc::Parser.parsers.unshift [/\.#{new_ext}$/, parser.last]
+
+    true
+  end
+
+  ##
+  # Return a parser that can handle a particular extension
+
+  def self.can_parse(file_name)
+    RDoc::Parser.parsers.find { |regexp, parser| regexp =~ file_name }.last
+  end
+
+  ##
+  # Find the correct parser for a particular file name. Return a SimpleParser
+  # for ones that we don't know
+
+  def self.for(top_level, file_name, body, options, stats)
+    # If no extension, look for shebang
+    if file_name !~ /\.\w+$/ && body =~ %r{\A#!(.+)} then
+      shebang = $1
+      case shebang
+      when %r{env\s+ruby}, %r{/ruby}
+        file_name = "dummy.rb"
+      end
+    end
+
+    parser = can_parse file_name
+
+    parser.new top_level, file_name, body, options, stats
+  end
+
+  ##
+  # Record which file types this parser can understand.
+
+  def self.parse_files_matching(regexp)
+    RDoc::Parser.parsers.unshift [regexp, self]
+  end
+
+  def initialize(top_level, file_name, content, options, stats)
+    @top_level = top_level
+    @file_name = file_name
+    @content = content
+    @options = options
+    @stats = stats
+    @progress = $stderr unless options.quiet
+  end
+
+end
+
+require 'rdoc/parser/simple'
+
Index: lib/rdoc/markup/attribute_manager.rb
===================================================================
--- lib/rdoc/markup/attribute_manager.rb	(revision 18120)
+++ lib/rdoc/markup/attribute_manager.rb	(revision 18121)
@@ -144,8 +144,6 @@
     add_html("b",  :BOLD)
     add_html("tt",   :TT)
     add_html("code", :TT)
-
-    add_special(/<!--(.*?)-->/, :COMMENT)
   end
 
   def add_word_pair(start, stop, name)
Index: lib/rdoc/markup/to_texinfo.rb
===================================================================
--- lib/rdoc/markup/to_texinfo.rb	(revision 0)
+++ lib/rdoc/markup/to_texinfo.rb	(revision 18121)
@@ -0,0 +1,69 @@
+require 'rdoc/markup/formatter'
+require 'rdoc/markup/fragments'
+require 'rdoc/markup/inline'
+
+require 'rdoc/markup'
+require 'rdoc/markup/formatter'
+
+##
+# Convert SimpleMarkup to basic TexInfo format
+#
+# TODO: WTF is AttributeManager for?
+#
+class RDoc::Markup::ToTexInfo < RDoc::Markup::Formatter
+
+  def start_accepting
+    @text = []
+  end
+
+  def end_accepting
+    @text.join("\n")
+  end
+
+  def accept_paragraph(attributes, text)
+    @text << format(text)
+  end
+
+  def accept_verbatim(attributes, text)
+    @text << "@verb{|#{format(text)}|}"
+  end
+
+  def accept_heading(attributes, text)
+    heading = ['@majorheading', '@chapheading'][text.head_level - 1] || '@heading'
+    @text << "#{heading}{#{format(text)}}"
+  end
+
+  def accept_list_start(attributes, text)
+    @text << '@itemize @bullet'
+  end
+
+  def accept_list_end(attributes, text)
+    @text << '@end itemize'
+  end
+
+  def accept_list_item(attributes, text)
+    @text << "@item\n#{format(text)}"
+  end
+
+  def accept_blank_line(attributes, text)
+    @text << "\n"
+  end
+
+  def accept_rule(attributes, text)
+    @text << '-----'
+  end
+
+  def format(text)
+    text.txt.
+      gsub(/@/, "@@").
+      gsub(/\{/, "@{").
+      gsub(/\}/, "@}").
+      # gsub(/,/, "@,"). # technically only required in cross-refs
+      gsub(/\+([\w]+)\+/, "@code{\\1}").
+      gsub(/\<tt\>([^<]+)\<\/tt\>/, "@code{\\1}").
+      gsub(/\*([\w]+)\*/, "@strong{\\1}").
+      gsub(/\<b\>([^<]+)\<\/b\>/, "@strong{\\1}").
+      gsub(/_([\w]+)_/, "@emph{\\1}").
+      gsub(/\<em\>([^<]+)\<\/em\>/, "@emph{\\1}")
+  end
+end
Index: lib/rdoc/markup/preprocess.rb
===================================================================
--- lib/rdoc/markup/preprocess.rb	(revision 18120)
+++ lib/rdoc/markup/preprocess.rb	(revision 18121)
@@ -14,21 +14,25 @@
 
   ##
   # Look for common options in a chunk of text. Options that we don't handle
-  # are passed back to our caller as |directive, param|
+  # are yielded to the caller.
 
   def handle(text)
-    text.gsub!(/^([ \t#]*):(\w+):\s*(.+)?\n/) do
+    text.gsub!(/^([ \t]*#?[ \t]*):(\w+):([ \t]*)(.+)?\n/) do
+      next $& if $3.empty? and $4 and $4[0, 1] == ':'
+
       prefix    = $1
       directive = $2.downcase
-      param     = $3
+      param     = $4
 
       case directive
-      when "include"
+      when 'include' then
         filename = param.split[0]
-        include_file(filename, prefix)
+        include_file filename, prefix
 
       else
-        yield(directive, param)
+        result = yield directive, param
+        result = "#{prefix}:#{directive}: #{param}\n" unless result
+        result
       end
     end
   end
Index: lib/rdoc/markup/fragments.rb
===================================================================
--- lib/rdoc/markup/fragments.rb	(revision 18120)
+++ lib/rdoc/markup/fragments.rb	(revision 18121)
@@ -11,8 +11,8 @@
     attr_reader   :level, :param, :txt
     attr_accessor :type
 
-    ######
-    # This is a simple factory system that lets us associate fragment
+    ##
+    # This is a simple factory system that lets us associate fragement
     # types (a string) with a subclass of fragment
 
     TYPE_MAP = {}
Index: lib/rdoc/markup/to_html.rb
===================================================================
--- lib/rdoc/markup/to_html.rb	(revision 18120)
+++ lib/rdoc/markup/to_html.rb	(revision 18121)
@@ -1,7 +1,6 @@
 require 'rdoc/markup/formatter'
 require 'rdoc/markup/fragments'
 require 'rdoc/markup/inline'
-require 'rdoc/generator'
 
 require 'cgi'
 
@@ -21,6 +20,11 @@
   def initialize
     super
 
+    # @in_tt - tt nested levels count
+    # @tt_bit - cache
+    @in_tt = 0
+    @tt_bit = RDoc::Markup::Attribute.bitmap_for :TT
+
     # external hyperlinks
     @markup.add_special(/((link:|https?:|mailto:|ftp:|www\.)\S+\w)/, :HYPERLINK)
 
@@ -31,6 +35,27 @@
   end
 
   ##
+  # Converts a target url to one that is relative to a given path
+
+  def self.gen_relative_url(path, target)
+    from        = File.dirname path
+    to, to_file = File.split target
+
+    from = from.split "/"
+    to   = to.split "/"
+
+    while from.size > 0 and to.size > 0 and from[0] == to[0] do
+      from.shift
+      to.shift
+    end
+
+    from.fill ".."
+    from.concat to
+    from << to_file
+    File.join(*from)
+  end
+
+  ##
   # Generate a hyperlink for url, labeled with text. Handle the
   # special cases for img: and link: described under handle_special_HYPEDLINK
 
@@ -48,7 +73,7 @@
       url = if path[0, 1] == '#' then # is this meaningful?
               path
             else
-              RDoc::Generator.gen_url @from_path, path
+              self.class.gen_relative_url @from_path, path
             end
     end
 
@@ -88,6 +113,20 @@
   end
 
   ##
+  # are we currently inside <tt> tags?
+
+  def in_tt?
+    @in_tt > 0
+  end
+
+  ##
+  # is +tag+ a <tt> tag?
+
+  def tt?(tag)
+    tag.bit == @tt_bit
+  end
+
+  ##
   # Set up the standard mapping of attributes to HTML tags
 
   def init_tags
@@ -216,6 +255,7 @@
     @attr_tags.each do |tag|
       if attr_mask & tag.bit != 0
         res << annotate(tag.on)
+        @in_tt += 1 if tt?(tag)
       end
     end
   end
@@ -226,6 +266,7 @@
 
     @attr_tags.reverse_each do |tag|
       if attr_mask & tag.bit != 0
+        @in_tt -= 1 if tt?(tag)
         res << annotate(tag.off)
       end
     end
@@ -251,27 +292,33 @@
     res
   end
 
+  def convert_string(item)
+    in_tt? ? convert_string_simple(item) : convert_string_fancy(item)
+  end
+
+  def convert_string_simple(item)
+    CGI.escapeHTML item
+  end
+
   ##
   # some of these patterns are taken from SmartyPants...
 
-  def convert_string(item)
-    CGI.escapeHTML(item).
-
+  def convert_string_fancy(item)
     # convert -- to em-dash, (-- to en-dash)
-      gsub(/---?/, '&#8212;'). #gsub(/--/, '&#8211;').
+    item.gsub(/---?/, '&#8212;'). #gsub(/--/, '&#8211;').
 
     # convert ... to elipsis (and make sure .... becomes .<elipsis>)
       gsub(/\.\.\.\./, '.&#8230;').gsub(/\.\.\./, '&#8230;').
 
     # convert single closing quote
-      gsub(%r{([^ \t\r\n\[\{\(])\'}, '\1&#8217;').
+      gsub(%r{([^ \t\r\n\[\{\(])\'}, '\1&#8217;'). # }
       gsub(%r{\'(?=\W|s\b)}, '&#8217;').
 
     # convert single opening quote
       gsub(/'/, '&#8216;').
 
     # convert double closing quote
-      gsub(%r{([^ \t\r\n\[\{\(])\'(?=\W)}, '\1&#8221;').
+      gsub(%r{([^ \t\r\n\[\{\(])\'(?=\W)}, '\1&#8221;'). # }
 
     # convert double opening quote
       gsub(/'/, '&#8220;').
@@ -281,7 +328,6 @@
 
     # convert and registered trademark
       gsub(/\(r\)/, '&#174;')
-
   end
 
   def convert_special(special)
Index: lib/rdoc/markup/to_html_crossref.rb
===================================================================
--- lib/rdoc/markup/to_html_crossref.rb	(revision 18120)
+++ lib/rdoc/markup/to_html_crossref.rb	(revision 18121)
@@ -14,6 +14,7 @@
   # correct relative paths for any hyperlinks that we find
 
   def initialize(from_path, context, show_hash)
+    raise ArgumentError, 'from_path cannot be nil' if from_path.nil?
     super()
 
     # class names, variable names, or instance variables
@@ -47,28 +48,43 @@
   def handle_special_CROSSREF(special)
     name = special.text
 
+    return name if name =~ /\A[a-z]*\z/
+
     return @seen[name] if @seen.include? name
 
-    if name[0,1] == '#' then
+    if name[0, 1] == '#' then
       lookup = name[1..-1]
       name = lookup unless @show_hash
     else
       lookup = name
     end
 
+
     # Find class, module, or method in class or module.
-    if /([A-Z]\w*)[.\#](\w+[!?=]?)/ =~ lookup then
+    #
+    # Do not, however, use an if/elsif/else chain to do so.  Instead, test
+    # each possible pattern until one matches.  The reason for this is that a
+    # string like "YAML.txt" could be the txt() class method of class YAML (in
+    # which case it would match the first pattern, which splits the string
+    # into container and method components and looks up both) or a filename
+    # (in which case it would match the last pattern, which just checks
+    # whether the string as a whole is a known symbol).
+
+    if /([A-Z][\w:]*)[.\#](\w+[!?=]?)/ =~ lookup then
       container = $1
       method = $2
       ref = @context.find_symbol container, method
-    elsif /([A-Za-z]\w*)[.\#](\w+(\([\.\w+\*\/\+\-\=\<\>]+\))?)/ =~ lookup then
+    end
+
+    if !ref and
+       /([A-Za-z][\w:]*)[.\#](\w+(\([\.\w+\*\/\+\-\=\<\>]+\))?)/ =~ lookup then
       container = $1
       method = $2
       ref = @context.find_symbol container, method
-    else
-      ref = @context.find_symbol lookup
     end
 
+    ref = @context.find_symbol lookup unless ref
+
     out = if lookup =~ /^\\/ then
             $'
           elsif ref and ref.document_self then
Index: lib/rdoc/ri/descriptions.rb
===================================================================
--- lib/rdoc/ri/descriptions.rb	(revision 18120)
+++ lib/rdoc/ri/descriptions.rb	(revision 18121)
@@ -2,11 +2,10 @@
 require 'rdoc/markup/fragments'
 require 'rdoc/ri'
 
-#--
+##
 # Descriptions are created by RDoc (in ri_generator) and written out in
 # serialized form into the documentation tree. ri then reads these to generate
 # the documentation
-#++
 
 class RDoc::RI::NamedThing
   attr_reader :name
Index: lib/rdoc/ri/driver.rb
===================================================================
--- lib/rdoc/ri/driver.rb	(revision 18120)
+++ lib/rdoc/ri/driver.rb	(revision 18121)
@@ -11,6 +11,64 @@
 
 class RDoc::RI::Driver
 
+  class Hash < ::Hash
+    def self.convert(hash)
+      hash = new.update hash
+
+      hash.each do |key, value|
+        hash[key] = case value
+                    when ::Hash then
+                      convert value
+                    when Array then
+                      value = value.map do |v|
+                        ::Hash === v ? convert(v) : v
+                      end
+                      value
+                    else
+                      value
+                    end
+      end
+
+      hash
+    end
+
+    def method_missing method, *args
+      self[method.to_s]
+    end
+
+    def merge_enums(other)
+      other.each do |k, v|
+        if self[k] then
+          case v
+          when Array then
+            # HACK dunno
+            if String === self[k] and self[k].empty? then
+              self[k] = v
+            else
+              self[k] += v
+            end
+          when Hash then
+            self[k].update v
+          else
+            # do nothing
+          end
+        else
+          self[k] = v
+        end
+      end
+    end
+  end
+
+  class Error < RDoc::RI::Error; end
+
+  class NotFoundError < Error
+    def message
+      "Nothing known about #{super}"
+    end
+  end
+
+  attr_accessor :homepath # :nodoc:
+
   def self.process_args(argv)
     options = {}
     options[:use_stdout] = !$stdout.tty?
@@ -234,7 +292,7 @@
     @class_cache = if up_to_date then
                      load_cache_for @class_cache_name
                    else
-                     class_cache = {}
+                     class_cache = RDoc::RI::Driver::Hash.new
 
                      classes = map_dirs('**/cdesc*.yaml', :sys) { |f| Dir[f] }
                      populate_class_cache class_cache, classes
@@ -261,16 +319,24 @@
 
   def display_class(name)
     klass = class_cache[name]
+    klass = RDoc::RI::Driver::Hash.convert klass
     @display.display_class_info klass, class_cache
   end
 
+  def get_info_for(arg)
+    @names = [arg]
+    run
+  end
+
   def load_cache_for(klassname)
     path = cache_file_for klassname
 
+    cache = nil
+
     if File.exist? path and
        File.mtime(path) >= File.mtime(class_cache_file_path) then
       File.open path, 'rb' do |fp|
-        Marshal.load fp.read
+        cache = Marshal.load fp.read
       end
     else
       class_cache = nil
@@ -283,7 +349,7 @@
       return nil unless klass
 
       method_files = klass["sources"]
-      cache = {}
+      cache = RDoc::RI::Driver::Hash.new
 
       sys_dir = @sys_dirs.first
       method_files.each do |f|
@@ -296,14 +362,30 @@
           ext_path = f
           ext_path = "gem #{$1}" if f =~ %r%gems/[\d.]+/doc/([^/]+)%
           method["source_path"] = ext_path unless system_file
-          cache[name] = method
+          cache[name] = RDoc::RI::Driver::Hash.convert method
         end
       end
 
       write_cache cache, path
     end
+
+    RDoc::RI::Driver::Hash.convert cache
   end
 
+  ##
+  # Finds the method
+
+  def lookup_method(name, klass)
+    cache = load_cache_for klass
+    raise NotFoundError, name unless cache
+
+    method = cache[name.gsub('.', '#')]
+    method = cache[name.gsub('.', '::')] unless method
+    raise NotFoundError, name unless method
+
+    method
+  end
+
   def map_dirs(file_name, system=false)
     dirs = if system == :all then
              @all_dirs
@@ -318,6 +400,22 @@
     dirs.map { |dir| yield File.join(dir, file_name) }.flatten.compact
   end
 
+  ##
+  # Extract the class and method name parts from +name+ like Foo::Bar#baz
+
+  def parse_name(name)
+    parts = name.split(/(::|\#|\.)/)
+
+    if parts[-2] != '::' or parts.last !~ /^[A-Z]/ then
+      meth = parts.pop
+      parts.pop
+    end
+
+    klass = parts.join
+
+    [klass, meth]
+  end
+
   def populate_class_cache(class_cache, classes, extension = false)
     classes.each do |cdesc|
       desc = read_yaml cdesc
@@ -351,11 +449,6 @@
     YAML.load data
   end
 
-  def get_info_for(arg)
-    @names = [arg]
-    run
-  end
-
   def run
     if @names.empty? then
       @display.list_known_classes class_cache.keys.sort
@@ -368,15 +461,10 @@
           else
             meth = nil
 
-            parts = name.split(/::|\#|\./)
-            meth = parts.pop unless parts.last =~ /^[A-Z]/
-            klass = parts.join '::'
+            klass, meth = parse_name name
 
-            cache = load_cache_for klass
-            # HACK Does not support F.n
-            abort "Nothing known about #{name}" unless cache
-            method = cache[name.gsub(/\./, '#')]
-            abort "Nothing known about #{name}" unless method
+            method = lookup_method name, klass
+
             @display.display_method_info method
           end
         else
@@ -385,7 +473,7 @@
           else
             methods = select_methods(/^#{name}/)
             if methods.size == 0
-              abort "Nothing known about #{name}"
+              raise NotFoundError, name
             elsif methods.size == 1
               @display.display_method_info methods.first
             else
@@ -395,6 +483,8 @@
         end
       end
     end
+  rescue NotFoundError => e
+    abort e.message
   end
 
   def select_methods(pattern)
@@ -422,31 +512,3 @@
 
 end
 
-class Hash # HACK don't add stuff to Hash.
-  def method_missing method, *args
-    self[method.to_s]
-  end
-
-  def merge_enums(other)
-    other.each do |k,v|
-      if self[k] then
-        case v
-        when Array then
-          # HACK dunno
-          if String === self[k] and self[k].empty? then
-            self[k] = v
-          else
-            self[k] += v
-          end
-        when Hash then
-          self[k].merge! v
-        else
-          # do nothing
-        end
-      else
-        self[k] = v
-      end
-    end
-  end
-end
-
Index: lib/rdoc/known_classes.rb
===================================================================
--- lib/rdoc/known_classes.rb	(revision 0)
+++ lib/rdoc/known_classes.rb	(revision 18121)
@@ -0,0 +1,69 @@
+module RDoc
+
+  ##
+  # Ruby's built-in classes, modules and exceptions
+
+  KNOWN_CLASSES = {
+    "rb_cArray"            => "Array",
+    "rb_cBignum"           => "Bignum",
+    "rb_cClass"            => "Class",
+    "rb_cData"             => "Data",
+    "rb_cDir"              => "Dir",
+    "rb_cFalseClass"       => "FalseClass",
+    "rb_cFile"             => "File",
+    "rb_cFixnum"           => "Fixnum",
+    "rb_cFloat"            => "Float",
+    "rb_cHash"             => "Hash",
+    "rb_cIO"               => "IO",
+    "rb_cInteger"          => "Integer",
+    "rb_cModule"           => "Module",
+    "rb_cNilClass"         => "NilClass",
+    "rb_cNumeric"          => "Numeric",
+    "rb_cObject"           => "Object",
+    "rb_cProc"             => "Proc",
+    "rb_cRange"            => "Range",
+    "rb_cRegexp"           => "Regexp",
+    "rb_cRubyVM"           => "RubyVM",
+    "rb_cString"           => "String",
+    "rb_cStruct"           => "Struct",
+    "rb_cSymbol"           => "Symbol",
+    "rb_cThread"           => "Thread",
+    "rb_cTime"             => "Time",
+    "rb_cTrueClass"        => "TrueClass",
+
+    "rb_eArgError"         => "ArgError",
+    "rb_eEOFError"         => "EOFError",
+    "rb_eException"        => "Exception",
+    "rb_eFatal"            => "Fatal",
+    "rb_eFloatDomainError" => "FloatDomainError",
+    "rb_eIOError"          => "IOError",
+    "rb_eIndexError"       => "IndexError",
+    "rb_eInterrupt"        => "Interrupt",
+    "rb_eLoadError"        => "LoadError",
+    "rb_eNameError"        => "NameError",
+    "rb_eNoMemError"       => "NoMemError",
+    "rb_eNotImpError"      => "NotImpError",
+    "rb_eRangeError"       => "RangeError",
+    "rb_eRuntimeError"     => "RuntimeError",
+    "rb_eScriptError"      => "ScriptError",
+    "rb_eSecurityError"    => "SecurityError",
+    "rb_eSignal"           => "Signal",
+    "rb_eStandardError"    => "StandardError",
+    "rb_eSyntaxError"      => "SyntaxError",
+    "rb_eSystemCallError"  => "SystemCallError",
+    "rb_eSystemExit"       => "SystemExit",
+    "rb_eTypeError"        => "TypeError",
+    "rb_eZeroDivError"     => "ZeroDivError",
+
+    "rb_mComparable"       => "Comparable",
+    "rb_mEnumerable"       => "Enumerable",
+    "rb_mErrno"            => "Errno",
+    "rb_mFileTest"         => "FileTest",
+    "rb_mGC"               => "GC",
+    "rb_mKernel"           => "Kernel",
+    "rb_mMath"             => "Math",
+    "rb_mPrecision"        => "Precision",
+    "rb_mProcess"          => "Process"
+  }
+
+end
Index: lib/rdoc/ri.rb
===================================================================
--- lib/rdoc/ri.rb	(revision 18120)
+++ lib/rdoc/ri.rb	(revision 18121)
@@ -1,4 +1,8 @@
 require 'rdoc'
 
-module RDoc::RI; end
+module RDoc::RI
 
+  class Error < RDoc::Error; end
+
+end
+
Index: test/rdoc/test_rdoc_c_parser.rb
===================================================================
--- test/rdoc/test_rdoc_c_parser.rb	(revision 18120)
+++ test/rdoc/test_rdoc_c_parser.rb	(revision 18121)
@@ -1,261 +0,0 @@
-require 'stringio'
-require 'tempfile'
-require 'test/unit'
-require 'rdoc/parsers/parse_c'
-
-class RDoc::C_Parser
-  attr_accessor :classes
-
-  public :do_classes, :do_constants
-end
-
-class TestRdocC_Parser < Test::Unit::TestCase
-
-  def setup
-    @tempfile = Tempfile.new self.class.name
-    filename = @tempfile.path
-
-    @top_level = RDoc::TopLevel.new filename
-    @fn = filename
-    @options = RDoc::Options.new Hash.new
-    @stats = RDoc::Stats.new
-
-    @progress = StringIO.new
-  end
-
-  def teardown
-    @tempfile.close
-  end
-
-  def test_do_classes_boot_class
-    content = <<-EOF
-/* Document-class: Foo
- * this is the Foo boot class
- */
-VALUE cFoo = boot_defclass("Foo", 0);
-    EOF
-
-    klass = util_get_class content, 'cFoo'
-    assert_equal "   this is the Foo boot class\n   ", klass.comment
-  end
-
-  def test_do_classes_class
-    content = <<-EOF
-/* Document-class: Foo
- * this is the Foo class
- */
-VALUE cFoo = rb_define_class("Foo", rb_cObject);
-    EOF
-
-    klass = util_get_class content, 'cFoo'
-    assert_equal "   this is the Foo class\n   ", klass.comment
-  end
-
-  def test_do_classes_class_under
-    content = <<-EOF
-/* Document-class: Kernel::Foo
- * this is the Foo class under Kernel
- */
-VALUE cFoo = rb_define_class_under(rb_mKernel, "Foo", rb_cObject);
-    EOF
-
-    klass = util_get_class content, 'cFoo'
-    assert_equal "   this is the Foo class under Kernel\n   ", klass.comment
-  end
-
-  def test_do_classes_module
-    content = <<-EOF
-/* Document-module: Foo
- * this is the Foo module
- */
-VALUE mFoo = rb_define_module("Foo");
-    EOF
-
-    klass = util_get_class content, 'mFoo'
-    assert_equal "   this is the Foo module\n   ", klass.comment
-  end
-
-  def test_do_classes_module_under
-    content = <<-EOF
-/* Document-module: Kernel::Foo
- * this is the Foo module under Kernel
- */
-VALUE mFoo = rb_define_module_under(rb_mKernel, "Foo");
-    EOF
-
-    klass = util_get_class content, 'mFoo'
-    assert_equal "   this is the Foo module under Kernel\n   ", klass.comment
-  end
-
-  def test_do_constants
-    content = <<-EOF
-#include <ruby.h>
-
-void Init_foo(){
-   VALUE cFoo = rb_define_class("Foo", rb_cObject);
-
-   /* 300: The highest possible score in bowling */
-   rb_define_const(cFoo, "PERFECT", INT2FIX(300));
-
-   /* Huzzah!: What you cheer when you roll a perfect game */
-   rb_define_const(cFoo, "CHEER", rb_str_new2("Huzzah!"));
-
-   /* TEST\:TEST: Checking to see if escaped semicolon works */
-   rb_define_const(cFoo, "TEST", rb_str_new2("TEST:TEST"));
-
-   /* \\: The file separator on MS Windows */
-   rb_define_const(cFoo, "MSEPARATOR", rb_str_new2("\\"));
-
-   /* /: The file separator on Unix */
-   rb_define_const(cFoo, "SEPARATOR", rb_str_new2("/"));
-
-   /* C:\\Program Files\\Stuff: A directory on MS Windows */
-   rb_define_const(cFoo, "STUFF", rb_str_new2("C:\\Program Files\\Stuff"));
-
-   /* Default definition */
-   rb_define_const(cFoo, "NOSEMI", INT2FIX(99));
-
-   rb_define_const(cFoo, "NOCOMMENT", rb_str_new2("No comment"));
-
-   /*
-    * Multiline comment goes here because this comment spans multiple lines.
-    * Multiline comment goes here because this comment spans multiple lines.
-    */
-   rb_define_const(cFoo, "MULTILINE", INT2FIX(1));
-
-   /*
-    * 1: Multiline comment goes here because this comment spans multiple lines.
-    * Multiline comment goes here because this comment spans multiple lines.
-    */
-   rb_define_const(cFoo, "MULTILINE_VALUE", INT2FIX(1));
-
-   /* Multiline comment goes here because this comment spans multiple lines.
-    * Multiline comment goes here because this comment spans multiple lines.
-    */
-   rb_define_const(cFoo, "MULTILINE_NOT_EMPTY", INT2FIX(1));
-
-}
-    EOF
-
-    parser = util_parser content
-
-    parser.do_classes
-    parser.do_constants
-
-    klass = parser.classes['cFoo']
-    assert klass
-
-    constants = klass.constants
-    assert !klass.constants.empty?
-
-    constants = constants.map { |c| [c.name, c.value, c.comment] }
-
-    assert_equal ['PERFECT', '300',
-                  "\n      The highest possible score in bowling   \n   "],
-                 constants.shift
-    assert_equal ['CHEER', 'Huzzah!',
-                  "\n      What you cheer when you roll a perfect game   \n   "],
-                 constants.shift
-    assert_equal ['TEST', 'TEST:TEST',
-                  "\n      Checking to see if escaped semicolon works   \n   "],
-                 constants.shift
-    assert_equal ['MSEPARATOR', '\\',
-                  "\n      The file separator on MS Windows   \n   "],
-                 constants.shift
-    assert_equal ['SEPARATOR', '/',
-                  "\n      The file separator on Unix   \n   "],
-                 constants.shift
-    assert_equal ['STUFF', 'C:\\Program Files\\Stuff',
-                  "\n      A directory on MS Windows   \n   "],
-                 constants.shift
-    assert_equal ['NOSEMI', 'INT2FIX(99)',
-                  "\n      Default definition   \n   "],
-                 constants.shift
-    assert_equal ['NOCOMMENT', 'rb_str_new2("No comment")', nil],
-                 constants.shift
-
-    comment = <<-EOF.chomp
-
-     
-      Multiline comment goes here because this comment spans multiple lines.
-      Multiline comment goes here because this comment spans multiple lines.
-      
-   
-    EOF
-    assert_equal ['MULTILINE', 'INT2FIX(1)', comment], constants.shift
-    assert_equal ['MULTILINE_VALUE', '1', comment], constants.shift
-
-    comment = <<-EOF.chomp
-
-      Multiline comment goes here because this comment spans multiple lines.
-      Multiline comment goes here because this comment spans multiple lines.
-      
-   
-    EOF
-    assert_equal ['MULTILINE_NOT_EMPTY', 'INT2FIX(1)', comment], constants.shift
-
-    assert constants.empty?, constants.inspect
-  end
-
-  def test_find_class_comment_init
-    content = <<-EOF
-/*
- * a comment for class Foo
- */
-void
-Init_Foo(void) {
-  VALUE foo = rb_define_class("Foo", rb_cObject);
-}
-    EOF
-
-    klass = util_get_class content, 'foo'
-
-    assert_equal "  \n   a comment for class Foo\n   \n", klass.comment
-  end
-
-  def test_find_class_comment_define_class
-    content = <<-EOF
-/*
- * a comment for class Foo
- */
-VALUE foo = rb_define_class("Foo", rb_cObject);
-    EOF
-
-    klass = util_get_class content, 'foo'
-
-    assert_equal "  \n   a comment for class Foo\n   ", klass.comment
-  end
-
-  def test_find_class_comment_define_class_Init_Foo
-    content = <<-EOF
-/*
- * a comment for class Foo on Init
- */
-void
-Init_Foo(void) {
-    /*
-     * a comment for class Foo on rb_define_class
-     */
-    VALUE foo = rb_define_class("Foo", rb_cObject);
-}
-    EOF
-
-    klass = util_get_class content, 'foo'
-
-    assert_equal "  \n   a comment for class Foo on Init\n   \n", klass.comment
-  end
-
-  def util_get_class(content, name)
-    parser = util_parser content
-    parser.do_classes
-    parser.classes[name]
-  end
-
-  def util_parser(content)
-    parser = RDoc::C_Parser.new @top_level, @fn, content, @options, @stats
-    parser.progress = @progress
-    parser
-  end
-
-end
-
Index: test/rdoc/test_rdoc_markup_to_html_crossref.rb
===================================================================
--- test/rdoc/test_rdoc_markup_to_html_crossref.rb	(revision 0)
+++ test/rdoc/test_rdoc_markup_to_html_crossref.rb	(revision 18121)
@@ -0,0 +1,18 @@
+require 'test/unit'
+require 'rdoc/generator'
+require 'rdoc/markup/to_html_crossref'
+
+class TestRdocMarkupToHtmlCrossref < Test::Unit::TestCase
+
+  def setup
+    @xref = RDoc::Markup::ToHtmlCrossref.new 'from_path', nil, nil
+  end
+
+  def test_handle_special_CROSSREF_no_underscore
+    out = @xref.convert 'foo'
+
+    assert_equal "<p>\nfoo\n</p>\n", out
+  end
+
+end
+
Index: test/rdoc/test_rdoc_parser_ruby.rb
===================================================================
--- test/rdoc/test_rdoc_parser_ruby.rb	(revision 0)
+++ test/rdoc/test_rdoc_parser_ruby.rb	(revision 18121)
@@ -0,0 +1,500 @@
+require 'stringio'
+require 'tempfile'
+require 'test/unit'
+
+require 'rdoc/options'
+require 'rdoc/parser/ruby'
+require 'rdoc/stats'
+
+class TestRdocParserRuby < Test::Unit::TestCase
+
+  def setup
+    @tempfile = Tempfile.new self.class.name
+    @filename = @tempfile.path
+
+    util_toplevel
+    @options = RDoc::Options.new Hash.new
+    @options.quiet = true
+    @stats = RDoc::Stats.new
+
+    @progress = StringIO.new
+  end
+
+  def teardown
+    @tempfile.unlink
+  end
+
+  def test_look_for_directives_in_commented
+    util_parser ""
+
+    comment = "# how to make a section:\n# # :section: new section\n"
+
+    @parser.look_for_directives_in @top_level, comment
+
+    section = @top_level.current_section
+    assert_equal nil, section.title
+    assert_equal nil, section.comment
+
+    assert_equal "# how to make a section:\n# # :section: new section\n",
+                 comment
+  end
+
+  def test_look_for_directives_in_enddoc
+    util_parser ""
+
+    assert_throws :enddoc do
+      @parser.look_for_directives_in @top_level, "# :enddoc:\n"
+    end
+  end
+
+  def test_look_for_directives_in_main
+    util_parser ""
+
+    @parser.look_for_directives_in @top_level, "# :main: new main page\n"
+
+    assert_equal 'new main page', @options.main_page
+  end
+
+  def test_look_for_directives_in_method
+    util_parser ""
+
+    comment = "# :method: my_method\n"
+
+    @parser.look_for_directives_in @top_level, comment
+
+    assert_equal "# :method: my_method\n", comment
+
+    comment = "# :singleton-method: my_method\n"
+
+    @parser.look_for_directives_in @top_level, comment
+
+    assert_equal "# :singleton-method: my_method\n", comment
+  end
+
+  def test_look_for_directives_in_startdoc
+    util_parser ""
+
+    @top_level.stop_doc
+    assert !@top_level.document_self
+    assert !@top_level.document_children
+    assert !@top_level.force_documentation
+
+    @parser.look_for_directives_in @top_level, "# :startdoc:\n"
+
+    assert @top_level.document_self
+    assert @top_level.document_children
+    assert @top_level.force_documentation
+  end
+
+  def test_look_for_directives_in_stopdoc
+    util_parser ""
+
+    assert @top_level.document_self
+    assert @top_level.document_children
+
+    @parser.look_for_directives_in @top_level, "# :stopdoc:\n"
+
+    assert !@top_level.document_self
+    assert !@top_level.document_children
+  end
+
+  def test_look_for_directives_in_section
+    util_parser ""
+
+    comment = "# :section: new section\n# woo stuff\n"
+
+    @parser.look_for_directives_in @top_level, comment
+
+    section = @top_level.current_section
+    assert_equal 'new section', section.title
+    assert_equal "# woo stuff\n", section.comment
+
+    assert_equal '', comment
+  end
+
+  def test_look_for_directives_in_title
+    util_parser ""
+
+    @parser.look_for_directives_in @top_level, "# :title: new title\n"
+
+    assert_equal 'new title', @options.title
+  end
+
+  def test_look_for_directives_in_unhandled
+    util_parser ""
+
+    comment = "# :unhandled: \n# :title: hi\n"
+
+    @parser.look_for_directives_in @top_level, comment
+
+    assert_equal "# :unhandled: \n", comment
+
+    assert_equal 'hi', @options.title
+  end
+
+  def test_parse_meta_method
+    klass = RDoc::NormalClass.new 'Foo'
+    klass.parent = @top_level
+
+    comment = "##\n# my method\n"
+
+    util_parser "add_my_method :foo, :bar\nadd_my_method :baz"
+
+    tk = @parser.get_tk
+
+    @parser.parse_meta_method klass, RDoc::Parser::Ruby::NORMAL, tk, comment
+
+    foo = klass.method_list.first
+    assert_equal 'foo', foo.name
+    assert_equal comment, foo.comment
+
+    assert_equal [],      foo.aliases
+    assert_equal nil,     foo.block_params
+    assert_equal nil,     foo.call_seq
+    assert_equal true,    foo.document_children
+    assert_equal true,    foo.document_self
+    assert_equal false,   foo.done_documenting
+    assert_equal false,   foo.dont_rename_initialize
+    assert_equal false,   foo.force_documentation
+    assert_equal nil,     foo.is_alias_for
+    assert_equal '',      foo.params
+    assert_equal klass,   foo.parent
+    assert_equal false,   foo.singleton
+    assert_equal 'add_my_method :foo', foo.text
+    assert_equal nil,     foo.viewer
+    assert_equal :public, foo.visibility
+    assert_equal klass.current_section, foo.section
+
+    stream = [
+      tk(:COMMENT, 1, 1, nil, "# File #{@top_level.file_absolute_name}, line 1"),
+      RDoc::Parser::Ruby::NEWLINE_TOKEN,
+      tk(:SPACE,      1, 1,  nil, ''),
+      tk(:IDENTIFIER, 1, 0,  'add_my_method', 'add_my_method'),
+      tk(:SPACE,      1, 13, nil, ' '),
+      tk(:SYMBOL,     1, 14, nil, ':foo'),
+      tk(:COMMA,      1, 18, nil, ','),
+      tk(:SPACE,      1, 19, nil, ' '),
+      tk(:SYMBOL,     1, 20, nil, ':bar'),
+      tk(:NL,         1, 24, nil, "\n"),
+    ]
+
+    assert_equal stream, foo.token_stream
+  end
+
+  def test_parse_meta_method_name
+    klass = RDoc::NormalClass.new 'Foo'
+    klass.parent = @top_level
+
+    comment = "##\n# :method: woo_hoo!\n# my method\n"
+
+    util_parser "add_my_method :foo, :bar\nadd_my_method :baz"
+
+    tk = @parser.get_tk
+
+    @parser.parse_meta_method klass, RDoc::Parser::Ruby::NORMAL, tk, comment
+
+    foo = klass.method_list.first
+    assert_equal 'woo_hoo!', foo.name
+    assert_equal "##\n# my method\n", foo.comment
+  end
+
+  def test_parse_meta_method_singleton
+    klass = RDoc::NormalClass.new 'Foo'
+    klass.parent = @top_level
+
+    comment = "##\n# :singleton-method:\n# my method\n"
+
+    util_parser "add_my_method :foo, :bar\nadd_my_method :baz"
+
+    tk = @parser.get_tk
+
+    @parser.parse_meta_method klass, RDoc::Parser::Ruby::NORMAL, tk, comment
+
+    foo = klass.method_list.first
+    assert_equal 'foo', foo.name
+    assert_equal true, foo.singleton, 'singleton method'
+    assert_equal "##\n# my method\n", foo.comment
+  end
+
+  def test_parse_meta_method_singleton_name
+    klass = RDoc::NormalClass.new 'Foo'
+    klass.parent = @top_level
+
+    comment = "##\n# :singleton-method: woo_hoo!\n# my method\n"
+
+    util_parser "add_my_method :foo, :bar\nadd_my_method :baz"
+
+    tk = @parser.get_tk
+
+    @parser.parse_meta_method klass, RDoc::Parser::Ruby::NORMAL, tk, comment
+
+    foo = klass.method_list.first
+    assert_equal 'woo_hoo!', foo.name
+    assert_equal true, foo.singleton, 'singleton method'
+    assert_equal "##\n# my method\n", foo.comment
+  end
+
+  def test_parse_meta_method_string_name
+    klass = RDoc::NormalClass.new 'Foo'
+    comment = "##\n# my method\n"
+
+    util_parser "add_my_method 'foo'"
+
+    tk = @parser.get_tk
+
+    @parser.parse_meta_method klass, RDoc::Parser::Ruby::NORMAL, tk, comment
+
+    foo = klass.method_list.first
+    assert_equal 'foo', foo.name
+    assert_equal comment, foo.comment
+  end
+
+  def test_parse_method
+    klass = RDoc::NormalClass.new 'Foo'
+    klass.parent = @top_level
+
+    comment = "##\n# my method\n"
+
+    util_parser "def foo() :bar end"
+
+    tk = @parser.get_tk
+
+    @parser.parse_method klass, RDoc::Parser::Ruby::NORMAL, tk, comment
+
+    foo = klass.method_list.first
+    assert_equal 'foo',     foo.name
+    assert_equal comment,   foo.comment
+
+    assert_equal [],        foo.aliases
+    assert_equal nil,       foo.block_params
+    assert_equal nil,       foo.call_seq
+    assert_equal nil,       foo.is_alias_for
+    assert_equal nil,       foo.viewer
+    assert_equal true,      foo.document_children
+    assert_equal true,      foo.document_self
+    assert_equal '()',      foo.params
+    assert_equal false,     foo.done_documenting
+    assert_equal false,     foo.dont_rename_initialize
+    assert_equal false,     foo.force_documentation
+    assert_equal klass,     foo.parent
+    assert_equal false,     foo.singleton
+    assert_equal :public,   foo.visibility
+    assert_equal 'def foo', foo.text
+    assert_equal klass.current_section, foo.section
+
+    stream = [
+      tk(:COMMENT, 1, 1, nil, "# File #{@top_level.file_absolute_name}, line 1"),
+      RDoc::Parser::Ruby::NEWLINE_TOKEN,
+      tk(:SPACE,      1, 1,  nil,   ''),
+      tk(:DEF,        1, 0,  'def', 'def'),
+      tk(:SPACE,      1, 3,  nil,   ' '),
+      tk(:IDENTIFIER, 1, 4,  'foo', 'foo'),
+      tk(:LPAREN,     1, 7,  nil,   '('),
+      tk(:RPAREN,     1, 8,  nil,   ')'),
+      tk(:SPACE,      1, 9,  nil,   ' '),
+      tk(:COLON,      1, 10, nil,   ':'),
+      tk(:IDENTIFIER, 1, 11, 'bar', 'bar'),
+      tk(:SPACE,      1, 14, nil,   ' '),
+      tk(:END,        1, 15, 'end', 'end'),
+    ]
+
+    assert_equal stream, foo.token_stream
+  end
+
+  def test_parse_statements_comment
+    content = <<-EOF
+class Foo
+  ##
+  # :method: my_method
+  # my method comment
+
+end
+    EOF
+    klass = RDoc::NormalClass.new 'Foo'
+    klass.parent = @top_level
+
+    comment = "##\n# :method: foo\n# my method\n"
+
+    util_parser "\n"
+
+    tk = @parser.get_tk
+
+    @parser.parse_comment klass, tk, comment
+
+    foo = klass.method_list.first
+    assert_equal 'foo',     foo.name
+    assert_equal comment,   foo.comment
+
+    assert_equal [],        foo.aliases
+    assert_equal nil,       foo.block_params
+    assert_equal nil,       foo.call_seq
+    assert_equal nil,       foo.is_alias_for
+    assert_equal nil,       foo.viewer
+    assert_equal true,      foo.document_children
+    assert_equal true,      foo.document_self
+    assert_equal '',        foo.params
+    assert_equal false,     foo.done_documenting
+    assert_equal false,     foo.dont_rename_initialize
+    assert_equal false,     foo.force_documentation
+    assert_equal klass,     foo.parent
+    assert_equal false,     foo.singleton
+    assert_equal :public,   foo.visibility
+    assert_equal "\n",      foo.text
+    assert_equal klass.current_section, foo.section
+
+    stream = [
+      tk(:COMMENT, 1, 1, nil, "# File #{@top_level.file_absolute_name}, line 1"),
+      RDoc::Parser::Ruby::NEWLINE_TOKEN,
+      tk(:SPACE,      1, 1,  nil,   ''),
+    ]
+
+    assert_equal stream, foo.token_stream
+  end
+
+  def test_parse_statements_identifier_meta_method
+    content = <<-EOF
+class Foo
+  ##
+  # this is my method
+  add_my_method :foo
+end
+    EOF
+
+    util_parser content
+
+    @parser.parse_statements @top_level, RDoc::Parser::Ruby::NORMAL, nil, ''
+
+    foo = @top_level.classes.first.method_list.first
+    assert_equal 'foo', foo.name
+  end
+
+  def test_parse_statements_identifier_alias_method
+    content = "class Foo def foo() end; alias_method :foo2, :foo end"
+
+    util_parser content
+
+    @parser.parse_statements @top_level, RDoc::Parser::Ruby::NORMAL, nil, ''
+
+    foo2 = @top_level.classes.first.method_list.last
+    assert_equal 'foo2', foo2.name
+    assert_equal 'foo', foo2.is_alias_for.name
+  end
+
+  def test_parse_statements_identifier_attr
+    content = "class Foo; attr :foo; end"
+
+    util_parser content
+
+    @parser.parse_statements @top_level, RDoc::Parser::Ruby::NORMAL, nil, ''
+
+    foo = @top_level.classes.first.attributes.first
+    assert_equal 'foo', foo.name
+    assert_equal 'R', foo.rw
+  end
+
+  def test_parse_statements_identifier_attr_accessor
+    content = "class Foo; attr_accessor :foo; end"
+
+    util_parser content
+
+    @parser.parse_statements @top_level, RDoc::Parser::Ruby::NORMAL, nil, ''
+
+    foo = @top_level.classes.first.attributes.first
+    assert_equal 'foo', foo.name
+    assert_equal 'RW', foo.rw
+  end
+
+  def test_parse_statements_identifier_extra_accessors
+    @options.extra_accessors = /^my_accessor$/
+
+    content = "class Foo; my_accessor :foo; end"
+
+    util_parser content
+
+    @parser.parse_statements @top_level, RDoc::Parser::Ruby::NORMAL, nil, ''
+
+    foo = @top_level.classes.first.attributes.first
+    assert_equal 'foo', foo.name
+    assert_equal '?', foo.rw
+  end
+
+  def test_parse_statements_identifier_include
+    content = "class Foo; include Bar; end"
+
+    util_parser content
+
+    @parser.parse_statements @top_level, RDoc::Parser::Ruby::NORMAL, nil, ''
+
+    foo = @top_level.classes.first
+    assert_equal 'Foo', foo.name
+    assert_equal 1, foo.includes.length
+  end
+
+  def test_parse_statements_identifier_module_function
+    content = "module Foo def foo() end; module_function :foo; end"
+
+    util_parser content
+
+    @parser.parse_statements @top_level, RDoc::Parser::Ruby::NORMAL, nil, ''
+
+    foo, s_foo = @top_level.modules.first.method_list
+    assert_equal 'foo', foo.name, 'instance method name'
+    assert_equal :private, foo.visibility, 'instance method visibility'
+    assert_equal false, foo.singleton, 'instance method singleton'
+
+    assert_equal 'foo', s_foo.name, 'module function name'
+    assert_equal :public, s_foo.visibility, 'module function visibility'
+    assert_equal true, s_foo.singleton, 'module function singleton'
+  end
+
+  def test_parse_statements_identifier_private
+    content = "class Foo private; def foo() end end"
+
+    util_parser content
+
+    @parser.parse_statements @top_level, RDoc::Parser::Ruby::NORMAL, nil, ''
+
+    foo = @top_level.classes.first.method_list.first
+    assert_equal 'foo', foo.name
+    assert_equal :private, foo.visibility
+  end
+
+  def test_parse_statements_identifier_require
+    content = "require 'bar'"
+
+    util_parser content
+
+    @parser.parse_statements @top_level, RDoc::Parser::Ruby::NORMAL, nil, ''
+
+    assert_equal 1, @top_level.requires.length
+  end
+
+  def tk(klass, line, char, name, text)
+    klass = RDoc::RubyToken.const_get "Tk#{klass.to_s.upcase}"
+
+    token = if klass.instance_method(:initialize).arity == 2 then
+              raise ArgumentError, "name not used for #{klass}" unless name.nil?
+              klass.new line, char
+            else
+              klass.new line, char, name
+            end
+
+    token.set_text text
+
+    token
+  end
+
+  def util_parser(content)
+    @parser = RDoc::Parser::Ruby.new @top_level, @filename, content, @options,
+                                     @stats
+    @parser.progress = @progress
+    @parser
+  end
+
+  def util_toplevel
+    RDoc::TopLevel.reset
+    @top_level = RDoc::TopLevel.new @filename
+  end
+
+end
+
Index: test/rdoc/test_rdoc_parser_c.rb
===================================================================
--- test/rdoc/test_rdoc_parser_c.rb	(revision 0)
+++ test/rdoc/test_rdoc_parser_c.rb	(revision 18121)
@@ -0,0 +1,262 @@
+require 'stringio'
+require 'tempfile'
+require 'test/unit'
+require 'rdoc/options'
+require 'rdoc/parser/c'
+
+class RDoc::Parser::C
+  attr_accessor :classes
+
+  public :do_classes, :do_constants
+end
+
+class TestRdocParserC < Test::Unit::TestCase
+
+  def setup
+    @tempfile = Tempfile.new self.class.name
+    filename = @tempfile.path
+
+    @top_level = RDoc::TopLevel.new filename
+    @fn = filename
+    @options = RDoc::Options.new Hash.new
+    @stats = RDoc::Stats.new
+
+    @progress = StringIO.new
+  end
+
+  def teardown
+    @tempfile.close
+  end
+
+  def test_do_classes_boot_class
+    content = <<-EOF
+/* Document-class: Foo
+ * this is the Foo boot class
+ */
+VALUE cFoo = boot_defclass("Foo", 0);
+    EOF
+
+    klass = util_get_class content, 'cFoo'
+    assert_equal "   this is the Foo boot class\n   ", klass.comment
+  end
+
+  def test_do_classes_class
+    content = <<-EOF
+/* Document-class: Foo
+ * this is the Foo class
+ */
+VALUE cFoo = rb_define_class("Foo", rb_cObject);
+    EOF
+
+    klass = util_get_class content, 'cFoo'
+    assert_equal "   this is the Foo class\n   ", klass.comment
+  end
+
+  def test_do_classes_class_under
+    content = <<-EOF
+/* Document-class: Kernel::Foo
+ * this is the Foo class under Kernel
+ */
+VALUE cFoo = rb_define_class_under(rb_mKernel, "Foo", rb_cObject);
+    EOF
+
+    klass = util_get_class content, 'cFoo'
+    assert_equal "   this is the Foo class under Kernel\n   ", klass.comment
+  end
+
+  def test_do_classes_module
+    content = <<-EOF
+/* Document-module: Foo
+ * this is the Foo module
+ */
+VALUE mFoo = rb_define_module("Foo");
+    EOF
+
+    klass = util_get_class content, 'mFoo'
+    assert_equal "   this is the Foo module\n   ", klass.comment
+  end
+
+  def test_do_classes_module_under
+    content = <<-EOF
+/* Document-module: Kernel::Foo
+ * this is the Foo module under Kernel
+ */
+VALUE mFoo = rb_define_module_under(rb_mKernel, "Foo");
+    EOF
+
+    klass = util_get_class content, 'mFoo'
+    assert_equal "   this is the Foo module under Kernel\n   ", klass.comment
+  end
+
+  def test_do_constants
+    content = <<-EOF
+#include <ruby.h>
+
+void Init_foo(){
+   VALUE cFoo = rb_define_class("Foo", rb_cObject);
+
+   /* 300: The highest possible score in bowling */
+   rb_define_const(cFoo, "PERFECT", INT2FIX(300));
+
+   /* Huzzah!: What you cheer when you roll a perfect game */
+   rb_define_const(cFoo, "CHEER", rb_str_new2("Huzzah!"));
+
+   /* TEST\:TEST: Checking to see if escaped semicolon works */
+   rb_define_const(cFoo, "TEST", rb_str_new2("TEST:TEST"));
+
+   /* \\: The file separator on MS Windows */
+   rb_define_const(cFoo, "MSEPARATOR", rb_str_new2("\\"));
+
+   /* /: The file separator on Unix */
+   rb_define_const(cFoo, "SEPARATOR", rb_str_new2("/"));
+
+   /* C:\\Program Files\\Stuff: A directory on MS Windows */
+   rb_define_const(cFoo, "STUFF", rb_str_new2("C:\\Program Files\\Stuff"));
+
+   /* Default definition */
+   rb_define_const(cFoo, "NOSEMI", INT2FIX(99));
+
+   rb_define_const(cFoo, "NOCOMMENT", rb_str_new2("No comment"));
+
+   /*
+    * Multiline comment goes here because this comment spans multiple lines.
+    * Multiline comment goes here because this comment spans multiple lines.
+    */
+   rb_define_const(cFoo, "MULTILINE", INT2FIX(1));
+
+   /*
+    * 1: Multiline comment goes here because this comment spans multiple lines.
+    * Multiline comment goes here because this comment spans multiple lines.
+    */
+   rb_define_const(cFoo, "MULTILINE_VALUE", INT2FIX(1));
+
+   /* Multiline comment goes here because this comment spans multiple lines.
+    * Multiline comment goes here because this comment spans multiple lines.
+    */
+   rb_define_const(cFoo, "MULTILINE_NOT_EMPTY", INT2FIX(1));
+
+}
+    EOF
+
+    parser = util_parser content
+
+    parser.do_classes
+    parser.do_constants
+
+    klass = parser.classes['cFoo']
+    assert klass
+
+    constants = klass.constants
+    assert !klass.constants.empty?
+
+    constants = constants.map { |c| [c.name, c.value, c.comment] }
+
+    assert_equal ['PERFECT', '300',
+                  "\n      The highest possible score in bowling   \n   "],
+                 constants.shift
+    assert_equal ['CHEER', 'Huzzah!',
+                  "\n      What you cheer when you roll a perfect game   \n   "],
+                 constants.shift
+    assert_equal ['TEST', 'TEST:TEST',
+                  "\n      Checking to see if escaped semicolon works   \n   "],
+                 constants.shift
+    assert_equal ['MSEPARATOR', '\\',
+                  "\n      The file separator on MS Windows   \n   "],
+                 constants.shift
+    assert_equal ['SEPARATOR', '/',
+                  "\n      The file separator on Unix   \n   "],
+                 constants.shift
+    assert_equal ['STUFF', 'C:\\Program Files\\Stuff',
+                  "\n      A directory on MS Windows   \n   "],
+                 constants.shift
+    assert_equal ['NOSEMI', 'INT2FIX(99)',
+                  "\n      Default definition   \n   "],
+                 constants.shift
+    assert_equal ['NOCOMMENT', 'rb_str_new2("No comment")', nil],
+                 constants.shift
+
+    comment = <<-EOF.chomp
+
+     
+      Multiline comment goes here because this comment spans multiple lines.
+      Multiline comment goes here because this comment spans multiple lines.
+      
+   
+    EOF
+    assert_equal ['MULTILINE', 'INT2FIX(1)', comment], constants.shift
+    assert_equal ['MULTILINE_VALUE', '1', comment], constants.shift
+
+    comment = <<-EOF.chomp
+
+      Multiline comment goes here because this comment spans multiple lines.
+      Multiline comment goes here because this comment spans multiple lines.
+      
+   
+    EOF
+    assert_equal ['MULTILINE_NOT_EMPTY', 'INT2FIX(1)', comment], constants.shift
+
+    assert constants.empty?, constants.inspect
+  end
+
+  def test_find_class_comment_init
+    content = <<-EOF
+/*
+ * a comment for class Foo
+ */
+void
+Init_Foo(void) {
+  VALUE foo = rb_define_class("Foo", rb_cObject);
+}
+    EOF
+
+    klass = util_get_class content, 'foo'
+
+    assert_equal "  \n   a comment for class Foo\n   \n", klass.comment
+  end
+
+  def test_find_class_comment_define_class
+    content = <<-EOF
+/*
+ * a comment for class Foo
+ */
+VALUE foo = rb_define_class("Foo", rb_cObject);
+    EOF
+
+    klass = util_get_class content, 'foo'
+
+    assert_equal "  \n   a comment for class Foo\n   ", klass.comment
+  end
+
+  def test_find_class_comment_define_class_Init_Foo
+    content = <<-EOF
+/*
+ * a comment for class Foo on Init
+ */
+void
+Init_Foo(void) {
+    /*
+     * a comment for class Foo on rb_define_class
+     */
+    VALUE foo = rb_define_class("Foo", rb_cObject);
+}
+    EOF
+
+    klass = util_get_class content, 'foo'
+
+    assert_equal "  \n   a comment for class Foo on Init\n   \n", klass.comment
+  end
+
+  def util_get_class(content, name)
+    parser = util_parser content
+    parser.do_classes
+    parser.classes[name]
+  end
+
+  def util_parser(content)
+    parser = RDoc::Parser::C.new @top_level, @fn, content, @options, @stats
+    parser.progress = @progress
+    parser
+  end
+
+end
+
Index: test/rdoc/test_rdoc_info_sections.rb
===================================================================
--- test/rdoc/test_rdoc_info_sections.rb	(revision 0)
+++ test/rdoc/test_rdoc_info_sections.rb	(revision 18121)
@@ -0,0 +1,93 @@
+$LOAD_PATH.unshift File.dirname(__FILE__) + '/../lib/'
+require 'fileutils'
+require 'test/unit'
+require 'rdoc/generator/texinfo'
+require 'yaml'
+
+# give us access to check this stuff before it's rendered
+class RDoc::Generator::Texinfo; attr_reader :files, :classes; end
+class RDoc::RDoc; attr_reader :options; attr_reader :gen; end
+
+class TestRdocInfoSections < Test::Unit::TestCase
+  OUTPUT_DIR = "/tmp/rdoc-#{$$}"
+
+  def setup
+    # supress stdout
+    $stdout = File.new('/dev/null','w')
+    $stderr = File.new('/dev/null','w')
+
+    @rdoc = RDoc::RDoc.new
+    @rdoc.document(['--fmt=texinfo',
+                    File.expand_path(File.dirname(__FILE__) + '/../lib/rdoc/generator/texinfo.rb'),
+                    File.expand_path(File.dirname(__FILE__) + '/../README.txt'),
+                    "--op=#{OUTPUT_DIR}"])
+    @text = File.read(OUTPUT_DIR + '/rdoc.texinfo')
+  end
+
+  def teardown
+    $stdout = STDOUT
+    $stderr = STDERR
+    FileUtils.rm_rf OUTPUT_DIR
+  end
+
+  def test_output_exists
+    assert ! @text.empty?
+  end
+
+  def test_each_class_has_a_chapter
+    assert_section "Class RDoc::Generator::Texinfo", '@chapter'
+    assert_section "Class RDoc::Generator::TexinfoTemplate", '@chapter'
+  end
+
+  def test_class_descriptions_are_given
+    assert_match(/This generates .*Texinfo.* files for viewing with GNU Info or Emacs from .*RDoc.* extracted from Ruby source files/, @text.gsub("\n", ' '))
+  end
+
+  def test_included_modules_are_given
+    assert_match(/Includes.* Generator::MarkUp/m, @text)
+  end
+
+  def test_class_methods_are_given
+    assert_match(/new\(options\)/, @text)
+  end
+
+  def test_classes_instance_methods_are_given
+    assert_section 'Class RDoc::Generator::Texinfo#generate'
+    assert_match(/generate\(toplevels\)/, @text)
+  end
+
+  def test_each_module_has_a_chapter
+    assert_section "RDoc", '@chapter'
+    assert_section "Generator", '@chapter'
+  end
+
+  def test_methods_are_shown_only_once
+    methods = @rdoc.gen.classes.map { |c| c.methods.map{ |m| c.name + '#' + m.name } }.flatten
+    assert_equal methods, methods.uniq
+  end
+
+#   if system "makeinfo --version > /dev/null"
+#     def test_compiles_to_info
+#       makeinfo_output = `cd #{OUTPUT_DIR} && makeinfo rdoc.texinfo`
+#       assert(File.exist?(File.join(OUTPUT_DIR, 'rdoc.info')),
+#              "Info file was not compiled: #{makeinfo_output}")
+#     end
+#   end
+
+#   def test_constants_are_documented_somehow
+#     assert_section 'DEFAULT_FILENAME' # what kind of section?
+#     assert_section 'DEFAULT_INFO_FILENAME'
+#   end
+
+#   def test_oh_yeah_dont_forget_files
+#   end
+
+  private
+  def assert_section(name, command = '@section')
+    assert_match Regexp.new("^#{command}.*#{Regexp.escape name}"), @text, "Could not find a #{command} #{name}"
+  end
+
+#   def puts(*args)
+#     @real_stdout.puts(*args)
+#   end
+end
Index: test/rdoc/test_rdoc_ri_default_display.rb
===================================================================
--- test/rdoc/test_rdoc_ri_default_display.rb	(revision 18120)
+++ test/rdoc/test_rdoc_ri_default_display.rb	(revision 18121)
@@ -4,7 +4,7 @@
 require 'rdoc/ri/display'
 require 'rdoc/ri/driver'
 
-class TestRDocRIDefaultDisplay < Test::Unit::TestCase
+class TestRdocRiDefaultDisplay < Test::Unit::TestCase
 
   def setup
     @output = StringIO.new
@@ -14,7 +14,7 @@
     @dd = RDoc::RI::DefaultDisplay.new RDoc::RI::Formatter, @width, true,
                                        @output
 
-    @some_method = {
+    @some_method = h \
       'aliases' => [{'name' => 'some_method_alias'}],
       'block_params' => 'block_param',
       'comment' => [RDoc::Markup::Flow::P.new('some comment')],
@@ -23,13 +23,12 @@
       'name' => 'some_method',
       'params' => '(arg1, arg2) {|block_param| ...}',
       'source_path' => '/nonexistent',
-      'visibility' => 'public',
-    }
+      'visibility' => 'public'
   end
 
   def test_display_class_info
     ri_reader = nil
-    klass = {
+    klass = h \
       'attributes' => [
         { 'name' => 'attribute', 'rw' => 'RW',
           'comment' => [RDoc::Markup::Flow::P.new('attribute comment')] },
@@ -58,8 +57,7 @@
       'instance_method_extensions' => [
         { 'name' => 'instance_method_extension' },
       ],
-      'superclass_string' => 'Object',
-    }
+      'superclass_string' => 'Object'
 
     @dd.display_class_info klass, ri_reader
 
@@ -154,7 +152,7 @@
   end
 
   def test_display_method_info_singleton
-    method = {
+    method = RDoc::RI::Driver::Hash.new.update \
       'aliases' => [],
       'block_params' => nil,
       'comment' => nil,
@@ -162,8 +160,7 @@
       'is_singleton' => true,
       'name' => 'some_method',
       'params' => '(arg1, arg2)',
-      'visibility' => 'public',
-    }
+      'visibility' => 'public'
 
     @dd.display_method_info method
 
@@ -179,7 +176,7 @@
 
   def test_display_method_list
     methods = [
-      {
+      RDoc::RI::Driver::Hash.new.update(
         "aliases" => [],
         "block_params" => nil,
         "comment" =>  nil,
@@ -187,9 +184,9 @@
         "is_singleton" => false,
         "name" => "some_method",
         "params" => "()",
-        "visibility" => "public",
-      },
-      {
+        "visibility" => "public"
+      ),
+      RDoc::RI::Driver::Hash.new.update(
         "aliases" => [],
         "block_params" => nil,
         "comment" => nil,
@@ -197,8 +194,8 @@
         "is_singleton" => false,
         "name" => "some_other_method",
         "params" => "()",
-        "visibility" => "public",
-      },
+        "visibility" => "public"
+      ),
     ]
 
     @dd.display_method_list methods
@@ -291,5 +288,9 @@
     assert_equal expected, @output.string
   end
 
+  def h(hash)
+    RDoc::RI::Driver::Hash.convert hash
+  end
+
 end
 
Index: test/rdoc/test_rdoc_info_formatting.rb
===================================================================
--- test/rdoc/test_rdoc_info_formatting.rb	(revision 0)
+++ test/rdoc/test_rdoc_info_formatting.rb	(revision 18121)
@@ -0,0 +1,179 @@
+$LOAD_PATH.unshift File.dirname(__FILE__) + '/../lib/'
+require 'fileutils'
+require 'test/unit'
+require 'rdoc/generator/texinfo'
+require 'yaml'
+
+# From chapter 18 of the Pickaxe 3rd ed. and the TexInfo manual.
+class TestRdocInfoFormatting < Test::Unit::TestCase
+  OUTPUT_DIR = "/tmp/rdoc-#{$$}"
+
+  def setup
+    # supress stdout
+    $stdout = File.new('/dev/null','w')
+    $stderr = File.new('/dev/null','w')
+
+    RDoc::RDoc.new.document(['--fmt=texinfo',
+                             File.expand_path(__FILE__),
+                             "--op=#{OUTPUT_DIR}"])
+    @text = File.read(OUTPUT_DIR + '/rdoc.texinfo')
+    # File.open('rdoc.texinfo', 'w') { |f| f.puts @text }
+  end
+
+  def teardown
+    $stdout = STDOUT
+    $stderr = STDERR
+    FileUtils.rm_rf OUTPUT_DIR
+  end
+
+  # Make sure tags like *this* do not make HTML
+  def test_descriptions_are_not_html
+    assert_no_match Regexp.new("\<b\>this\<\/b\>"), @text, "We had some HTML; icky!"
+  end
+
+  # Ensure we get a reasonable amount
+  #
+  # of space in between paragraphs.
+  def test_paragraphs_are_spaced
+    assert_match(/amount\n\n\nof space/, @text)
+  end
+
+  # @ and {} should be at-sign-prefixed
+  def test_escaping
+    assert_match(/@@ and @\{@\} should be at-sign-prefixed/)
+  end
+  
+  # This tests that *bold* and <b>bold me</b> become @strong{bolded}
+  def test_bold
+    # Seems like a limitation of the Info format: @strong{bold}
+    # becomes *bold* when read in Info or M-x info. highly lame!
+    assert_match(/@strong\{bold\}/)
+    assert_match(/@strong\{bold me\}/)
+  end
+
+  # Test that _italics_ and <em>italicize me</em> becomes @emph{italicized}
+  def test_italics
+    assert_match(/@emph\{italics\}/)
+    assert_match(/@emph\{italicize me\}/)
+  end
+
+  # And that typewriter +text+ and <tt>typewriter me</tt> becomes @code{typewriter}
+  def test_tt
+    assert_match(/@code\{text\}/)
+    assert_match(/@code\{typewriter me\}/)
+  end
+
+  # Check that
+  #   anything indented is
+  #   verbatim @verb{|foo bar baz|}
+  def test_literal_code
+    assert_match("@verb{|  anything indented is
+  verbatim @@verb@{|foo bar baz|@}
+|}")
+  end
+
+  # = Huge heading should be a @majorheading
+  # == There is also @chapheading
+  # === Everything deeper becomes a regular @heading
+  # ====== Regardless of its nesting level
+  def test_headings
+    assert_match(/@majorheading\{Huge heading should be a @@majorheading\}/)
+    assert_match(/@chapheading\{There is also @@chapheading\}/)
+    assert_match(/@heading\{Everything deeper becomes a regular @@heading\}/)
+    assert_match(/@heading\{Regardless of its nesting level\}/)
+  end
+
+  # * list item
+  # * list item2
+  #
+  # with a paragraph in between
+  #
+  # - hyphen lists
+  # - are also allowed
+  #   and items may flow over lines
+  def test_bullet_lists
+    assert_match("@itemize @bullet
+@item
+list item
+@item
+list item2
+@end itemize")
+    assert_match("@itemize @bullet
+@item
+hyphen lists
+@item
+are also allowed and items may flow over lines
+@end itemize")
+  end
+
+  # 2. numbered lists
+  # 8. are made by
+  # 9. a digit followed by a period
+  def test_numbered_lists
+  end
+
+  # a. alpha lists
+  # b. should be parsed too
+  def test_alpha_lists
+  end
+
+  # [cat]   small domestic animal
+  # [+cat+] command to copy standard input
+  #         to standard output
+  def test_labelled_lists
+  end
+
+  # * First item.
+  #   * Inner item.
+  #   * Second inner item.
+  # * Second outer item.
+  def test_nested_lists
+    assert_match("@itemize @bullet
+@item
+First item.
+@itemize @bullet
+@item
+Inner item.
+@item
+Second inner item.
+@end itemize
+@item
+Second outer item.
+@end itemize")
+  end
+  
+  def test_internal_hyperlinks
+    # be sure to test multi-word hyperlinks as well.
+  end
+
+  def test_hyperlink_targets
+  end
+
+  def test_web_links
+    # An example of the two-argument form: The official
+    # @uref{ftp://ftp.gnu.org/gnu, GNU ftp site} holds programs and texts.
+
+    # produces:
+    #      The official GNU ftp site (ftp://ftp.gnu.org/gnu)
+    #      holds programs and texts.
+    # and the HTML output is this:
+    #      The official <a href="ftp://ftp.gnu.org/gnu">GNU ftp site</a>
+    #      holds programs and texts.
+  end
+
+  # three or more hyphens
+  # ----
+  # should produce a horizontal rule
+  def test_horizontal_rule
+    # gah; not sure texinfo supports horizontal rules
+  end
+
+  private
+
+  # We don't want the whole string inspected if we pass our own
+  # message in.
+  def assert_match(regex, string = @text,
+                   message = "Didn't find #{regex.inspect} in #{string}.")
+    assert string[regex] #, message
+  end
+end
Index: test/rdoc/test_rdoc_markup_to_html.rb
===================================================================
--- test/rdoc/test_rdoc_markup_to_html.rb	(revision 0)
+++ test/rdoc/test_rdoc_markup_to_html.rb	(revision 18121)
@@ -0,0 +1,30 @@
+require 'test/unit'
+require 'rdoc/markup'
+require 'rdoc/markup/to_html'
+
+class TestRdocMarkupToHtml < Test::Unit::TestCase
+
+  def setup
+    @am = RDoc::Markup::AttributeManager.new
+    @th = RDoc::Markup::ToHtml.new
+  end
+
+  def test_tt_formatting
+    assert_equal "<p>\n<tt>--</tt> &#8212; <tt>(c)</tt> &#169;\n</p>\n",
+                 util_format("<tt>--</tt> -- <tt>(c)</tt> (c)")
+    assert_equal "<p>\n<b>&#8212;</b>\n</p>\n", util_format("<b>--</b>")
+  end
+
+  def util_fragment(text)
+    RDoc::Markup::Fragment.new 0, nil, nil, text
+  end
+
+  def util_format(text)
+    fragment = util_fragment text
+
+    @th.start_accepting
+    @th.accept_paragraph @am, fragment
+    @th.end_accepting
+  end
+
+end
Index: test/rdoc/test_rdoc_ri_driver.rb
===================================================================
--- test/rdoc/test_rdoc_ri_driver.rb	(revision 0)
+++ test/rdoc/test_rdoc_ri_driver.rb	(revision 18121)
@@ -0,0 +1,100 @@
+require 'test/unit'
+require 'tmpdir'
+require 'rdoc/ri/driver'
+
+class TestRDocRIDriver < Test::Unit::TestCase
+
+  def setup
+    @tmpdir = File.join Dir.tmpdir, "test_rdoc_ri_driver_#{$$}"
+    @home_ri = File.join @tmpdir, 'dot_ri'
+    @cache_dir = File.join @home_ri, 'cache'
+    @class_cache = File.join @cache_dir, 'classes'
+
+    FileUtils.mkdir_p @tmpdir
+    FileUtils.mkdir_p @home_ri
+    FileUtils.mkdir_p @cache_dir
+
+    @driver = RDoc::RI::Driver.new
+    @driver.homepath = @home_ri
+  end
+
+  def teardown
+    FileUtils.rm_rf @tmpdir
+  end
+
+  def test_lookup_method
+    def @driver.load_cache_for(klassname)
+      { 'Foo#bar' => :found }
+    end
+
+    assert @driver.lookup_method('Foo#bar',  'Foo')
+  end
+
+  def test_lookup_method_class_method
+    def @driver.load_cache_for(klassname)
+      { 'Foo::Bar' => :found }
+    end
+
+    assert @driver.lookup_method('Foo::Bar', 'Foo::Bar')
+  end
+
+  def test_lookup_method_class_missing
+    def @driver.load_cache_for(klassname) end
+
+    e = assert_raise RDoc::RI::Driver::NotFoundError do
+      @driver.lookup_method 'Foo#bar', 'Foo'
+    end
+
+    assert_equal 'Nothing known about Foo#bar', e.message
+  end
+
+  def test_lookup_method_dot_instance
+    def @driver.load_cache_for(klassname)
+      { 'Foo#bar' => :instance, 'Foo::bar' => :klass }
+    end
+
+    assert_equal :instance, @driver.lookup_method('Foo.bar', 'Foo')
+  end
+
+  def test_lookup_method_dot_class
+    def @driver.load_cache_for(klassname)
+      { 'Foo::bar' => :found }
+    end
+
+    assert @driver.lookup_method('Foo.bar', 'Foo')
+  end
+
+  def test_lookup_method_method_missing
+    def @driver.load_cache_for(klassname) {} end
+
+    e = assert_raise RDoc::RI::Driver::NotFoundError do
+      @driver.lookup_method 'Foo#bar', 'Foo'
+    end
+
+    assert_equal 'Nothing known about Foo#bar', e.message
+  end
+
+  def test_parse_name
+    klass, meth = @driver.parse_name 'Foo::Bar'
+
+    assert_equal 'Foo::Bar', klass, 'Foo::Bar class'
+    assert_equal nil,        meth,  'Foo::Bar method'
+
+    klass, meth = @driver.parse_name 'Foo#Bar'
+
+    assert_equal 'Foo', klass, 'Foo#Bar class'
+    assert_equal 'Bar', meth,  'Foo#Bar method'
+
+    klass, meth = @driver.parse_name 'Foo.Bar'
+
+    assert_equal 'Foo', klass, 'Foo#Bar class'
+    assert_equal 'Bar', meth,  'Foo#Bar method'
+
+    klass, meth = @driver.parse_name 'Foo::bar'
+
+    assert_equal 'Foo', klass, 'Foo::bar class'
+    assert_equal 'bar', meth,  'Foo::bar method'
+  end
+
+end
+

--
ML: ruby-changes@q...
Info: http://www.atdot.net/~ko1/quickml/

[前][次][番号順一覧][スレッド一覧]