https://git.ruby-lang.org/ruby.git/commit/?id=723a37d038

From 723a37d0386bc20efedf516656c2ccafa889c89d Mon Sep 17 00:00:00 2001
From: Yusuke Endoh <mame@r...>
Date: Wed, 7 Aug 2019 02:32:03 +0900
Subject: Separate RDoc::TokenStream#add_tokens and #add_token

The old version of `add_tokens` accepts an array of tokens, and
multiple arguments of tokens by using `Array#flatten`.
And `add_token` was an alias to `add_tokens`.

I think it is unnecessarily flexible; in fact, all callsites of
`add_tokens` (except test) passes only an array of tokens.
And the code created a lot of temporal arrays.

This change makes `add_tokens` accept only one array of tokens,
and does `add_token` accept one token.  It is a bit faster (about
1 second in Ruby's `make rdoc`), and it ls also cleaner in my point of
view.

diff --git a/lib/rdoc/token_stream.rb b/lib/rdoc/token_stream.rb
index dbe6c5a..f428e24 100644
--- a/lib/rdoc/token_stream.rb
+++ b/lib/rdoc/token_stream.rb
@@ -74,11 +74,16 @@ module RDoc::TokenStream https://github.com/ruby/ruby/blob/trunk/lib/rdoc/token_stream.rb#L74
   ##
   # Adds +tokens+ to the collected tokens
 
-  def add_tokens(*tokens)
-    tokens.flatten.each { |token| @token_stream << token }
+  def add_tokens(tokens)
+    @token_stream.concat(tokens)
   end
 
-  alias add_token add_tokens
+  ##
+  # Adds one +token+ to the collected tokens
+
+  def add_token(token)
+    @token_stream.push(token)
+  end
 
   ##
   # Starts collecting tokens
diff --git a/test/rdoc/test_rdoc_any_method.rb b/test/rdoc/test_rdoc_any_method.rb
index 6dd46b0..615789d 100644
--- a/test/rdoc/test_rdoc_any_method.rb
+++ b/test/rdoc/test_rdoc_any_method.rb
@@ -78,7 +78,7 @@ method(a, b) { |c, d| ... } https://github.com/ruby/ruby/blob/trunk/test/rdoc/test_rdoc_any_method.rb#L78
     ]
 
     @c2_a.collect_tokens
-    @c2_a.add_tokens(*tokens)
+    @c2_a.add_tokens(tokens)
 
     expected = '<span class="ruby-constant">CONSTANT</span>'
 
@@ -96,7 +96,7 @@ method(a, b) { |c, d| ... } https://github.com/ruby/ruby/blob/trunk/test/rdoc/test_rdoc_any_method.rb#L96
     ]
 
     @c2_a.collect_tokens
-    @c2_a.add_tokens(*tokens)
+    @c2_a.add_tokens(tokens)
 
     assert_equal <<-EXPECTED.chomp, @c2_a.markup_code
 <span class="ruby-comment"># File xref_data.rb, line 1</span>
-- 
cgit v0.10.2


--
ML: ruby-changes@q...
Info: http://www.atdot.net/~ko1/quickml/