ruby-changes:70514
From: Sutou <ko1@a...>
Date: Fri, 24 Dec 2021 14:41:18 +0900 (JST)
Subject: [ruby-changes:70514] 002ce9f515 (master): [ruby/csv] parser: use row separator explicitly
https://git.ruby-lang.org/ruby.git/commit/?id=002ce9f515 From 002ce9f515d984aa0fe43c51448b36ea3651a395 Mon Sep 17 00:00:00 2001 From: Sutou Kouhei <kou@c...> Date: Fri, 24 Dec 2021 10:18:35 +0900 Subject: [ruby/csv] parser: use row separator explicitly It will improve performance a bit. (But I haven't confirmed it yet...) https://github.com/ruby/csv/commit/06a65b0302 --- lib/csv/parser.rb | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/lib/csv/parser.rb b/lib/csv/parser.rb index e1fe559a41b..78ff363d692 100644 --- a/lib/csv/parser.rb +++ b/lib/csv/parser.rb @@ -85,9 +85,10 @@ class CSV https://github.com/ruby/ruby/blob/trunk/lib/csv/parser.rb#L85 # If there is no more data (eos? = true), it returns "". # class InputsScanner - def initialize(inputs, encoding, chunk_size: 8192) + def initialize(inputs, encoding, row_separator, chunk_size: 8192) @inputs = inputs.dup @encoding = encoding + @row_separator = row_separator @chunk_size = chunk_size @last_scanner = @inputs.empty? @keeps = [] @@ -250,7 +251,7 @@ class CSV https://github.com/ruby/ruby/blob/trunk/lib/csv/parser.rb#L251 @last_scanner = @inputs.empty? true else - chunk = input.gets(nil, @chunk_size) + chunk = input.gets(@row_separator, @chunk_size) if chunk raise InvalidEncoding unless chunk.valid_encoding? @scanner = StringScanner.new(chunk) @@ -778,6 +779,7 @@ class CSV https://github.com/ruby/ruby/blob/trunk/lib/csv/parser.rb#L779 Integer((ENV["CSV_PARSER_SCANNER_TEST_CHUNK_SIZE"] || "1"), 10) InputsScanner.new(inputs, @encoding, + @row_separator, chunk_size: chunk_size) end else @@ -807,7 +809,7 @@ class CSV https://github.com/ruby/ruby/blob/trunk/lib/csv/parser.rb#L809 StringIO.new(sample) end inputs << @input - InputsScanner.new(inputs, @encoding) + InputsScanner.new(inputs, @encoding, @row_separator) end end end -- cgit v1.2.1 -- ML: ruby-changes@q... Info: http://www.atdot.net/~ko1/quickml/