[前][次][番号順一覧][スレッド一覧]

ruby-changes:62308

From: Sutou <ko1@a...>
Date: Mon, 20 Jul 2020 03:35:57 +0900 (JST)
Subject: [ruby-changes:62308] 4fcfa85cb6 (master): [ruby/csv] CSV.generate_line: use the encoding of the first non ASCII field as the expected encoding

https://git.ruby-lang.org/ruby.git/commit/?id=4fcfa85cb6

From 4fcfa85cb6a44b4e8c7a47578b50064711dff404 Mon Sep 17 00:00:00 2001
From: Sutou Kouhei <kou@c...>
Date: Sun, 19 Jul 2020 06:25:05 +0900
Subject: [ruby/csv] CSV.generate_line: use the encoding of the first non ASCII
 field as the expected encoding

See also: https://github.com/ruby/stringio/issues/13#issuecomment-660543554

https://github.com/ruby/csv/commit/004cf49d18

diff --git a/lib/csv.rb b/lib/csv.rb
index 8389889..63309f3 100644
--- a/lib/csv.rb
+++ b/lib/csv.rb
@@ -1289,8 +1289,20 @@ class CSV https://github.com/ruby/ruby/blob/trunk/lib/csv.rb#L1289
       str = +""
       if options[:encoding]
         str.force_encoding(options[:encoding])
-      elsif field = row.find {|f| f.is_a?(String)}
-        str.force_encoding(field.encoding)
+      else
+        fallback_encoding = nil
+        output_encoding = nil
+        row.each do |field|
+          next unless field.is_a?(String)
+          fallback_encoding ||= field.encoding
+          next if field.ascii_only?
+          output_encoding = field.encoding
+          break
+        end
+        output_encoding ||= fallback_encoding
+        if output_encoding
+          str.force_encoding(output_encoding)
+        end
       end
       (new(str, **options) << row).string
     end
diff --git a/test/csv/test_encodings.rb b/test/csv/test_encodings.rb
index cd63af6..8d228c0 100644
--- a/test/csv/test_encodings.rb
+++ b/test/csv/test_encodings.rb
@@ -242,6 +242,18 @@ class TestCSVEncodings < Test::Unit::TestCase https://github.com/ruby/ruby/blob/trunk/test/csv/test_encodings.rb#L242
     assert_equal("UTF-8",      data.to_csv.encoding.name)
   end
 
+  def test_encoding_is_not_upgraded_for_non_ascii_content_during_writing_as_needed
+    data = ["\u00c0".encode("ISO-8859-1"), "\u3042"]
+    assert_equal([
+                   "ISO-8859-1",
+                   "UTF-8",
+                 ],
+                 data.collect {|field| field.encoding.name})
+    assert_raise(Encoding::CompatibilityError) do
+      data.to_csv
+    end
+  end
+
   def test_explicit_encoding
     bug9766 = '[ruby-core:62113] [Bug #9766]'
     s = CSV.generate(encoding: "Windows-31J") do |csv|
-- 
cgit v0.10.2


--
ML: ruby-changes@q...
Info: http://www.atdot.net/~ko1/quickml/

[前][次][番号順一覧][スレッド一覧]