[前][次][番号順一覧][スレッド一覧]

ruby-changes:15529

From: naruse <ko1@a...>
Date: Wed, 21 Apr 2010 15:37:41 +0900 (JST)
Subject: [ruby-changes:15529] Ruby:r27432 (trunk): * io.c (io_getc): fix incomplete character immediately before EOF

naruse	2010-04-21 15:37:18 +0900 (Wed, 21 Apr 2010)

  New Revision: 27432

  http://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=rev&revision=27432

  Log:
    * io.c (io_getc): fix incomplete character immediately before EOF
      with newline converter. [ruby-dev:41024]
      run test only when "Create Shortcut (&S)" menu is found.
    * lib/rexml/parsers/xpathparser.rb (PathExpr): ditto.
    * lib/matrix.rb (Vector#each): make Vector enumerable.

  Modified files:
    trunk/ChangeLog
    trunk/io.c
    trunk/test/ruby/test_io_m17n.rb

Index: ChangeLog
===================================================================
--- ChangeLog	(revision 27431)
+++ ChangeLog	(revision 27432)
@@ -1,3 +1,8 @@
+Wed Apr 21 15:13:10 2010  NARUSE, Yui  <naruse@r...>
+
+	* io.c (io_getc): fix incomplete character immediately before EOF
+	  with newline converter. [ruby-dev:41024]
+
 Wed Apr 21 13:44:54 2010  Nobuyoshi Nakada  <nobu@r...>
 
 	* parse.y (regexp): check each fragments.   [ruby-core:27374]
@@ -131,7 +136,7 @@
 	* parse.y (string_content): preserve cond_stack and cmdarg_stack.
 	  [ruby-core:29579]
 
-Sun Apr 18 05:50:58 2010  Aaron Patterson <aaron@t...>
+Sun Apr 18 05:50:58 2010  Aaron Patterson  <aaron@t...>
 
 	* ext/psych/lib/psych/visitors/yaml_tree.rb: defaulting binary string
 	  format to "literal" format.
@@ -175,13 +180,13 @@
 
 	* NEWS: add Array#repeated_{combinationpermutation} [Feature #2981]
 
-Sat Apr 17 05:30:22 2010  Aaron Patterson <aaron@t...>
+Sat Apr 17 05:30:22 2010  Aaron Patterson  <aaron@t...>
 
 	* ext/psych/lib/psych/core_ext.rb (yaml_as): supporting deprecated
 	  "yaml_as" method
 	* ext/syck/lib/syck/tag.rb: ditto
 
-Sat Apr 17 05:25:15 2010  Aaron Patterson <aaron@t...>
+Sat Apr 17 05:25:15 2010  Aaron Patterson  <aaron@t...>
 
 	* ext/psych/lib/psych/coder.rb (scalar): supporting deprecated methods
 	* ext/psych/lib/psych/deprecated.rb: supporting deprecated
@@ -192,10 +197,10 @@
 	* io.c (rb_io_rewind, rb_io_eof): add rdoc.  based on a patch from
 	  Roger Pack in [ruby-core:26771] [Bug #2377].
 
-Fri Apr 16 23:42:56 2010  Masaki Suketa <masaki.suketa@n...>
+Fri Apr 16 23:42:56 2010  Masaki Suketa  <masaki.suketa@n...>
 
 	* test/win32ole/test_folderitem2_invokeverb.rb (test_invokeverb):
-	  run test only when "Create Shortcut (&S)" menu is found. 
+	  run test only when "Create Shortcut (&S)" menu is found.
 	  [ruby-core:29550] [Bug #1602]
 
 Fri Apr 16 21:52:16 2010  Yusuke Endoh  <mame@t...>
@@ -422,7 +427,7 @@
 
 	* lib/rdoc:  Update to RDoc 2.5.3.  Includes r27288 and r27290.
 
-Sun Apr 11 09:31:39 2010  Aaron Patterson <aaron@t...>
+Sun Apr 11 09:31:39 2010  Aaron Patterson  <aaron@t...>
 
 	* test/syck/*: Moved test/yaml to test/syck since it's actually
 	  testing the syck YAML engine.
@@ -469,7 +474,7 @@
 	* vm_method.c (rb_add_method): clear attr.location before rb_ary_new3.
 	  a patch from Masaya TARUI in [Bug #3108]
 
-Sat Apr 10 05:27:26 2010  Aaron Patterson <aaron@t...>
+Sat Apr 10 05:27:26 2010  Aaron Patterson  <aaron@t...>
 
 	* ext/psych/lib/psych/deprecated.rb: implementing Psych.quick_emit and
 	  adding deprecation warnings.
@@ -478,7 +483,7 @@
 	* ext/psych/lib/psych/visitors/yaml_tree.rb: supporting deprecated
 	  to_yaml api.
 
-Sat Apr 10 01:08:53 2010  Aaron Patterson <aaron@t...>
+Sat Apr 10 01:08:53 2010  Aaron Patterson  <aaron@t...>
 
 	* ext/psych/parser.c: parse raises a TypeError when nil is passed in.
 
@@ -533,7 +538,7 @@
 	* dir.c (push_glob): add RB_GC_GUARD to prevent str being free'ed.
 	  a patch from Masaya TARUI in [ruby-dev:40939].  [ruby-dev:40514]
 
-Fri Apr  9 10:53:00 2010  Kenta Murata <mrkn@m...>
+Fri Apr  9 10:53:00 2010  Kenta Murata  <mrkn@m...>
 
 	* compile.c (iseq_compile_each): stop hiding, and freeze unpopped
 	string nodes to allow method redefinition.  [ruby-dev:40641]
@@ -623,6 +628,14 @@
 	* thread.c (do_select): Use UNINITIALIZED_VAR() instead FAKE_FD_ZERO().
 	  Also, remove FAKE_FD_ZERO completely. [Feature #3018]
 
+Tue Apr  6 14:53:17 2010  NAKAMURA Usaku  <usa@r...>
+
+	* include/ruby/win32.h: check definition existence before defining
+	  errno macros.
+
+	* win32/win32.c (errmap): define winsock errors mappings.
+	  these are VC++10 support. see [ruby-core:29278]
+
 Tue Apr  6 21:55:25 2010  Tanaka Akira  <akr@f...>
 
 	* configure.in: test localtime(3) overflow.  [ruby-dev:40910]
@@ -633,15 +646,7 @@
 	  there is overflow problem.
 	  (rb_gmtime_r2): call rb_gmtime_r and validate the result if there
 	  is overflow problem.
-	  
-Tue Apr  6 14:53:17 2010  NAKAMURA Usaku  <usa@r...>
 
-	* include/ruby/win32.h: check definition existence before defining
-	  errno macros.
-
-	* win32/win32.c (errmap): define winsock errors mappings.
-	  these are VC++10 support. see [ruby-core:29278]
-
 Tue Apr  6 11:21:23 2010  Nobuyoshi Nakada  <nobu@r...>
 
 	* template/fake.rb.in: hooks for extconf.rb.
@@ -661,7 +666,7 @@
 
 	* lib/rexml/xmltokens.rb (NCNAME_STR, NAMECHAR): ditto.
 
-	* lib/rexml/parsers/xpathparser.rb (PathExpr): ditto. 
+	* lib/rexml/parsers/xpathparser.rb (PathExpr): ditto.
 
 	* lib/rexml/text.rb (REXML::Text#initialize):
 	  initialize @parent.
@@ -727,7 +732,7 @@
 
 	* time.c (lt): call <=> instead of <.
 
-Sun Apr  4 06:46:16 2010  Aaron Patterson <aaron@t...>
+Sun Apr  4 06:46:16 2010  Aaron Patterson  <aaron@t...>
 
 	* lib/yaml: Moved to ext/syck/lib, Syck only uses Syck constant.
 	* lib/yaml.rb: Added an engine manager for choosing YAML engine.
@@ -1042,17 +1047,17 @@
 	* tool/file2lastrev.rb (VCS#get_revisions): particular commands do
 	  not depend on instance.
 
-Tue Mar 30 08:55:50 2010  Aaron Patterson <aaron@t...>
+Tue Mar 30 08:55:50 2010  Aaron Patterson  <aaron@t...>
 
 	* ext/psych/extconf.rb: Making library detection more agnostic.
 
-Tue Mar 30 08:10:59 2010  Aaron Patterson <aaron@t...>
+Tue Mar 30 08:10:59 2010  Aaron Patterson  <aaron@t...>
 
 	* lib/psych.rb: Fix problem with empty and white-space only strings.
 	  Thanks Peter McLain!
 	* test/psych/test_psych.rb: tests for change.
 
-Tue Mar 30 05:31:39 2010  Aaron Patterson <aaron@t...>
+Tue Mar 30 05:31:39 2010  Aaron Patterson  <aaron@t...>
 
 	* lib/psych.rb: documentation updates.  Thanks Peter McLain!
 	* lib/psych/{coder,tree_builder}.rb: ditto
@@ -1089,7 +1094,7 @@
 
 	* random.c: change include order; ruby.h should be at first.
 
-Mon Mar 29 06:47:25 2010  Aaron Patterson <aaron@t...>
+Mon Mar 29 06:47:25 2010  Aaron Patterson  <aaron@t...>
 
 	* ext/psych/*: importing Psych to trunk
 	* test/psych/*: ditto
@@ -1163,13 +1168,13 @@
 	* test/ruby/test_hash.rb (test_recursive_key): recursive keys are
 	  permitted now.  [ruby-dev:40735]
 
-Fri Mar 26 12:36:10 2010  Aaron Patterson <aaron@t...>
+Fri Mar 26 12:36:10 2010  Aaron Patterson  <aaron@t...>
 
 	* lib/yaml/basenode.rb: deprecating YPath methods
 
 	* lib/yaml/stream.rb: deprecating YAML::Stream#edit
 
-Fri Mar 26 12:29:28 2010  Aaron Patterson <aaron@t...>
+Fri Mar 26 12:29:28 2010  Aaron Patterson  <aaron@t...>
 
 	* lib/yaml/encoding.rb: YAML.encode, YAML.decode are deprecated.
 
@@ -1870,7 +1875,7 @@
 
 	* test/dl/test_method.rb: delete a residual test for dl on libffi.
 
-Tue Mar  9 18:00:00 2010  Kenta Murata <mrkn@m...>
+Tue Mar  9 18:00:00 2010  Kenta Murata  <mrkn@m...>
 
 	* enc/trans/EMOJI/*.src, enc/trans/emoji*,
 	  enc/x-emoji.c, test/ruby/enc/test_emoji.rb,
@@ -2044,7 +2049,7 @@
 
 Wed Mar  3 16:14:24 2010  Yukihiro Matsumoto  <matz@r...>
 
-	* lib/matrix.rb (Vector#each): make Vector enumerable.  
+	* lib/matrix.rb (Vector#each): make Vector enumerable.
 	  [ruby-core:28405]
 
 Wed Mar  3 14:28:23 2010  Yukihiro Matsumoto  <matz@r...>
@@ -2076,7 +2081,7 @@
 
 	* regcomp.c (renumber_by_map): ditto.
 
-Tue Mar  2 14:42:00 2010  Kenta Murata <mrkn@m...>
+Tue Mar  2 14:42:00 2010  Kenta Murata  <mrkn@m...>
 
 	* test/dl/test_closure.rb: forget to revert at r26764.
 
@@ -2433,7 +2438,7 @@
 	* test/dl/test_import.rb (DL::LIBC::BoundQsortCallback): renamed
 	  to get rid of overwriting warning.
 
-Tue Feb 16 11:03:19 2010  Aaron Patterson <aaron@t...>
+Tue Feb 16 11:03:19 2010  Aaron Patterson  <aaron@t...>
 
 	* ext/dl/method.c: Adding DL::Method as a superclass for DL::Function
 
@@ -2732,7 +2737,7 @@
 
 	* bignum.c (big_op): remove unused variables.
 
-Fri Feb  5 02:06:57 2010  Aaron Patterson <tenderlove@r...>
+Fri Feb  5 02:06:57 2010  Aaron Patterson  <tenderlove@r...>
 
 	* lib/yaml/rubytypes.rb: Struct members are emitted without a leading
 	  colon.  Thanks Yusuke Endoh! [ruby-core:28052]
@@ -2816,7 +2821,7 @@
 
 	* ext/dl/{closure,function}.c: removed C99 features and warnings.
 
-Wed Feb  3 10:12:09 2010  Aaron Patterson <tenderlove@r...>
+Wed Feb  3 10:12:09 2010  Aaron Patterson  <tenderlove@r...>
 
 	* ext/dl/function.c: DL::Function now uses libffi
 
@@ -3216,7 +3221,7 @@
 	  make the rdoc(generating html) run faster and use less memory.
 	  patch by Tetsu Soh [ruby-core:27656]
 
-Sat Jan 23 19:54:48 2010  Masaki Suketa <masaki.suketa@n...>
+Sat Jan 23 19:54:48 2010  Masaki Suketa  <masaki.suketa@n...>
 
 	* ext/win32ole/win32ole.c: add WIN32OLE.ole_initialize,
 	  WIN32OLE.ole_uninitialize to use in win32ole.rb.
@@ -3342,7 +3347,7 @@
 	* ext/zlib/{extconf.rb, zlib.c): crc32_combine and adler32_combine is
 	  supported on Zlib 1.2.2.1, so check them for old zlib.
 
-Tue Jan 19 09:03:37 2010  Aaron Patterson <tenderlove@r...>
+Tue Jan 19 09:03:37 2010  Aaron Patterson  <tenderlove@r...>
 
 	* ext/zlib/zlib.c: added Zlib.crc32_combine and Zlib.adler32_combine
 
@@ -3918,7 +3923,7 @@
 
 	* tool/install-sh: wrote the intention. [ruby-dev:39928]
 
-Sun Dec 27 10:45:00 2009  Masaki Suketa <masaki.suketa@n...>
+Sun Dec 27 10:45:00 2009  Masaki Suketa  <masaki.suketa@n...>
 
 	* ext/win32ole/win32ole.c (foleparam_initialize): add foleparam_initialize
 	  to check argument of WIN32OLE_PARAM.new
@@ -4307,7 +4312,7 @@
 
 	* vm_eval.c (check_funcall_failed): pass ID.  [ruby-core:26934]
 
-Sun Nov 29 06:37:53 2009  Aaron Patterson <tenderlove@r...>
+Sun Nov 29 06:37:53 2009  Aaron Patterson  <tenderlove@r...>
 
 	* lib/rexml/formatters/default.rb (write_attribute): fix an
 	  exception when printing a document when duplicate namespaced
Index: io.c
===================================================================
--- io.c	(revision 27431)
+++ io.c	(revision 27432)
@@ -2756,15 +2756,15 @@
             }
 
             if (more_char(fptr) == MORE_CHAR_FINISHED) {
-                clear_readconv(fptr);
-                if (fptr->cbuf_len == 0)
-                    return Qnil;
+                if (fptr->cbuf_len == 0) {
+		    clear_readconv(fptr);
+		    return Qnil;
+		}
                 /* return an unit of an incomplete character just before EOF */
-		r = rb_enc_mbclen(fptr->cbuf+fptr->cbuf_off,
-			fptr->cbuf+fptr->cbuf_off+fptr->cbuf_len,
-			read_enc);
-		io_shift_cbuf(fptr, r, &str);
-		str = io_enc_str(str, fptr);
+		str = rb_enc_str_new(fptr->cbuf+fptr->cbuf_off, 1, read_enc);
+		fptr->cbuf_off += 1;
+		fptr->cbuf_len -= 1;
+                if (fptr->cbuf_len == 0) clear_readconv(fptr);
 		ENC_CODERANGE_SET(str, ENC_CODERANGE_BROKEN);
 		return str;
             }
Index: test/ruby/test_io_m17n.rb
===================================================================
--- test/ruby/test_io_m17n.rb	(revision 27431)
+++ test/ruby/test_io_m17n.rb	(revision 27432)
@@ -400,6 +400,25 @@
     }
   end
 
+  def test_getc_newlineconv_invalid
+    with_tmpdir {
+      src = "\xE3\x81"
+      generate_file('tmp', src)
+      defext = Encoding.default_external
+      Encoding.default_external = Encoding::UTF_8
+      open("tmp", "rt") {|f|
+        p f.read;f.rewind
+        s = f.getc
+        assert_equal(false, s.valid_encoding?)
+        assert_equal("\xE3".force_encoding("UTF-8"), s)
+        s = f.getc
+        assert_equal(false, s.valid_encoding?)
+        assert_equal("\x81".force_encoding("UTF-8"), s)
+      }
+      Encoding.default_external = defext
+    }
+  end
+
   def test_ungetc_stateful_conversion
     with_tmpdir {
       src = "before \e$B\x23\x30\x23\x31\e(B after".force_encoding("iso-2022-jp")

--
ML: ruby-changes@q...
Info: http://www.atdot.net/~ko1/quickml/

[前][次][番号順一覧][スレッド一覧]