ruby-changes:15529
From: naruse <ko1@a...>
Date: Wed, 21 Apr 2010 15:37:41 +0900 (JST)
Subject: [ruby-changes:15529] Ruby:r27432 (trunk): * io.c (io_getc): fix incomplete character immediately before EOF
naruse 2010-04-21 15:37:18 +0900 (Wed, 21 Apr 2010) New Revision: 27432 http://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=rev&revision=27432 Log: * io.c (io_getc): fix incomplete character immediately before EOF with newline converter. [ruby-dev:41024] run test only when "Create Shortcut (&S)" menu is found. * lib/rexml/parsers/xpathparser.rb (PathExpr): ditto. * lib/matrix.rb (Vector#each): make Vector enumerable. Modified files: trunk/ChangeLog trunk/io.c trunk/test/ruby/test_io_m17n.rb Index: ChangeLog =================================================================== --- ChangeLog (revision 27431) +++ ChangeLog (revision 27432) @@ -1,3 +1,8 @@ +Wed Apr 21 15:13:10 2010 NARUSE, Yui <naruse@r...> + + * io.c (io_getc): fix incomplete character immediately before EOF + with newline converter. [ruby-dev:41024] + Wed Apr 21 13:44:54 2010 Nobuyoshi Nakada <nobu@r...> * parse.y (regexp): check each fragments. [ruby-core:27374] @@ -131,7 +136,7 @@ * parse.y (string_content): preserve cond_stack and cmdarg_stack. [ruby-core:29579] -Sun Apr 18 05:50:58 2010 Aaron Patterson <aaron@t...> +Sun Apr 18 05:50:58 2010 Aaron Patterson <aaron@t...> * ext/psych/lib/psych/visitors/yaml_tree.rb: defaulting binary string format to "literal" format. @@ -175,13 +180,13 @@ * NEWS: add Array#repeated_{combinationpermutation} [Feature #2981] -Sat Apr 17 05:30:22 2010 Aaron Patterson <aaron@t...> +Sat Apr 17 05:30:22 2010 Aaron Patterson <aaron@t...> * ext/psych/lib/psych/core_ext.rb (yaml_as): supporting deprecated "yaml_as" method * ext/syck/lib/syck/tag.rb: ditto -Sat Apr 17 05:25:15 2010 Aaron Patterson <aaron@t...> +Sat Apr 17 05:25:15 2010 Aaron Patterson <aaron@t...> * ext/psych/lib/psych/coder.rb (scalar): supporting deprecated methods * ext/psych/lib/psych/deprecated.rb: supporting deprecated @@ -192,10 +197,10 @@ * io.c (rb_io_rewind, rb_io_eof): add rdoc. based on a patch from Roger Pack in [ruby-core:26771] [Bug #2377]. -Fri Apr 16 23:42:56 2010 Masaki Suketa <masaki.suketa@n...> +Fri Apr 16 23:42:56 2010 Masaki Suketa <masaki.suketa@n...> * test/win32ole/test_folderitem2_invokeverb.rb (test_invokeverb): - run test only when "Create Shortcut (&S)" menu is found. + run test only when "Create Shortcut (&S)" menu is found. [ruby-core:29550] [Bug #1602] Fri Apr 16 21:52:16 2010 Yusuke Endoh <mame@t...> @@ -422,7 +427,7 @@ * lib/rdoc: Update to RDoc 2.5.3. Includes r27288 and r27290. -Sun Apr 11 09:31:39 2010 Aaron Patterson <aaron@t...> +Sun Apr 11 09:31:39 2010 Aaron Patterson <aaron@t...> * test/syck/*: Moved test/yaml to test/syck since it's actually testing the syck YAML engine. @@ -469,7 +474,7 @@ * vm_method.c (rb_add_method): clear attr.location before rb_ary_new3. a patch from Masaya TARUI in [Bug #3108] -Sat Apr 10 05:27:26 2010 Aaron Patterson <aaron@t...> +Sat Apr 10 05:27:26 2010 Aaron Patterson <aaron@t...> * ext/psych/lib/psych/deprecated.rb: implementing Psych.quick_emit and adding deprecation warnings. @@ -478,7 +483,7 @@ * ext/psych/lib/psych/visitors/yaml_tree.rb: supporting deprecated to_yaml api. -Sat Apr 10 01:08:53 2010 Aaron Patterson <aaron@t...> +Sat Apr 10 01:08:53 2010 Aaron Patterson <aaron@t...> * ext/psych/parser.c: parse raises a TypeError when nil is passed in. @@ -533,7 +538,7 @@ * dir.c (push_glob): add RB_GC_GUARD to prevent str being free'ed. a patch from Masaya TARUI in [ruby-dev:40939]. [ruby-dev:40514] -Fri Apr 9 10:53:00 2010 Kenta Murata <mrkn@m...> +Fri Apr 9 10:53:00 2010 Kenta Murata <mrkn@m...> * compile.c (iseq_compile_each): stop hiding, and freeze unpopped string nodes to allow method redefinition. [ruby-dev:40641] @@ -623,6 +628,14 @@ * thread.c (do_select): Use UNINITIALIZED_VAR() instead FAKE_FD_ZERO(). Also, remove FAKE_FD_ZERO completely. [Feature #3018] +Tue Apr 6 14:53:17 2010 NAKAMURA Usaku <usa@r...> + + * include/ruby/win32.h: check definition existence before defining + errno macros. + + * win32/win32.c (errmap): define winsock errors mappings. + these are VC++10 support. see [ruby-core:29278] + Tue Apr 6 21:55:25 2010 Tanaka Akira <akr@f...> * configure.in: test localtime(3) overflow. [ruby-dev:40910] @@ -633,15 +646,7 @@ there is overflow problem. (rb_gmtime_r2): call rb_gmtime_r and validate the result if there is overflow problem. - -Tue Apr 6 14:53:17 2010 NAKAMURA Usaku <usa@r...> - * include/ruby/win32.h: check definition existence before defining - errno macros. - - * win32/win32.c (errmap): define winsock errors mappings. - these are VC++10 support. see [ruby-core:29278] - Tue Apr 6 11:21:23 2010 Nobuyoshi Nakada <nobu@r...> * template/fake.rb.in: hooks for extconf.rb. @@ -661,7 +666,7 @@ * lib/rexml/xmltokens.rb (NCNAME_STR, NAMECHAR): ditto. - * lib/rexml/parsers/xpathparser.rb (PathExpr): ditto. + * lib/rexml/parsers/xpathparser.rb (PathExpr): ditto. * lib/rexml/text.rb (REXML::Text#initialize): initialize @parent. @@ -727,7 +732,7 @@ * time.c (lt): call <=> instead of <. -Sun Apr 4 06:46:16 2010 Aaron Patterson <aaron@t...> +Sun Apr 4 06:46:16 2010 Aaron Patterson <aaron@t...> * lib/yaml: Moved to ext/syck/lib, Syck only uses Syck constant. * lib/yaml.rb: Added an engine manager for choosing YAML engine. @@ -1042,17 +1047,17 @@ * tool/file2lastrev.rb (VCS#get_revisions): particular commands do not depend on instance. -Tue Mar 30 08:55:50 2010 Aaron Patterson <aaron@t...> +Tue Mar 30 08:55:50 2010 Aaron Patterson <aaron@t...> * ext/psych/extconf.rb: Making library detection more agnostic. -Tue Mar 30 08:10:59 2010 Aaron Patterson <aaron@t...> +Tue Mar 30 08:10:59 2010 Aaron Patterson <aaron@t...> * lib/psych.rb: Fix problem with empty and white-space only strings. Thanks Peter McLain! * test/psych/test_psych.rb: tests for change. -Tue Mar 30 05:31:39 2010 Aaron Patterson <aaron@t...> +Tue Mar 30 05:31:39 2010 Aaron Patterson <aaron@t...> * lib/psych.rb: documentation updates. Thanks Peter McLain! * lib/psych/{coder,tree_builder}.rb: ditto @@ -1089,7 +1094,7 @@ * random.c: change include order; ruby.h should be at first. -Mon Mar 29 06:47:25 2010 Aaron Patterson <aaron@t...> +Mon Mar 29 06:47:25 2010 Aaron Patterson <aaron@t...> * ext/psych/*: importing Psych to trunk * test/psych/*: ditto @@ -1163,13 +1168,13 @@ * test/ruby/test_hash.rb (test_recursive_key): recursive keys are permitted now. [ruby-dev:40735] -Fri Mar 26 12:36:10 2010 Aaron Patterson <aaron@t...> +Fri Mar 26 12:36:10 2010 Aaron Patterson <aaron@t...> * lib/yaml/basenode.rb: deprecating YPath methods * lib/yaml/stream.rb: deprecating YAML::Stream#edit -Fri Mar 26 12:29:28 2010 Aaron Patterson <aaron@t...> +Fri Mar 26 12:29:28 2010 Aaron Patterson <aaron@t...> * lib/yaml/encoding.rb: YAML.encode, YAML.decode are deprecated. @@ -1870,7 +1875,7 @@ * test/dl/test_method.rb: delete a residual test for dl on libffi. -Tue Mar 9 18:00:00 2010 Kenta Murata <mrkn@m...> +Tue Mar 9 18:00:00 2010 Kenta Murata <mrkn@m...> * enc/trans/EMOJI/*.src, enc/trans/emoji*, enc/x-emoji.c, test/ruby/enc/test_emoji.rb, @@ -2044,7 +2049,7 @@ Wed Mar 3 16:14:24 2010 Yukihiro Matsumoto <matz@r...> - * lib/matrix.rb (Vector#each): make Vector enumerable. + * lib/matrix.rb (Vector#each): make Vector enumerable. [ruby-core:28405] Wed Mar 3 14:28:23 2010 Yukihiro Matsumoto <matz@r...> @@ -2076,7 +2081,7 @@ * regcomp.c (renumber_by_map): ditto. -Tue Mar 2 14:42:00 2010 Kenta Murata <mrkn@m...> +Tue Mar 2 14:42:00 2010 Kenta Murata <mrkn@m...> * test/dl/test_closure.rb: forget to revert at r26764. @@ -2433,7 +2438,7 @@ * test/dl/test_import.rb (DL::LIBC::BoundQsortCallback): renamed to get rid of overwriting warning. -Tue Feb 16 11:03:19 2010 Aaron Patterson <aaron@t...> +Tue Feb 16 11:03:19 2010 Aaron Patterson <aaron@t...> * ext/dl/method.c: Adding DL::Method as a superclass for DL::Function @@ -2732,7 +2737,7 @@ * bignum.c (big_op): remove unused variables. -Fri Feb 5 02:06:57 2010 Aaron Patterson <tenderlove@r...> +Fri Feb 5 02:06:57 2010 Aaron Patterson <tenderlove@r...> * lib/yaml/rubytypes.rb: Struct members are emitted without a leading colon. Thanks Yusuke Endoh! [ruby-core:28052] @@ -2816,7 +2821,7 @@ * ext/dl/{closure,function}.c: removed C99 features and warnings. -Wed Feb 3 10:12:09 2010 Aaron Patterson <tenderlove@r...> +Wed Feb 3 10:12:09 2010 Aaron Patterson <tenderlove@r...> * ext/dl/function.c: DL::Function now uses libffi @@ -3216,7 +3221,7 @@ make the rdoc(generating html) run faster and use less memory. patch by Tetsu Soh [ruby-core:27656] -Sat Jan 23 19:54:48 2010 Masaki Suketa <masaki.suketa@n...> +Sat Jan 23 19:54:48 2010 Masaki Suketa <masaki.suketa@n...> * ext/win32ole/win32ole.c: add WIN32OLE.ole_initialize, WIN32OLE.ole_uninitialize to use in win32ole.rb. @@ -3342,7 +3347,7 @@ * ext/zlib/{extconf.rb, zlib.c): crc32_combine and adler32_combine is supported on Zlib 1.2.2.1, so check them for old zlib. -Tue Jan 19 09:03:37 2010 Aaron Patterson <tenderlove@r...> +Tue Jan 19 09:03:37 2010 Aaron Patterson <tenderlove@r...> * ext/zlib/zlib.c: added Zlib.crc32_combine and Zlib.adler32_combine @@ -3918,7 +3923,7 @@ * tool/install-sh: wrote the intention. [ruby-dev:39928] -Sun Dec 27 10:45:00 2009 Masaki Suketa <masaki.suketa@n...> +Sun Dec 27 10:45:00 2009 Masaki Suketa <masaki.suketa@n...> * ext/win32ole/win32ole.c (foleparam_initialize): add foleparam_initialize to check argument of WIN32OLE_PARAM.new @@ -4307,7 +4312,7 @@ * vm_eval.c (check_funcall_failed): pass ID. [ruby-core:26934] -Sun Nov 29 06:37:53 2009 Aaron Patterson <tenderlove@r...> +Sun Nov 29 06:37:53 2009 Aaron Patterson <tenderlove@r...> * lib/rexml/formatters/default.rb (write_attribute): fix an exception when printing a document when duplicate namespaced Index: io.c =================================================================== --- io.c (revision 27431) +++ io.c (revision 27432) @@ -2756,15 +2756,15 @@ } if (more_char(fptr) == MORE_CHAR_FINISHED) { - clear_readconv(fptr); - if (fptr->cbuf_len == 0) - return Qnil; + if (fptr->cbuf_len == 0) { + clear_readconv(fptr); + return Qnil; + } /* return an unit of an incomplete character just before EOF */ - r = rb_enc_mbclen(fptr->cbuf+fptr->cbuf_off, - fptr->cbuf+fptr->cbuf_off+fptr->cbuf_len, - read_enc); - io_shift_cbuf(fptr, r, &str); - str = io_enc_str(str, fptr); + str = rb_enc_str_new(fptr->cbuf+fptr->cbuf_off, 1, read_enc); + fptr->cbuf_off += 1; + fptr->cbuf_len -= 1; + if (fptr->cbuf_len == 0) clear_readconv(fptr); ENC_CODERANGE_SET(str, ENC_CODERANGE_BROKEN); return str; } Index: test/ruby/test_io_m17n.rb =================================================================== --- test/ruby/test_io_m17n.rb (revision 27431) +++ test/ruby/test_io_m17n.rb (revision 27432) @@ -400,6 +400,25 @@ } end + def test_getc_newlineconv_invalid + with_tmpdir { + src = "\xE3\x81" + generate_file('tmp', src) + defext = Encoding.default_external + Encoding.default_external = Encoding::UTF_8 + open("tmp", "rt") {|f| + p f.read;f.rewind + s = f.getc + assert_equal(false, s.valid_encoding?) + assert_equal("\xE3".force_encoding("UTF-8"), s) + s = f.getc + assert_equal(false, s.valid_encoding?) + assert_equal("\x81".force_encoding("UTF-8"), s) + } + Encoding.default_external = defext + } + end + def test_ungetc_stateful_conversion with_tmpdir { src = "before \e$B\x23\x30\x23\x31\e(B after".force_encoding("iso-2022-jp") -- ML: ruby-changes@q... Info: http://www.atdot.net/~ko1/quickml/