ruby-changes:40460
From: normal <ko1@a...>
Date: Thu, 12 Nov 2015 11:01:04 +0900 (JST)
Subject: [ruby-changes:40460] normal:r52541 (trunk): io.c: avoid kwarg parsing in C API
normal 2015-11-12 11:00:41 +0900 (Thu, 12 Nov 2015) New Revision: 52541 http://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=52541 Log: io.c: avoid kwarg parsing in C API * benchmark/bm_io_nonblock_noex2.rb: new benchmark based on bm_io_nonblock_noex.rb * io.c (io_read_nonblock): move documentation to prelude.rb (io_write_nonblock): ditto (Init_io): private, internal methods for prelude.rb use only * prelude.rb (IO#read_nonblock): wrapper + documentation (IO#write_nonblock): ditto [ruby-core:71439] [Feature #11339] rb_scan_args and hash lookups for kwargs in the C API are clumsy and slow. Instead of improving the C API for performance, use Ruby instead :) Implement IO#read_nonblock and IO#write_nonblock in prelude.rb to avoid argument parsing via rb_scan_args and hash lookups. This speeds up IO#write_nonblock and IO#read_nonblock benchmarks in both cases, including the original non-idiomatic case where the `exception: false' hash is pre-allocated to avoid GC pressure. Now, writing the kwargs in natural, idiomatic Ruby is fastest. I've added the noex2 benchmark to show this. 2015-11-12 01:41:12 +0000 target 0: a (ruby 2.3.0dev (2015-11-11 trunk 52540) [x86_64-linux]) target 1: b (ruby 2.3.0dev (2015-11-11 avoid-kwarg-capi 52540) ----------------------------------------------------------- benchmark results: minimum results in each 10 measurements. Execution time (sec) name a b io_nonblock_noex 2.508 2.382 io_nonblock_noex2 2.950 1.882 Speedup ratio: compare with the result of `a' (greater is better) name b io_nonblock_noex 1.053 io_nonblock_noex2 1.567 Added files: trunk/benchmark/bm_io_nonblock_noex2.rb Modified files: trunk/ChangeLog trunk/io.c trunk/prelude.rb Index: prelude.rb =================================================================== --- prelude.rb (revision 52540) +++ prelude.rb (revision 52541) @@ -13,3 +13,117 @@ class Thread https://github.com/ruby/ruby/blob/trunk/prelude.rb#L13 } end end + +class IO + + # call-seq: + # ios.read_nonblock(maxlen) -> string + # ios.read_nonblock(maxlen, outbuf) -> outbuf + # + # Reads at most <i>maxlen</i> bytes from <em>ios</em> using + # the read(2) system call after O_NONBLOCK is set for + # the underlying file descriptor. + # + # If the optional <i>outbuf</i> argument is present, + # it must reference a String, which will receive the data. + # The <i>outbuf</i> will contain only the received data after the method call + # even if it is not empty at the beginning. + # + # read_nonblock just calls the read(2) system call. + # It causes all errors the read(2) system call causes: Errno::EWOULDBLOCK, Errno::EINTR, etc. + # The caller should care such errors. + # + # If the exception is Errno::EWOULDBLOCK or Errno::EAGAIN, + # it is extended by IO::WaitReadable. + # So IO::WaitReadable can be used to rescue the exceptions for retrying + # read_nonblock. + # + # read_nonblock causes EOFError on EOF. + # + # If the read byte buffer is not empty, + # read_nonblock reads from the buffer like readpartial. + # In this case, the read(2) system call is not called. + # + # When read_nonblock raises an exception kind of IO::WaitReadable, + # read_nonblock should not be called + # until io is readable for avoiding busy loop. + # This can be done as follows. + # + # # emulates blocking read (readpartial). + # begin + # result = io.read_nonblock(maxlen) + # rescue IO::WaitReadable + # IO.select([io]) + # retry + # end + # + # Although IO#read_nonblock doesn't raise IO::WaitWritable. + # OpenSSL::Buffering#read_nonblock can raise IO::WaitWritable. + # If IO and SSL should be used polymorphically, + # IO::WaitWritable should be rescued too. + # See the document of OpenSSL::Buffering#read_nonblock for sample code. + # + # Note that this method is identical to readpartial + # except the non-blocking flag is set. + def read_nonblock(len, buf = nil, exception: true) + __read_nonblock(len, buf, exception) + end + + # call-seq: + # ios.write_nonblock(string) -> integer + # ios.write_nonblock(string [, options]) -> integer + # + # Writes the given string to <em>ios</em> using + # the write(2) system call after O_NONBLOCK is set for + # the underlying file descriptor. + # + # It returns the number of bytes written. + # + # write_nonblock just calls the write(2) system call. + # It causes all errors the write(2) system call causes: Errno::EWOULDBLOCK, Errno::EINTR, etc. + # The result may also be smaller than string.length (partial write). + # The caller should care such errors and partial write. + # + # If the exception is Errno::EWOULDBLOCK or Errno::EAGAIN, + # it is extended by IO::WaitWritable. + # So IO::WaitWritable can be used to rescue the exceptions for retrying write_nonblock. + # + # # Creates a pipe. + # r, w = IO.pipe + # + # # write_nonblock writes only 65536 bytes and return 65536. + # # (The pipe size is 65536 bytes on this environment.) + # s = "a" #100000 + # p w.write_nonblock(s) #=> 65536 + # + # # write_nonblock cannot write a byte and raise EWOULDBLOCK (EAGAIN). + # p w.write_nonblock("b") # Resource temporarily unavailable (Errno::EAGAIN) + # + # If the write buffer is not empty, it is flushed at first. + # + # When write_nonblock raises an exception kind of IO::WaitWritable, + # write_nonblock should not be called + # until io is writable for avoiding busy loop. + # This can be done as follows. + # + # begin + # result = io.write_nonblock(string) + # rescue IO::WaitWritable, Errno::EINTR + # IO.select(nil, [io]) + # retry + # end + # + # Note that this doesn't guarantee to write all data in string. + # The length written is reported as result and it should be checked later. + # + # On some platforms such as Windows, write_nonblock is not supported + # according to the kind of the IO object. + # In such cases, write_nonblock raises <code>Errno::EBADF</code>. + # + # By specifying `exception: false`, the options hash allows you to indicate + # that write_nonblock should not raise an IO::WaitWritable exception, but + # return the symbol :wait_writable instead. + def write_nonblock(buf, exception: true) + __write_nonblock(buf, exception) + end +end Index: ChangeLog =================================================================== --- ChangeLog (revision 52540) +++ ChangeLog (revision 52541) @@ -1,3 +1,14 @@ https://github.com/ruby/ruby/blob/trunk/ChangeLog#L1 +Thu Nov 12 10:53:41 2015 Eric Wong <e@8...> + + * benchmark/bm_io_nonblock_noex2.rb: new benchmark based + on bm_io_nonblock_noex.rb + * io.c (io_read_nonblock): move documentation to prelude.rb + (io_write_nonblock): ditto + (Init_io): private, internal methods for prelude.rb use only + * prelude.rb (IO#read_nonblock): wrapper + documentation + (IO#write_nonblock): ditto + [ruby-core:71439] [Feature #11339] + Wed Nov 11 18:30:28 2015 Nobuyoshi Nakada <nobu@r...> * sprintf.c (rb_str_format): look up the key, then get default Index: io.c =================================================================== --- io.c (revision 52540) +++ io.c (revision 52541) @@ -2632,74 +2632,56 @@ io_nonblock_eof(VALUE opts) https://github.com/ruby/ruby/blob/trunk/io.c#L2632 return Qnil; } -/* - * call-seq: - * ios.read_nonblock(maxlen) -> string - * ios.read_nonblock(maxlen, outbuf) -> outbuf - * - * Reads at most <i>maxlen</i> bytes from <em>ios</em> using - * the read(2) system call after O_NONBLOCK is set for - * the underlying file descriptor. - * - * If the optional <i>outbuf</i> argument is present, - * it must reference a String, which will receive the data. - * The <i>outbuf</i> will contain only the received data after the method call - * even if it is not empty at the beginning. - * - * read_nonblock just calls the read(2) system call. - * It causes all errors the read(2) system call causes: Errno::EWOULDBLOCK, Errno::EINTR, etc. - * The caller should care such errors. - * - * If the exception is Errno::EWOULDBLOCK or Errno::EAGAIN, - * it is extended by IO::WaitReadable. - * So IO::WaitReadable can be used to rescue the exceptions for retrying read_nonblock. - * - * read_nonblock causes EOFError on EOF. - * - * If the read byte buffer is not empty, - * read_nonblock reads from the buffer like readpartial. - * In this case, the read(2) system call is not called. - * - * When read_nonblock raises an exception kind of IO::WaitReadable, - * read_nonblock should not be called - * until io is readable for avoiding busy loop. - * This can be done as follows. - * - * # emulates blocking read (readpartial). - * begin - * result = io.read_nonblock(maxlen) - * rescue IO::WaitReadable - * IO.select([io]) - * retry - * end - * - * Although IO#read_nonblock doesn't raise IO::WaitWritable. - * OpenSSL::Buffering#read_nonblock can raise IO::WaitWritable. - * If IO and SSL should be used polymorphically, - * IO::WaitWritable should be rescued too. - * See the document of OpenSSL::Buffering#read_nonblock for sample code. - * - * Note that this method is identical to readpartial - * except the non-blocking flag is set. - */ - +/* :nodoc: */ static VALUE -io_read_nonblock(int argc, VALUE *argv, VALUE io) +io_read_nonblock(VALUE io, VALUE length, VALUE str, VALUE ex) { - VALUE ret, opts; + rb_io_t *fptr; + long n, len; + struct read_internal_arg arg; - rb_scan_args(argc, argv, "11:", NULL, NULL, &opts); + if ((len = NUM2LONG(length)) < 0) { + rb_raise(rb_eArgError, "negative length %ld given", len); + } - ret = io_getpartial(argc, argv, io, opts, 1); + io_setstrbuf(&str,len); + OBJ_TAINT(str); + GetOpenFile(io, fptr); + rb_io_check_byte_readable(fptr); + + if (len == 0) + return str; - if (NIL_P(ret)) { - return io_nonblock_eof(opts); + n = read_buffered_data(RSTRING_PTR(str), len, fptr); + if (n <= 0) { + rb_io_set_nonblock(fptr); + io_setstrbuf(&str, len); + arg.fd = fptr->fd; + arg.str_ptr = RSTRING_PTR(str); + arg.len = len; + rb_str_locktmp_ensure(str, read_internal_call, (VALUE)&arg); + n = arg.len; + if (n < 0) { + if ((errno == EWOULDBLOCK || errno == EAGAIN)) { + if (ex == Qfalse) return sym_wait_readable; + rb_readwrite_sys_fail(RB_IO_WAIT_READABLE, "read would block"); + } + rb_sys_fail_path(fptr->pathv); + } } - return ret; + io_set_read_length(str, n); + + if (n == 0) { + if (ex == Qfalse) return Qnil; + rb_eof_error(); + } + + return str; } +/* :nodoc: */ static VALUE -io_write_nonblock(VALUE io, VALUE str, VALUE opts) +io_write_nonblock(VALUE io, VALUE str, VALUE ex) { rb_io_t *fptr; long n; @@ -2719,7 +2701,7 @@ io_write_nonblock(VALUE io, VALUE str, V https://github.com/ruby/ruby/blob/trunk/io.c#L2701 if (n == -1) { if (errno == EWOULDBLOCK || errno == EAGAIN) { - if (no_exception_p(opts)) { + if (ex == Qfalse) { return sym_wait_writable; } else { @@ -2734,74 +2716,6 @@ io_write_nonblock(VALUE io, VALUE str, V https://github.com/ruby/ruby/blob/trunk/io.c#L2716 /* * call-seq: - * ios.write_nonblock(string) -> integer - * ios.write_nonblock(string [, options]) -> integer - * - * Writes the given string to <em>ios</em> using - * the write(2) system call after O_NONBLOCK is set for - * the underlying file descriptor. - * - * It returns the number of bytes written. - * - * write_nonblock just calls the write(2) system call. - * It causes all errors the write(2) system call causes: Errno::EWOULDBLOCK, Errno::EINTR, etc. - * The result may also be smaller than string.length (partial write). - * The caller should care such errors and partial write. - * - * If the exception is Errno::EWOULDBLOCK or Errno::EAGAIN, - * it is extended by IO::WaitWritable. - * So IO::WaitWritable can be used to rescue the exceptions for retrying write_nonblock. - * - * # Creates a pipe. - * r, w = IO.pipe - * - * # write_nonblock writes only 65536 bytes and return 65536. - * # (The pipe size is 65536 bytes on this environment.) - * s = "a" * 100000 - * p w.write_nonblock(s) #=> 65536 - * - * # write_nonblock cannot write a byte and raise EWOULDBLOCK (EAGAIN). - * p w.write_nonblock("b") # Resource temporarily unavailable (Errno::EAGAIN) - * - * If the write buffer is not empty, it is flushed at first. - * - * When write_nonblock raises an exception kind of IO::WaitWritable, - * write_nonblock should not be called - * until io is writable for avoiding busy loop. - * This can be done as follows. - * - * begin - * result = io.write_nonblock(string) - * rescue IO::WaitWritable, Errno::EINTR - * IO.select(nil, [io]) - * retry - * end - * - * Note that this doesn't guarantee to write all data in string. - * The length written is reported as result and it should be checked later. - * - * On some platforms such as Windows, write_nonblock is not supported - * according to the kind of the IO object. - * In such cases, write_nonblock raises <code>Errno::EBADF</code>. - * - * By specifying `exception: false`, the options hash allows you to indicate - * that write_nonblock should not raise an IO::WaitWritable exception, but - * return the symbol :wait_writable instead. - * - */ - -static VALUE -rb_io_write_nonblock(int argc, VALUE *argv, VALUE io) -{ - VALUE str, opts; - - rb_scan_args(argc, argv, "10:", &str, &opts); - - return io_write_nonblock(io, str, opts); -} - -/* - * call-seq: * ios.read([length [, outbuf]]) -> string, outbuf, or nil * * Reads <i>length</i> bytes from the I/O stream. @@ -12386,8 +12300,10 @@ Init_IO(void) https://github.com/ruby/ruby/blob/trunk/io.c#L12300 rb_define_method(rb_cIO, "readlines", rb_io_readlines, -1); - rb_define_method(rb_cIO, "read_nonblock", io_read_nonblock, -1); - rb_define_method(rb_cIO, "write_nonblock", rb_io_write_nonblock, -1); + /* for prelude.rb use only: */ + rb_define_private_method(rb_cIO, "__read_nonblock", io_read_nonblock, 3); + rb_define_private_method(rb_cIO, "__write_nonblock", io_write_nonblock, 2); + rb_define_method(rb_cIO, "readpartial", io_readpartial, -1); rb_define_method(rb_cIO, "read", io_read, -1); rb_define_method(rb_cIO, "write", io_write_m, 1); Index: benchmark/bm_io_nonblock_noex2.rb =================================================================== --- benchmark/bm_io_nonblock_noex2.rb (revision 0) +++ benchmark/bm_io_nonblock_noex2.rb (revision 52541) @@ -0,0 +1,21 @@ https://github.com/ruby/ruby/blob/trunk/benchmark/bm_io_nonblock_noex2.rb#L1 +nr = 1_000_000 +i = 0 +msg = '.' +buf = '.' +begin + r, w = IO.pipe + while i < nr + i += 1 + w.write_nonblock(msg, exception: false) + r.read_nonblock(1, buf, exception: false) + end +rescue ArgumentError # old Rubies + while i < nr + i += 1 + w.write_nonblock(msg) + r.read_nonblock(1, buf) + end +ensure + r.close + w.close +end -- ML: ruby-changes@q... Info: http://www.atdot.net/~ko1/quickml/