[前][次][番号順一覧][スレッド一覧]

ruby-changes:50806

From: nagachika <ko1@a...>
Date: Wed, 28 Mar 2018 22:55:06 +0900 (JST)
Subject: [ruby-changes:50806] nagachika:r63012 (ruby_2_4): merge revision(s) 60584, 62954, 62955, 62956, 62957, 62958, 62959, 63008:

nagachika	2018-03-28 22:54:58 +0900 (Wed, 28 Mar 2018)

  New Revision: 63012

  https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=63012

  Log:
    merge revision(s) 60584,62954,62955,62956,62957,62958,62959,63008:
    
    webrick: support Proc objects as body responses
    
    * lib/webrick/httpresponse.rb (send_body): call send_body_proc
      (send_body_proc): new method
      (class ChunkedWrapper): new class
    
    * test/webrick/test_httpresponse.rb (test_send_body_proc): new test
      (test_send_body_proc_chunked): ditto
      [Feature #855]
    
    webrick/httpresponse: IO.copy_stream for regular files
    
    Remove the redundant _send_file method since its functionality
    is unnecessary with IO.copy_stream.  IO.copy_stream also allows
    the use of sendfile under some OSes to speed up copies to
    non-TLS sockets.
    
    Testing with "curl >/dev/null" and "ruby -run -e httpd" to
    read a 1G file over Linux loopback reveals a reduction from
    around ~0.770 to ~0.490 seconds on the client side.
    
    * lib/webrick/httpresponse.rb (send_body_io): use IO.copy_stream
      (_send_file): remove
      [Feature #14237]
    
    webrick: use IO.copy_stream for single range response
    
    This is also compatible with range responses generated
    by Rack::File (tested with rack 2.0.3).
    
    * lib/webrick/httpresponse.rb (send_body_io): use Content-Range
    * lib/webrick/httpservlet/filehandler.rb (make_partial_content):
      use File object for the single range case
    * test/webrick/test_filehandler.rb (get_res_body): use send_body
      to test result
    
    test/webrick/test_filehandler.rb: stricter multipart range test
    
    We need to ensure we generate compatibile output in
    the face of future changes
    
    * test/webrick/test_filehandler.rb (test_make_partial_content):
      check response body
    
    webrick: quiet warning for multi-part ranges
    
    Content-Length is ignored by WEBrick::HTTPResponse even if we
    calculate it, so instead we chunk responses to HTTP/1.1 clients
    and terminate HTTP/1.0 connections.
    
    * lib/webrick/httpservlet/filehandler.rb (make_partial_content):
      quiet warning
    
    webrick/httpresponse: make ChunkedWrapper copy_stream-compatible
    
    The .write method needs to return the number of bytes written
    to avoid confusing IO.copy_stream.
    
    * lib/webrick/httpresponse.rb (ChunkedWrapper#write): return bytes written
      (ChunkedWrapper#<<): return self
    
    webrick: use IO.copy_stream for multipart response
    
    Use the new Proc response body feature to generate a multipart
    range response dynamically.  We use a flat array to minimize
    object overhead as much as possible; as many ranges may fit
    into an HTTP request header.
    
    * lib/webrick/httpservlet/filehandler.rb (multipart_body): new method
      (make_partial_content): use multipart_body
    
    get rid of test error/failure on Windows introduced at r62955
    
    * lib/webrick/httpresponse.rb (send_body_io): use seek if NotImplementedError
      is raised in IO.copy_stream with offset.
    
    * lib/webrick/httpservlet/filehandler.rb (multipart_body): ditto.

  Modified directories:
    branches/ruby_2_4/
  Modified files:
    branches/ruby_2_4/lib/webrick/httpresponse.rb
    branches/ruby_2_4/lib/webrick/httpservlet/filehandler.rb
    branches/ruby_2_4/test/webrick/test_filehandler.rb
    branches/ruby_2_4/test/webrick/test_httpresponse.rb
    branches/ruby_2_4/version.h
Index: ruby_2_4/lib/webrick/httpresponse.rb
===================================================================
--- ruby_2_4/lib/webrick/httpresponse.rb	(revision 63011)
+++ ruby_2_4/lib/webrick/httpresponse.rb	(revision 63012)
@@ -310,6 +310,8 @@ module WEBrick https://github.com/ruby/ruby/blob/trunk/ruby_2_4/lib/webrick/httpresponse.rb#L310
     def send_body(socket) # :nodoc:
       if @body.respond_to? :readpartial then
         send_body_io(socket)
+      elsif @body.respond_to?(:call) then
+        send_body_proc(socket)
       else
         send_body_string(socket)
       end
@@ -424,9 +426,20 @@ module WEBrick https://github.com/ruby/ruby/blob/trunk/ruby_2_4/lib/webrick/httpresponse.rb#L426
           end
           socket.write("0#{CRLF}#{CRLF}")
         else
-          size = @header['content-length'].to_i
-          _send_file(socket, @body, 0, size)
-          @sent_size = size
+          if %r{\Abytes (\d+)-(\d+)/\d+\z} =~ @header['content-range']
+            offset = $1.to_i
+            size = $2.to_i - offset + 1
+          else
+            offset = nil
+            size = @header['content-length']
+            size = size.to_i if size
+          end
+          begin
+            @sent_size = IO.copy_stream(@body, socket, size, offset)
+          rescue NotImplementedError
+            @body.seek(offset, IO::SEEK_SET)
+            @sent_size = IO.copy_stream(@body, socket, size)
+          end
         end
       ensure
         @body.close
@@ -455,24 +468,41 @@ module WEBrick https://github.com/ruby/ruby/blob/trunk/ruby_2_4/lib/webrick/httpresponse.rb#L468
       end
     end
 
-    def _send_file(output, input, offset, size)
-      while offset > 0
-        sz = @buffer_size < size ? @buffer_size : size
-        buf = input.read(sz)
-        offset -= buf.bytesize
+    def send_body_proc(socket)
+      if @request_method == "HEAD"
+        # do nothing
+      elsif chunked?
+        @body.call(ChunkedWrapper.new(socket, self))
+        _write_data(socket, "0#{CRLF}#{CRLF}")
+      else
+        size = @header['content-length'].to_i
+        @body.call(socket)
+        @sent_size = size
       end
+    end
 
-      if size == 0
-        while buf = input.read(@buffer_size)
-          output.write(buf)
-        end
-      else
-        while size > 0
-          sz = @buffer_size < size ? @buffer_size : size
-          buf = input.read(sz)
-          output.write(buf)
-          size -= buf.bytesize
-        end
+    class ChunkedWrapper
+      def initialize(socket, resp)
+        @socket = socket
+        @resp = resp
+      end
+
+      def write(buf)
+        return 0 if buf.empty?
+        socket = @socket
+        @resp.instance_eval {
+          size = buf.bytesize
+          data = "#{size.to_s(16)}#{CRLF}#{buf}#{CRLF}"
+          _write_data(socket, data)
+          data.clear
+          @sent_size += size
+          size
+        }
+      end
+
+      def <<(*buf)
+        write(buf)
+        self
       end
     end
 
Index: ruby_2_4/lib/webrick/httpservlet/filehandler.rb
===================================================================
--- ruby_2_4/lib/webrick/httpservlet/filehandler.rb	(revision 63011)
+++ ruby_2_4/lib/webrick/httpservlet/filehandler.rb	(revision 63012)
@@ -87,6 +87,35 @@ module WEBrick https://github.com/ruby/ruby/blob/trunk/ruby_2_4/lib/webrick/httpservlet/filehandler.rb#L87
         return false
       end
 
+      # returns a lambda for webrick/httpresponse.rb send_body_proc
+      def multipart_body(body, parts, boundary, mtype, filesize)
+        lambda do |socket|
+          begin
+            begin
+              first = parts.shift
+              last = parts.shift
+              socket.write(
+                "--#{boundary}#{CRLF}" \
+                "Content-Type: #{mtype}#{CRLF}" \
+                "Content-Range: bytes #{first}-#{last}/#{filesize}#{CRLF}" \
+                "#{CRLF}"
+              )
+
+              begin
+                IO.copy_stream(body, socket, last - first + 1, first)
+              rescue NotImplementedError
+                body.seek(first, IO::SEEK_SET)
+                IO.copy_stream(body, socket, last - first + 1)
+              end
+              socket.write(CRLF)
+            end while parts[0]
+            socket.write("--#{boundary}--#{CRLF}")
+          ensure
+            body.close
+          end
+        end
+      end
+
       def make_partial_content(req, res, filename, filesize)
         mtype = HTTPUtils::mime_type(filename, @config[:MimeTypes])
         unless ranges = HTTPUtils::parse_range_header(req['range'])
@@ -97,37 +126,27 @@ module WEBrick https://github.com/ruby/ruby/blob/trunk/ruby_2_4/lib/webrick/httpservlet/filehandler.rb#L126
           if ranges.size > 1
             time = Time.now
             boundary = "#{time.sec}_#{time.usec}_#{Process::pid}"
-            body = ''
-            ranges.each{|range|
-              first, last = prepare_range(range, filesize)
-              next if first < 0
-              io.pos = first
-              content = io.read(last-first+1)
-              body << "--" << boundary << CRLF
-              body << "Content-Type: #{mtype}" << CRLF
-              body << "Content-Range: bytes #{first}-#{last}/#{filesize}" << CRLF
-              body << CRLF
-              body << content
-              body << CRLF
+            parts = []
+            ranges.each {|range|
+              prange = prepare_range(range, filesize)
+              next if prange[0] < 0
+              parts.concat(prange)
             }
-            raise HTTPStatus::RequestRangeNotSatisfiable if body.empty?
-            body << "--" << boundary << "--" << CRLF
+            raise HTTPStatus::RequestRangeNotSatisfiable if parts.empty?
             res["content-type"] = "multipart/byteranges; boundary=#{boundary}"
-            res.body = body
+            if req.http_version < '1.1'
+              res['connection'] = 'close'
+            else
+              res.chunked = true
+            end
+            res.body = multipart_body(io.dup, parts, boundary, mtype, filesize)
           elsif range = ranges[0]
             first, last = prepare_range(range, filesize)
             raise HTTPStatus::RequestRangeNotSatisfiable if first < 0
-            if last == filesize - 1
-              content = io.dup
-              content.pos = first
-            else
-              io.pos = first
-              content = io.read(last-first+1)
-            end
             res['content-type'] = mtype
             res['content-range'] = "bytes #{first}-#{last}/#{filesize}"
             res['content-length'] = last - first + 1
-            res.body = content
+            res.body = io.dup
           else
             raise HTTPStatus::BadRequest
           end
Index: ruby_2_4/version.h
===================================================================
--- ruby_2_4/version.h	(revision 63011)
+++ ruby_2_4/version.h	(revision 63012)
@@ -1,6 +1,6 @@ https://github.com/ruby/ruby/blob/trunk/ruby_2_4/version.h#L1
 #define RUBY_VERSION "2.4.4"
 #define RUBY_RELEASE_DATE "2018-03-28"
-#define RUBY_PATCHLEVEL 295
+#define RUBY_PATCHLEVEL 296
 
 #define RUBY_RELEASE_YEAR 2018
 #define RUBY_RELEASE_MONTH 3
Index: ruby_2_4/test/webrick/test_filehandler.rb
===================================================================
--- ruby_2_4/test/webrick/test_filehandler.rb	(revision 63011)
+++ ruby_2_4/test/webrick/test_filehandler.rb	(revision 63012)
@@ -20,16 +20,10 @@ class WEBrick::TestFileHandler < Test::U https://github.com/ruby/ruby/blob/trunk/ruby_2_4/test/webrick/test_filehandler.rb#L20
   end
 
   def get_res_body(res)
-    body = res.body
-    if defined? body.read
-      begin
-        body.read
-      ensure
-        body.close
-      end
-    else
-      body
-    end
+    sio = StringIO.new
+    sio.binmode
+    res.send_body(sio)
+    sio.string
   end
 
   def make_range_request(range_spec)
@@ -81,6 +75,23 @@ class WEBrick::TestFileHandler < Test::U https://github.com/ruby/ruby/blob/trunk/ruby_2_4/test/webrick/test_filehandler.rb#L75
 
     res = make_range_response(filename, "bytes=0-0, -2")
     assert_match(%r{^multipart/byteranges}, res["content-type"])
+    body = get_res_body(res)
+    boundary = /; boundary=(.+)/.match(res['content-type'])[1]
+    off = filesize - 2
+    last = filesize - 1
+
+    exp = "--#{boundary}\r\n" \
+          "Content-Type: text/plain\r\n" \
+          "Content-Range: bytes 0-0/#{filesize}\r\n" \
+          "\r\n" \
+          "#{IO.read(__FILE__, 1)}\r\n" \
+          "--#{boundary}\r\n" \
+          "Content-Type: text/plain\r\n" \
+          "Content-Range: bytes #{off}-#{last}/#{filesize}\r\n" \
+          "\r\n" \
+          "#{IO.read(__FILE__, 2, off)}\r\n" \
+          "--#{boundary}--\r\n"
+    assert_equal exp, body
   end
 
   def test_filehandler
Index: ruby_2_4/test/webrick/test_httpresponse.rb
===================================================================
--- ruby_2_4/test/webrick/test_httpresponse.rb	(revision 63011)
+++ ruby_2_4/test/webrick/test_httpresponse.rb	(revision 63012)
@@ -169,6 +169,29 @@ module WEBrick https://github.com/ruby/ruby/blob/trunk/ruby_2_4/test/webrick/test_httpresponse.rb#L169
       assert_equal 0, logger.messages.length
     end
 
+    def test_send_body_proc
+      @res.body = Proc.new { |out| out.write('hello') }
+      IO.pipe do |r, w|
+        @res.send_body(w)
+        w.close
+        r.binmode
+        assert_equal 'hello', r.read
+      end
+      assert_equal 0, logger.messages.length
+    end
+
+    def test_send_body_proc_chunked
+      @res.body = Proc.new { |out| out.write('hello') }
+      @res.chunked = true
+      IO.pipe do |r, w|
+        @res.send_body(w)
+        w.close
+        r.binmode
+        assert_equal "5\r\nhello\r\n0\r\n\r\n", r.read
+      end
+      assert_equal 0, logger.messages.length
+    end
+
     def test_set_error
       status = 400
       message = 'missing attribute'
Index: ruby_2_4
===================================================================
--- ruby_2_4	(revision 63011)
+++ ruby_2_4	(revision 63012)

Property changes on: ruby_2_4
___________________________________________________________________
Modified: svn:mergeinfo
## -0,0 +0,1 ##
   Merged /trunk:r60584,62954-62959,63008

--
ML: ruby-changes@q...
Info: http://www.atdot.net/~ko1/quickml/

[前][次][番号順一覧][スレッド一覧]