ruby-changes:50326
From: usa <ko1@a...>
Date: Sat, 17 Feb 2018 01:28:01 +0900 (JST)
Subject: [ruby-changes:50326] usa:r62441 (ruby_2_2): merge revision(s) 58471, 58493, 62436: [Backport #13505]
usa 2018-02-17 01:27:56 +0900 (Sat, 17 Feb 2018) New Revision: 62441 https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=62441 Log: merge revision(s) 58471,58493,62436: [Backport #13505] load.c: backtrace of circular require * load.c (load_lock): print backtrace of circular require via `Warning.warn` [ruby-core:80850] [Bug #13505] Send the backtrace of the circular require warning as a single String to Warning.warn * load.c: send as a single string. * error.c: expose the string formatted by rb_warning as rb_warning_string(). * test/ruby/test_exception.rb: update tests. [ruby-core:80850] [Bug #13505] fix regexp literal warning. * test/rubygems/test_gem_server.rb: eliminate duplicated character class warning. [Bug #14481] Modified files: branches/ruby_2_2/ChangeLog branches/ruby_2_2/lib/rubygems/package/tar_header.rb branches/ruby_2_2/lib/rubygems/package/tar_writer.rb branches/ruby_2_2/lib/rubygems/package.rb branches/ruby_2_2/lib/rubygems/server.rb branches/ruby_2_2/lib/rubygems/specification.rb branches/ruby_2_2/lib/rubygems.rb branches/ruby_2_2/test/rubygems/test_gem_package.rb branches/ruby_2_2/test/rubygems/test_gem_package_tar_header.rb branches/ruby_2_2/test/rubygems/test_gem_server.rb branches/ruby_2_2/test/rubygems/test_gem_specification.rb branches/ruby_2_2/version.h Index: ruby_2_2/version.h =================================================================== --- ruby_2_2/version.h (revision 62440) +++ ruby_2_2/version.h (revision 62441) @@ -1,10 +1,10 @@ https://github.com/ruby/ruby/blob/trunk/ruby_2_2/version.h#L1 #define RUBY_VERSION "2.2.9" -#define RUBY_RELEASE_DATE "2017-12-15" -#define RUBY_PATCHLEVEL 480 +#define RUBY_RELEASE_DATE "2018-02-17" +#define RUBY_PATCHLEVEL 481 -#define RUBY_RELEASE_YEAR 2017 -#define RUBY_RELEASE_MONTH 12 -#define RUBY_RELEASE_DAY 15 +#define RUBY_RELEASE_YEAR 2018 +#define RUBY_RELEASE_MONTH 2 +#define RUBY_RELEASE_DAY 17 #include "ruby/version.h" Index: ruby_2_2/ChangeLog =================================================================== --- ruby_2_2/ChangeLog (revision 62440) +++ ruby_2_2/ChangeLog (revision 62441) @@ -1,3 +1,22 @@ https://github.com/ruby/ruby/blob/trunk/ruby_2_2/ChangeLog#L1 +Sat Feb 17 01:24:49 2018 SHIBATA Hiroshi <hsbt@r...> + + load.c: backtrace of circular require + + * load.c (load_lock): print backtrace of circular require via + `Warning.warn` [ruby-core:80850] [Bug #13505] + + Send the backtrace of the circular require warning as a single String to Warning.warn + + * load.c: send as a single string. + * error.c: expose the string formatted by rb_warning as rb_warning_string(). + * test/ruby/test_exception.rb: update tests. + [ruby-core:80850] [Bug #13505] + + fix regexp literal warning. + + * test/rubygems/test_gem_server.rb: eliminate duplicated character class warning. + [Bug #14481] + Fri Dec 15 00:08:26 2017 NAKAMURA Usaku <usa@r...> * test/net/ftp/test_ftp.rb (process_port_or_eprt): merge a part of Index: ruby_2_2/lib/rubygems.rb =================================================================== --- ruby_2_2/lib/rubygems.rb (revision 62440) +++ ruby_2_2/lib/rubygems.rb (revision 62441) @@ -9,7 +9,7 @@ require 'rbconfig' https://github.com/ruby/ruby/blob/trunk/ruby_2_2/lib/rubygems.rb#L9 require 'thread' module Gem - VERSION = '2.4.5.4' + VERSION = '2.4.5.5' end # Must be first since it unloads the prelude from 1.9.2 Index: ruby_2_2/lib/rubygems/package.rb =================================================================== --- ruby_2_2/lib/rubygems/package.rb (revision 62440) +++ ruby_2_2/lib/rubygems/package.rb (revision 62441) @@ -408,7 +408,7 @@ EOM https://github.com/ruby/ruby/blob/trunk/ruby_2_2/lib/rubygems/package.rb#L408 destination = File.expand_path destination raise Gem::Package::PathError.new(destination, destination_dir) unless - destination.start_with? destination_dir + destination.start_with? destination_dir + '/' destination.untaint destination @@ -587,6 +587,10 @@ EOM https://github.com/ruby/ruby/blob/trunk/ruby_2_2/lib/rubygems/package.rb#L587 raise Gem::Package::FormatError.new \ 'package content (data.tar.gz) is missing', @gem end + + if duplicates = @files.group_by {|f| f }.select {|k,v| v.size > 1 }.map(&:first) and duplicates.any? + raise Gem::Security::Exception, "duplicate files in the package: (#{duplicates.map(&:inspect).join(', ')})" + end end ## Index: ruby_2_2/lib/rubygems/specification.rb =================================================================== --- ruby_2_2/lib/rubygems/specification.rb (revision 62440) +++ ruby_2_2/lib/rubygems/specification.rb (revision 62441) @@ -13,6 +13,7 @@ require 'rubygems/deprecate' https://github.com/ruby/ruby/blob/trunk/ruby_2_2/lib/rubygems/specification.rb#L13 require 'rubygems/basic_specification' require 'rubygems/stub_specification' require 'rubygems/util/stringio' +require 'uri' ## # The Specification class contains the information for a Gem. Typically @@ -2609,10 +2610,16 @@ http://opensource.org/licenses/alphabeti https://github.com/ruby/ruby/blob/trunk/ruby_2_2/lib/rubygems/specification.rb#L2610 raise Gem::InvalidSpecificationException, "#{lazy} is not a summary" end - if homepage and not homepage.empty? and - homepage !~ /\A[a-z][a-z\d+.-]*:/i then - raise Gem::InvalidSpecificationException, - "\"#{homepage}\" is not a URI" + # Make sure a homepage is valid HTTP/HTTPS URI + if homepage and not homepage.empty? + begin + homepage_uri = URI.parse(homepage) + unless [URI::HTTP, URI::HTTPS].member? homepage_uri.class + raise Gem::InvalidSpecificationException, "\"#{homepage}\" is not a valid HTTP URI" + end + rescue URI::InvalidURIError + raise Gem::InvalidSpecificationException, "\"#{homepage}\" is not a valid HTTP URI" + end end # Warnings Index: ruby_2_2/lib/rubygems/package/tar_header.rb =================================================================== --- ruby_2_2/lib/rubygems/package/tar_header.rb (revision 62440) +++ ruby_2_2/lib/rubygems/package/tar_header.rb (revision 62441) @@ -103,25 +103,30 @@ class Gem::Package::TarHeader https://github.com/ruby/ruby/blob/trunk/ruby_2_2/lib/rubygems/package/tar_header.rb#L103 fields = header.unpack UNPACK_FORMAT new :name => fields.shift, - :mode => fields.shift.oct, - :uid => fields.shift.oct, - :gid => fields.shift.oct, - :size => fields.shift.oct, - :mtime => fields.shift.oct, - :checksum => fields.shift.oct, + :mode => strict_oct(fields.shift), + :uid => strict_oct(fields.shift), + :gid => strict_oct(fields.shift), + :size => strict_oct(fields.shift), + :mtime => strict_oct(fields.shift), + :checksum => strict_oct(fields.shift), :typeflag => fields.shift, :linkname => fields.shift, :magic => fields.shift, - :version => fields.shift.oct, + :version => strict_oct(fields.shift), :uname => fields.shift, :gname => fields.shift, - :devmajor => fields.shift.oct, - :devminor => fields.shift.oct, + :devmajor => strict_oct(fields.shift), + :devminor => strict_oct(fields.shift), :prefix => fields.shift, :empty => empty end + def self.strict_oct(str) + return str.oct if str =~ /\A[0-7]*\z/ + raise ArgumentError, "#{str.inspect} is not an octal string" + end + ## # Creates a new TarHeader using +vals+ Index: ruby_2_2/lib/rubygems/package/tar_writer.rb =================================================================== --- ruby_2_2/lib/rubygems/package/tar_writer.rb (revision 62440) +++ ruby_2_2/lib/rubygems/package/tar_writer.rb (revision 62441) @@ -195,6 +195,8 @@ class Gem::Package::TarWriter https://github.com/ruby/ruby/blob/trunk/ruby_2_2/lib/rubygems/package/tar_writer.rb#L195 digest_name == signer.digest_name end + raise "no #{signer.digest_name} in #{digests.values.compact}" unless signature_digest + if signer.key then signature = signer.sign signature_digest.digest Index: ruby_2_2/lib/rubygems/server.rb =================================================================== --- ruby_2_2/lib/rubygems/server.rb (revision 62440) +++ ruby_2_2/lib/rubygems/server.rb (revision 62441) @@ -625,6 +625,18 @@ div.method-source-code pre { color: #ffd https://github.com/ruby/ruby/blob/trunk/ruby_2_2/lib/rubygems/server.rb#L625 executables = nil if executables.empty? executables.last["is_last"] = true if executables + # Pre-process spec homepage for safety reasons + begin + homepage_uri = URI.parse(spec.homepage) + if [URI::HTTP, URI::HTTPS].member? homepage_uri.class + homepage_uri = spec.homepage + else + homepage_uri = "." + end + rescue URI::InvalidURIError + homepage_uri = "." + end + specs << { "authors" => spec.authors.sort.join(", "), "date" => spec.date.to_s, @@ -634,7 +646,7 @@ div.method-source-code pre { color: #ffd https://github.com/ruby/ruby/blob/trunk/ruby_2_2/lib/rubygems/server.rb#L646 "only_one_executable" => (executables && executables.size == 1), "full_name" => spec.full_name, "has_deps" => !deps.empty?, - "homepage" => spec.homepage, + "homepage" => homepage_uri, "name" => spec.name, "rdoc_installed" => Gem::RDoc.new(spec).rdoc_installed?, "ri_installed" => Gem::RDoc.new(spec).ri_installed?, Index: ruby_2_2/test/rubygems/test_gem_server.rb =================================================================== --- ruby_2_2/test/rubygems/test_gem_server.rb (revision 62440) +++ ruby_2_2/test/rubygems/test_gem_server.rb (revision 62441) @@ -331,6 +331,171 @@ class TestGemServer < Gem::TestCase https://github.com/ruby/ruby/blob/trunk/ruby_2_2/test/rubygems/test_gem_server.rb#L331 assert_match 'z 9', @res.body end + + def test_xss_homepage_fix_289313 + data = StringIO.new "GET / HTTP/1.0\r\n\r\n" + dir = "#{@gemhome}2" + + spec = util_spec 'xsshomepagegem', 1 + spec.homepage = "javascript:confirm(document.domain)" + + specs_dir = File.join dir, 'specifications' + FileUtils.mkdir_p specs_dir + + open File.join(specs_dir, spec.spec_name), 'w' do |io| + io.write spec.to_ruby + end + + server = Gem::Server.new dir, process_based_port, false + + @req.parse data + + server.root @req, @res + + assert_equal 200, @res.status + assert_match 'xsshomepagegem 1', @res.body + + # This verifies that the homepage for this spec is not displayed and is set to ".", because it's not a + # valid HTTP/HTTPS URL and could be unsafe in an HTML context. We would prefer to throw an exception here, + # but spec.homepage is currently free form and not currently required to be a URL, this behavior may be + # validated in future versions of Gem::Specification. + # + # There are two variant we're checking here, one where rdoc is not present, and one where rdoc is present in the same regex: + # + # Variant #1 - rdoc not installed + # + # <b>xsshomepagegem 1</b> + # + # + # <span title="rdoc not installed">[rdoc]</span> + # + # + # + # <a href="." title=".">[www]</a> + # + # Variant #2 - rdoc installed + # + # <b>xsshomepagegem 1</b> + # + # + # <a href="\/doc_root\/xsshomepagegem-1\/">\[rdoc\]<\/a> + # + # + # + # <a href="." title=".">[www]</a> + regex_match = /xsshomepagegem 1<\/b>[\s]+(<span title="rdoc not installed">\[rdoc\]<\/span>|<a href="\/doc_root\/xsshomepagegem-1\/">\[rdoc\]<\/a>)[\s]+<a href="\." title="\.">\[www\]<\/a>/ + assert_match regex_match, @res.body + end + + def test_invalid_homepage + data = StringIO.new "GET / HTTP/1.0\r\n\r\n" + dir = "#{@gemhome}2" + + spec = util_spec 'invalidhomepagegem', 1 + spec.homepage = "notavalidhomepageurl" + + specs_dir = File.join dir, 'specifications' + FileUtils.mkdir_p specs_dir + + open File.join(specs_dir, spec.spec_name), 'w' do |io| + io.write spec.to_ruby + end + + server = Gem::Server.new dir, process_based_port, false + + @req.parse data + + server.root @req, @res + + assert_equal 200, @res.status + assert_match 'invalidhomepagegem 1', @res.body + + # This verifies that the homepage for this spec is not displayed and is set to ".", because it's not a + # valid HTTP/HTTPS URL and could be unsafe in an HTML context. We would prefer to throw an exception here, + # but spec.homepage is currently free form and not currently required to be a URL, this behavior may be + # validated in future versions of Gem::Specification. + # + # There are two variant we're checking here, one where rdoc is not present, and one where rdoc is present in the same regex: + # + # Variant #1 - rdoc not installed + # + # <b>invalidhomepagegem 1</b> + # + # + # <span title="rdoc not installed">[rdoc]</span> + # + # + # + # <a href="." title=".">[www]</a> + # + # Variant #2 - rdoc installed + # + # <b>invalidhomepagegem 1</b> + # + # + # <a href="\/doc_root\/invalidhomepagegem-1\/">\[rdoc\]<\/a> + # + # + # + # <a href="." title=".">[www]</a> + regex_match = /invalidhomepagegem 1<\/b>[\s]+(<span title="rdoc not installed">\[rdoc\]<\/span>|<a href="\/doc_root\/invalidhomepagegem-1\/">\[rdoc\]<\/a>)[\s]+<a href="\." title="\.">\[www\]<\/a>/ + assert_match regex_match, @res.body + end + + def test_valid_homepage_http + data = StringIO.new "GET / HTTP/1.0\r\n\r\n" + dir = "#{@gemhome}2" + + spec = util_spec 'validhomepagegemhttp', 1 + spec.homepage = "http://rubygems.org" + + specs_dir = File.join dir, 'specifications' + FileUtils.mkdir_p specs_dir + + open File.join(specs_dir, spec.spec_name), 'w' do |io| + io.write spec.to_ruby + end + + server = Gem::Server.new dir, process_based_port, false + + @req.parse data + + server.root @req, @res + + assert_equal 200, @res.status + assert_match 'validhomepagegemhttp 1', @res.body + + regex_match = /validhomepagegemhttp 1<\/b>[\s]+(<span title="rdoc not installed">\[rdoc\]<\/span>|<a href="\/doc_root\/validhomepagegemhttp-1\/">\[rdoc\]<\/a>)[\s]+<a href="http:\/\/rubygems\.org" title="http:\/\/rubygems\.org">\[www\]<\/a>/ + assert_match regex_match, @res.body + end + + def test_valid_homepage_https + data = StringIO.new "GET / HTTP/1.0\r\n\r\n" + dir = "#{@gemhome}2" + + spec = util_spec 'validhomepagegemhttps', 1 + spec.homepage = "https://rubygems.org" + + specs_dir = File.join dir, 'specifications' + FileUtils.mkdir_p specs_dir + + open File.join(specs_dir, spec.spec_name), 'w' do |io| + io.write spec.to_ruby + end + + server = Gem::Server.new dir, process_based_port, false + + @req.parse data + + server.root @req, @res + + assert_equal 200, @res.status + assert_match 'validhomepagegemhttps 1', @res.body + + regex_match = /validhomepagegemhttps 1<\/b>[\s]+(<span title="rdoc not installed">\[rdoc\]<\/span>|<a href="\/doc_root\/validhomepagegemhttps-1\/">\[rdoc\]<\/a>)[\s]+<a href="https:\/\/rubygems\.org" title="https:\/\/rubygems\.org">\[www\]<\/a>/ + assert_match regex_match, @res.body + end + def test_specs data = StringIO.new "GET /specs.#{Gem.marshal_version} HTTP/1.0\r\n\r\n" @req.parse data Index: ruby_2_2/test/rubygems/test_gem_package.rb =================================================================== --- ruby_2_2/test/rubygems/test_gem_package.rb (revision 62440) +++ ruby_2_2/test/rubygems/test_gem_package.rb (revision 62441) @@ -507,6 +507,21 @@ class TestGemPackage < Gem::Package::Tar https://github.com/ruby/ruby/blob/trunk/ruby_2_2/test/rubygems/test_gem_package.rb#L507 "#{@destination} is not allowed", e.message) end + def test_install_location_suffix + package = Gem::Package.new @gem + + filename = "../#{File.basename(@destination)}suffix.rb" + + e = assert_raises Gem::Package::PathError do + package.install_location filename, @destination + end + + parent = File.expand_path File.join @destination, filename + + assert_equal("installing into parent path #{parent} of " + + "#{@destination} is not allowed", e.message) + end + def test_load_spec entry = StringIO.new Gem.gzip @spec.to_yaml def entry.full_name() 'metadata.gz' end @@ -664,6 +679,32 @@ class TestGemPackage < Gem::Package::Tar https://github.com/ruby/ruby/blob/trunk/ruby_2_2/test/rubygems/test_gem_package.rb#L679 assert_match %r%nonexistent.gem$%, e.message end + def test_verify_duplicate_file + FileUtils.mkdir_p 'lib' + FileUtils.touch 'lib/code.rb' + + build = Gem::Package.new @gem + build.spec = @spec + build.setup_signer + open @gem, 'wb' do |gem_io| + Gem::Package::TarWriter.new gem_io do |gem| + build.add_metadata gem + build.add_contents gem + + gem.add_file_simple 'a.sig', 0444, 0 + gem.add_file_simple 'a.sig', 0444, 0 + end + end + + package = Gem::Package.new @gem + + e = assert_raises Gem::Security::Exception do + package.verify + end + + assert_equal 'duplicate files in the package: ("a.sig")', e.message + end + def test_verify_security_policy skip 'openssl is missing' unless defined?(OpenSSL::SSL) @@ -721,7 +762,13 @@ class TestGemPackage < Gem::Package::Tar https://github.com/ruby/ruby/blob/trunk/ruby_2_2/test/rubygems/test_gem_package.rb#L762 # write bogus data.tar.gz to foil signature bogus_data = Gem.gzip 'hello' - gem.add_file_simple 'data.tar.gz', 0444, bogus_data.length do |io| + fake_signer = Class.new do + def digest_name; 'SHA512'; end + def digest_algorithm; Digest(:SHA512); end + def key; 'key'; end + def sign(*); 'fake_sig'; end + end + gem.add_file_signed 'data2.tar.gz', 0444, fake_signer.new do |io| io.write bogus_data end Index: ruby_2_2/test/rubygems/test_gem_specification.rb =================================================================== --- ruby_2_2/test/rubygems/test_gem_specification.rb (revision 62440) +++ ruby_2_2/test/rubygems/test_gem_specification.rb (revision 62441) @@ -2584,7 +2584,22 @@ duplicate dependency on b (>= 1.2.3), (~ https://github.com/ruby/ruby/blob/trunk/ruby_2_2/test/rubygems/test_gem_specification.rb#L2584 @a1.validate end - assert_equal '"over at my cool site" is not a URI', e.message + assert_equal '"over at my cool site" is not a valid HTTP URI', e.message + + @a1.homepage = 'ftp://rubygems.org' + + e = assert_raises Gem::InvalidSpecificationException do + @a1.validate + end + + assert_equal '"ftp://rubygems.org" is not a valid HTTP URI', e.message + + @a1.homepage = 'http://rubygems.org' + assert_equal true, @a1.validate + + @a1.homepage = 'https://rubygems.org' + assert_equal true, @a1.validate + end end Index: ruby_2_2/test/rubygems/test_gem_package_tar_header.rb =================================================================== --- ruby_2_2/test/rubygems/test_gem_package_tar_header.rb (revision 62440) +++ ruby_2_2/test/rubygems/test_gem_package_tar_header.rb (revision 62441) @@ -142,5 +142,26 @@ group\000\000\000\000\000\000\000\000\00 https://github.com/ruby/ruby/blob/trunk/ruby_2_2/test/rubygems/test_gem_package_tar_header.rb#L142 assert_equal '012467', @tar_header.checksum end + def test_from_bad_octal + test_cases = [ + "00000006,44\000", # bogus character + "00000006789\000", # non-octal digit + "+0000001234\000", # positive sign + "-0000001000\000", # negative sign + "0x000123abc\000", # radix prefix + ] + + test_cases.each do |val| + header_s = @tar_header.to_s + # overwrite the size field + header_s[124, 12] = val + io = TempIO.new header_s + assert_raises ArgumentError do + new_header = Gem::Package::TarHeader.from io + end + io.close! if io.respond_to? :close! + end + end + end -- ML: ruby-changes@q... Info: http://www.atdot.net/~ko1/quickml/