ruby-changes:50519
From: k0kubun <ko1@a...>
Date: Sun, 4 Mar 2018 16:04:45 +0900 (JST)
Subject: [ruby-changes:50519] k0kubun:r62655 (trunk): mjit_compile.c: use local variables for stack
k0kubun 2018-03-04 16:04:40 +0900 (Sun, 04 Mar 2018) New Revision: 62655 https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=62655 Log: mjit_compile.c: use local variables for stack if catch_except_p is FALSE. If catch_except_p is TRUE, stack values should be on VM's stack when exception is thrown and the JIT-ed frame is re-executed by VM's exception handler. If it's FALSE, the JIT-ed frame won't be re-executed and don't need to keep values on VM's stack. Using local variables allows us to reduce cfp->sp motion. Moving cfp->sp is needed only for insns whose handles_frame? is false. So it improves performance. _mjit_compile_insn.erb: Prepare `stack_size` variable for GET_SP, STACK_ADDR_FROM_TOP, TOPN macros. Share pc and sp motion partial view. Use cancel handler created in mjit_compile.c. _mjit_compile_send.erb: ditto. Also, when iseq->body->catch_except_p is TRUE, this stops to call mjit_exec directly. I described the reason in vm_insnhelper.h's comment for EXEC_EC_CFP. _mjit_compile_pc_and_sp.erb: Shared logic for moving sp and pc. As you can see from thsi file, when status->local_stack_p is TRUE and insn.handles_frame? is false, moving sp is skipped. But if insn.handles_frame? is true, values should be rolled back to VM's stack. common.mk: add dependency for the file _mjit_compile_insn_body.erb: Set sp value before canceling JIT on DISPATCH_ORIGINAL_INSN. Replace GET_SP, STACK_ADDR_FROM_TOP, TOPN macros for the case ocal_stack_p is TRUE and insn.handles_frame? is false. In that case, values are not available on VM's stack and those macros should be replaced. mjit_compile.inc.erb: updated comments of macros which are supported by JIT compiler. All references to `cfp->sp` should be replaced and thus INC_SP, SET_SV, PUSH are no longer supported for now, because they are not used now. vm_exec.h: moved EXEC_EC_CFP definition to vm_insnhelper.h because it's tighly coupled to CALL_METHOD. vm_insnhelper.h: Have revised EXEC_EC_CFP definition moved from vm_exec.h. Now it triggers mjit_exec for VM, and has the guard for catch_except_p on JIT-ed code. See comments for details. CALL_METHOD delegates triggering mjit_exec to EXEC_EC_CFP. insns.def: Stopped using EXEC_EC_CFP for the case we don't want to trigger mjit_exec. Those insns (defineclass, opt_call_c_function) are not supported by JIT and it's safe to use RESTORE_REGS(), NEXT_INSN(). expandarray is changed to pass GET_SP() to replace the macro in _mjit_compile_insn_body.erb. vm_insnhelper.c: change to take sp for the above reason. [close https://github.com/ruby/ruby/pull/1828] This patch resurrects the performance which was attached in [Feature #14235]. * Benchmark Optcarrot (with configuration for benchmark_driver.gem) https://github.com/benchmark-driver/optcarrot $ benchmark-driver benchmark.yml --verbose 1 --rbenv 'before;before+JIT::before,--jit;after;after+JIT::after,--jit' --repeat-count 10 before: ruby 2.6.0dev (2018-03-04 trunk 62652) [x86_64-linux] before+JIT: ruby 2.6.0dev (2018-03-04 trunk 62652) +JIT [x86_64-linux] after: ruby 2.6.0dev (2018-03-04 local-variable.. 62652) [x86_64-linux] last_commit=mjit_compile.c: use local variables for stack after+JIT: ruby 2.6.0dev (2018-03-04 local-variable.. 62652) +JIT [x86_64-linux] last_commit=mjit_compile.c: use local variables for stack Calculating ------------------------------------- before before+JIT after after+JIT optcarrot 53.552 59.680 53.697 63.358 fps Comparison: optcarrot after+JIT: 63.4 fps before+JIT: 59.7 fps - 1.06x slower after: 53.7 fps - 1.18x slower before: 53.6 fps - 1.18x slower Added files: trunk/tool/ruby_vm/views/_mjit_compile_pc_and_sp.erb Modified files: trunk/common.mk trunk/insns.def trunk/mjit_compile.c trunk/test/ruby/test_jit.rb trunk/tool/ruby_vm/views/_mjit_compile_insn.erb trunk/tool/ruby_vm/views/_mjit_compile_insn_body.erb trunk/tool/ruby_vm/views/_mjit_compile_send.erb trunk/tool/ruby_vm/views/mjit_compile.inc.erb trunk/vm_exec.h trunk/vm_insnhelper.c trunk/vm_insnhelper.h Index: tool/ruby_vm/views/_mjit_compile_insn.erb =================================================================== --- tool/ruby_vm/views/_mjit_compile_insn.erb (revision 62654) +++ tool/ruby_vm/views/_mjit_compile_insn.erb (revision 62655) @@ -20,6 +20,11 @@ https://github.com/ruby/ruby/blob/trunk/tool/ruby_vm/views/_mjit_compile_insn.erb#L20 MAYBE_UNUSED(<%= ope.fetch(:decl) %>) = (<%= ope.fetch(:type) %>)operands[<%= i %>]; % end % +% # JIT: Declare stack_size to be used in some macro of _mjit_compile_insn_body.erb + if (status->local_stack_p) { + fprintf(f, " MAYBE_UNUSED(unsigned int) stack_size = %u;\n", b->stack_size); + } +% % # JIT: Declare variables for operands, popped values and return values % ret_decls = insn.rets.map { |r| "MAYBE_UNUSED(#{r.fetch(:type)}) #{r.fetch(:name)}"} # TODO: fix #declarations to return Hash... % insn.declarations.each do |decl| @@ -52,13 +57,7 @@ https://github.com/ruby/ruby/blob/trunk/tool/ruby_vm/views/_mjit_compile_insn.erb#L57 % end % % # JIT: move sp and pc if necessary -% if insn.handles_frame? - fprintf(f, " reg_cfp->pc = original_body_iseq + %d;\n", next_pos); /* ADD_PC(INSN_ATTR(width)); */ - fprintf(f, " reg_cfp->sp = (VALUE *)reg_cfp->bp + %d;\n", b->stack_size + 1 - <%= insn.pops.size %>); /* POPN(INSN_ATTR(popn)); */ -% else - fprintf(f, " reg_cfp->pc = original_body_iseq + %d;\n", pos); - fprintf(f, " reg_cfp->sp = (VALUE *)reg_cfp->bp + %d;\n", b->stack_size + 1); -% end +<%= render 'mjit_compile_pc_and_sp', locals: { insn: insn } -%> % % # JIT: Print insn body in insns.def <%= render 'mjit_compile_insn_body', locals: { insn: insn } -%> @@ -75,7 +74,7 @@ https://github.com/ruby/ruby/blob/trunk/tool/ruby_vm/views/_mjit_compile_insn.erb#L74 % if trace_enablable_insns.include?(insn.name) fprintf(f, " if (UNLIKELY(ruby_vm_event_enabled_flags & ISEQ_TRACE_EVENTS)) {\n"); fprintf(f, " reg_cfp->sp = (VALUE *)reg_cfp->bp + %d;\n", b->stack_size + (int)<%= insn.call_attribute('sp_inc') %> + 1); - fprintf(f, " return Qundef; /* cancel JIT */\n"); + fprintf(f, " goto cancel;\n"); fprintf(f, " }\n"); % end % Index: tool/ruby_vm/views/_mjit_compile_insn_body.erb =================================================================== --- tool/ruby_vm/views/_mjit_compile_insn_body.erb (revision 62654) +++ tool/ruby_vm/views/_mjit_compile_insn_body.erb (revision 62655) @@ -17,9 +17,6 @@ https://github.com/ruby/ruby/blob/trunk/tool/ruby_vm/views/_mjit_compile_insn_body.erb#L17 % # % expand_simple_macros = lambda do |arg_expr| % arg_expr.dup.tap do |expr| -% # For `opt_xxx`'s fallbacks. -% expr.gsub!(/\bDISPATCH_ORIGINAL_INSN\([^)]+\);/, 'return Qundef; /* cancel JIT */') -% % # For `leave`. We can't proceed next ISeq in the same JIT function. % expr.gsub!(/^(?<indent>\s*)RESTORE_REGS\(\);\n/) do % indent = Regexp.last_match[:indent] @@ -42,7 +39,9 @@ https://github.com/ruby/ruby/blob/trunk/tool/ruby_vm/views/_mjit_compile_insn_body.erb#L39 % # % # Expand dynamic macro here (only JUMP for now) % # -% if line =~ /\A\s+JUMP\((?<dest>[^)]+)\);\s+\z/ +% # TODO: support combination of following macros in the same line +% case line +% when /\A\s+JUMP\((?<dest>[^)]+)\);\s+\z/ % dest = Regexp.last_match[:dest] % % if insn.name == 'opt_case_dispatch' # special case... TODO: use another macro to avoid checking name @@ -68,7 +67,38 @@ https://github.com/ruby/ruby/blob/trunk/tool/ruby_vm/views/_mjit_compile_insn_body.erb#L67 next_pos = pos + insn_len(insn) + (unsigned int)<%= dest %>; fprintf(f, " goto label_%d;\n", next_pos); % end +% when /\A\s+DISPATCH_ORIGINAL_INSN\([^)]+\);\s+\z/ +% # For `opt_xxx`'s fallbacks. + if (status->local_stack_p) { + fprintf(f, " reg_cfp->sp = (VALUE *)reg_cfp->bp + %d;\n", b->stack_size + 1); + } + fprintf(f, " goto cancel;\n"); % else +% if insn.handles_frame? +% # If insn.handles_frame? is true, cfp->sp might be changed inside insns (like vm_caller_setup_arg_block) +% # and thus we need to use cfp->sp, even when local_stack_p is TRUE. When insn.handles_frame? is true, +% # cfp->sp should be available too because _mjit_compile_pc_and_sp.erb sets it. fprintf(f, <%= to_cstr.call(line) %>); +% else +% # If local_stack_p is TRUE and insn.handles_frame? is false, stack values are only available in local variables +% # for stack. So we need to replace those macros if local_stack_p is TRUE here. +% case line +% when /\bGET_SP\(\)/ +% # reg_cfp->sp + fprintf(f, <%= to_cstr.call(line.sub(/\bGET_SP\(\)/, '%s')) %>, (status->local_stack_p ? "(stack + stack_size)" : "GET_SP()")); +% when /\bSTACK_ADDR_FROM_TOP\((?<num>[^)]+)\)/ +% # #define STACK_ADDR_FROM_TOP(n) (GET_SP()-(n)) +% num = Regexp.last_match[:num] + fprintf(f, <%= to_cstr.call(line.sub(/\bSTACK_ADDR_FROM_TOP\(([^)]+)\)/, '%s')) %>, + (status->local_stack_p ? "stack + (stack_size - (<%= num %>))" : "STACK_ADDR_FROM_TOP(<%= num %>)")); +% when /\bTOPN\((?<num>[^)]+)\)/ +% # #define TOPN(n) (*(GET_SP()-(n)-1)) +% num = Regexp.last_match[:num] + fprintf(f, <%= to_cstr.call(line.sub(/\bTOPN\(([^)]+)\)/, '%s')) %>, + (status->local_stack_p ? "*(stack + (stack_size - (<%= num %>) - 1))" : "TOPN(<%= num %>)")); +% else + fprintf(f, <%= to_cstr.call(line) %>); +% end +% end % end % end Index: tool/ruby_vm/views/_mjit_compile_pc_and_sp.erb =================================================================== --- tool/ruby_vm/views/_mjit_compile_pc_and_sp.erb (nonexistent) +++ tool/ruby_vm/views/_mjit_compile_pc_and_sp.erb (revision 62655) @@ -0,0 +1,38 @@ https://github.com/ruby/ruby/blob/trunk/tool/ruby_vm/views/_mjit_compile_pc_and_sp.erb#L1 +% # Copyright (c) 2018 Takashi Kokubun. All rights reserved. +% # +% # This file is a part of the programming language Ruby. Permission is hereby +% # granted, to either redistribute and/or modify this file, provided that the +% # conditions mentioned in the file COPYING are met. Consult the file for +% # details. +% +% # JIT: move pc so that catch table lookup condition is met +% if insn.handles_frame? + fprintf(f, " reg_cfp->pc = original_body_iseq + %d;\n", next_pos); /* ADD_PC(INSN_ATTR(width)); */ +% else + fprintf(f, " reg_cfp->pc = original_body_iseq + %d;\n", pos); +% end +% +% # JIT: move sp to use or preserve stack variables + if (status->local_stack_p) { +% # sp motion is optimized away for `handles_frame? #=> false` case. +% # Thus sp should be set properly before `goto cancel`. +% if insn.handles_frame? + fprintf(f, " reg_cfp->sp = (VALUE *)reg_cfp->bp + stack_size + 1 - <%= insn.pops.size %>;\n"); /* POPN(INSN_ATTR(popn)); */ +% +% # JIT-only behavior (pushing JIT's local variables to VM's stack): + { + rb_snum_t i, push_size; + push_size = -<%= insn.call_attribute('sp_inc') %> + <%= insn.rets.size %> - <%= insn.pops.size %>; + for (i = 0; i < push_size; i++) { /* TODO: use memcpy? */ + fprintf(f, " *(reg_cfp->sp + %ld) = stack[%ld];\n", i - push_size, (rb_snum_t)b->stack_size - push_size + i); + } + } +% end + } + else { +% if insn.handles_frame? + fprintf(f, " reg_cfp->sp = (VALUE *)reg_cfp->bp + %d;\n", b->stack_size + 1 - <%= insn.pops.size %>); /* POPN(INSN_ATTR(popn)); */ +% else + fprintf(f, " reg_cfp->sp = (VALUE *)reg_cfp->bp + %d;\n", b->stack_size + 1); +% end + } Index: tool/ruby_vm/views/mjit_compile.inc.erb =================================================================== --- tool/ruby_vm/views/mjit_compile.inc.erb (revision 62654) +++ tool/ruby_vm/views/mjit_compile.inc.erb (revision 62655) @@ -31,16 +31,13 @@ https://github.com/ruby/ruby/blob/trunk/tool/ruby_vm/views/mjit_compile.inc.erb#L31 % # reg_cfp: the second argument of _mjitXXX % # GET_CFP(): refers to `reg_cfp` % # GET_EP(): refers to `reg_cfp->ep` -% # GET_SP(): refers to `reg_cfp->sp` -% # INC_SP(): refers to `reg_cfp->sp` -% # SET_SV(): refers to `reg_cfp->sp` -% # PUSH(): refers to `SET_SV()`, `INC_SP()` +% # GET_SP(): refers to `reg_cfp->sp`, or `(stack + stack_size)` if local_stack_p % # GET_SELF(): refers to `reg_cfp->self` % # GET_LEP(): refers to `VM_EP_LEP(reg_cfp->ep)` % # EXEC_EC_CFP(): refers to `val = vm_exec(ec, TRUE)` with frame setup % # CALL_METHOD(): using `GET_CFP()` and `EXEC_EC_CFP()` -% # TOPN(): refers to `reg_cfp->sp`, which needs to have correct sp (of course) -% # STACK_ADDR_FROM_TOP(): refers to `reg_cfp->sp`, same problem here +% # TOPN(): refers to `reg_cfp->sp`, or `*(stack + (stack_size - num - 1))` if local_stack_p +% # STACK_ADDR_FROM_TOP(): refers to `reg_cfp->sp`, or `stack + (stack_size - num)` if local_stack_p % # DISPATCH_ORIGINAL_INSN(): expanded in _mjit_compile_insn.erb % # THROW_EXCEPTION(): specially defined for JIT % # RESTORE_REGS(): specially defined for `leave` Index: tool/ruby_vm/views/_mjit_compile_send.erb =================================================================== --- tool/ruby_vm/views/_mjit_compile_send.erb (revision 62654) +++ tool/ruby_vm/views/_mjit_compile_send.erb (revision 62655) @@ -22,15 +22,20 @@ https://github.com/ruby/ruby/blob/trunk/tool/ruby_vm/views/_mjit_compile_send.erb#L22 if (inlinable_iseq_p(ci, cc, iseq = get_iseq_if_available(cc))) { int param_size = iseq->body->param.size; /* TODO: check calling->argc for argument_arity_error */ + fprintf(f, "{\n"); +% # JIT: Declare stack_size to be used in some macro of _mjit_compile_insn_body.erb + if (status->local_stack_p) { + fprintf(f, " MAYBE_UNUSED(unsigned int) stack_size = %u;\n", b->stack_size); + } + % # JIT: move sp and pc if necessary - fprintf(f, " reg_cfp->pc = original_body_iseq + %d;\n", next_pos); /* ADD_PC(INSN_ATTR(width)); */ - fprintf(f, " reg_cfp->sp = (VALUE *)reg_cfp->bp + %d;\n", b->stack_size + 1 - <%= insn.pops.size %>); /* POPN(INSN_ATTR(popn)); */ +<%= render 'mjit_compile_pc_and_sp', locals: { insn: insn } -%> % # JIT: Invalidate call cache if it requires vm_search_method. This allows to inline some of following things. fprintf(f, " if (UNLIKELY(GET_GLOBAL_METHOD_STATE() != %"PRI_SERIALT_PREFIX"u ||\n", cc->method_state); fprintf(f, " RCLASS_SERIAL(CLASS_OF(stack[%d])) != %"PRI_SERIALT_PREFIX"u)) {\n", b->stack_size - 1 - argc, cc->class_serial); fprintf(f, " reg_cfp->pc = original_body_iseq + %d;\n", pos); - fprintf(f, " return Qundef; /* cancel JIT */\n"); + fprintf(f, " goto cancel;\n"); fprintf(f, " }\n"); % # JIT: Print insn body in insns.def @@ -52,10 +57,16 @@ https://github.com/ruby/ruby/blob/trunk/tool/ruby_vm/views/_mjit_compile_send.erb#L57 fprintf(f, " vm_push_frame(ec, (const rb_iseq_t *)0x%"PRIxVALUE", VM_FRAME_MAGIC_METHOD | VM_ENV_FLAG_LOCAL, calling.recv, " "calling.block_handler, 0x%"PRIxVALUE", (const VALUE *)0x%"PRIxVALUE", argv + %d, %d, %d);\n", (VALUE)iseq, (VALUE)cc->me, (VALUE)iseq->body->iseq_encoded, param_size, iseq->body->local_table_size - param_size, iseq->body->stack_max); - fprintf(f, " if ((v = mjit_exec(ec)) == Qundef) {\n"); - fprintf(f, " VM_ENV_FLAGS_SET(ec->cfp->ep, VM_FRAME_FLAG_FINISH);\n"); /* This is vm_call0_body's code after vm_call_iseq_setup */ - fprintf(f, " v = vm_exec(ec, FALSE);\n"); - fprintf(f, " }\n"); + if (iseq->body->catch_except_p) { + fprintf(f, " VM_ENV_FLAGS_SET(ec->cfp->ep, VM_FRAME_FLAG_FINISH);\n"); + fprintf(f, " v = vm_exec(ec, TRUE);\n"); + } + else { + fprintf(f, " if ((v = mjit_exec(ec)) == Qundef) {\n"); + fprintf(f, " VM_ENV_FLAGS_SET(ec->cfp->ep, VM_FRAME_FLAG_FINISH);\n"); /* This is vm_call0_body's code after vm_call_iseq_setup */ + fprintf(f, " v = vm_exec(ec, FALSE);\n"); + fprintf(f, " }\n"); + } fprintf(f, " stack[%d] = v;\n", b->stack_size - argc - 1); fprintf(f, " }\n"); @@ -64,12 +75,13 @@ https://github.com/ruby/ruby/blob/trunk/tool/ruby_vm/views/_mjit_compile_send.erb#L75 % # JIT: We should evaluate ISeq modified for TracePoint if it's enabled. Note: This is slow. fprintf(f, " if (UNLIKELY(ruby_vm_event_enabled_flags & ISEQ_TRACE_EVENTS)) {\n"); fprintf(f, " reg_cfp->sp = (VALUE *)reg_cfp->bp + %d;\n", b->stack_size + (int)<%= insn.call_attribute('sp_inc') %> + 1); - fprintf(f, " return Qundef; /* cancel JIT */\n"); + fprintf(f, " goto cancel;\n"); fprintf(f, " }\n"); % # compiler: Move JIT compiler's internal stack pointer b->stack_size += <%= insn.call_attribute('sp_inc') %>; + fprintf(f, "}\n"); break; } } Index: test/ruby/test_jit.rb =================================================================== --- test/ruby/test_jit.rb (revision 62654) +++ test/ruby/test_jit.rb (revision 62655) @@ -131,7 +131,15 @@ class TestJIT < Test::Unit::TestCase https://github.com/ruby/ruby/blob/trunk/test/ruby/test_jit.rb#L131 assert_compile_once('/#{true}/ =~ "true"', result_inspect: '0') end - def test_compile_insn_intern_newarray_duparray + def test_compile_insn_newarray + assert_compile_once("#{<<~"begin;"}\n#{<<~"end;"}", result_inspect: '[1, 2, 3]') + begin; + a, b, c = 1, 2, 3 + [a, b, c] + end; + end + + def test_compile_insn_intern_duparray assert_compile_once('[:"#{0}"] + [1,2,3]', result_inspect: '[:"0", 1, 2, 3]') end @@ -462,6 +470,46 @@ class TestJIT < Test::Unit::TestCase https://github.com/ruby/ruby/blob/trunk/test/ruby/test_jit.rb#L470 assert_match(/^Successful MJIT finish$/, err) end + def test_local_stack_on_exception + assert_eval_with_jit("#{<<~"begin;"}\n#{<<~"end;"}", stdout: '3', success_count: 2) + begin; + def b + raise + rescue + 2 + end + + def a + # Calling #b should be vm_exec, not direct mjit_exec. + # Otherwise `1` on local variable would be purged. + 1 + b + end + + print a + end; + end + + def test_local_stack_with_sp_motion_by_blockargs + assert_eval_with_jit("#{<<~"begin;"}\n#{<<~"end;"}", stdout: '1', success_count: 2) + begin; + def b(base) + 1 + end + + # This method is simple enough to have false in catch_except_p. + # So local_stack_p would be true in JIT compiler. + def a + m = method(:b) + + # ci->flag has VM_CALL_ARGS_BLOCKARG and cfp->sp is moved in vm_caller_setup_arg_block. + # So, for this send insn, JIT-ed code should use cfp->sp instead of local variables for stack. + Module.module_eval(&m) + end + + print a + end; + end + private # The shortest way to test one proc Index: vm_insnhelper.c =================================================================== --- vm_insnhelper.c (revision 62654) +++ vm_insnhelper.c (revision 62655) @@ -1232,11 +1232,11 @@ vm_throw(const rb_execution_context_t *e https://github.com/ruby/ruby/blob/trunk/vm_insnhelper.c#L1232 } static inline void -vm_expandarray(rb_control_frame_t *cfp, VALUE ary, rb_num_t num, int flag) +vm_expandarray(VALUE *sp, VALUE ary, rb_num_t num, int flag) { int is_splat = flag & 0x01; rb_num_t space_size = num + is_splat; - VALUE *base = cfp->sp - 1; + VALUE *base = sp - 1; const VALUE *ptr; rb_num_t len; const VALUE obj = ary; Index: vm_insnhelper.h =================================================================== --- vm_insnhelper.h (revision 62654) +++ vm_insnhelper.h (revision 62655) @@ -127,10 +127,36 @@ enum vm_regan_acttype { https://github.com/ruby/ruby/blob/trunk/vm_insnhelper.h#L127 /* deal with control flow 2: method/iterator */ /**********************************************************/ +#ifdef MJIT_HEADER +/* When calling ISeq which may catch an exception from JIT-ed code, we should not call + mjit_exec directly to prevent the caller frame from being canceled. That's because + the caller frame may have stack values in the local variables and the cancelling + the caller frame will purge them. But directly calling mjit_exec is faster... */ +#define EXEC_EC_CFP(val) do { \ + if (ec->cfp->iseq->body->catch_except_p) { \ + VM_ENV_FLAGS_SET(ec->cfp->ep, VM_FRAME_FLAG_FINISH); \ + val = vm_exec(ec, TRUE); \ + } \ + else if ((val = mjit_exec(ec)) == Qundef) { \ + VM_ENV_FLAGS_SET(ec->cfp->ep, VM_FRAME_FLAG_FINISH); \ + val = vm_exec(ec, FALSE); \ + } \ +} while (0) +#else +/* When calling from VM, longjmp in the callee won't purge any JIT-ed caller frames. + So it's safe to directly call mjit_exec. */ +#define EXEC_EC_CFP(val) do { \ + if ((val = mjit_exec(ec)) == Qundef) { \ + RESTORE_REGS(); \ + NEXT_INSN(); \ + } \ +} while (0) +#endif + #define CALL_METHOD(calling, ci, cc) do { \ VALUE v = (*(cc)->call)(ec, GET_CFP(), (calling), (ci), (cc)); \ - if (v == Qundef && (v = mjit_exec(ec)) == Qundef) { \ - EXEC_EC_CFP(FALSE); \ + if (v == Qundef) { \ + EXEC_EC_CFP(val); \ } \ else { \ val = v; \ Index: insns.def =================================================================== --- insns.def (revision 62654) +++ insns.def (revision 62655) @@ -453,7 +453,7 @@ expandarray https://github.com/ruby/ruby/blob/trunk/insns.def#L453 (...) // attr rb_snum_t sp_inc = num - 1 + (flag & 1 ? 1 : 0); { - vm_expandarray(GET_CFP(), ary, num, (int)flag); + vm_expandarray(GET_SP(), ary, num, (int)flag); } /* concat two arrays */ @@ -693,7 +693,8 @@ defineclass https://github.com/ruby/ruby/blob/trunk/insns.def#L693 class_iseq->body->iseq_encoded, GET_SP(), class_iseq->body->local_table_size, class_iseq->body->stack_max); - EXEC_EC_CFP(TRUE); + RESTORE_REGS(); + NEXT_INSN(); } /**********************************************************/ @@ -819,8 +820,8 @@ invokeblock https://github.com/ruby/ruby/blob/trunk/insns.def#L820 } val = vm_invoke_block(ec, GET_CFP(), &calling, ci, block_handler); - if (val == Qundef && (val = mjit_exec(ec)) == Qundef) { - EXEC_EC_CFP(FALSE); + if (val == Qundef) { + EXEC_EC_CFP(val); } } @@ -1344,7 +1345,8 @@ opt_call_c_function https://github.com/ruby/ruby/blob/trunk/insns.def#L1345 THROW_EXCEPTION(err); } - EXEC_EC_CFP(TRUE); + RESTORE_REGS(); + NEXT_INSN(); } /* BLT */ Index: common.mk =================================================================== --- common.mk (revision 62654) +++ common.mk (revision 62655) @@ -924,7 +924,7 @@ $(srcs_vpath)vmtc.inc: $(srcdir)/tool/ru https://github.com/ruby/ruby/blob/trunk/common.mk#L924 $(srcs_vpath)vm.inc: $(srcdir)/tool/ruby_vm/views/vm.inc.erb $(srcs_vpath)mjit_compile.inc: $(srcdir)/tool/ruby_vm/views/mjit_compile.inc.erb \ $(srcdir)/tool/ruby_vm/views/_mjit_compile_insn.erb $(srcdir)/tool/ruby_vm/views/_mjit_compile_send.erb \ - $(srcdir)/tool/ruby_vm/views/_mjit_compile_insn_body.erb + $(srcdir)/tool/ruby_vm/views/_mjit (... truncated) -- ML: ruby-changes@q... Info: http://www.atdot.net/~ko1/quickml/