title: CbCによるMoarVMの改良
author: Takahiro Shimizu
profile:
lang: Japanese


# 研究目的
- Perl5の後継言語として開発されているPerl6はMoarVMと呼ばれるVMを搭載している．
- Perl6はMoarVM,JVM,JavaScript上で動くRakudoと呼ばれる実装と，コンパイラ開発者用のサブセットであるNQPが主な実装となっている．
- 現在Perl6及びMoarVMは全体的な速度がPerl5と比較し低下しており，実務として利用できるレベルに達していない．
- さらにPerl6の実装自体巨大なcase-switch文など見通しが悪くなっている．
- この問題を解決するために現在当研究室で開発している継続を中心にしたContinuation based Cを用いて改良を行う
- CbCの設計理念からVMの実装と親和性が高い事も推測できる為，実際にCbCを用いてどのようにVMが実装できるかを検証する

# 今週の進捗
* 油断していたら一週間経ってました
* DynAsmに手がかりが無いかなと思って読んでいます

# DynAsm

* minilua経由で`dynasm.lua`などが呼ばれている模様
* `dasm_init`でbreak pointを書けたら止まった

```
(lldb) bt
* thread #2, stop reason = breakpoint 1.1
  * frame #0: 0x000000010029982f libmoar.dylib`dasm_init(compiler=0x00007000010bbc60, maxsection=2) at dasm_x86.h:93
    frame #1: 0x0000000100287d6a libmoar.dylib`MVM_jit_compiler_init(tc=0x0000000100a4b8f0, cl=0x00007000010bbc60, jg=0x000000010183ac7a) at compile.c:19
    frame #2: 0x0000000100287ff5 libmoar.dylib`MVM_jit_compile_graph(tc=0x0000000100a4b8f0, jg=0x000000010183ac7a) at compile.c:52
    frame #3: 0x00000001001b50da libmoar.dylib`MVM_spesh_candidate_add(tc=0x0000000100a4b8f0, p=0x0000000100a4c480) at candidate.c:101
    frame #4: 0x00000001001cf991 libmoar.dylib`worker(tc=0x0000000100a4b8f0, callsite=0x00000001006c9150, args=0x0000000000000000) at worker.c:16
    frame #5: 0x000000010014e8e2 libmoar.dylib`invoke_handler(tc=0x0000000100a4b8f0, invokee=0x000000010181ae40, callsite=0x00000001006c9150, args=0x0000000000000000) at MVMCFunction.c:9
    frame #6: 0x00000001000e8494 libmoar.dylib`thread_initial_invoke(tc=0x0000000100a4b8f0, data=0x0000000100a4c070) at threads.c:59
    frame #7: 0x00000001000aefee libmoar.dylib`MVM_interp_run(tc=0x0000000100a4b8f0, initial_invoke=(libmoar.dylib`thread_initial_invoke at threads.c:50), invoke_data=0x0000000100a4c070) at interp.c:93
    frame #8: 0x00000001000e7a35 libmoar.dylib`start_thread(data=0x0000000100a4c070) at threads.c:87
    frame #9: 0x00007fff7abf8661 libsystem_pthread.dylib`_pthread_body + 340
    frame #10: 0x00007fff7abf850d libsystem_pthread.dylib`_pthread_start + 377
    frame #11: 0x00007fff7abf7bf9 libsystem_pthread.dylib`thread_start + 13
```

# dasm_free

* freeしている箇所で一度止めてupしていく

```
(lldb) b dasm_free
Breakpoint 2: where = libmoar.dylib`dasm_free + 12 at dasm_x86.h:118, address = 0x0000000100299a0c
(lldb) c
Process 42568 resuming
Process 42568 stopped
* thread #2, stop reason = breakpoint 2.1
    frame #0: 0x0000000100299a0c libmoar.dylib`dasm_free(compiler=0x00007000010bbc60) at dasm_x86.h:118
   115 	void
   116 	dasm_free (Dst_DECL)
   117 	{
-> 118 	  dasm_State *D = Dst_REF;
   119 	  int i;
   120 	  for (i = 0; i < D->maxsection; i++)
   121 	    if (D->sections[i].buf)
Target 0: (moar) stopped.
(lldb) bt
* thread #2, stop reason = breakpoint 2.1
  * frame #0: 0x0000000100299a0c libmoar.dylib`dasm_free(compiler=0x00007000010bbc60) at dasm_x86.h:118
    frame #1: 0x0000000100287f39 libmoar.dylib`MVM_jit_compiler_deinit(tc=0x0000000100a4b8f0, cl=0x00007000010bbc60) at compile.c:40
    frame #2: 0x00000001002881d7 libmoar.dylib`MVM_jit_compile_graph(tc=0x0000000100a4b8f0, jg=0x000000010183ac7a) at compile.c:99
    frame #3: 0x00000001001b50da libmoar.dylib`MVM_spesh_candidate_add(tc=0x0000000100a4b8f0, p=0x0000000100a4c480) at candidate.c:101
    frame #4: 0x00000001001cf991 libmoar.dylib`worker(tc=0x0000000100a4b8f0, callsite=0x00000001006c9150, args=0x0000000000000000) at worker.c:16
    frame #5: 0x000000010014e8e2 libmoar.dylib`invoke_handler(tc=0x0000000100a4b8f0, invokee=0x000000010181ae40, callsite=0x00000001006c9150, args=0x0000000000000000) at MVMCFunction.c:9
    frame #6: 0x00000001000e8494 libmoar.dylib`thread_initial_invoke(tc=0x0000000100a4b8f0, data=0x0000000100a4c070) at threads.c:59
    frame #7: 0x00000001000aefee libmoar.dylib`MVM_interp_run(tc=0x0000000100a4b8f0, initial_invoke=(libmoar.dylib`thread_initial_invoke at threads.c:50), invoke_data=0x0000000100a4c070) at interp.c:93
    frame #8: 0x00000001000e7a35 libmoar.dylib`start_thread(data=0x0000000100a4c070) at threads.c:87
    frame #9: 0x00007fff7abf8661 libsystem_pthread.dylib`_pthread_body + 340
    frame #10: 0x00007fff7abf850d libsystem_pthread.dylib`_pthread_start + 377
    frame #11: 0x00007fff7abf7bf9 libsystem_pthread.dylib`thread_start + 13
```

# MBM_jit__compile_graph

```
(lldb) f
frame #2: 0x00000001002881d7 libmoar.dylib`MVM_jit_compile_graph(tc=0x0000000100a4b8f0, jg=0x000000010183ac7a) at compile.c:99
   96  	    code = MVM_jit_compiler_assemble(tc, &cl, jg);
   97
   98  	    /* Clear up the compiler */
-> 99  	    MVM_jit_compiler_deinit(tc, &cl);
   100
   101 	    /* Logging for insight */
   102 	    if (tc->instance->jit_bytecode_dir) {
(lldb)
```

```
(lldb) p code
(MVMJitCode *) $2 = 0x00000001047196a0
(lldb) p *code
(MVMJitCode) $3 = {
  func_ptr = 0x00000001007ff000
  size = 1723
  bytecode = 0x0000000100560cf0 "R\x03"
  sf = 0x0000000103248300
  local_types = 0x0000000104719920
  num_locals = 18
  num_labels = 7
  labels = 0x0000000102902530
  num_deopts = 0
  num_inlines = 0
  num_handlers = 0
  deopts = 0x0000000000000000
  inlines = 0x0000000000000000
  handlers = 0x0000000000000000
  spill_size = 1
  seq_nr = 0
}
```

# disassしてみる

* `MAGIC_BYTECODE`という謎のbytecodeを吐いている

```
(lldb) disass -a code->bytecode
libmoar.dylib`MAGIC_BYTECODE:
    0x100560cf0 <+0>:  pushq  %rdx
    0x100560cf1 <+1>:  addl   (%rax), %eax
    0x100560cf3 <+3>:  addb   %al, (%rax)
    0x100560cf5 <+5>:  addb   %al, (%rax)
    0x100560cf7 <+7>:  addb   %al, (%rax)
    0x100560cf9 <+9>:  addb   %al, (%rax)
    0x100560cfb <+11>: addb   %al, (%rax)
    0x100560cfd <+13>: addb   %al, (%rax)
    0x100560cff <+15>: addb   %al, (%rbx)
    0x100560d01 <+17>: addb   %al, (%rax)
    0x100560d03 <+19>: addb   %al, (%rax)
    0x100560d05 <+21>: addb   %al, (%rax)
    0x100560d07 <+23>: addb   %al, (%rax)
    0x100560d09 <+25>: addb   %al, (%rax)
    0x100560d0b <+27>: addb   %al, (%rax)
    0x100560d0d <+29>: addb   %al, (%rax)
    0x100560d0f <+31>: addb   %al, (%rax)
    0x100560d11 <+33>: addb   %al, (%rax)
    0x100560d13 <+35>: addb   %al, (%rax)
    0x100560d15 <+37>: addb   %al, (%rax)
    0x100560d17 <+39>: addb   %al, (%rax)
    0x100560d19 <+41>: addb   %al, (%rax)
    0x100560d1b <+43>: addb   %al, (%rax)
    0x100560d1d <+45>: addb   %al, (%rax)
```

# 作っていそう

```
   45  	MVMJitCode * MVM_jit_compile_graph(MVMThreadContext *tc, MVMJitGraph *jg) {
   46  	    MVMJitCompiler cl;
   47  	    MVMJitCode *code;
   48  	    MVMJitNode *node = jg->first_node;
   49
   50  	    MVM_jit_log(tc, "Starting compilation\n");
(lldb)
   51  	    /* initialation */
   52  	    MVM_jit_compiler_init(tc, &cl, jg);
   53  	    /* generate code */
   54  	    MVM_jit_emit_prologue(tc, &cl, jg);
   55  	    while (node) {
   56  	        switch(node->type) {
   57  	        case MVM_JIT_NODE_LABEL:
   58  	            MVM_jit_emit_label(tc, &cl, jg, node->u.label.name);
   59  	            break;
   60  	        case MVM_JIT_NODE_PRIMITIVE:
   61  	            MVM_jit_emit_primitive(tc, &cl, jg, &node->u.prim);
(lldb)
   62  	            break;
   63  	        case MVM_JIT_NODE_BRANCH:
   64  	            MVM_jit_emit_block_branch(tc, &cl, jg, &node->u.branch);
   65  	            break;
   66  	        case MVM_JIT_NODE_CALL_C:
   67  	            MVM_jit_emit_call_c(tc, &cl, jg, &node->u.call);
   68  	            break;
   69  	        case MVM_JIT_NODE_GUARD:
   70  	            MVM_jit_emit_guard(tc, &cl, jg, &node->u.guard);
   71  	            break;
   72  	        case MVM_JIT_NODE_INVOKE:
(lldb)
   73  	            MVM_jit_emit_invoke(tc, &cl, jg, &node->u.invoke);
   74  	            break;
   75  	        case MVM_JIT_NODE_JUMPLIST:
   76  	            MVM_jit_emit_jumplist(tc, &cl, jg, &node->u.jumplist);
   77  	            break;
   78  	        case MVM_JIT_NODE_CONTROL:
   79  	            MVM_jit_emit_control(tc, &cl, &node->u.control, NULL);
   80  	            break;
   81  	        case MVM_JIT_NODE_EXPR_TREE:
   82  	            MVM_jit_compile_expr_tree(tc, &cl, jg, node->u.tree);
   83  	            break;
(lldb)
   84  	        case MVM_JIT_NODE_DATA:
   85  	            MVM_jit_emit_data(tc, &cl, &node->u.data);
   86  	            break;
   87  	        case MVM_JIT_NODE_SAVE_RV:
   88  	            MVM_jit_emit_save_rv(tc, &cl, node->u.stack.slot);
   89  	            break;
   90  	        }
   91  	        node = node->next;
   92  	    }
   93  	    MVM_jit_emit_epilogue(tc, &cl, jg);
   94
(lldb)
   95  	    /* Generate code */
   96  	    code = MVM_jit_compiler_assemble(tc, &cl, jg);
   97
   98  	    /* Clear up the compiler */
   99  	    MVM_jit_compiler_deinit(tc, &cl);
   100
   101 	    /* Logging for insight */
   102 	    if (tc->instance->jit_bytecode_dir) {
   103 	        MVM_jit_log_bytecode(tc, code);
   104 	    }
   105 	    if (tc->instance->jit_log_fh)
(lldb)
   106 	        fflush(tc->instance->jit_log_fh);
   107 	    return code;
   108 	}
   109
```

# 実際にアセンブルしている箇所

```
110 MVMJitCode * MVM_jit_compiler_assemble(MVMThreadContext *tc, MVMJitCompiler *cl, MVMJitGraph *jg) {
111     MVMJitCode * code;
112     MVMint32 i;
113     char * memory;
114     size_t codesize;
115
116     MVMint32 dasm_error = 0;
117
118    /* compile the function */
119     if ((dasm_error = dasm_link(cl, &codesize)) != 0) {
120         MVM_jit_log(tc, "DynASM could not link, error: %d\n", dasm_error);
121         return NULL;
122     }
123
124     memory = MVM_platform_alloc_pages(codesize, MVM_PAGE_READ|MVM_PAGE_WRITE);
125     if ((dasm_error = dasm_encode(cl, memory)) != 0) {
126         MVM_jit_log(tc, "DynASM could not encode, error: %d\n", dasm_error);
127         return NULL;
128     }
129
130     /* set memory readable + executable */
131     if (!MVM_platform_set_page_mode(memory, codesize, MVM_PAGE_READ|MVM_PAGE_EXEC)) {
132         MVM_jit_log(tc, "Setting jit page executable failed or was denied. deactivating jit.\n");
133         /* our caller allocated the compiler and our caller must clean it up */
134         tc->instance->jit_enabled = 0;
135         return NULL;
136     }
137
138     MVM_jit_log(tc, "Bytecode size: %"MVM_PRSz"\n", codesize);
139
140     /* Create code segment */
141     code = MVM_malloc(sizeof(MVMJitCode));
142     code->func_ptr   = (void (*)(MVMThreadContext*,MVMCompUnit*,void*)) memory;
143     code->size       = codesize;
144     code->bytecode   = (MVMuint8*)MAGIC_BYTECODE;
145     code->sf         = jg->sg->sf;
146     code->spill_size = cl->spills_num;
147     if (cl->spills_num > 0) {
148         MVMint32 sg_num_locals = jg->sg->num_locals;
149         code->num_locals  = sg_num_locals + cl->spills_num;
150         code->local_types = MVM_malloc(code->num_locals * sizeof(MVMuint16));
151         if (jg->sg->local_types != NULL) {
152             memcpy(code->local_types, jg->sg->local_types, sizeof(MVMuint16)*sg_num_locals);
153         } else {
154             memcpy(code->local_types, code->sf->body.local_types, sizeof(MVMuint16)*sg_num_locals);
155         }
156         for (i = 0; i < cl->spills_num; i++) {
157             code->local_types[sg_num_locals + i] = cl->spills[i].reg_type;
158         }
159     } else {
160         code->local_types = NULL;
161         code->num_locals  = 0;
162     }
```


# メモリのプロテクトを解除

* `memory = MVM_platform_alloc_pages(codesize, MVM_PAGE_READ|MVM_PAGE_WRITE);`で呼び出している
* `platform/posix/mmap.c`で主に定義されている

```
 42 void *MVM_platform_alloc_pages(size_t size, int page_mode)
 43 {
 44     int prot_mode = page_mode_to_prot_mode(page_mode);
 45     void *block = mmap(NULL, size, prot_mode, MVM_MAP_ANON | MAP_PRIVATE, -1, 0);
 46
 47     if (block == MAP_FAILED)
 48         MVM_panic(1, "MVM_platform_alloc_pages failed: %d", errno);
 49
 50     return block;
 51 }
```

* mmapを変数をラップする用に使用している

```
 21 static int page_mode_to_prot_mode(int page_mode) {
 22     switch (page_mode) {
 23     case MVM_PAGE_READ:
 24         return PROT_READ;
 25     case MVM_PAGE_WRITE:
 26         return PROT_WRITE;
 27     case (MVM_PAGE_READ|MVM_PAGE_WRITE):
 28         return PROT_READ|PROT_WRITE;
 29     case MVM_PAGE_EXEC:
 30         return PROT_EXEC;
 31     case (MVM_PAGE_READ|MVM_PAGE_EXEC):
 32         return PROT_READ|PROT_EXEC;
 33     case (MVM_PAGE_WRITE|MVM_PAGE_EXEC):
 34         return PROT_WRITE|PROT_EXEC;
 35     case (MVM_PAGE_READ|MVM_PAGE_WRITE|MVM_PAGE_EXEC):
 36         return PROT_READ|PROT_WRITE|PROT_EXEC;
 37     default:
 38         return PROT_NONE;
 39     }
 40 }
```


# JITに突入する箇所

* ` MVM_jit_enter_code`が怪しいが複数宣言されている
*  `src/jit/compile.c`

```
324 /* Enter the JIT code segment. The label is a continuation point where control
325  * is resumed after the frame is properly setup. */
326 void MVM_jit_enter_code(MVMThreadContext *tc, MVMCompUnit *cu,
327                         MVMJitCode *code) {
328     void *label = tc->cur_frame->jit_entry_label;
329     MVMint32 ofs = (char*)label - (char*)code->func_ptr;
330     if (ofs < 0 || ofs >= code->size)
331         MVM_oops(tc, "JIT entry label out of range for code!\n"
332                  "(label %p, func_ptr %p, code size %lui, offset %li, frame_nr %i, seq nr %i)",
333                  label, code->func_ptr, code->size, ((char*)label) - ((char*)code->func_ptr),
334                  tc->cur_frame->sequence_nr, code->seq_nr);
335     code->func_ptr(tc, cu, label);
336 }
```

# コード実行箇所

* `code->func_ptr`でgrepした
* src/core/nativecall.cがそれっぽい

```
1149 void MVM_nativecall_invoke_jit(MVMThreadContext *tc, MVMObject *site) {
1150     MVMNativeCallBody *body = MVM_nativecall_get_nc_body(tc, site);
1151     MVMJitCode * const jitcode = body->jitcode;
1152
1153     jitcode->func_ptr(tc, *tc->interp_cu, jitcode->labels[0]);
1154 }
```

* ただしサンプルコードでは止まらない
    * JITされてない?
* 呼ばれている箇所を探索
    * jit/graph.c
    * op_to_funcで呼ばれているが巨大なcase文
    * opcodeによって判定している
    * これ自体は`consume_reprop`で呼ばれているが謎
* 多分`MVM_interp_run`をちゃんと読む必要がありそう