Mercurial > hg > Members > masakoha > masa

comparison 14/Cerium_IO/IO_result.txt @ 94:db476b9fadd9

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
result
author Masataka Kohagura <kohagura@cr.ie.u-ryukyu.ac.jp>
date Thu, 08 May 2014 19:44:53 +0900
parents 4db3a0797aa7
children 735feeaf7279
comparison
equal deleted inserted replaced
93:65b817490735 94:db476b9fadd9
1 --------------------------------------------------------------
2 2014/05/08 (Thu)
3 [result]
4 firefly にて
5
6 firefly を再起動してリフレッシュした後の計測
7 実験 CPU 数は全て 12
8
9
10 bread size = division_size * block_num
11 block num は 48 固定
12 division size を変化させてみた
13
14 division_size ->| 4k | 8k | 16k | 32k | 64k | 128k | 256k |
15 --------------------------------------------------------------------------------
16 mmap | 11.867 | 10.570 | 11.803 | 14.915 | 16.626 | 16.923 | 18.474 |
17 read | 12.020 | 11.585 | 11.729 | 11.661 | 12.497 | 11.347 | 11.658 |
18 bread & SPE_ANY | 11.508 | 15.932 | 11.407 | 12.816 | 12.454 | 12.891 | 11.962 |
19 bread & IO_0 | 11.342 | 12.242 | 11.636 | 12.331 | 10.870 | 11.295 | 11.723 |
20 --------------------------------------------------------------------------------
21
22 [考察]
23 mmap の遅くなる境目は 16k*48 = 768k と 32k * 48 = 1,536k の間
24 2^9 < bread size < 2^11
25
26 read, bread & IO_0 は安定(まぁ一気に読み込んでるわけだし。)
27 mmap は、一度に読み込む範囲が狭いほうが速度は出る。
28 -> ランダムアクセス
29
30 ###########################################
31 ./word_count -file 1gb.txt -cpu 12 -division 4 -block 48 -m
32
33 division size を 4k に変更
34 Blocked Read size 4k * 48
35
36 [mmap]
37 13.955924 10.593330 10.076561 13.343254 11.368239
38 ave : 11.8674616
39
40 [read]
41 10.874073 11.181728 12.574511 12.639683 12.830423
42 ave : 12.0200836
43
44 [blocked Read & SPE_ANY]
45 10.139217 10.343027 10.868068 15.226670 10.960942
46 ave : 11.5075848
47
48 [blocked Read & IO_0]
49 10.071492 12.267685 11.409233 10.417327 12.542598
50 ave : 11.341667
51
52 ###########################################
53 ./word_count -file 1gb.txt -cpu 12 -division 8 -block 48 -m
54
55 division size を 8k に変更
56 Blocked Read size 8k * 48
57
58 [mmap]
59 10.019870 10.182644 10.865943 11.464948 10.316064
60 ave : 10.5698938
61
62 [read]
63 10.929073 10.494944 13.320447 11.696959 11.485670
64 ave : 11.5854186
65
66 [blocked Read & SPE_ANY]
67 10.524184 16.851492 18.499222 17.166419 16.617646
68 ave : 15.9317926
69
70 [blocked Read & IO_0]
71 12.594326 10.974664 9.791322 14.108371 13.741705
72 ave : 12.2420776
73
74
75 ###########################################
76 ./word_count -file 1gb.txt -cpu 12 -division 16 -block 48 -m
77
78 division size を 16k に変更
79 Blocked Read size 16k * 48
80
81 [mmap]
82 10.169057 10.540739 15.449075 10.957382 11.899007
83 ave : 11.803052
84
85 [read]
86 11.462661 12.633221 11.207322 10.525345 12.815037
87 ave : 11.7287172
88
89 [blocked Read & SPE_ANY]
90 11.794426 11.285127 10.987724 11.539787 11.426118
91 ave : 11.4066364
92
93 [blocked Read & IO_0]
94 9.902556 12.431267 11.206498 10.889500 13.748140
95 ave : 11.6355922
96
97
98 ###########################################
99 ./word_count -file 1gb.txt -cpu 12 -division 32 -block 48 -m
100
101 division size を 32k に変更
102 Blocked Read size 32k * 48
103
104 [mmap]
105 15.069456 15.122584 14.922003 14.683691 14.777088
106 ave : 14.9149644
107
108 [read]
109 11.028623 11.035131 11.373129 12.821938 12.046853
110 ave : 11.6611348
111
112 [blocked Read & SPE_ANY]
113 11.919220 12.329987 12.332933 14.418972 13.078229
114 ave : 12.8158682
115
116 [blocked Read & IO_0]
117 10.530808 10.747448 13.123480 10.487363 16.765442
118 ave : 12.3309082
119
120 ###########################################
121 ./word_count -file 1gb.txt -cpu 12 -division 64 -block 48 -m
122
123 division size を 64k に変更
124 Blocked Read size 64k * 48
125
126 [mmap]
127 16.111595 16.025707 17.268757 15.955856 17.766358
128 ave : 16.6256546
129
130 [read]
131 11.808742 12.239779 12.924552 13.963415 11.550749
132 ave : 12.4974474
133
134 [blocked Read & SPE_ANY]
135 11.352159 13.369793 12.610295 12.669730 12.267202
136 ave : 12.4538358
137
138 [blocked Read & IO_0]
139 10.797157 10.301865 11.943482 10.176760 11.135116
140 ave : 10.870876
141
142 ###########################################
143 ./word_count -file 1gb.txt -cpu 12 -division 128 -block 48 -m
144
145 division size を 128k に変更
146 Blocked Read size 128k * 48
147
148 [mmap]
149 16.034226 16.560303 16.890796 19.045418 16.082843
150 ave : 16.9227172
151
152 [read]
153 10.536881 10.504687 11.637468 12.857552 11.196838
154 ave : 11.3466852
155
156 [blocked Read & SPE_ANY]
157 12.481490 13.197780 12.647181 13.424816 12.707132
158 ave : 12.8916798
159
160 [blocked Read & IO_0]
161 10.342932 10.488933 12.881027 11.114004 11.648817
162 ave : 11.2951426
163
164 ###########################################
165 ./word_count -file 1gb.txt -cpu 12 -division 256 -block 48 -m
166
167 division size を 256k に変更
168 Blocked Read size 256k * 48
169
170 [mmap]
171 16.861397 16.332805 21.959199 19.286259 17.929253
172 ave : 18.4737826
173
174 [read]
175 11.290773 10.484917 12.768376 11.330529 12.415259
176 ave : 11.6579708
177
178 [blocked Read & SPE_ANY]
179 11.277959 11.902750 12.208483 11.148553 13.273579
180 ave : 11.9622648
181
182 [blocked Read & IO_0]
183 10.603377 10.891932 11.916977 12.061559 13.142021
184 ave : 11.7231732
185
186 ###########################################
187 リフレッシュ後はここまで
188 #############################################
189
190 ###########################################
191 ./word_count -file 1gb.txt -cpu 12 -division 32 -block 48 -m
192
193 mmap only で Blocked Read の様々な大きさを試してみる
194
195 Blocked Read size 16k * 48
196 11.225793 11.516904 13.259233 12.324640 10.343828
197 ave : 11.7340796
198
199 ###########################################
200 ./word_count -file 1gb.txt -cpu 12 -division 32 -block 48 -m
201
202 division size を 32k に変更
203 Blocked Read size 256k * 48
204
205 [mmap]
206 16.648177 20.314135 22.318274 20.390665 21.174707
207 ave : 20.1691916
208
209 [read]
210 12.048092 13.031264 15.055066 13.631955 14.629920
211 ave : 13.6792594
212
213 [blocked Read & SPE_ANY]
214 11.673849 14.027035 13.938218 14.581857 15.435502
215 ave : 13.9312922
216
217 [blocked Read & IO_0]
218 15.310970 12.431799 13.015167 12.111157 13.268908
219 ave : 13.2276002
220
221 ###########################################
222 ./word_count -file 1gb.txt -cpu 12 -division 256 -block 48 -m
223
224 division size を 256k に変更
225 Blocked Read size 256k * 48
226
227 [mmap]
228 16.533852 22.421299 23.928809 20.837954 21.757904
229 ave : 21.0959636
230
231 [read]
232 11.522632 15.494991 14.413933 15.421523 14.346096
233 ave : 14.239835
234
235 [blocked Read & SPE_ANY]
236 11.645137 14.220895 13.072463 13.235456 13.830988
237 ave : 13.2009878
238
239 [blocked Read & IO_0]
240 13.405995 13.243454 12.777046 13.805675 13.137011
241 ave : 13.2738362
242
243 ###########################################
244 ./word_count -file 1gb.txt -cpu 12 -division 64 -block 48 -m
245
246 division size を 64k に変更
247 Blocked Read size 64k * 48
248
249 [mmap]
250 16.563569 20.158053 21.448670 21.293077 20.550916
251 ave : 20.002857
252
253 [read]
254 11.368613 14.068283 13.444215 15.334056 14.554330
255 ave : 13.7538994
256
257 [blocked Read & SPE_ANY]
258 11.169437 14.419505 15.024782 17.806684 17.591594
259 ave : 15.2024004
260
261 [blocked Read & IO_0]
262 11.116777 14.260367 13.418558 13.119003 13.319392
263 ave : 13.0468194
264
265
266 --------------------------------------------------------------
267 2014/05/07 (Wed)
268 [result]
269 nkmr 研サーバにて
270
271 ###########################################
272 (2014/05/01 の結果)
273 cpu 8 other division_size = 16k block_num = 48
274 cache 0.840
275 [mmap]
276 include I/O 7.886 7.919 7.604 7.412 7.443 7.631
277 ave 7.649
278
279 [read]
280 include I/O 7.541 7.553 7.963 7.501 7.477
281 ave 7.607
282
283 [blocked Read & SPE_ANY]
284 include I/O 14.887 14.486 14.453 14.608 15.279
285 ave 14.723
286
287 [blocked Read & IO_0]
288 include I/O 6.719 6.835 6.819 6.663 6.738
289 ave 6.755
290
291 以下 2014/05/07 に計測
292
293 ###########################################
294 cpu 4 other division_size = 16k block_num = 48
295 cache 1.088943
296 [mmap]
297 11.394516 12.045339 13.763432 13.429961 11.886197
298 12.503889
299
300 [read]
301 7.900163 8.210646 8.174739 8.221865 8.257309
302 8.1529444
303
304 [blocked Read & SPE_ANY]
305 11.583224 13.696717 11.232450 10.875663 11.609224
306 11.7994556
307
308 [blocked Read & IO_0]
309 6.862518 7.163325 7.196333 6.887726 6.972422
310 7.0164648
311
312 ###########################################
313 cpu 3 other division_size = 16k block_num = 48
314 cache 2.049192
315 [mmap]
316 7.210944 7.754113 7.971077 7.704215 7.520623
317 7.6321944
318
319 [read]
320 8.232689 8.645796 8.895464 8.685775 8.873369
321 8.6666186
322
323 [blocked Read & SPE_ANY]
324 12.253558 13.401002 13.410819 12.614162 13.296896
325 12.9952874
326
327 [blocked Read & IO_0]
328 6.753707 7.472933 7.339476 7.597705 7.389503
329 7.3106648
330
331 ###########################################
332 cpu 2 other division_size = 16k block_num = 48
333 cache 2.049192
334 [mmap]
335 6.702701 6.761172 6.753862 6.736276 6.869777
336 6.7647576
337
338 [read]
339 8.690265 8.691177 8.712588 8.695770 8.706455
340 8.699251
341
342 [blocked Read & SPE_ANY]
343 10.189692 10.972406 10.679654 10.344061 10.843843
344 10.6059312
345
346 [blocked Read & IO_0]
347 6.688520 6.831342 6.813723 6.787828 6.881037
348 6.80049
349
350 ###########################################
351 cpu 1 other division_size = 16k block_num = 48
352 cache 3.956206
353 [mmap]
354 6.894841 6.736401 6.896121 7.003213 6.729732
355 6.8520616
356
357 [read]
358 10.526449 10.556775 10.552578 10.569887 10.518269
359 10.5447916
360
361 [blocked Read & SPE_ANY]
362 8.655361 8.710293 8.661651 8.634304 8.768228
363 8.6859674
364
365 [blocked Read & IO_0]
366 6.724043 6.680827 6.790831 6.724226 6.756765 6.831216
367 6.751318
368
369
1 2014/05/01 (Thu) 370 2014/05/01 (Thu)
2 [result] 371 [result]
3 nkmr 研 namako 鯖 372 nkmr 研 namako 鯖
4 OS : CentOS 6.5 373 OS : CentOS 6.5
5 CPU : Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz 374 CPU : Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
10 HDD : WD1001FALS-41Y6A1 1TB 7200 rpm 32MB Cache SATA 3.0 Gbps 379 HDD : WD1001FALS-41Y6A1 1TB 7200 rpm 32MB Cache SATA 3.0 Gbps
11 380
12 381
13 ########################################### 382 ###########################################
14 cpu 8 other division_size = 16k block_num = 48 383 cpu 8 other division_size = 16k block_num = 48
15 ache 0.840 384 cache 0.840
16 [mmap] 385 [mmap]
17 include I/O 7.886 7.919 7.604 7.412 7.443 7.631 386 include I/O 7.886 7.919 7.604 7.412 7.443 7.631
387 ave 7.649
18 388
19 [read] 389 [read]
20 include I/O 7.541 7.553 7.963 7.501 7.477 390 include I/O 7.541 7.553 7.963 7.501 7.477
391 ave 7.607
21 392
22 [blocked Read & SPE_ANY] 393 [blocked Read & SPE_ANY]
23 include I/O 14.887 14.486 14.453 14.608 15.279 394 include I/O 14.887 14.486 14.453 14.608 15.279
395 ave 14.723
24 396
25 [blocked Read & IO_0] 397 [blocked Read & IO_0]
26 include I/O 6.719 6.835 6.819 6.663 6.738 398 include I/O 6.719 6.835 6.819 6.663 6.738
27 399 ave 6.755
28 400
29 ########################################### 401 ###########################################
30 cpu 8 division_size = 128k block_num = 48 402 cpu 8 division_size = 128k block_num = 48
31 403
32 cache 0.698 404 cache 0.698
33 [mmap] 405 [mmap]
34 include I/O 18.380 19.356 19.033 19.007 19.641 406 include I/O 18.380 19.356 19.033 19.007 19.641
407 ave 19.083
35 408
36 [read] 409 [read]
37 include I/O 7.389 7.419 7.480 7.409 7.455 410 include I/O 7.389 7.419 7.480 7.409 7.455
411 ave 7.430
38 412
39 [blocked Read & SPE_ANY] 413 [blocked Read & SPE_ANY]
40 include I/O 10.488 10.254 10.833 10.513 10.937 414 include I/O 10.488 10.254 10.833 10.513 10.937
415 ave 10.605
41 416
42 [blocked Read & IO_0] 417 [blocked Read & IO_0]
43 include I/O 6.722 6.822 6.747 7.339 6.790 418 include I/O 6.722 6.822 6.747 7.339 6.790
419 ave 6.884
44 420
45 421
46 ########################################### 422 ###########################################
47 cpu 8 division_size = 16k block_num = 96 423 cpu 8 division_size = 16k block_num = 96
48 424
68 444
69 [read] 445 [read]
70 include I/O 7.369 7.444 7.493 7.393 7.478 7.539 446 include I/O 7.369 7.444 7.493 7.393 7.478 7.539
71 447
72 [blocked Read & SPE_ANY] 448 [blocked Read & SPE_ANY]
449 include I/O 9.301 9.453 9.362 9.426 9.272
450
451 [blocked Read & IO_0]
73 include I/O 6.784 6.741 6.737 6.801 6.743 452 include I/O 6.784 6.741 6.737 6.801 6.743
74 453
75 [blocked Read & IO_0] 454
76 include I/O 9.301 9.453 9.362 9.426 9.272 455 -------------------------------------------------------------
456
457 2014/04/18 (Fri)
458 [memo]
459 word count の再計測
460
461 kono 研 firefly
462 HDD : WD1001FALS-41Y6A1 1TB 7200 rpm 32MB Cache SATA 3.0 Gbps
463
464 ./word_count -file 1gb.txt -cpu 12 -division 128 -block 48 -m
465 (CPU num = 12)
466 < mmap > 22.713 0.878
467 < read > 12.520 1.469
468 < bread & IO_0 > 10.295 0.886
469 < bread & SPE_ANY > 14.028 0.882
470
471 (CPU num = 8)
472 < mmap > 22.789 0.952
473 < read > 14.785 1.571
474 < bread & IO_0 > 14.774 0.857
475 < bread & SPE_ANY > 15.287 1.009
476
477 (CPU num = 4)
478 < mmap > 22.861 1.739
479 < read > 15.737 2.303
480 < bread & IO_0 > 14.634 1.565
481 < bread & SPE_ANY > 15.646 1.941
482
483 (CPU num = 1)
484 < mmap > 20.179 6.521
485 < read > 21.351 6.832
486 < bread & IO_0 > 13.930 6.009
487 < bread & IO_0 > 18.531 6.775
488
489
490
491 ./word_count -file X gb.txt -cpu 12 -division 128 -block 48 -m
492 (file size 2GB)
493 < mmap > 34.072
494 < read > 24.755
495 < bread & IO_0 > 21.396
496 < bread & SPE_ANY > 23.909
497
498 (file size 3GB)
499 < mmap > 52.819
500 < read > 33.398
501 < bread & IO_0 > 32.798
502 < bread & SPE_ANY > 37.140
503
504 (file size 4GB)
505 < mmap > 68.951
506 < read > 43.578
507 < bread & IO_0 > 42.491
508 < bread & SPE_ANY > 48.078
509
510
511
512
513 ./word_count -file 1gb.txt -cpu 12 -division 16 -block 48 -m
514 < mmap > 11.137 0.854
515 < read > 11.437 1.487
516 < bread & IO_0 > 11.412 0.847
517 < bread & SPE_ANY > 11.146 0.866
518
519 ./word_count -file 1gb.txt -cpu 1 -division 16 -block 48 -m
520 < mmap > 15.353
521 < read > 16.846
522 < bread & IO_0 > 11.503 6.992
523 < bread & SPE_ANY > 13.297
524
525 ./word_count -file 1gb.txt -cpu 8 -division 16 -block 48 -m
526 < mmap > 11.707
527 < read > 11.487
528 < bread & IO_0 > 11.365
529 < bread & SPE_ANY > 10.887
530
531
532
533 ./word_count -file 1gb.txt -cpu 1 -division 128 -block 48 -br
534 14.210562 15.025856 13.118798 14.155912 13.137139 13.930
535
536 ./word_count -file 1gb.txt -cpu 1 -division 128 -block 48 -br spe_any
537 21.347 18.110 15.926 18.128 19.143 18.531
538
539 ./word_count -file 1gb.txt -cpu 1 -division 128 -block 48 -r
540 19.669 19.612 22.830 22.116 22.528 21.351
541
542 ./word_count -file 1gb.txt -cpu 1 -division 128 -block 48 -m
543 18.604 22.576 15.844 22.019 21.852 20.179
544
545 ./word_count -file 1gb.txt -cpu 4 -division 128 -block 48 -m
546 20.963 24.989 19.770 20.893 27.692 22.861
547
548 ./word_count -file 1gb.txt -cpu 4 -division 128 -block 48 -r
549 14.331 14.403 16.493 15.652 17.808 15.737
550
551 ./word_count -file 1gb.txt -cpu 4 -division 128 -block 48 -br spe_any
552 14.369 13.818 17.198 15.796 17.051 15.646
553
554 ./word_count -file 1gb.txt -cpu 4 -division 128 -block 48 -br IO_0
555 12.898 14.056 15.115 15.839 15.263 14.634
556
557 ./word_count -file 1gb.txt -cpu 4 -division 128 -block 48 -br IO_0
558 12.898 14.056 15.115 15.839 15.263 14.634
559
560 ./word_count -file 1gb.txt -cpu 8 -division 128 -block 48 -br spe_any
561 14.995 13.622 15.332 16.686 15.802 A 15.287
562
563 ./word_count -file 1gb.txt -cpu 8 -division 128 -block 48 -br IO_0
564 12.351 15.303 17.811 14.310 14.094 14.774
565
566 ./word_count -file 1gb.txt -cpu 8 -division 128 -block 48 -m
567 20.071 20.652 27.683 23.919 22.070 22.789
568
569 ./word_count -file 1gb.txt -cpu 8 -division 128 -block 48 -r
570 15.542 14.062 14.255 14.541 15.524 14.785
571
572 ./word_count -file 1gb.txt -cpu 12 -division 128 -block 48 -m
573 20.415 23.049 27.368 20.539 22.195 22.713
574
575 ./word_count -file 2gb.txt -cpu 12 -division 128 -block 48 -m
576 40.646 41.631 34.220 36.335 38.208
577
578 ./word_count -file 2gb.txt -cpu 12 -division 128 -block 48 -br
579 27.800 30.437 26.995 28.411
580
581 ./word_count -file 2gb.txt -cpu 12 -division 128 -block 48 -br io_0
582 21.977 23.126 28.174 25.038 22.688 24.201
583
584 ./word_count -file 3gb.txt -cpu 12 -division 128 -block 48 -m
585 51.671 61.249 59.457 56.704 63.490 58.514
586
587 ./word_count -file 4gb.txt -cpu 12 -division 128 -block 48 -m
588 65.671 73.202 77.090 76.386 76.017 73.767
589
590 ./word_count -file 4gb.txt -cpu 12 -division 128 -block 48 -br io_0
591 43.864 51.867 49.750 49.981 44.271 47.967
592
593 ./word_count -file 3gb.txt -cpu 12 -division 128 -block 48 -br io_0
594 29.918 35.544 36.423 39.121 41.158 36.397
595
596 ./word_count -file 4gb.txt -cpu 12 -division 128 -block 48 -br SPE_ANY
597 50.935 57.769 54.234 54.155 52.010 53.813
598
599 ./word_count -file 3gb.txt -cpu 12 -division 128 -block 48 -br SPE_ANY
600 45.513 40.117 42.841 38.430 41.798 41.740
601
602 ./word_count -file 2gb.txt -cpu 12 -division 128 -block 48 -br SPE_ANY
603 23.304 24.006 22.301 23.292 25.107 23.602
604
605 -------------------------------------------------------------
606