Mercurial > hg > CbC > CbC_gcc
comparison gcc/sel-sched.c @ 55:77e2b8dfacca gcc-4.4.5
update it from 4.4.3 to 4.5.0
author | ryoma <e075725@ie.u-ryukyu.ac.jp> |
---|---|
date | Fri, 12 Feb 2010 23:39:51 +0900 |
parents | a06113de4d67 |
children | b7f97abdc517 |
comparison
equal
deleted
inserted
replaced
52:c156f1bd5cd9 | 55:77e2b8dfacca |
---|---|
54 | 54 |
55 /* Implementation of selective scheduling approach. | 55 /* Implementation of selective scheduling approach. |
56 The below implementation follows the original approach with the following | 56 The below implementation follows the original approach with the following |
57 changes: | 57 changes: |
58 | 58 |
59 o the scheduler works after register allocation (but can be also tuned | 59 o the scheduler works after register allocation (but can be also tuned |
60 to work before RA); | 60 to work before RA); |
61 o some instructions are not copied or register renamed; | 61 o some instructions are not copied or register renamed; |
62 o conditional jumps are not moved with code duplication; | 62 o conditional jumps are not moved with code duplication; |
63 o several jumps in one parallel group are not supported; | 63 o several jumps in one parallel group are not supported; |
64 o when pipelining outer loops, code motion through inner loops | 64 o when pipelining outer loops, code motion through inner loops |
67 o some improvements for better compile time/performance were made. | 67 o some improvements for better compile time/performance were made. |
68 | 68 |
69 Terminology | 69 Terminology |
70 =========== | 70 =========== |
71 | 71 |
72 A vinsn, or virtual insn, is an insn with additional data characterizing | 72 A vinsn, or virtual insn, is an insn with additional data characterizing |
73 insn pattern, such as LHS, RHS, register sets used/set/clobbered, etc. | 73 insn pattern, such as LHS, RHS, register sets used/set/clobbered, etc. |
74 Vinsns also act as smart pointers to save memory by reusing them in | 74 Vinsns also act as smart pointers to save memory by reusing them in |
75 different expressions. A vinsn is described by vinsn_t type. | 75 different expressions. A vinsn is described by vinsn_t type. |
76 | 76 |
77 An expression is a vinsn with additional data characterizing its properties | 77 An expression is a vinsn with additional data characterizing its properties |
78 at some point in the control flow graph. The data may be its usefulness, | 78 at some point in the control flow graph. The data may be its usefulness, |
79 priority, speculative status, whether it was renamed/subsituted, etc. | 79 priority, speculative status, whether it was renamed/subsituted, etc. |
80 An expression is described by expr_t type. | 80 An expression is described by expr_t type. |
81 | 81 |
82 Availability set (av_set) is a set of expressions at a given control flow | 82 Availability set (av_set) is a set of expressions at a given control flow |
83 point. It is represented as av_set_t. The expressions in av sets are kept | 83 point. It is represented as av_set_t. The expressions in av sets are kept |
84 sorted in the terms of expr_greater_p function. It allows to truncate | 84 sorted in the terms of expr_greater_p function. It allows to truncate |
85 the set while leaving the best expressions. | 85 the set while leaving the best expressions. |
86 | 86 |
87 A fence is a point through which code motion is prohibited. On each step, | 87 A fence is a point through which code motion is prohibited. On each step, |
88 we gather a parallel group of insns at a fence. It is possible to have | 88 we gather a parallel group of insns at a fence. It is possible to have |
89 multiple fences. A fence is represented via fence_t. | 89 multiple fences. A fence is represented via fence_t. |
90 | 90 |
91 A boundary is the border between the fence group and the rest of the code. | 91 A boundary is the border between the fence group and the rest of the code. |
92 Currently, we never have more than one boundary per fence, as we finalize | 92 Currently, we never have more than one boundary per fence, as we finalize |
93 the fence group when a jump is scheduled. A boundary is represented | 93 the fence group when a jump is scheduled. A boundary is represented |
94 via bnd_t. | 94 via bnd_t. |
95 | 95 |
96 High-level overview | 96 High-level overview |
97 =================== | 97 =================== |
98 | 98 |
99 The scheduler finds regions to schedule, schedules each one, and finalizes. | 99 The scheduler finds regions to schedule, schedules each one, and finalizes. |
100 The regions are formed starting from innermost loops, so that when the inner | 100 The regions are formed starting from innermost loops, so that when the inner |
101 loop is pipelined, its prologue can be scheduled together with yet unprocessed | 101 loop is pipelined, its prologue can be scheduled together with yet unprocessed |
102 outer loop. The rest of acyclic regions are found using extend_rgns: | 102 outer loop. The rest of acyclic regions are found using extend_rgns: |
103 the blocks that are not yet allocated to any regions are traversed in top-down | 103 the blocks that are not yet allocated to any regions are traversed in top-down |
104 order, and a block is added to a region to which all its predecessors belong; | 104 order, and a block is added to a region to which all its predecessors belong; |
105 otherwise, the block starts its own region. | 105 otherwise, the block starts its own region. |
106 | 106 |
107 The main scheduling loop (sel_sched_region_2) consists of just | 107 The main scheduling loop (sel_sched_region_2) consists of just |
108 scheduling on each fence and updating fences. For each fence, | 108 scheduling on each fence and updating fences. For each fence, |
109 we fill a parallel group of insns (fill_insns) until some insns can be added. | 109 we fill a parallel group of insns (fill_insns) until some insns can be added. |
110 First, we compute available exprs (av-set) at the boundary of the current | 110 First, we compute available exprs (av-set) at the boundary of the current |
111 group. Second, we choose the best expression from it. If the stall is | 111 group. Second, we choose the best expression from it. If the stall is |
112 required to schedule any of the expressions, we advance the current cycle | 112 required to schedule any of the expressions, we advance the current cycle |
113 appropriately. So, the final group does not exactly correspond to a VLIW | 113 appropriately. So, the final group does not exactly correspond to a VLIW |
114 word. Third, we move the chosen expression to the boundary (move_op) | 114 word. Third, we move the chosen expression to the boundary (move_op) |
115 and update the intermediate av sets and liveness sets. We quit fill_insns | 115 and update the intermediate av sets and liveness sets. We quit fill_insns |
116 when either no insns left for scheduling or we have scheduled enough insns | 116 when either no insns left for scheduling or we have scheduled enough insns |
117 so we feel like advancing a scheduling point. | 117 so we feel like advancing a scheduling point. |
118 | 118 |
119 Computing available expressions | 119 Computing available expressions |
120 =============================== | 120 =============================== |
121 | 121 |
122 The computation (compute_av_set) is a bottom-up traversal. At each insn, | 122 The computation (compute_av_set) is a bottom-up traversal. At each insn, |
123 we're moving the union of its successors' sets through it via | 123 we're moving the union of its successors' sets through it via |
124 moveup_expr_set. The dependent expressions are removed. Local | 124 moveup_expr_set. The dependent expressions are removed. Local |
125 transformations (substitution, speculation) are applied to move more | 125 transformations (substitution, speculation) are applied to move more |
126 exprs. Then the expr corresponding to the current insn is added. | 126 exprs. Then the expr corresponding to the current insn is added. |
127 The result is saved on each basic block header. | 127 The result is saved on each basic block header. |
128 | 128 |
129 When traversing the CFG, we're moving down for no more than max_ws insns. | 129 When traversing the CFG, we're moving down for no more than max_ws insns. |
130 Also, we do not move down to ineligible successors (is_ineligible_successor), | 130 Also, we do not move down to ineligible successors (is_ineligible_successor), |
131 which include moving along a back-edge, moving to already scheduled code, | 131 which include moving along a back-edge, moving to already scheduled code, |
132 and moving to another fence. The first two restrictions are lifted during | 132 and moving to another fence. The first two restrictions are lifted during |
133 pipelining, which allows us to move insns along a back-edge. We always have | 133 pipelining, which allows us to move insns along a back-edge. We always have |
134 an acyclic region for scheduling because we forbid motion through fences. | 134 an acyclic region for scheduling because we forbid motion through fences. |
135 | 135 |
136 Choosing the best expression | 136 Choosing the best expression |
137 ============================ | 137 ============================ |
138 | 138 |
139 We sort the final availability set via sel_rank_for_schedule, then we remove | 139 We sort the final availability set via sel_rank_for_schedule, then we remove |
140 expressions which are not yet ready (tick_check_p) or which dest registers | 140 expressions which are not yet ready (tick_check_p) or which dest registers |
141 cannot be used. For some of them, we choose another register via | 141 cannot be used. For some of them, we choose another register via |
142 find_best_reg. To do this, we run find_used_regs to calculate the set of | 142 find_best_reg. To do this, we run find_used_regs to calculate the set of |
143 registers which cannot be used. The find_used_regs function performs | 143 registers which cannot be used. The find_used_regs function performs |
144 a traversal of code motion paths for an expr. We consider for renaming | 144 a traversal of code motion paths for an expr. We consider for renaming |
145 only registers which are from the same regclass as the original one and | 145 only registers which are from the same regclass as the original one and |
146 using which does not interfere with any live ranges. Finally, we convert | 146 using which does not interfere with any live ranges. Finally, we convert |
147 the resulting set to the ready list format and use max_issue and reorder* | 147 the resulting set to the ready list format and use max_issue and reorder* |
148 hooks similarly to the Haifa scheduler. | 148 hooks similarly to the Haifa scheduler. |
149 | 149 |
150 Scheduling the best expression | 150 Scheduling the best expression |
151 ============================== | 151 ============================== |
152 | 152 |
153 We run the move_op routine to perform the same type of code motion paths | 153 We run the move_op routine to perform the same type of code motion paths |
154 traversal as in find_used_regs. (These are working via the same driver, | 154 traversal as in find_used_regs. (These are working via the same driver, |
155 code_motion_path_driver.) When moving down the CFG, we look for original | 155 code_motion_path_driver.) When moving down the CFG, we look for original |
156 instruction that gave birth to a chosen expression. We undo | 156 instruction that gave birth to a chosen expression. We undo |
157 the transformations performed on an expression via the history saved in it. | 157 the transformations performed on an expression via the history saved in it. |
158 When found, we remove the instruction or leave a reg-reg copy/speculation | 158 When found, we remove the instruction or leave a reg-reg copy/speculation |
159 check if needed. On a way up, we insert bookkeeping copies at each join | 159 check if needed. On a way up, we insert bookkeeping copies at each join |
160 point. If a copy is not needed, it will be removed later during this | 160 point. If a copy is not needed, it will be removed later during this |
161 traversal. We update the saved av sets and liveness sets on the way up, too. | 161 traversal. We update the saved av sets and liveness sets on the way up, too. |
162 | 162 |
163 Finalizing the schedule | 163 Finalizing the schedule |
164 ======================= | 164 ======================= |
165 | 165 |
166 When pipelining, we reschedule the blocks from which insns were pipelined | 166 When pipelining, we reschedule the blocks from which insns were pipelined |
167 to get a tighter schedule. On Itanium, we also perform bundling via | 167 to get a tighter schedule. On Itanium, we also perform bundling via |
168 the same routine from ia64.c. | 168 the same routine from ia64.c. |
169 | 169 |
170 Dependence analysis changes | 170 Dependence analysis changes |
171 =========================== | 171 =========================== |
172 | 172 |
173 We augmented the sched-deps.c with hooks that get called when a particular | 173 We augmented the sched-deps.c with hooks that get called when a particular |
174 dependence is found in a particular part of an insn. Using these hooks, we | 174 dependence is found in a particular part of an insn. Using these hooks, we |
175 can do several actions such as: determine whether an insn can be moved through | 175 can do several actions such as: determine whether an insn can be moved through |
176 another (has_dependence_p, moveup_expr); find out whether an insn can be | 176 another (has_dependence_p, moveup_expr); find out whether an insn can be |
177 scheduled on the current cycle (tick_check_p); find out registers that | 177 scheduled on the current cycle (tick_check_p); find out registers that |
178 are set/used/clobbered by an insn and find out all the strange stuff that | 178 are set/used/clobbered by an insn and find out all the strange stuff that |
179 restrict its movement, like SCHED_GROUP_P or CANT_MOVE (done in | 179 restrict its movement, like SCHED_GROUP_P or CANT_MOVE (done in |
180 init_global_and_expr_for_insn). | 180 init_global_and_expr_for_insn). |
181 | 181 |
182 Initialization changes | 182 Initialization changes |
183 ====================== | 183 ====================== |
184 | 184 |
185 There are parts of haifa-sched.c, sched-deps.c, and sched-rgn.c that are | 185 There are parts of haifa-sched.c, sched-deps.c, and sched-rgn.c that are |
186 reused in all of the schedulers. We have split up the initialization of data | 186 reused in all of the schedulers. We have split up the initialization of data |
187 of such parts into different functions prefixed with scheduler type and | 187 of such parts into different functions prefixed with scheduler type and |
188 postfixed with the type of data initialized: {,sel_,haifa_}sched_{init,finish}, | 188 postfixed with the type of data initialized: {,sel_,haifa_}sched_{init,finish}, |
189 sched_rgn_init/finish, sched_deps_init/finish, sched_init_{luids/bbs}, etc. | 189 sched_rgn_init/finish, sched_deps_init/finish, sched_init_{luids/bbs}, etc. |
190 The same splitting is done with current_sched_info structure: | 190 The same splitting is done with current_sched_info structure: |
191 dependence-related parts are in sched_deps_info, common part is in | 191 dependence-related parts are in sched_deps_info, common part is in |
192 common_sched_info, and haifa/sel/etc part is in current_sched_info. | 192 common_sched_info, and haifa/sel/etc part is in current_sched_info. |
193 | 193 |
194 Target contexts | 194 Target contexts |
195 =============== | 195 =============== |
196 | 196 |
197 As we now have multiple-point scheduling, this would not work with backends | 197 As we now have multiple-point scheduling, this would not work with backends |
198 which save some of the scheduler state to use it in the target hooks. | 198 which save some of the scheduler state to use it in the target hooks. |
199 For this purpose, we introduce a concept of target contexts, which | 199 For this purpose, we introduce a concept of target contexts, which |
200 encapsulate such information. The backend should implement simple routines | 200 encapsulate such information. The backend should implement simple routines |
201 of allocating/freeing/setting such a context. The scheduler calls these | 201 of allocating/freeing/setting such a context. The scheduler calls these |
202 as target hooks and handles the target context as an opaque pointer (similar | 202 as target hooks and handles the target context as an opaque pointer (similar |
203 to the DFA state type, state_t). | 203 to the DFA state type, state_t). |
204 | 204 |
205 Various speedups | 205 Various speedups |
206 ================ | 206 ================ |
207 | 207 |
208 As the correct data dependence graph is not supported during scheduling (which | 208 As the correct data dependence graph is not supported during scheduling (which |
209 is to be changed in mid-term), we cache as much of the dependence analysis | 209 is to be changed in mid-term), we cache as much of the dependence analysis |
210 results as possible to avoid reanalyzing. This includes: bitmap caches on | 210 results as possible to avoid reanalyzing. This includes: bitmap caches on |
211 each insn in stream of the region saying yes/no for a query with a pair of | 211 each insn in stream of the region saying yes/no for a query with a pair of |
212 UIDs; hashtables with the previously done transformations on each insn in | 212 UIDs; hashtables with the previously done transformations on each insn in |
213 stream; a vector keeping a history of transformations on each expr. | 213 stream; a vector keeping a history of transformations on each expr. |
214 | 214 |
215 Also, we try to minimize the dependence context used on each fence to check | 215 Also, we try to minimize the dependence context used on each fence to check |
216 whether the given expression is ready for scheduling by removing from it | 216 whether the given expression is ready for scheduling by removing from it |
217 insns that are definitely completed the execution. The results of | 217 insns that are definitely completed the execution. The results of |
218 tick_check_p checks are also cached in a vector on each fence. | 218 tick_check_p checks are also cached in a vector on each fence. |
219 | 219 |
220 We keep a valid liveness set on each insn in a region to avoid the high | 220 We keep a valid liveness set on each insn in a region to avoid the high |
221 cost of recomputation on large basic blocks. | 221 cost of recomputation on large basic blocks. |
222 | 222 |
223 Finally, we try to minimize the number of needed updates to the availability | 223 Finally, we try to minimize the number of needed updates to the availability |
224 sets. The updates happen in two cases: when fill_insns terminates, | 224 sets. The updates happen in two cases: when fill_insns terminates, |
225 we advance all fences and increase the stage number to show that the region | 225 we advance all fences and increase the stage number to show that the region |
226 has changed and the sets are to be recomputed; and when the next iteration | 226 has changed and the sets are to be recomputed; and when the next iteration |
227 of a loop in fill_insns happens (but this one reuses the saved av sets | 227 of a loop in fill_insns happens (but this one reuses the saved av sets |
228 on bb headers.) Thus, we try to break the fill_insns loop only when | 228 on bb headers.) Thus, we try to break the fill_insns loop only when |
229 "significant" number of insns from the current scheduling window was | 229 "significant" number of insns from the current scheduling window was |
230 scheduled. This should be made a target param. | 230 scheduled. This should be made a target param. |
231 | 231 |
232 | 232 |
233 TODO: correctly support the data dependence graph at all stages and get rid | 233 TODO: correctly support the data dependence graph at all stages and get rid |
234 of all caches. This should speed up the scheduler. | 234 of all caches. This should speed up the scheduler. |
235 TODO: implement moving cond jumps with bookkeeping copies on both targets. | 235 TODO: implement moving cond jumps with bookkeeping copies on both targets. |
236 TODO: tune the scheduler before RA so it does not create too much pseudos. | 236 TODO: tune the scheduler before RA so it does not create too much pseudos. |
237 | 237 |
238 | 238 |
239 References: | 239 References: |
240 S.-M. Moon and K. Ebcioglu. Parallelizing nonnumerical code with | 240 S.-M. Moon and K. Ebcioglu. Parallelizing nonnumerical code with |
241 selective scheduling and software pipelining. | 241 selective scheduling and software pipelining. |
242 ACM TOPLAS, Vol 19, No. 6, pages 853--898, Nov. 1997. | 242 ACM TOPLAS, Vol 19, No. 6, pages 853--898, Nov. 1997. |
243 | 243 |
244 Andrey Belevantsev, Maxim Kuvyrkov, Vladimir Makarov, Dmitry Melnik, | 244 Andrey Belevantsev, Maxim Kuvyrkov, Vladimir Makarov, Dmitry Melnik, |
245 and Dmitry Zhurikhin. An interblock VLIW-targeted instruction scheduler | 245 and Dmitry Zhurikhin. An interblock VLIW-targeted instruction scheduler |
246 for GCC. In Proceedings of GCC Developers' Summit 2006. | 246 for GCC. In Proceedings of GCC Developers' Summit 2006. |
247 | 247 |
248 Arutyun Avetisyan, Andrey Belevantsev, and Dmitry Melnik. GCC Instruction | 248 Arutyun Avetisyan, Andrey Belevantsev, and Dmitry Melnik. GCC Instruction |
249 Scheduler and Software Pipeliner on the Itanium Platform. EPIC-7 Workshop. | 249 Scheduler and Software Pipeliner on the Itanium Platform. EPIC-7 Workshop. |
250 http://rogue.colorado.edu/EPIC7/. | 250 http://rogue.colorado.edu/EPIC7/. |
251 | 251 |
252 */ | 252 */ |
253 | 253 |
254 /* True when pipelining is enabled. */ | 254 /* True when pipelining is enabled. */ |
255 bool pipelining_p; | 255 bool pipelining_p; |
256 | 256 |
262 | 262 |
263 | 263 |
264 /* Definitions of local types and macros. */ | 264 /* Definitions of local types and macros. */ |
265 | 265 |
266 /* Represents possible outcomes of moving an expression through an insn. */ | 266 /* Represents possible outcomes of moving an expression through an insn. */ |
267 enum MOVEUP_EXPR_CODE | 267 enum MOVEUP_EXPR_CODE |
268 { | 268 { |
269 /* The expression is not changed. */ | 269 /* The expression is not changed. */ |
270 MOVEUP_EXPR_SAME, | 270 MOVEUP_EXPR_SAME, |
271 | 271 |
272 /* Not changed, but requires a new destination register. */ | 272 /* Not changed, but requires a new destination register. */ |
273 MOVEUP_EXPR_AS_RHS, | 273 MOVEUP_EXPR_AS_RHS, |
274 | 274 |
275 /* Cannot be moved. */ | 275 /* Cannot be moved. */ |
276 MOVEUP_EXPR_NULL, | 276 MOVEUP_EXPR_NULL, |
277 | 277 |
278 /* Changed (substituted or speculated). */ | 278 /* Changed (substituted or speculated). */ |
279 MOVEUP_EXPR_CHANGED | 279 MOVEUP_EXPR_CHANGED |
280 }; | 280 }; |
281 | 281 |
282 /* The container to be passed into rtx search & replace functions. */ | 282 /* The container to be passed into rtx search & replace functions. */ |
283 struct rtx_search_arg | 283 struct rtx_search_arg |
284 { | 284 { |
289 int n; | 289 int n; |
290 }; | 290 }; |
291 | 291 |
292 typedef struct rtx_search_arg *rtx_search_arg_p; | 292 typedef struct rtx_search_arg *rtx_search_arg_p; |
293 | 293 |
294 /* This struct contains precomputed hard reg sets that are needed when | 294 /* This struct contains precomputed hard reg sets that are needed when |
295 computing registers available for renaming. */ | 295 computing registers available for renaming. */ |
296 struct hard_regs_data | 296 struct hard_regs_data |
297 { | 297 { |
298 /* For every mode, this stores registers available for use with | 298 /* For every mode, this stores registers available for use with |
299 that mode. */ | 299 that mode. */ |
300 HARD_REG_SET regs_for_mode[NUM_MACHINE_MODES]; | 300 HARD_REG_SET regs_for_mode[NUM_MACHINE_MODES]; |
301 | 301 |
302 /* True when regs_for_mode[mode] is initialized. */ | 302 /* True when regs_for_mode[mode] is initialized. */ |
303 bool regs_for_mode_ok[NUM_MACHINE_MODES]; | 303 bool regs_for_mode_ok[NUM_MACHINE_MODES]; |
305 /* For every register, it has regs that are ok to rename into it. | 305 /* For every register, it has regs that are ok to rename into it. |
306 The register in question is always set. If not, this means | 306 The register in question is always set. If not, this means |
307 that the whole set is not computed yet. */ | 307 that the whole set is not computed yet. */ |
308 HARD_REG_SET regs_for_rename[FIRST_PSEUDO_REGISTER]; | 308 HARD_REG_SET regs_for_rename[FIRST_PSEUDO_REGISTER]; |
309 | 309 |
310 /* For every mode, this stores registers not available due to | 310 /* For every mode, this stores registers not available due to |
311 call clobbering. */ | 311 call clobbering. */ |
312 HARD_REG_SET regs_for_call_clobbered[NUM_MACHINE_MODES]; | 312 HARD_REG_SET regs_for_call_clobbered[NUM_MACHINE_MODES]; |
313 | 313 |
314 /* All registers that are used or call used. */ | 314 /* All registers that are used or call used. */ |
315 HARD_REG_SET regs_ever_used; | 315 HARD_REG_SET regs_ever_used; |
332 | 332 |
333 /* Whether this code motion path crosses a call. */ | 333 /* Whether this code motion path crosses a call. */ |
334 bool crosses_call; | 334 bool crosses_call; |
335 }; | 335 }; |
336 | 336 |
337 /* A global structure that contains the needed information about harg | 337 /* A global structure that contains the needed information about harg |
338 regs. */ | 338 regs. */ |
339 static struct hard_regs_data sel_hrd; | 339 static struct hard_regs_data sel_hrd; |
340 | 340 |
341 | 341 |
342 /* This structure holds local data used in code_motion_path_driver hooks on | 342 /* This structure holds local data used in code_motion_path_driver hooks on |
343 the same or adjacent levels of recursion. Here we keep those parameters | 343 the same or adjacent levels of recursion. Here we keep those parameters |
344 that are not used in code_motion_path_driver routine itself, but only in | 344 that are not used in code_motion_path_driver routine itself, but only in |
345 its hooks. Moreover, all parameters that can be modified in hooks are | 345 its hooks. Moreover, all parameters that can be modified in hooks are |
346 in this structure, so all other parameters passed explicitly to hooks are | 346 in this structure, so all other parameters passed explicitly to hooks are |
347 read-only. */ | 347 read-only. */ |
348 struct cmpd_local_params | 348 struct cmpd_local_params |
349 { | 349 { |
350 /* Local params used in move_op_* functions. */ | 350 /* Local params used in move_op_* functions. */ |
351 | 351 |
359 /* Copy of the ORIGINAL_INSN list, stores the original insns already | 359 /* Copy of the ORIGINAL_INSN list, stores the original insns already |
360 found before entering the current level of code_motion_path_driver. */ | 360 found before entering the current level of code_motion_path_driver. */ |
361 def_list_t old_original_insns; | 361 def_list_t old_original_insns; |
362 | 362 |
363 /* Local params used in move_op_* functions. */ | 363 /* Local params used in move_op_* functions. */ |
364 /* True when we have removed last insn in the block which was | 364 /* True when we have removed last insn in the block which was |
365 also a boundary. Do not update anything or create bookkeeping copies. */ | 365 also a boundary. Do not update anything or create bookkeeping copies. */ |
366 BOOL_BITFIELD removed_last_insn : 1; | 366 BOOL_BITFIELD removed_last_insn : 1; |
367 }; | 367 }; |
368 | 368 |
369 /* Stores the static parameters for move_op_* calls. */ | 369 /* Stores the static parameters for move_op_* calls. */ |
428 | 428 |
429 /* Called on the backward stage of recursion to do moveup_expr. | 429 /* Called on the backward stage of recursion to do moveup_expr. |
430 Used only with move_op_*. */ | 430 Used only with move_op_*. */ |
431 void (*ascend) (insn_t, void *); | 431 void (*ascend) (insn_t, void *); |
432 | 432 |
433 /* Called on the ascending pass, before returning from the current basic | 433 /* Called on the ascending pass, before returning from the current basic |
434 block or from the whole traversal. */ | 434 block or from the whole traversal. */ |
435 void (*at_first_insn) (insn_t, cmpd_local_params_p, void *); | 435 void (*at_first_insn) (insn_t, cmpd_local_params_p, void *); |
436 | 436 |
437 /* When processing successors in move_op we need only descend into | 437 /* When processing successors in move_op we need only descend into |
438 SUCCS_NORMAL successors, while in find_used_regs we need SUCCS_ALL. */ | 438 SUCCS_NORMAL successors, while in find_used_regs we need SUCCS_ALL. */ |
439 int succ_flags; | 439 int succ_flags; |
440 | 440 |
441 /* The routine name to print in dumps ("move_op" of "find_used_regs"). */ | 441 /* The routine name to print in dumps ("move_op" of "find_used_regs"). */ |
442 const char *routine_name; | 442 const char *routine_name; |
446 FUR_HOOKS. */ | 446 FUR_HOOKS. */ |
447 struct code_motion_path_driver_info_def *code_motion_path_driver_info; | 447 struct code_motion_path_driver_info_def *code_motion_path_driver_info; |
448 | 448 |
449 /* Set of hooks for performing move_op and find_used_regs routines with | 449 /* Set of hooks for performing move_op and find_used_regs routines with |
450 code_motion_path_driver. */ | 450 code_motion_path_driver. */ |
451 struct code_motion_path_driver_info_def move_op_hooks, fur_hooks; | 451 extern struct code_motion_path_driver_info_def move_op_hooks, fur_hooks; |
452 | 452 |
453 /* True if/when we want to emulate Haifa scheduler in the common code. | 453 /* True if/when we want to emulate Haifa scheduler in the common code. |
454 This is used in sched_rgn_local_init and in various places in | 454 This is used in sched_rgn_local_init and in various places in |
455 sched-deps.c. */ | 455 sched-deps.c. */ |
456 int sched_emulate_haifa_p; | 456 int sched_emulate_haifa_p; |
457 | 457 |
458 /* GLOBAL_LEVEL is used to discard information stored in basic block headers | 458 /* GLOBAL_LEVEL is used to discard information stored in basic block headers |
459 av_sets. Av_set of bb header is valid if its (bb header's) level is equal | 459 av_sets. Av_set of bb header is valid if its (bb header's) level is equal |
467 /* True when separable insns should be scheduled as RHSes. */ | 467 /* True when separable insns should be scheduled as RHSes. */ |
468 static bool enable_schedule_as_rhs_p; | 468 static bool enable_schedule_as_rhs_p; |
469 | 469 |
470 /* Used in verify_target_availability to assert that target reg is reported | 470 /* Used in verify_target_availability to assert that target reg is reported |
471 unavailabile by both TARGET_UNAVAILABLE and find_used_regs only if | 471 unavailabile by both TARGET_UNAVAILABLE and find_used_regs only if |
472 we haven't scheduled anything on the previous fence. | 472 we haven't scheduled anything on the previous fence. |
473 if scheduled_something_on_previous_fence is true, TARGET_UNAVAILABLE can | 473 if scheduled_something_on_previous_fence is true, TARGET_UNAVAILABLE can |
474 have more conservative value than the one returned by the | 474 have more conservative value than the one returned by the |
475 find_used_regs, thus we shouldn't assert that these values are equal. */ | 475 find_used_regs, thus we shouldn't assert that these values are equal. */ |
476 static bool scheduled_something_on_previous_fence; | 476 static bool scheduled_something_on_previous_fence; |
477 | 477 |
478 /* All newly emitted insns will have their uids greater than this value. */ | 478 /* All newly emitted insns will have their uids greater than this value. */ |
479 static int first_emitted_uid; | 479 static int first_emitted_uid; |
516 can't be moved up due to bookkeeping created during code motion to another | 516 can't be moved up due to bookkeeping created during code motion to another |
517 fence. See comment near the call to update_and_record_unavailable_insns | 517 fence. See comment near the call to update_and_record_unavailable_insns |
518 for the detailed explanations. */ | 518 for the detailed explanations. */ |
519 static vinsn_vec_t vec_bookkeeping_blocked_vinsns = NULL; | 519 static vinsn_vec_t vec_bookkeeping_blocked_vinsns = NULL; |
520 | 520 |
521 /* This vector has vinsns which are scheduled with renaming on the first fence | 521 /* This vector has vinsns which are scheduled with renaming on the first fence |
522 and then seen on the second. For expressions with such vinsns, target | 522 and then seen on the second. For expressions with such vinsns, target |
523 availability information may be wrong. */ | 523 availability information may be wrong. */ |
524 static vinsn_vec_t vec_target_unavailable_vinsns = NULL; | 524 static vinsn_vec_t vec_target_unavailable_vinsns = NULL; |
525 | 525 |
526 /* Vector to store temporary nops inserted in move_op to prevent removal | 526 /* Vector to store temporary nops inserted in move_op to prevent removal |
527 of empty bbs. */ | 527 of empty bbs. */ |
528 DEF_VEC_P(insn_t); | 528 DEF_VEC_P(insn_t); |
529 DEF_VEC_ALLOC_P(insn_t,heap); | 529 DEF_VEC_ALLOC_P(insn_t,heap); |
530 static VEC(insn_t, heap) *vec_temp_moveop_nops = NULL; | 530 static VEC(insn_t, heap) *vec_temp_moveop_nops = NULL; |
531 | 531 |
532 /* These bitmaps record original instructions scheduled on the current | 532 /* These bitmaps record original instructions scheduled on the current |
533 iteration and bookkeeping copies created by them. */ | 533 iteration and bookkeeping copies created by them. */ |
534 static bitmap current_originators = NULL; | 534 static bitmap current_originators = NULL; |
535 static bitmap current_copies = NULL; | 535 static bitmap current_copies = NULL; |
536 | 536 |
537 /* This bitmap marks the blocks visited by code_motion_path_driver so we don't | 537 /* This bitmap marks the blocks visited by code_motion_path_driver so we don't |
538 visit them afterwards. */ | 538 visit them afterwards. */ |
555 | 555 |
556 /* Forward declarations of static functions. */ | 556 /* Forward declarations of static functions. */ |
557 static bool rtx_ok_for_substitution_p (rtx, rtx); | 557 static bool rtx_ok_for_substitution_p (rtx, rtx); |
558 static int sel_rank_for_schedule (const void *, const void *); | 558 static int sel_rank_for_schedule (const void *, const void *); |
559 static av_set_t find_sequential_best_exprs (bnd_t, expr_t, bool); | 559 static av_set_t find_sequential_best_exprs (bnd_t, expr_t, bool); |
560 static basic_block find_block_for_bookkeeping (edge e1, edge e2, bool lax); | |
560 | 561 |
561 static rtx get_dest_from_orig_ops (av_set_t); | 562 static rtx get_dest_from_orig_ops (av_set_t); |
562 static basic_block generate_bookkeeping_insn (expr_t, edge, edge); | 563 static basic_block generate_bookkeeping_insn (expr_t, edge, edge); |
563 static bool find_used_regs (insn_t, av_set_t, regset, struct reg_rename *, | 564 static bool find_used_regs (insn_t, av_set_t, regset, struct reg_rename *, |
564 def_list_t *); | 565 def_list_t *); |
565 static bool move_op (insn_t, av_set_t, expr_t, rtx, expr_t, bool*); | 566 static bool move_op (insn_t, av_set_t, expr_t, rtx, expr_t, bool*); |
566 static int code_motion_path_driver (insn_t, av_set_t, ilist_t, | 567 static int code_motion_path_driver (insn_t, av_set_t, ilist_t, |
567 cmpd_local_params_p, void *); | 568 cmpd_local_params_p, void *); |
568 static void sel_sched_region_1 (void); | 569 static void sel_sched_region_1 (void); |
579 advance_one_cycle (fence_t fence) | 580 advance_one_cycle (fence_t fence) |
580 { | 581 { |
581 unsigned i; | 582 unsigned i; |
582 int cycle; | 583 int cycle; |
583 rtx insn; | 584 rtx insn; |
584 | 585 |
585 advance_state (FENCE_STATE (fence)); | 586 advance_state (FENCE_STATE (fence)); |
586 cycle = ++FENCE_CYCLE (fence); | 587 cycle = ++FENCE_CYCLE (fence); |
587 FENCE_ISSUED_INSNS (fence) = 0; | 588 FENCE_ISSUED_INSNS (fence) = 0; |
588 FENCE_STARTS_CYCLE_P (fence) = 1; | 589 FENCE_STARTS_CYCLE_P (fence) = 1; |
589 can_issue_more = issue_rate; | 590 can_issue_more = issue_rate; |
624 bb = bb->next_bb; | 625 bb = bb->next_bb; |
625 | 626 |
626 return bb == BLOCK_FOR_INSN (succ); | 627 return bb == BLOCK_FOR_INSN (succ); |
627 } | 628 } |
628 | 629 |
629 /* Construct successor fences from OLD_FENCEs and put them in NEW_FENCES. | 630 /* Construct successor fences from OLD_FENCEs and put them in NEW_FENCES. |
630 When a successor will continue a ebb, transfer all parameters of a fence | 631 When a successor will continue a ebb, transfer all parameters of a fence |
631 to the new fence. ORIG_MAX_SEQNO is the maximal seqno before this round | 632 to the new fence. ORIG_MAX_SEQNO is the maximal seqno before this round |
632 of scheduling helping to distinguish between the old and the new code. */ | 633 of scheduling helping to distinguish between the old and the new code. */ |
633 static void | 634 static void |
634 extract_new_fences_from (flist_t old_fences, flist_tail_t new_fences, | 635 extract_new_fences_from (flist_t old_fences, flist_tail_t new_fences, |
648 gcc_assert (!was_here_p); | 649 gcc_assert (!was_here_p); |
649 was_here_p = true; | 650 was_here_p = true; |
650 } | 651 } |
651 gcc_assert (was_here_p && insn != NULL_RTX); | 652 gcc_assert (was_here_p && insn != NULL_RTX); |
652 | 653 |
653 /* When in the "middle" of the block, just move this fence | 654 /* When in the "middle" of the block, just move this fence |
654 to the new list. */ | 655 to the new list. */ |
655 bb = BLOCK_FOR_INSN (insn); | 656 bb = BLOCK_FOR_INSN (insn); |
656 if (! sel_bb_end_p (insn) | 657 if (! sel_bb_end_p (insn) |
657 || (single_succ_p (bb) | 658 || (single_succ_p (bb) |
658 && single_pred_p (single_succ (bb)))) | 659 && single_pred_p (single_succ (bb)))) |
659 { | 660 { |
660 insn_t succ; | 661 insn_t succ; |
661 | 662 |
662 succ = (sel_bb_end_p (insn) | 663 succ = (sel_bb_end_p (insn) |
663 ? sel_bb_head (single_succ (bb)) | 664 ? sel_bb_head (single_succ (bb)) |
664 : NEXT_INSN (insn)); | 665 : NEXT_INSN (insn)); |
665 | 666 |
666 if (INSN_SEQNO (succ) > 0 | 667 if (INSN_SEQNO (succ) > 0 |
667 && INSN_SEQNO (succ) <= orig_max_seqno | 668 && INSN_SEQNO (succ) <= orig_max_seqno |
668 && INSN_SCHED_TIMES (succ) <= 0) | 669 && INSN_SCHED_TIMES (succ) <= 0) |
669 { | 670 { |
670 FENCE_INSN (fence) = succ; | 671 FENCE_INSN (fence) = succ; |
671 move_fence_to_fences (old_fences, new_fences); | 672 move_fence_to_fences (old_fences, new_fences); |
672 | 673 |
673 if (sched_verbose >= 1) | 674 if (sched_verbose >= 1) |
674 sel_print ("Fence %d continues as %d[%d] (state continue)\n", | 675 sel_print ("Fence %d continues as %d[%d] (state continue)\n", |
675 INSN_UID (insn), INSN_UID (succ), BLOCK_NUM (succ)); | 676 INSN_UID (insn), INSN_UID (succ), BLOCK_NUM (succ)); |
676 } | 677 } |
677 return; | 678 return; |
678 } | 679 } |
679 | 680 |
684 | 685 |
685 if (0 < seqno && seqno <= orig_max_seqno | 686 if (0 < seqno && seqno <= orig_max_seqno |
686 && (pipelining_p || INSN_SCHED_TIMES (succ) <= 0)) | 687 && (pipelining_p || INSN_SCHED_TIMES (succ) <= 0)) |
687 { | 688 { |
688 bool b = (in_same_ebb_p (insn, succ) | 689 bool b = (in_same_ebb_p (insn, succ) |
689 || in_fallthru_bb_p (insn, succ)); | 690 || in_fallthru_bb_p (insn, succ)); |
690 | 691 |
691 if (sched_verbose >= 1) | 692 if (sched_verbose >= 1) |
692 sel_print ("Fence %d continues as %d[%d] (state %s)\n", | 693 sel_print ("Fence %d continues as %d[%d] (state %s)\n", |
693 INSN_UID (insn), INSN_UID (succ), | 694 INSN_UID (insn), INSN_UID (succ), |
694 BLOCK_NUM (succ), b ? "continue" : "reset"); | 695 BLOCK_NUM (succ), b ? "continue" : "reset"); |
695 | 696 |
696 if (b) | 697 if (b) |
697 add_dirty_fence_to_fences (new_fences, succ, fence); | 698 add_dirty_fence_to_fences (new_fences, succ, fence); |
698 else | 699 else |
706 } | 707 } |
707 | 708 |
708 | 709 |
709 /* Functions to support substitution. */ | 710 /* Functions to support substitution. */ |
710 | 711 |
711 /* Returns whether INSN with dependence status DS is eligible for | 712 /* Returns whether INSN with dependence status DS is eligible for |
712 substitution, i.e. it's a copy operation x := y, and RHS that is | 713 substitution, i.e. it's a copy operation x := y, and RHS that is |
713 moved up through this insn should be substituted. */ | 714 moved up through this insn should be substituted. */ |
714 static bool | 715 static bool |
715 can_substitute_through_p (insn_t insn, ds_t ds) | 716 can_substitute_through_p (insn_t insn, ds_t ds) |
716 { | 717 { |
717 /* We can substitute only true dependencies. */ | 718 /* We can substitute only true dependencies. */ |
719 || (ds & DEP_ANTI) | 720 || (ds & DEP_ANTI) |
720 || ! INSN_RHS (insn) | 721 || ! INSN_RHS (insn) |
721 || ! INSN_LHS (insn)) | 722 || ! INSN_LHS (insn)) |
722 return false; | 723 return false; |
723 | 724 |
724 /* Now we just need to make sure the INSN_RHS consists of only one | 725 /* Now we just need to make sure the INSN_RHS consists of only one |
725 simple REG rtx. */ | 726 simple REG rtx. */ |
726 if (REG_P (INSN_LHS (insn)) | 727 if (REG_P (INSN_LHS (insn)) |
727 && REG_P (INSN_RHS (insn))) | 728 && REG_P (INSN_RHS (insn))) |
728 return true; | 729 return true; |
729 return false; | 730 return false; |
730 } | 731 } |
731 | 732 |
732 /* Substitute all occurences of INSN's destination in EXPR' vinsn with INSN's | 733 /* Substitute all occurences of INSN's destination in EXPR' vinsn with INSN's |
733 source (if INSN is eligible for substitution). Returns TRUE if | 734 source (if INSN is eligible for substitution). Returns TRUE if |
734 substitution was actually performed, FALSE otherwise. Substitution might | 735 substitution was actually performed, FALSE otherwise. Substitution might |
735 be not performed because it's either EXPR' vinsn doesn't contain INSN's | 736 be not performed because it's either EXPR' vinsn doesn't contain INSN's |
736 destination or the resulting insn is invalid for the target machine. | 737 destination or the resulting insn is invalid for the target machine. |
737 When UNDO is true, perform unsubstitution instead (the difference is in | 738 When UNDO is true, perform unsubstitution instead (the difference is in |
738 the part of rtx on which validate_replace_rtx is called). */ | 739 the part of rtx on which validate_replace_rtx is called). */ |
739 static bool | 740 static bool |
740 substitute_reg_in_expr (expr_t expr, insn_t insn, bool undo) | 741 substitute_reg_in_expr (expr_t expr, insn_t insn, bool undo) |
741 { | 742 { |
744 vinsn_t *vi = &EXPR_VINSN (expr); | 745 vinsn_t *vi = &EXPR_VINSN (expr); |
745 bool has_rhs = VINSN_RHS (*vi) != NULL; | 746 bool has_rhs = VINSN_RHS (*vi) != NULL; |
746 rtx old, new_rtx; | 747 rtx old, new_rtx; |
747 | 748 |
748 /* Do not try to replace in SET_DEST. Although we'll choose new | 749 /* Do not try to replace in SET_DEST. Although we'll choose new |
749 register for the RHS, we don't want to change RHS' original reg. | 750 register for the RHS, we don't want to change RHS' original reg. |
750 If the insn is not SET, we may still be able to substitute something | 751 If the insn is not SET, we may still be able to substitute something |
751 in it, and if we're here (don't have deps), it doesn't write INSN's | 752 in it, and if we're here (don't have deps), it doesn't write INSN's |
752 dest. */ | 753 dest. */ |
753 where = (has_rhs | 754 where = (has_rhs |
754 ? &VINSN_RHS (*vi) | 755 ? &VINSN_RHS (*vi) |
755 : &PATTERN (VINSN_INSN_RTX (*vi))); | 756 : &PATTERN (VINSN_INSN_RTX (*vi))); |
756 old = undo ? INSN_RHS (insn) : INSN_LHS (insn); | 757 old = undo ? INSN_RHS (insn) : INSN_LHS (insn); |
763 | 764 |
764 /* We should copy these rtxes before substitution. */ | 765 /* We should copy these rtxes before substitution. */ |
765 new_rtx = copy_rtx (undo ? INSN_LHS (insn) : INSN_RHS (insn)); | 766 new_rtx = copy_rtx (undo ? INSN_LHS (insn) : INSN_RHS (insn)); |
766 new_insn = create_copy_of_insn_rtx (VINSN_INSN_RTX (*vi)); | 767 new_insn = create_copy_of_insn_rtx (VINSN_INSN_RTX (*vi)); |
767 | 768 |
768 /* Where we'll replace. | 769 /* Where we'll replace. |
769 WHERE_REPLACE should point inside NEW_INSN, so INSN_RHS couldn't be | 770 WHERE_REPLACE should point inside NEW_INSN, so INSN_RHS couldn't be |
770 used instead of SET_SRC. */ | 771 used instead of SET_SRC. */ |
771 where_replace = (has_rhs | 772 where_replace = (has_rhs |
772 ? &SET_SRC (PATTERN (new_insn)) | 773 ? &SET_SRC (PATTERN (new_insn)) |
773 : &PATTERN (new_insn)); | 774 : &PATTERN (new_insn)); |
774 | 775 |
775 new_insn_valid | 776 new_insn_valid |
776 = validate_replace_rtx_part_nosimplify (old, new_rtx, where_replace, | 777 = validate_replace_rtx_part_nosimplify (old, new_rtx, where_replace, |
777 new_insn); | 778 new_insn); |
778 | 779 |
779 /* ??? Actually, constrain_operands result depends upon choice of | 780 /* ??? Actually, constrain_operands result depends upon choice of |
780 destination register. E.g. if we allow single register to be an rhs, | 781 destination register. E.g. if we allow single register to be an rhs, |
781 and if we try to move dx=ax(as rhs) through ax=dx, we'll result | 782 and if we try to move dx=ax(as rhs) through ax=dx, we'll result |
782 in invalid insn dx=dx, so we'll loose this rhs here. | 783 in invalid insn dx=dx, so we'll loose this rhs here. |
783 Just can't come up with significant testcase for this, so just | 784 Just can't come up with significant testcase for this, so just |
784 leaving it for now. */ | 785 leaving it for now. */ |
785 if (new_insn_valid) | 786 if (new_insn_valid) |
786 { | 787 { |
787 change_vinsn_in_expr (expr, | 788 change_vinsn_in_expr (expr, |
788 create_vinsn_from_insn_rtx (new_insn, false)); | 789 create_vinsn_from_insn_rtx (new_insn, false)); |
789 | 790 |
790 /* Do not allow clobbering the address register of speculative | 791 /* Do not allow clobbering the address register of speculative |
791 insns. */ | 792 insns. */ |
792 if ((EXPR_SPEC_DONE_DS (expr) & SPECULATIVE) | 793 if ((EXPR_SPEC_DONE_DS (expr) & SPECULATIVE) |
793 && bitmap_bit_p (VINSN_REG_USES (EXPR_VINSN (expr)), | 794 && bitmap_bit_p (VINSN_REG_USES (EXPR_VINSN (expr)), |
794 expr_dest_regno (expr))) | 795 expr_dest_regno (expr))) |
795 EXPR_TARGET_AVAILABLE (expr) = false; | 796 EXPR_TARGET_AVAILABLE (expr) = false; |
796 | 797 |
797 return true; | 798 return true; |
798 } | 799 } |
802 else | 803 else |
803 return false; | 804 return false; |
804 } | 805 } |
805 | 806 |
806 /* Helper function for count_occurences_equiv. */ | 807 /* Helper function for count_occurences_equiv. */ |
807 static int | 808 static int |
808 count_occurrences_1 (rtx *cur_rtx, void *arg) | 809 count_occurrences_1 (rtx *cur_rtx, void *arg) |
809 { | 810 { |
810 rtx_search_arg_p p = (rtx_search_arg_p) arg; | 811 rtx_search_arg_p p = (rtx_search_arg_p) arg; |
811 | 812 |
812 /* The last param FOR_GCSE is true, because otherwise it performs excessive | 813 /* The last param FOR_GCSE is true, because otherwise it performs excessive |
835 if (GET_CODE (*cur_rtx) == SUBREG | 836 if (GET_CODE (*cur_rtx) == SUBREG |
836 && REG_P (p->x) | 837 && REG_P (p->x) |
837 && REGNO (SUBREG_REG (*cur_rtx)) == REGNO (p->x)) | 838 && REGNO (SUBREG_REG (*cur_rtx)) == REGNO (p->x)) |
838 { | 839 { |
839 /* ??? Do not support substituting regs inside subregs. In that case, | 840 /* ??? Do not support substituting regs inside subregs. In that case, |
840 simplify_subreg will be called by validate_replace_rtx, and | 841 simplify_subreg will be called by validate_replace_rtx, and |
841 unsubstitution will fail later. */ | 842 unsubstitution will fail later. */ |
842 p->n = 0; | 843 p->n = 0; |
843 return 1; | 844 return 1; |
844 } | 845 } |
845 | 846 |
846 /* Continue search. */ | 847 /* Continue search. */ |
847 return 0; | 848 return 0; |
848 } | 849 } |
849 | 850 |
850 /* Return the number of places WHAT appears within WHERE. | 851 /* Return the number of places WHAT appears within WHERE. |
851 Bail out when we found a reference occupying several hard registers. */ | 852 Bail out when we found a reference occupying several hard registers. */ |
852 static int | 853 static int |
853 count_occurrences_equiv (rtx what, rtx where) | 854 count_occurrences_equiv (rtx what, rtx where) |
854 { | 855 { |
855 struct rtx_search_arg arg; | 856 struct rtx_search_arg arg; |
856 | 857 |
857 arg.x = what; | 858 arg.x = what; |
887 insn_rtx = create_insn_rtx_from_pattern (pattern, NULL_RTX); | 888 insn_rtx = create_insn_rtx_from_pattern (pattern, NULL_RTX); |
888 | 889 |
889 return insn_rtx; | 890 return insn_rtx; |
890 } | 891 } |
891 | 892 |
892 /* Returns whether INSN's src can be replaced with register number | 893 /* Returns whether INSN's src can be replaced with register number |
893 NEW_SRC_REG. E.g. the following insn is valid for i386: | 894 NEW_SRC_REG. E.g. the following insn is valid for i386: |
894 | 895 |
895 (insn:HI 2205 6585 2207 727 ../../gcc/libiberty/regex.c:3337 | 896 (insn:HI 2205 6585 2207 727 ../../gcc/libiberty/regex.c:3337 |
896 (set (mem/s:QI (plus:SI (plus:SI (reg/f:SI 7 sp) | 897 (set (mem/s:QI (plus:SI (plus:SI (reg/f:SI 7 sp) |
897 (reg:SI 0 ax [orig:770 c1 ] [770])) | 898 (reg:SI 0 ax [orig:770 c1 ] [770])) |
898 (const_int 288 [0x120])) [0 str S1 A8]) | 899 (const_int 288 [0x120])) [0 str S1 A8]) |
899 (const_int 0 [0x0])) 43 {*movqi_1} (nil) | 900 (const_int 0 [0x0])) 43 {*movqi_1} (nil) |
900 (nil)) | 901 (nil)) |
901 | 902 |
902 But if we change (const_int 0 [0x0]) to (reg:QI 4 si), it will be invalid | 903 But if we change (const_int 0 [0x0]) to (reg:QI 4 si), it will be invalid |
903 because of operand constraints: | 904 because of operand constraints: |
904 | 905 |
905 (define_insn "*movqi_1" | 906 (define_insn "*movqi_1" |
906 [(set (match_operand:QI 0 "nonimmediate_operand" "=q,q ,q ,r,r ,?r,m") | 907 [(set (match_operand:QI 0 "nonimmediate_operand" "=q,q ,q ,r,r ,?r,m") |
907 (match_operand:QI 1 "general_operand" " q,qn,qm,q,rn,qm,qn") | 908 (match_operand:QI 1 "general_operand" " q,qn,qm,q,rn,qm,qn") |
908 )] | 909 )] |
909 | 910 |
910 So do constrain_operands here, before choosing NEW_SRC_REG as best | 911 So do constrain_operands here, before choosing NEW_SRC_REG as best |
911 reg for rhs. */ | 912 reg for rhs. */ |
912 | 913 |
913 static bool | 914 static bool |
914 replace_src_with_reg_ok_p (insn_t insn, rtx new_src_reg) | 915 replace_src_with_reg_ok_p (insn_t insn, rtx new_src_reg) |
915 { | 916 { |
968 insn_rtx = create_insn_rtx_from_pattern (pattern, NULL_RTX); | 969 insn_rtx = create_insn_rtx_from_pattern (pattern, NULL_RTX); |
969 | 970 |
970 return insn_rtx; | 971 return insn_rtx; |
971 } | 972 } |
972 | 973 |
973 /* Substitute lhs in the given expression EXPR for the register with number | 974 /* Substitute lhs in the given expression EXPR for the register with number |
974 NEW_REGNO. SET_DEST may be arbitrary rtx, not only register. */ | 975 NEW_REGNO. SET_DEST may be arbitrary rtx, not only register. */ |
975 static void | 976 static void |
976 replace_dest_with_reg_in_expr (expr_t expr, rtx new_reg) | 977 replace_dest_with_reg_in_expr (expr_t expr, rtx new_reg) |
977 { | 978 { |
978 rtx insn_rtx; | 979 rtx insn_rtx; |
987 } | 988 } |
988 | 989 |
989 /* Returns whether VI writes either one of the USED_REGS registers or, | 990 /* Returns whether VI writes either one of the USED_REGS registers or, |
990 if a register is a hard one, one of the UNAVAILABLE_HARD_REGS registers. */ | 991 if a register is a hard one, one of the UNAVAILABLE_HARD_REGS registers. */ |
991 static bool | 992 static bool |
992 vinsn_writes_one_of_regs_p (vinsn_t vi, regset used_regs, | 993 vinsn_writes_one_of_regs_p (vinsn_t vi, regset used_regs, |
993 HARD_REG_SET unavailable_hard_regs) | 994 HARD_REG_SET unavailable_hard_regs) |
994 { | 995 { |
995 unsigned regno; | 996 unsigned regno; |
996 reg_set_iterator rsi; | 997 reg_set_iterator rsi; |
997 | 998 |
1014 } | 1015 } |
1015 | 1016 |
1016 return false; | 1017 return false; |
1017 } | 1018 } |
1018 | 1019 |
1019 /* Returns register class of the output register in INSN. | 1020 /* Returns register class of the output register in INSN. |
1020 Returns NO_REGS for call insns because some targets have constraints on | 1021 Returns NO_REGS for call insns because some targets have constraints on |
1021 destination register of a call insn. | 1022 destination register of a call insn. |
1022 | 1023 |
1023 Code adopted from regrename.c::build_def_use. */ | 1024 Code adopted from regrename.c::build_def_use. */ |
1024 static enum reg_class | 1025 static enum reg_class |
1025 get_reg_class (rtx insn) | 1026 get_reg_class (rtx insn) |
1026 { | 1027 { |
1027 int alt, i, n_ops; | 1028 int alt, i, n_ops; |
1060 { | 1061 { |
1061 for (i = 0; i < n_ops + recog_data.n_dups; i++) | 1062 for (i = 0; i < n_ops + recog_data.n_dups; i++) |
1062 { | 1063 { |
1063 int opn = i < n_ops ? i : recog_data.dup_num[i - n_ops]; | 1064 int opn = i < n_ops ? i : recog_data.dup_num[i - n_ops]; |
1064 enum reg_class cl = recog_op_alt[opn][alt].cl; | 1065 enum reg_class cl = recog_op_alt[opn][alt].cl; |
1065 | 1066 |
1066 if (recog_data.operand_type[opn] == OP_OUT || | 1067 if (recog_data.operand_type[opn] == OP_OUT || |
1067 recog_data.operand_type[opn] == OP_INOUT) | 1068 recog_data.operand_type[opn] == OP_INOUT) |
1068 return cl; | 1069 return cl; |
1069 } | 1070 } |
1070 } | 1071 } |
1095 SET_HARD_REG_BIT (sel_hrd.regs_for_rename[regno], cur_reg); | 1096 SET_HARD_REG_BIT (sel_hrd.regs_for_rename[regno], cur_reg); |
1096 } | 1097 } |
1097 } | 1098 } |
1098 #endif | 1099 #endif |
1099 | 1100 |
1100 /* A wrapper around HARD_REGNO_RENAME_OK that will look into the hard regs | 1101 /* A wrapper around HARD_REGNO_RENAME_OK that will look into the hard regs |
1101 data first. */ | 1102 data first. */ |
1102 static inline bool | 1103 static inline bool |
1103 sel_hard_regno_rename_ok (int from ATTRIBUTE_UNUSED, int to ATTRIBUTE_UNUSED) | 1104 sel_hard_regno_rename_ok (int from ATTRIBUTE_UNUSED, int to ATTRIBUTE_UNUSED) |
1104 { | 1105 { |
1105 #ifdef HARD_REGNO_RENAME_OK | 1106 #ifdef HARD_REGNO_RENAME_OK |
1118 /* Calculate set of registers that are capable of holding MODE. */ | 1119 /* Calculate set of registers that are capable of holding MODE. */ |
1119 static void | 1120 static void |
1120 init_regs_for_mode (enum machine_mode mode) | 1121 init_regs_for_mode (enum machine_mode mode) |
1121 { | 1122 { |
1122 int cur_reg; | 1123 int cur_reg; |
1123 | 1124 |
1124 CLEAR_HARD_REG_SET (sel_hrd.regs_for_mode[mode]); | 1125 CLEAR_HARD_REG_SET (sel_hrd.regs_for_mode[mode]); |
1125 CLEAR_HARD_REG_SET (sel_hrd.regs_for_call_clobbered[mode]); | 1126 CLEAR_HARD_REG_SET (sel_hrd.regs_for_call_clobbered[mode]); |
1126 | 1127 |
1127 for (cur_reg = 0; cur_reg < FIRST_PSEUDO_REGISTER; cur_reg++) | 1128 for (cur_reg = 0; cur_reg < FIRST_PSEUDO_REGISTER; cur_reg++) |
1128 { | 1129 { |
1129 int nregs = hard_regno_nregs[cur_reg][mode]; | 1130 int nregs = hard_regno_nregs[cur_reg][mode]; |
1130 int i; | 1131 int i; |
1131 | 1132 |
1132 for (i = nregs - 1; i >= 0; --i) | 1133 for (i = nregs - 1; i >= 0; --i) |
1133 if (fixed_regs[cur_reg + i] | 1134 if (fixed_regs[cur_reg + i] |
1134 || global_regs[cur_reg + i] | 1135 || global_regs[cur_reg + i] |
1135 /* Can't use regs which aren't saved by | 1136 /* Can't use regs which aren't saved by |
1136 the prologue. */ | 1137 the prologue. */ |
1137 || !TEST_HARD_REG_BIT (sel_hrd.regs_ever_used, cur_reg + i) | 1138 || !TEST_HARD_REG_BIT (sel_hrd.regs_ever_used, cur_reg + i) |
1138 #ifdef LEAF_REGISTERS | 1139 #ifdef LEAF_REGISTERS |
1139 /* We can't use a non-leaf register if we're in a | 1140 /* We can't use a non-leaf register if we're in a |
1140 leaf function. */ | 1141 leaf function. */ |
1141 || (current_function_is_leaf | 1142 || (current_function_is_leaf |
1142 && !LEAF_REGISTERS[cur_reg + i]) | 1143 && !LEAF_REGISTERS[cur_reg + i]) |
1143 #endif | 1144 #endif |
1144 ) | 1145 ) |
1145 break; | 1146 break; |
1146 | 1147 |
1147 if (i >= 0) | 1148 if (i >= 0) |
1148 continue; | 1149 continue; |
1149 | 1150 |
1150 /* See whether it accepts all modes that occur in | 1151 /* See whether it accepts all modes that occur in |
1151 original insns. */ | 1152 original insns. */ |
1152 if (! HARD_REGNO_MODE_OK (cur_reg, mode)) | 1153 if (! HARD_REGNO_MODE_OK (cur_reg, mode)) |
1153 continue; | 1154 continue; |
1154 | 1155 |
1155 if (HARD_REGNO_CALL_PART_CLOBBERED (cur_reg, mode)) | 1156 if (HARD_REGNO_CALL_PART_CLOBBERED (cur_reg, mode)) |
1156 SET_HARD_REG_BIT (sel_hrd.regs_for_call_clobbered[mode], | 1157 SET_HARD_REG_BIT (sel_hrd.regs_for_call_clobbered[mode], |
1157 cur_reg); | 1158 cur_reg); |
1158 | 1159 |
1159 /* If the CUR_REG passed all the checks above, | 1160 /* If the CUR_REG passed all the checks above, |
1160 then it's ok. */ | 1161 then it's ok. */ |
1161 SET_HARD_REG_BIT (sel_hrd.regs_for_mode[mode], cur_reg); | 1162 SET_HARD_REG_BIT (sel_hrd.regs_for_mode[mode], cur_reg); |
1162 } | 1163 } |
1163 | 1164 |
1164 sel_hrd.regs_for_mode_ok[mode] = true; | 1165 sel_hrd.regs_for_mode_ok[mode] = true; |
1167 /* Init all register sets gathered in HRD. */ | 1168 /* Init all register sets gathered in HRD. */ |
1168 static void | 1169 static void |
1169 init_hard_regs_data (void) | 1170 init_hard_regs_data (void) |
1170 { | 1171 { |
1171 int cur_reg = 0; | 1172 int cur_reg = 0; |
1172 enum machine_mode cur_mode = 0; | 1173 int cur_mode = 0; |
1173 | 1174 |
1174 CLEAR_HARD_REG_SET (sel_hrd.regs_ever_used); | 1175 CLEAR_HARD_REG_SET (sel_hrd.regs_ever_used); |
1175 for (cur_reg = 0; cur_reg < FIRST_PSEUDO_REGISTER; cur_reg++) | 1176 for (cur_reg = 0; cur_reg < FIRST_PSEUDO_REGISTER; cur_reg++) |
1176 if (df_regs_ever_live_p (cur_reg) || call_used_regs[cur_reg]) | 1177 if (df_regs_ever_live_p (cur_reg) || call_used_regs[cur_reg]) |
1177 SET_HARD_REG_BIT (sel_hrd.regs_ever_used, cur_reg); | 1178 SET_HARD_REG_BIT (sel_hrd.regs_ever_used, cur_reg); |
1178 | 1179 |
1179 /* Initialize registers that are valid based on mode when this is | 1180 /* Initialize registers that are valid based on mode when this is |
1180 really needed. */ | 1181 really needed. */ |
1181 for (cur_mode = 0; cur_mode < NUM_MACHINE_MODES; cur_mode++) | 1182 for (cur_mode = 0; cur_mode < NUM_MACHINE_MODES; cur_mode++) |
1182 sel_hrd.regs_for_mode_ok[cur_mode] = false; | 1183 sel_hrd.regs_for_mode_ok[cur_mode] = false; |
1183 | 1184 |
1184 /* Mark that all HARD_REGNO_RENAME_OK is not calculated. */ | 1185 /* Mark that all HARD_REGNO_RENAME_OK is not calculated. */ |
1185 for (cur_reg = 0; cur_reg < FIRST_PSEUDO_REGISTER; cur_reg++) | 1186 for (cur_reg = 0; cur_reg < FIRST_PSEUDO_REGISTER; cur_reg++) |
1186 CLEAR_HARD_REG_SET (sel_hrd.regs_for_rename[cur_reg]); | 1187 CLEAR_HARD_REG_SET (sel_hrd.regs_for_rename[cur_reg]); |
1187 | 1188 |
1188 #ifdef STACK_REGS | 1189 #ifdef STACK_REGS |
1189 CLEAR_HARD_REG_SET (sel_hrd.stack_regs); | 1190 CLEAR_HARD_REG_SET (sel_hrd.stack_regs); |
1190 | 1191 |
1191 for (cur_reg = FIRST_STACK_REG; cur_reg <= LAST_STACK_REG; cur_reg++) | 1192 for (cur_reg = FIRST_STACK_REG; cur_reg <= LAST_STACK_REG; cur_reg++) |
1192 SET_HARD_REG_BIT (sel_hrd.stack_regs, cur_reg); | 1193 SET_HARD_REG_BIT (sel_hrd.stack_regs, cur_reg); |
1193 #endif | 1194 #endif |
1194 } | 1195 } |
1195 | 1196 |
1196 /* Mark hardware regs in REG_RENAME_P that are not suitable | 1197 /* Mark hardware regs in REG_RENAME_P that are not suitable |
1197 for renaming rhs in INSN due to hardware restrictions (register class, | 1198 for renaming rhs in INSN due to hardware restrictions (register class, |
1198 modes compatibility etc). This doesn't affect original insn's dest reg, | 1199 modes compatibility etc). This doesn't affect original insn's dest reg, |
1199 if it isn't in USED_REGS. DEF is a definition insn of rhs for which the | 1200 if it isn't in USED_REGS. DEF is a definition insn of rhs for which the |
1200 destination register is sought. LHS (DEF->ORIG_INSN) may be REG or MEM. | 1201 destination register is sought. LHS (DEF->ORIG_INSN) may be REG or MEM. |
1201 Registers that are in used_regs are always marked in | 1202 Registers that are in used_regs are always marked in |
1213 | 1214 |
1214 gcc_assert (GET_CODE (PATTERN (def->orig_insn)) == SET); | 1215 gcc_assert (GET_CODE (PATTERN (def->orig_insn)) == SET); |
1215 gcc_assert (reg_rename_p); | 1216 gcc_assert (reg_rename_p); |
1216 | 1217 |
1217 orig_dest = SET_DEST (PATTERN (def->orig_insn)); | 1218 orig_dest = SET_DEST (PATTERN (def->orig_insn)); |
1218 | 1219 |
1219 /* We have decided not to rename 'mem = something;' insns, as 'something' | 1220 /* We have decided not to rename 'mem = something;' insns, as 'something' |
1220 is usually a register. */ | 1221 is usually a register. */ |
1221 if (!REG_P (orig_dest)) | 1222 if (!REG_P (orig_dest)) |
1222 return; | 1223 return; |
1223 | 1224 |
1227 if (!reload_completed && !HARD_REGISTER_NUM_P (regno)) | 1228 if (!reload_completed && !HARD_REGISTER_NUM_P (regno)) |
1228 return; | 1229 return; |
1229 | 1230 |
1230 mode = GET_MODE (orig_dest); | 1231 mode = GET_MODE (orig_dest); |
1231 | 1232 |
1232 /* Stop when mode is not supported for renaming. Also can't proceed | 1233 /* Stop when mode is not supported for renaming. Also can't proceed |
1233 if the original register is one of the fixed_regs, global_regs or | 1234 if the original register is one of the fixed_regs, global_regs or |
1234 frame pointer. */ | 1235 frame pointer. */ |
1235 if (fixed_regs[regno] | 1236 if (fixed_regs[regno] |
1236 || global_regs[regno] | 1237 || global_regs[regno] |
1237 #if FRAME_POINTER_REGNUM != HARD_FRAME_POINTER_REGNUM | 1238 #if FRAME_POINTER_REGNUM != HARD_FRAME_POINTER_REGNUM |
1238 || (frame_pointer_needed && regno == HARD_FRAME_POINTER_REGNUM) | 1239 || (frame_pointer_needed && regno == HARD_FRAME_POINTER_REGNUM) |
1239 #else | 1240 #else |
1240 || (frame_pointer_needed && regno == FRAME_POINTER_REGNUM) | 1241 || (frame_pointer_needed && regno == FRAME_POINTER_REGNUM) |
1249 | 1250 |
1250 return; | 1251 return; |
1251 } | 1252 } |
1252 | 1253 |
1253 /* If something allocated on stack in this function, mark frame pointer | 1254 /* If something allocated on stack in this function, mark frame pointer |
1254 register unavailable, considering also modes. | 1255 register unavailable, considering also modes. |
1255 FIXME: it is enough to do this once per all original defs. */ | 1256 FIXME: it is enough to do this once per all original defs. */ |
1256 if (frame_pointer_needed) | 1257 if (frame_pointer_needed) |
1257 { | 1258 { |
1258 int i; | 1259 int i; |
1259 | 1260 |
1260 for (i = hard_regno_nregs[FRAME_POINTER_REGNUM][Pmode]; i--;) | 1261 for (i = hard_regno_nregs[FRAME_POINTER_REGNUM][Pmode]; i--;) |
1261 SET_HARD_REG_BIT (reg_rename_p->unavailable_hard_regs, | 1262 SET_HARD_REG_BIT (reg_rename_p->unavailable_hard_regs, |
1262 FRAME_POINTER_REGNUM + i); | 1263 FRAME_POINTER_REGNUM + i); |
1263 | 1264 |
1264 #if FRAME_POINTER_REGNUM != HARD_FRAME_POINTER_REGNUM | 1265 #if FRAME_POINTER_REGNUM != HARD_FRAME_POINTER_REGNUM |
1265 for (i = hard_regno_nregs[HARD_FRAME_POINTER_REGNUM][Pmode]; i--;) | 1266 for (i = hard_regno_nregs[HARD_FRAME_POINTER_REGNUM][Pmode]; i--;) |
1266 SET_HARD_REG_BIT (reg_rename_p->unavailable_hard_regs, | 1267 SET_HARD_REG_BIT (reg_rename_p->unavailable_hard_regs, |
1267 HARD_FRAME_POINTER_REGNUM + i); | 1268 HARD_FRAME_POINTER_REGNUM + i); |
1268 #endif | 1269 #endif |
1269 } | 1270 } |
1270 | 1271 |
1271 #ifdef STACK_REGS | 1272 #ifdef STACK_REGS |
1272 /* For the stack registers the presence of FIRST_STACK_REG in USED_REGS | 1273 /* For the stack registers the presence of FIRST_STACK_REG in USED_REGS |
1273 is equivalent to as if all stack regs were in this set. | 1274 is equivalent to as if all stack regs were in this set. |
1274 I.e. no stack register can be renamed, and even if it's an original | 1275 I.e. no stack register can be renamed, and even if it's an original |
1275 register here we make sure it won't be lifted over it's previous def | 1276 register here we make sure it won't be lifted over it's previous def |
1276 (it's previous def will appear as if it's a FIRST_STACK_REG def. | 1277 (it's previous def will appear as if it's a FIRST_STACK_REG def. |
1277 The HARD_REGNO_RENAME_OK covers other cases in condition below. */ | 1278 The HARD_REGNO_RENAME_OK covers other cases in condition below. */ |
1278 if (IN_RANGE (REGNO (orig_dest), FIRST_STACK_REG, LAST_STACK_REG) | 1279 if (IN_RANGE (REGNO (orig_dest), FIRST_STACK_REG, LAST_STACK_REG) |
1279 && REGNO_REG_SET_P (used_regs, FIRST_STACK_REG)) | 1280 && REGNO_REG_SET_P (used_regs, FIRST_STACK_REG)) |
1280 IOR_HARD_REG_SET (reg_rename_p->unavailable_hard_regs, | 1281 IOR_HARD_REG_SET (reg_rename_p->unavailable_hard_regs, |
1281 sel_hrd.stack_regs); | 1282 sel_hrd.stack_regs); |
1282 #endif | 1283 #endif |
1283 | 1284 |
1284 /* If there's a call on this path, make regs from call_used_reg_set | 1285 /* If there's a call on this path, make regs from call_used_reg_set |
1285 unavailable. */ | 1286 unavailable. */ |
1286 if (def->crosses_call) | 1287 if (def->crosses_call) |
1287 IOR_HARD_REG_SET (reg_rename_p->unavailable_hard_regs, | 1288 IOR_HARD_REG_SET (reg_rename_p->unavailable_hard_regs, |
1288 call_used_reg_set); | 1289 call_used_reg_set); |
1289 | 1290 |
1290 /* Stop here before reload: we need FRAME_REGS, STACK_REGS, and crosses_call, | 1291 /* Stop here before reload: we need FRAME_REGS, STACK_REGS, and crosses_call, |
1291 but not register classes. */ | 1292 but not register classes. */ |
1292 if (!reload_completed) | 1293 if (!reload_completed) |
1293 return; | 1294 return; |
1294 | 1295 |
1295 /* Leave regs as 'available' only from the current | 1296 /* Leave regs as 'available' only from the current |
1296 register class. */ | 1297 register class. */ |
1297 cl = get_reg_class (def->orig_insn); | 1298 cl = get_reg_class (def->orig_insn); |
1298 gcc_assert (cl != NO_REGS); | 1299 gcc_assert (cl != NO_REGS); |
1299 COPY_HARD_REG_SET (reg_rename_p->available_for_renaming, | 1300 COPY_HARD_REG_SET (reg_rename_p->available_for_renaming, |
1300 reg_class_contents[cl]); | 1301 reg_class_contents[cl]); |
1301 | 1302 |
1302 /* Leave only registers available for this mode. */ | 1303 /* Leave only registers available for this mode. */ |
1303 if (!sel_hrd.regs_for_mode_ok[mode]) | 1304 if (!sel_hrd.regs_for_mode_ok[mode]) |
1304 init_regs_for_mode (mode); | 1305 init_regs_for_mode (mode); |
1305 AND_HARD_REG_SET (reg_rename_p->available_for_renaming, | 1306 AND_HARD_REG_SET (reg_rename_p->available_for_renaming, |
1306 sel_hrd.regs_for_mode[mode]); | 1307 sel_hrd.regs_for_mode[mode]); |
1307 | 1308 |
1308 /* Exclude registers that are partially call clobbered. */ | 1309 /* Exclude registers that are partially call clobbered. */ |
1309 if (def->crosses_call | 1310 if (def->crosses_call |
1310 && ! HARD_REGNO_CALL_PART_CLOBBERED (regno, mode)) | 1311 && ! HARD_REGNO_CALL_PART_CLOBBERED (regno, mode)) |
1311 AND_COMPL_HARD_REG_SET (reg_rename_p->available_for_renaming, | 1312 AND_COMPL_HARD_REG_SET (reg_rename_p->available_for_renaming, |
1312 sel_hrd.regs_for_call_clobbered[mode]); | 1313 sel_hrd.regs_for_call_clobbered[mode]); |
1313 | 1314 |
1314 /* Leave only those that are ok to rename. */ | 1315 /* Leave only those that are ok to rename. */ |
1315 EXECUTE_IF_SET_IN_HARD_REG_SET (reg_rename_p->available_for_renaming, | 1316 EXECUTE_IF_SET_IN_HARD_REG_SET (reg_rename_p->available_for_renaming, |
1316 0, cur_reg, hrsi) | 1317 0, cur_reg, hrsi) |
1323 | 1324 |
1324 for (i = nregs - 1; i >= 0; --i) | 1325 for (i = nregs - 1; i >= 0; --i) |
1325 if (! sel_hard_regno_rename_ok (regno + i, cur_reg + i)) | 1326 if (! sel_hard_regno_rename_ok (regno + i, cur_reg + i)) |
1326 break; | 1327 break; |
1327 | 1328 |
1328 if (i >= 0) | 1329 if (i >= 0) |
1329 CLEAR_HARD_REG_BIT (reg_rename_p->available_for_renaming, | 1330 CLEAR_HARD_REG_BIT (reg_rename_p->available_for_renaming, |
1330 cur_reg); | 1331 cur_reg); |
1331 } | 1332 } |
1332 | 1333 |
1333 AND_COMPL_HARD_REG_SET (reg_rename_p->available_for_renaming, | 1334 AND_COMPL_HARD_REG_SET (reg_rename_p->available_for_renaming, |
1334 reg_rename_p->unavailable_hard_regs); | 1335 reg_rename_p->unavailable_hard_regs); |
1335 | 1336 |
1336 /* Regno is always ok from the renaming part of view, but it really | 1337 /* Regno is always ok from the renaming part of view, but it really |
1337 could be in *unavailable_hard_regs already, so set it here instead | 1338 could be in *unavailable_hard_regs already, so set it here instead |
1338 of there. */ | 1339 of there. */ |
1344 static int reg_rename_tick[FIRST_PSEUDO_REGISTER]; | 1345 static int reg_rename_tick[FIRST_PSEUDO_REGISTER]; |
1345 | 1346 |
1346 /* Indicates the number of times renaming happened before the current one. */ | 1347 /* Indicates the number of times renaming happened before the current one. */ |
1347 static int reg_rename_this_tick; | 1348 static int reg_rename_this_tick; |
1348 | 1349 |
1349 /* Choose the register among free, that is suitable for storing | 1350 /* Choose the register among free, that is suitable for storing |
1350 the rhs value. | 1351 the rhs value. |
1351 | 1352 |
1352 ORIGINAL_INSNS is the list of insns where the operation (rhs) | 1353 ORIGINAL_INSNS is the list of insns where the operation (rhs) |
1353 originally appears. There could be multiple original operations | 1354 originally appears. There could be multiple original operations |
1354 for single rhs since we moving it up and merging along different | 1355 for single rhs since we moving it up and merging along different |
1355 paths. | 1356 paths. |
1356 | 1357 |
1357 Some code is adapted from regrename.c (regrename_optimize). | 1358 Some code is adapted from regrename.c (regrename_optimize). |
1358 If original register is available, function returns it. | 1359 If original register is available, function returns it. |
1359 Otherwise it performs the checks, so the new register should | 1360 Otherwise it performs the checks, so the new register should |
1360 comply with the following: | 1361 comply with the following: |
1361 - it should not violate any live ranges (such registers are in | 1362 - it should not violate any live ranges (such registers are in |
1362 REG_RENAME_P->available_for_renaming set); | 1363 REG_RENAME_P->available_for_renaming set); |
1363 - it should not be in the HARD_REGS_USED regset; | 1364 - it should not be in the HARD_REGS_USED regset; |
1364 - it should be in the class compatible with original uses; | 1365 - it should be in the class compatible with original uses; |
1365 - it should not be clobbered through reference with different mode; | 1366 - it should not be clobbered through reference with different mode; |
1366 - if we're in the leaf function, then the new register should | 1367 - if we're in the leaf function, then the new register should |
1367 not be in the LEAF_REGISTERS; | 1368 not be in the LEAF_REGISTERS; |
1368 - etc. | 1369 - etc. |
1369 | 1370 |
1370 If several registers meet the conditions, the register with smallest | 1371 If several registers meet the conditions, the register with smallest |
1371 tick is returned to achieve more even register allocation. | 1372 tick is returned to achieve more even register allocation. |
1372 | 1373 |
1373 If original register seems to be ok, we set *IS_ORIG_REG_P_PTR to true. | 1374 If original register seems to be ok, we set *IS_ORIG_REG_P_PTR to true. |
1374 | 1375 |
1375 If no register satisfies the above conditions, NULL_RTX is returned. */ | 1376 If no register satisfies the above conditions, NULL_RTX is returned. */ |
1376 static rtx | 1377 static rtx |
1377 choose_best_reg_1 (HARD_REG_SET hard_regs_used, | 1378 choose_best_reg_1 (HARD_REG_SET hard_regs_used, |
1378 struct reg_rename *reg_rename_p, | 1379 struct reg_rename *reg_rename_p, |
1379 def_list_t original_insns, bool *is_orig_reg_p_ptr) | 1380 def_list_t original_insns, bool *is_orig_reg_p_ptr) |
1380 { | 1381 { |
1381 int best_new_reg; | 1382 int best_new_reg; |
1382 unsigned cur_reg; | 1383 unsigned cur_reg; |
1383 enum machine_mode mode = VOIDmode; | 1384 enum machine_mode mode = VOIDmode; |
1393 { | 1394 { |
1394 rtx orig_dest = SET_DEST (PATTERN (def->orig_insn)); | 1395 rtx orig_dest = SET_DEST (PATTERN (def->orig_insn)); |
1395 | 1396 |
1396 gcc_assert (REG_P (orig_dest)); | 1397 gcc_assert (REG_P (orig_dest)); |
1397 | 1398 |
1398 /* Check that all original operations have the same mode. | 1399 /* Check that all original operations have the same mode. |
1399 This is done for the next loop; if we'd return from this | 1400 This is done for the next loop; if we'd return from this |
1400 loop, we'd check only part of them, but in this case | 1401 loop, we'd check only part of them, but in this case |
1401 it doesn't matter. */ | 1402 it doesn't matter. */ |
1402 if (mode == VOIDmode) | 1403 if (mode == VOIDmode) |
1403 mode = GET_MODE (orig_dest); | 1404 mode = GET_MODE (orig_dest); |
1404 gcc_assert (mode == GET_MODE (orig_dest)); | 1405 gcc_assert (mode == GET_MODE (orig_dest)); |
1405 | 1406 |
1410 | 1411 |
1411 /* All hard registers are available. */ | 1412 /* All hard registers are available. */ |
1412 if (i == n) | 1413 if (i == n) |
1413 { | 1414 { |
1414 gcc_assert (mode != VOIDmode); | 1415 gcc_assert (mode != VOIDmode); |
1415 | 1416 |
1416 /* Hard registers should not be shared. */ | 1417 /* Hard registers should not be shared. */ |
1417 return gen_rtx_REG (mode, regno); | 1418 return gen_rtx_REG (mode, regno); |
1418 } | 1419 } |
1419 } | 1420 } |
1420 | 1421 |
1421 *is_orig_reg_p_ptr = false; | 1422 *is_orig_reg_p_ptr = false; |
1422 best_new_reg = -1; | 1423 best_new_reg = -1; |
1423 | 1424 |
1424 /* Among all available regs choose the register that was | 1425 /* Among all available regs choose the register that was |
1425 allocated earliest. */ | 1426 allocated earliest. */ |
1426 EXECUTE_IF_SET_IN_HARD_REG_SET (reg_rename_p->available_for_renaming, | 1427 EXECUTE_IF_SET_IN_HARD_REG_SET (reg_rename_p->available_for_renaming, |
1427 0, cur_reg, hrsi) | 1428 0, cur_reg, hrsi) |
1428 if (! TEST_HARD_REG_BIT (hard_regs_used, cur_reg)) | 1429 if (! TEST_HARD_REG_BIT (hard_regs_used, cur_reg)) |
1429 { | 1430 { |
1430 /* All hard registers are available. */ | 1431 /* All hard registers are available. */ |
1431 if (best_new_reg < 0 | 1432 if (best_new_reg < 0 |
1432 || reg_rename_tick[cur_reg] < reg_rename_tick[best_new_reg]) | 1433 || reg_rename_tick[cur_reg] < reg_rename_tick[best_new_reg]) |
1433 { | 1434 { |
1434 best_new_reg = cur_reg; | 1435 best_new_reg = cur_reg; |
1435 | 1436 |
1436 /* Return immediately when we know there's no better reg. */ | 1437 /* Return immediately when we know there's no better reg. */ |
1437 if (! reg_rename_tick[best_new_reg]) | 1438 if (! reg_rename_tick[best_new_reg]) |
1438 break; | 1439 break; |
1439 } | 1440 } |
1440 } | 1441 } |
1450 } | 1451 } |
1451 | 1452 |
1452 /* A wrapper around choose_best_reg_1 () to verify that we make correct | 1453 /* A wrapper around choose_best_reg_1 () to verify that we make correct |
1453 assumptions about available registers in the function. */ | 1454 assumptions about available registers in the function. */ |
1454 static rtx | 1455 static rtx |
1455 choose_best_reg (HARD_REG_SET hard_regs_used, struct reg_rename *reg_rename_p, | 1456 choose_best_reg (HARD_REG_SET hard_regs_used, struct reg_rename *reg_rename_p, |
1456 def_list_t original_insns, bool *is_orig_reg_p_ptr) | 1457 def_list_t original_insns, bool *is_orig_reg_p_ptr) |
1457 { | 1458 { |
1458 rtx best_reg = choose_best_reg_1 (hard_regs_used, reg_rename_p, | 1459 rtx best_reg = choose_best_reg_1 (hard_regs_used, reg_rename_p, |
1459 original_insns, is_orig_reg_p_ptr); | 1460 original_insns, is_orig_reg_p_ptr); |
1460 | 1461 |
1461 gcc_assert (best_reg == NULL_RTX | 1462 gcc_assert (best_reg == NULL_RTX |
1462 || TEST_HARD_REG_BIT (sel_hrd.regs_ever_used, REGNO (best_reg))); | 1463 || TEST_HARD_REG_BIT (sel_hrd.regs_ever_used, REGNO (best_reg))); |
1463 | 1464 |
1464 return best_reg; | 1465 return best_reg; |
1465 } | 1466 } |
1466 | 1467 |
1467 /* Choose the pseudo register for storing rhs value. As this is supposed | 1468 /* Choose the pseudo register for storing rhs value. As this is supposed |
1468 to work before reload, we return either the original register or make | 1469 to work before reload, we return either the original register or make |
1469 the new one. The parameters are the same that in choose_nest_reg_1 | 1470 the new one. The parameters are the same that in choose_nest_reg_1 |
1470 functions, except that USED_REGS may contain pseudos. | 1471 functions, except that USED_REGS may contain pseudos. |
1471 If we work with hard regs, check also REG_RENAME_P->UNAVAILABLE_HARD_REGS. | 1472 If we work with hard regs, check also REG_RENAME_P->UNAVAILABLE_HARD_REGS. |
1472 | 1473 |
1473 TODO: take into account register pressure while doing this. Up to this | 1474 TODO: take into account register pressure while doing this. Up to this |
1474 moment, this function would never return NULL for pseudos, but we should | 1475 moment, this function would never return NULL for pseudos, but we should |
1475 not rely on this. */ | 1476 not rely on this. */ |
1476 static rtx | 1477 static rtx |
1477 choose_best_pseudo_reg (regset used_regs, | 1478 choose_best_pseudo_reg (regset used_regs, |
1478 struct reg_rename *reg_rename_p, | 1479 struct reg_rename *reg_rename_p, |
1479 def_list_t original_insns, bool *is_orig_reg_p_ptr) | 1480 def_list_t original_insns, bool *is_orig_reg_p_ptr) |
1480 { | 1481 { |
1481 def_list_iterator i; | 1482 def_list_iterator i; |
1482 def_t def; | 1483 def_t def; |
1483 enum machine_mode mode = VOIDmode; | 1484 enum machine_mode mode = VOIDmode; |
1484 bool bad_hard_regs = false; | 1485 bool bad_hard_regs = false; |
1485 | 1486 |
1486 /* We should not use this after reload. */ | 1487 /* We should not use this after reload. */ |
1487 gcc_assert (!reload_completed); | 1488 gcc_assert (!reload_completed); |
1488 | 1489 |
1489 /* If original register is available, return it. */ | 1490 /* If original register is available, return it. */ |
1490 *is_orig_reg_p_ptr = true; | 1491 *is_orig_reg_p_ptr = true; |
1491 | 1492 |
1492 FOR_EACH_DEF (def, i, original_insns) | 1493 FOR_EACH_DEF (def, i, original_insns) |
1493 { | 1494 { |
1494 rtx dest = SET_DEST (PATTERN (def->orig_insn)); | 1495 rtx dest = SET_DEST (PATTERN (def->orig_insn)); |
1495 int orig_regno; | 1496 int orig_regno; |
1496 | 1497 |
1497 gcc_assert (REG_P (dest)); | 1498 gcc_assert (REG_P (dest)); |
1498 | 1499 |
1499 /* Check that all original operations have the same mode. */ | 1500 /* Check that all original operations have the same mode. */ |
1500 if (mode == VOIDmode) | 1501 if (mode == VOIDmode) |
1501 mode = GET_MODE (dest); | 1502 mode = GET_MODE (dest); |
1502 else | 1503 else |
1503 gcc_assert (mode == GET_MODE (dest)); | 1504 gcc_assert (mode == GET_MODE (dest)); |
1504 orig_regno = REGNO (dest); | 1505 orig_regno = REGNO (dest); |
1505 | 1506 |
1506 if (!REGNO_REG_SET_P (used_regs, orig_regno)) | 1507 if (!REGNO_REG_SET_P (used_regs, orig_regno)) |
1507 { | 1508 { |
1508 if (orig_regno < FIRST_PSEUDO_REGISTER) | 1509 if (orig_regno < FIRST_PSEUDO_REGISTER) |
1509 { | 1510 { |
1510 gcc_assert (df_regs_ever_live_p (orig_regno)); | 1511 gcc_assert (df_regs_ever_live_p (orig_regno)); |
1511 | 1512 |
1512 /* For hard registers, we have to check hardware imposed | 1513 /* For hard registers, we have to check hardware imposed |
1513 limitations (frame/stack registers, calls crossed). */ | 1514 limitations (frame/stack registers, calls crossed). */ |
1514 if (!TEST_HARD_REG_BIT (reg_rename_p->unavailable_hard_regs, | 1515 if (!TEST_HARD_REG_BIT (reg_rename_p->unavailable_hard_regs, |
1515 orig_regno)) | 1516 orig_regno)) |
1516 { | 1517 { |
1517 /* Don't let register cross a call if it doesn't already | 1518 /* Don't let register cross a call if it doesn't already |
1518 cross one. This condition is written in accordance with | 1519 cross one. This condition is written in accordance with |
1519 that in sched-deps.c sched_analyze_reg(). */ | 1520 that in sched-deps.c sched_analyze_reg(). */ |
1520 if (!reg_rename_p->crosses_call | 1521 if (!reg_rename_p->crosses_call |
1521 || REG_N_CALLS_CROSSED (orig_regno) > 0) | 1522 || REG_N_CALLS_CROSSED (orig_regno) > 0) |
1522 return gen_rtx_REG (mode, orig_regno); | 1523 return gen_rtx_REG (mode, orig_regno); |
1523 } | 1524 } |
1524 | 1525 |
1525 bad_hard_regs = true; | 1526 bad_hard_regs = true; |
1526 } | 1527 } |
1527 else | 1528 else |
1528 return dest; | 1529 return dest; |
1529 } | 1530 } |
1530 } | 1531 } |
1531 | 1532 |
1532 *is_orig_reg_p_ptr = false; | 1533 *is_orig_reg_p_ptr = false; |
1533 | 1534 |
1534 /* We had some original hard registers that couldn't be used. | 1535 /* We had some original hard registers that couldn't be used. |
1535 Those were likely special. Don't try to create a pseudo. */ | 1536 Those were likely special. Don't try to create a pseudo. */ |
1536 if (bad_hard_regs) | 1537 if (bad_hard_regs) |
1537 return NULL_RTX; | 1538 return NULL_RTX; |
1538 | 1539 |
1539 /* We haven't found a register from original operations. Get a new one. | 1540 /* We haven't found a register from original operations. Get a new one. |
1540 FIXME: control register pressure somehow. */ | 1541 FIXME: control register pressure somehow. */ |
1541 { | 1542 { |
1542 rtx new_reg = gen_reg_rtx (mode); | 1543 rtx new_reg = gen_reg_rtx (mode); |
1543 | 1544 |
1544 gcc_assert (mode != VOIDmode); | 1545 gcc_assert (mode != VOIDmode); |
1552 } | 1553 } |
1553 | 1554 |
1554 /* True when target of EXPR is available due to EXPR_TARGET_AVAILABLE, | 1555 /* True when target of EXPR is available due to EXPR_TARGET_AVAILABLE, |
1555 USED_REGS and REG_RENAME_P->UNAVAILABLE_HARD_REGS. */ | 1556 USED_REGS and REG_RENAME_P->UNAVAILABLE_HARD_REGS. */ |
1556 static void | 1557 static void |
1557 verify_target_availability (expr_t expr, regset used_regs, | 1558 verify_target_availability (expr_t expr, regset used_regs, |
1558 struct reg_rename *reg_rename_p) | 1559 struct reg_rename *reg_rename_p) |
1559 { | 1560 { |
1560 unsigned n, i, regno; | 1561 unsigned n, i, regno; |
1561 enum machine_mode mode; | 1562 enum machine_mode mode; |
1562 bool target_available, live_available, hard_available; | 1563 bool target_available, live_available, hard_available; |
1563 | 1564 |
1564 if (!REG_P (EXPR_LHS (expr)) || EXPR_TARGET_AVAILABLE (expr) < 0) | 1565 if (!REG_P (EXPR_LHS (expr)) || EXPR_TARGET_AVAILABLE (expr) < 0) |
1565 return; | 1566 return; |
1566 | 1567 |
1567 regno = expr_dest_regno (expr); | 1568 regno = expr_dest_regno (expr); |
1568 mode = GET_MODE (EXPR_LHS (expr)); | 1569 mode = GET_MODE (EXPR_LHS (expr)); |
1569 target_available = EXPR_TARGET_AVAILABLE (expr) == 1; | 1570 target_available = EXPR_TARGET_AVAILABLE (expr) == 1; |
1570 n = reload_completed ? hard_regno_nregs[regno][mode] : 1; | 1571 n = reload_completed ? hard_regno_nregs[regno][mode] : 1; |
1571 | 1572 |
1576 live_available = false; | 1577 live_available = false; |
1577 if (TEST_HARD_REG_BIT (reg_rename_p->unavailable_hard_regs, regno + i)) | 1578 if (TEST_HARD_REG_BIT (reg_rename_p->unavailable_hard_regs, regno + i)) |
1578 hard_available = false; | 1579 hard_available = false; |
1579 } | 1580 } |
1580 | 1581 |
1581 /* When target is not available, it may be due to hard register | 1582 /* When target is not available, it may be due to hard register |
1582 restrictions, e.g. crosses calls, so we check hard_available too. */ | 1583 restrictions, e.g. crosses calls, so we check hard_available too. */ |
1583 if (target_available) | 1584 if (target_available) |
1584 gcc_assert (live_available); | 1585 gcc_assert (live_available); |
1585 else | 1586 else |
1586 /* Check only if we haven't scheduled something on the previous fence, | 1587 /* Check only if we haven't scheduled something on the previous fence, |
1587 cause due to MAX_SOFTWARE_LOOKAHEAD_WINDOW_SIZE issues | 1588 cause due to MAX_SOFTWARE_LOOKAHEAD_WINDOW_SIZE issues |
1588 and having more than one fence, we may end having targ_un in a block | 1589 and having more than one fence, we may end having targ_un in a block |
1589 in which successors target register is actually available. | 1590 in which successors target register is actually available. |
1590 | 1591 |
1591 The last condition handles the case when a dependence from a call insn | 1592 The last condition handles the case when a dependence from a call insn |
1592 was created in sched-deps.c for insns with destination registers that | 1593 was created in sched-deps.c for insns with destination registers that |
1593 never crossed a call before, but do cross one after our code motion. | 1594 never crossed a call before, but do cross one after our code motion. |
1594 | 1595 |
1595 FIXME: in the latter case, we just uselessly called find_used_regs, | 1596 FIXME: in the latter case, we just uselessly called find_used_regs, |
1596 because we can't move this expression with any other register | 1597 because we can't move this expression with any other register |
1597 as well. */ | 1598 as well. */ |
1598 gcc_assert (scheduled_something_on_previous_fence || !live_available | 1599 gcc_assert (scheduled_something_on_previous_fence || !live_available |
1599 || !hard_available | 1600 || !hard_available |
1600 || (!reload_completed && reg_rename_p->crosses_call | 1601 || (!reload_completed && reg_rename_p->crosses_call |
1601 && REG_N_CALLS_CROSSED (regno) == 0)); | 1602 && REG_N_CALLS_CROSSED (regno) == 0)); |
1602 } | 1603 } |
1603 | 1604 |
1604 /* Collect unavailable registers due to liveness for EXPR from BNDS | 1605 /* Collect unavailable registers due to liveness for EXPR from BNDS |
1605 into USED_REGS. Save additional information about available | 1606 into USED_REGS. Save additional information about available |
1606 registers and unavailable due to hardware restriction registers | 1607 registers and unavailable due to hardware restriction registers |
1607 into REG_RENAME_P structure. Save original insns into ORIGINAL_INSNS | 1608 into REG_RENAME_P structure. Save original insns into ORIGINAL_INSNS |
1608 list. */ | 1609 list. */ |
1609 static void | 1610 static void |
1610 collect_unavailable_regs_from_bnds (expr_t expr, blist_t bnds, regset used_regs, | 1611 collect_unavailable_regs_from_bnds (expr_t expr, blist_t bnds, regset used_regs, |
1666 register. */ | 1667 register. */ |
1667 replace_dest_with_reg_in_expr (expr, best_reg); | 1668 replace_dest_with_reg_in_expr (expr, best_reg); |
1668 return true; | 1669 return true; |
1669 } | 1670 } |
1670 | 1671 |
1671 /* Select and assign best register to EXPR searching from BNDS. | 1672 /* Select and assign best register to EXPR searching from BNDS. |
1672 Set *IS_ORIG_REG_P to TRUE if original register was selected. | 1673 Set *IS_ORIG_REG_P to TRUE if original register was selected. |
1673 Return FALSE if no register can be chosen, which could happen when: | 1674 Return FALSE if no register can be chosen, which could happen when: |
1674 * EXPR_SEPARABLE_P is true but we were unable to find suitable register; | 1675 * EXPR_SEPARABLE_P is true but we were unable to find suitable register; |
1675 * EXPR_SEPARABLE_P is false but the insn sets/clobbers one of the registers | 1676 * EXPR_SEPARABLE_P is false but the insn sets/clobbers one of the registers |
1676 that are used on the moving path. */ | 1677 that are used on the moving path. */ |
1677 static bool | 1678 static bool |
1696 collect_unavailable_regs_from_bnds (expr, bnds, used_regs, ®_rename_data, | 1697 collect_unavailable_regs_from_bnds (expr, bnds, used_regs, ®_rename_data, |
1697 &original_insns); | 1698 &original_insns); |
1698 | 1699 |
1699 #ifdef ENABLE_CHECKING | 1700 #ifdef ENABLE_CHECKING |
1700 /* If after reload, make sure we're working with hard regs here. */ | 1701 /* If after reload, make sure we're working with hard regs here. */ |
1701 if (reload_completed) | 1702 if (reload_completed) |
1702 { | 1703 { |
1703 reg_set_iterator rsi; | 1704 reg_set_iterator rsi; |
1704 unsigned i; | 1705 unsigned i; |
1705 | 1706 |
1706 EXECUTE_IF_SET_IN_REG_SET (used_regs, FIRST_PSEUDO_REGISTER, i, rsi) | 1707 EXECUTE_IF_SET_IN_REG_SET (used_regs, FIRST_PSEUDO_REGISTER, i, rsi) |
1707 gcc_unreachable (); | 1708 gcc_unreachable (); |
1708 } | 1709 } |
1709 #endif | 1710 #endif |
1710 | 1711 |
1873 In case of control speculation we must convert C_EXPR to control | 1874 In case of control speculation we must convert C_EXPR to control |
1874 speculative mode, because failing to do so will bring us an exception | 1875 speculative mode, because failing to do so will bring us an exception |
1875 thrown by the non-control-speculative load. */ | 1876 thrown by the non-control-speculative load. */ |
1876 check_ds = ds_get_max_dep_weak (check_ds); | 1877 check_ds = ds_get_max_dep_weak (check_ds); |
1877 speculate_expr (c_expr, check_ds); | 1878 speculate_expr (c_expr, check_ds); |
1878 | 1879 |
1879 return insn; | 1880 return insn; |
1880 } | 1881 } |
1881 | 1882 |
1882 /* True when INSN is a "regN = regN" copy. */ | 1883 /* True when INSN is a "regN = regN" copy. */ |
1883 static bool | 1884 static bool |
1899 return false; | 1900 return false; |
1900 | 1901 |
1901 return REGNO (lhs) == REGNO (rhs); | 1902 return REGNO (lhs) == REGNO (rhs); |
1902 } | 1903 } |
1903 | 1904 |
1904 /* Undo all transformations on *AV_PTR that were done when | 1905 /* Undo all transformations on *AV_PTR that were done when |
1905 moving through INSN. */ | 1906 moving through INSN. */ |
1906 static void | 1907 static void |
1907 undo_transformations (av_set_t *av_ptr, rtx insn) | 1908 undo_transformations (av_set_t *av_ptr, rtx insn) |
1908 { | 1909 { |
1909 av_set_iterator av_iter; | 1910 av_set_iterator av_iter; |
1910 expr_t expr; | 1911 expr_t expr; |
1911 av_set_t new_set = NULL; | 1912 av_set_t new_set = NULL; |
1912 | 1913 |
1913 /* First, kill any EXPR that uses registers set by an insn. This is | 1914 /* First, kill any EXPR that uses registers set by an insn. This is |
1914 required for correctness. */ | 1915 required for correctness. */ |
1915 FOR_EACH_EXPR_1 (expr, av_iter, av_ptr) | 1916 FOR_EACH_EXPR_1 (expr, av_iter, av_ptr) |
1916 if (!sched_insns_conditions_mutex_p (insn, EXPR_INSN_RTX (expr)) | 1917 if (!sched_insns_conditions_mutex_p (insn, EXPR_INSN_RTX (expr)) |
1917 && bitmap_intersect_p (INSN_REG_SETS (insn), | 1918 && bitmap_intersect_p (INSN_REG_SETS (insn), |
1918 VINSN_REG_USES (EXPR_VINSN (expr))) | 1919 VINSN_REG_USES (EXPR_VINSN (expr))) |
1919 /* When an insn looks like 'r1 = r1', we could substitute through | 1920 /* When an insn looks like 'r1 = r1', we could substitute through |
1920 it, but the above condition will still hold. This happened with | 1921 it, but the above condition will still hold. This happened with |
1921 gcc.c-torture/execute/961125-1.c. */ | 1922 gcc.c-torture/execute/961125-1.c. */ |
1922 && !identical_copy_p (insn)) | 1923 && !identical_copy_p (insn)) |
1923 { | 1924 { |
1924 if (sched_verbose >= 6) | 1925 if (sched_verbose >= 6) |
1925 sel_print ("Expr %d removed due to use/set conflict\n", | 1926 sel_print ("Expr %d removed due to use/set conflict\n", |
1926 INSN_UID (EXPR_INSN_RTX (expr))); | 1927 INSN_UID (EXPR_INSN_RTX (expr))); |
1927 av_set_iter_remove (&av_iter); | 1928 av_set_iter_remove (&av_iter); |
1928 } | 1929 } |
1929 | 1930 |
1930 /* Undo transformations looking at the history vector. */ | 1931 /* Undo transformations looking at the history vector. */ |
1935 | 1936 |
1936 if (index >= 0) | 1937 if (index >= 0) |
1937 { | 1938 { |
1938 expr_history_def *phist; | 1939 expr_history_def *phist; |
1939 | 1940 |
1940 phist = VEC_index (expr_history_def, | 1941 phist = VEC_index (expr_history_def, |
1941 EXPR_HISTORY_OF_CHANGES (expr), | 1942 EXPR_HISTORY_OF_CHANGES (expr), |
1942 index); | 1943 index); |
1943 | 1944 |
1944 switch (phist->type) | 1945 switch (phist->type) |
1945 { | 1946 { |
1946 case TRANS_SPECULATION: | 1947 case TRANS_SPECULATION: |
1947 { | 1948 { |
1948 ds_t old_ds, new_ds; | 1949 ds_t old_ds, new_ds; |
1949 | 1950 |
1950 /* Compute the difference between old and new speculative | 1951 /* Compute the difference between old and new speculative |
1951 statuses: that's what we need to check. | 1952 statuses: that's what we need to check. |
1952 Earlier we used to assert that the status will really | 1953 Earlier we used to assert that the status will really |
1953 change. This no longer works because only the probability | 1954 change. This no longer works because only the probability |
1954 bits in the status may have changed during compute_av_set, | 1955 bits in the status may have changed during compute_av_set, |
1955 and in the case of merging different probabilities of the | 1956 and in the case of merging different probabilities of the |
1956 same speculative status along different paths we do not | 1957 same speculative status along different paths we do not |
1957 record this in the history vector. */ | 1958 record this in the history vector. */ |
1958 old_ds = phist->spec_ds; | 1959 old_ds = phist->spec_ds; |
1959 new_ds = EXPR_SPEC_DONE_DS (expr); | 1960 new_ds = EXPR_SPEC_DONE_DS (expr); |
1960 | 1961 |
1961 old_ds &= SPECULATIVE; | 1962 old_ds &= SPECULATIVE; |
1962 new_ds &= SPECULATIVE; | 1963 new_ds &= SPECULATIVE; |
1963 new_ds &= ~old_ds; | 1964 new_ds &= ~old_ds; |
1964 | 1965 |
1965 EXPR_SPEC_TO_CHECK_DS (expr) |= new_ds; | 1966 EXPR_SPEC_TO_CHECK_DS (expr) |= new_ds; |
1966 break; | 1967 break; |
1967 } | 1968 } |
1968 case TRANS_SUBSTITUTION: | 1969 case TRANS_SUBSTITUTION: |
1969 { | 1970 { |
1970 expr_def _tmp_expr, *tmp_expr = &_tmp_expr; | 1971 expr_def _tmp_expr, *tmp_expr = &_tmp_expr; |
1971 vinsn_t new_vi; | 1972 vinsn_t new_vi; |
1972 bool add = true; | 1973 bool add = true; |
1973 | 1974 |
1974 new_vi = phist->old_expr_vinsn; | 1975 new_vi = phist->old_expr_vinsn; |
1975 | 1976 |
1976 gcc_assert (VINSN_SEPARABLE_P (new_vi) | 1977 gcc_assert (VINSN_SEPARABLE_P (new_vi) |
1977 == EXPR_SEPARABLE_P (expr)); | 1978 == EXPR_SEPARABLE_P (expr)); |
1978 copy_expr (tmp_expr, expr); | 1979 copy_expr (tmp_expr, expr); |
1979 | 1980 |
1980 if (vinsn_equal_p (phist->new_expr_vinsn, | 1981 if (vinsn_equal_p (phist->new_expr_vinsn, |
1981 EXPR_VINSN (tmp_expr))) | 1982 EXPR_VINSN (tmp_expr))) |
1982 change_vinsn_in_expr (tmp_expr, new_vi); | 1983 change_vinsn_in_expr (tmp_expr, new_vi); |
1983 else | 1984 else |
1984 /* This happens when we're unsubstituting on a bookkeeping | 1985 /* This happens when we're unsubstituting on a bookkeeping |
1985 copy, which was in turn substituted. The history is wrong | 1986 copy, which was in turn substituted. The history is wrong |
1992 } | 1993 } |
1993 default: | 1994 default: |
1994 gcc_unreachable (); | 1995 gcc_unreachable (); |
1995 } | 1996 } |
1996 } | 1997 } |
1997 | 1998 |
1998 } | 1999 } |
1999 | 2000 |
2000 av_set_union_and_clear (av_ptr, &new_set, NULL); | 2001 av_set_union_and_clear (av_ptr, &new_set, NULL); |
2001 } | 2002 } |
2002 | 2003 |
2003 | 2004 |
2004 /* Moveup_* helpers for code motion and computing av sets. */ | 2005 /* Moveup_* helpers for code motion and computing av sets. */ |
2005 | 2006 |
2006 /* Propagates EXPR inside an insn group through THROUGH_INSN. | 2007 /* Propagates EXPR inside an insn group through THROUGH_INSN. |
2007 The difference from the below function is that only substitution is | 2008 The difference from the below function is that only substitution is |
2008 performed. */ | 2009 performed. */ |
2009 static enum MOVEUP_EXPR_CODE | 2010 static enum MOVEUP_EXPR_CODE |
2010 moveup_expr_inside_insn_group (expr_t expr, insn_t through_insn) | 2011 moveup_expr_inside_insn_group (expr_t expr, insn_t through_insn) |
2011 { | 2012 { |
2012 vinsn_t vi = EXPR_VINSN (expr); | 2013 vinsn_t vi = EXPR_VINSN (expr); |
2023 /* Substitution is the possible choice in this case. */ | 2024 /* Substitution is the possible choice in this case. */ |
2024 if (has_dep_p[DEPS_IN_RHS]) | 2025 if (has_dep_p[DEPS_IN_RHS]) |
2025 { | 2026 { |
2026 /* Can't substitute UNIQUE VINSNs. */ | 2027 /* Can't substitute UNIQUE VINSNs. */ |
2027 gcc_assert (!VINSN_UNIQUE_P (vi)); | 2028 gcc_assert (!VINSN_UNIQUE_P (vi)); |
2028 | 2029 |
2029 if (can_substitute_through_p (through_insn, | 2030 if (can_substitute_through_p (through_insn, |
2030 has_dep_p[DEPS_IN_RHS]) | 2031 has_dep_p[DEPS_IN_RHS]) |
2031 && substitute_reg_in_expr (expr, through_insn, false)) | 2032 && substitute_reg_in_expr (expr, through_insn, false)) |
2032 { | 2033 { |
2033 EXPR_WAS_SUBSTITUTED (expr) = true; | 2034 EXPR_WAS_SUBSTITUTED (expr) = true; |
2034 return MOVEUP_EXPR_CHANGED; | 2035 return MOVEUP_EXPR_CHANGED; |
2040 } | 2041 } |
2041 | 2042 |
2042 /* This can catch output dependencies in COND_EXECs. */ | 2043 /* This can catch output dependencies in COND_EXECs. */ |
2043 if (has_dep_p[DEPS_IN_INSN]) | 2044 if (has_dep_p[DEPS_IN_INSN]) |
2044 return MOVEUP_EXPR_NULL; | 2045 return MOVEUP_EXPR_NULL; |
2045 | 2046 |
2046 /* This is either an output or an anti dependence, which usually have | 2047 /* This is either an output or an anti dependence, which usually have |
2047 a zero latency. Allow this here, if we'd be wrong, tick_check_p | 2048 a zero latency. Allow this here, if we'd be wrong, tick_check_p |
2048 will fix this. */ | 2049 will fix this. */ |
2049 gcc_assert (has_dep_p[DEPS_IN_LHS]); | 2050 gcc_assert (has_dep_p[DEPS_IN_LHS]); |
2050 return MOVEUP_EXPR_AS_RHS; | 2051 return MOVEUP_EXPR_AS_RHS; |
2057 && !sel_insn_is_speculation_check (through_insn)) | 2058 && !sel_insn_is_speculation_check (through_insn)) |
2058 | 2059 |
2059 /* True when a conflict on a target register was found during moveup_expr. */ | 2060 /* True when a conflict on a target register was found during moveup_expr. */ |
2060 static bool was_target_conflict = false; | 2061 static bool was_target_conflict = false; |
2061 | 2062 |
2063 /* Return true when moving a debug INSN across THROUGH_INSN will | |
2064 create a bookkeeping block. We don't want to create such blocks, | |
2065 for they would cause codegen differences between compilations with | |
2066 and without debug info. */ | |
2067 | |
2068 static bool | |
2069 moving_insn_creates_bookkeeping_block_p (insn_t insn, | |
2070 insn_t through_insn) | |
2071 { | |
2072 basic_block bbi, bbt; | |
2073 edge e1, e2; | |
2074 edge_iterator ei1, ei2; | |
2075 | |
2076 if (!bookkeeping_can_be_created_if_moved_through_p (through_insn)) | |
2077 { | |
2078 if (sched_verbose >= 9) | |
2079 sel_print ("no bookkeeping required: "); | |
2080 return FALSE; | |
2081 } | |
2082 | |
2083 bbi = BLOCK_FOR_INSN (insn); | |
2084 | |
2085 if (EDGE_COUNT (bbi->preds) == 1) | |
2086 { | |
2087 if (sched_verbose >= 9) | |
2088 sel_print ("only one pred edge: "); | |
2089 return TRUE; | |
2090 } | |
2091 | |
2092 bbt = BLOCK_FOR_INSN (through_insn); | |
2093 | |
2094 FOR_EACH_EDGE (e1, ei1, bbt->succs) | |
2095 { | |
2096 FOR_EACH_EDGE (e2, ei2, bbi->preds) | |
2097 { | |
2098 if (find_block_for_bookkeeping (e1, e2, TRUE)) | |
2099 { | |
2100 if (sched_verbose >= 9) | |
2101 sel_print ("found existing block: "); | |
2102 return FALSE; | |
2103 } | |
2104 } | |
2105 } | |
2106 | |
2107 if (sched_verbose >= 9) | |
2108 sel_print ("would create bookkeeping block: "); | |
2109 | |
2110 return TRUE; | |
2111 } | |
2112 | |
2062 /* Modifies EXPR so it can be moved through the THROUGH_INSN, | 2113 /* Modifies EXPR so it can be moved through the THROUGH_INSN, |
2063 performing necessary transformations. Record the type of transformation | 2114 performing necessary transformations. Record the type of transformation |
2064 made in PTRANS_TYPE, when it is not NULL. When INSIDE_INSN_GROUP, | 2115 made in PTRANS_TYPE, when it is not NULL. When INSIDE_INSN_GROUP, |
2065 permit all dependencies except true ones, and try to remove those | 2116 permit all dependencies except true ones, and try to remove those |
2066 too via forward substitution. All cases when a non-eliminable | 2117 too via forward substitution. All cases when a non-eliminable |
2067 non-zero cost dependency exists inside an insn group will be fixed | 2118 non-zero cost dependency exists inside an insn group will be fixed |
2068 in tick_check_p instead. */ | 2119 in tick_check_p instead. */ |
2069 static enum MOVEUP_EXPR_CODE | 2120 static enum MOVEUP_EXPR_CODE |
2070 moveup_expr (expr_t expr, insn_t through_insn, bool inside_insn_group, | 2121 moveup_expr (expr_t expr, insn_t through_insn, bool inside_insn_group, |
2071 enum local_trans_type *ptrans_type) | 2122 enum local_trans_type *ptrans_type) |
2072 { | 2123 { |
2089 dependencies allow to do so and jump is not speculative). */ | 2140 dependencies allow to do so and jump is not speculative). */ |
2090 if (control_flow_insn_p (insn)) | 2141 if (control_flow_insn_p (insn)) |
2091 { | 2142 { |
2092 basic_block fallthru_bb; | 2143 basic_block fallthru_bb; |
2093 | 2144 |
2094 /* Do not move checks and do not move jumps through other | 2145 /* Do not move checks and do not move jumps through other |
2095 jumps. */ | 2146 jumps. */ |
2096 if (control_flow_insn_p (through_insn) | 2147 if (control_flow_insn_p (through_insn) |
2097 || sel_insn_is_speculation_check (insn)) | 2148 || sel_insn_is_speculation_check (insn)) |
2098 return MOVEUP_EXPR_NULL; | 2149 return MOVEUP_EXPR_NULL; |
2099 | 2150 |
2100 /* Don't move jumps through CFG joins. */ | 2151 /* Don't move jumps through CFG joins. */ |
2101 if (bookkeeping_can_be_created_if_moved_through_p (through_insn)) | 2152 if (bookkeeping_can_be_created_if_moved_through_p (through_insn)) |
2102 return MOVEUP_EXPR_NULL; | 2153 return MOVEUP_EXPR_NULL; |
2103 | 2154 |
2104 /* The jump should have a clear fallthru block, and | 2155 /* The jump should have a clear fallthru block, and |
2105 this block should be in the current region. */ | 2156 this block should be in the current region. */ |
2106 if ((fallthru_bb = fallthru_bb_of_jump (insn)) == NULL | 2157 if ((fallthru_bb = fallthru_bb_of_jump (insn)) == NULL |
2107 || ! in_current_region_p (fallthru_bb)) | 2158 || ! in_current_region_p (fallthru_bb)) |
2108 return MOVEUP_EXPR_NULL; | 2159 return MOVEUP_EXPR_NULL; |
2109 | 2160 |
2110 /* And it should be mutually exclusive with through_insn, or | 2161 /* And it should be mutually exclusive with through_insn, or |
2111 be an unconditional jump. */ | 2162 be an unconditional jump. */ |
2112 if (! any_uncondjump_p (insn) | 2163 if (! any_uncondjump_p (insn) |
2113 && ! sched_insns_conditions_mutex_p (insn, through_insn)) | 2164 && ! sched_insns_conditions_mutex_p (insn, through_insn) |
2165 && ! DEBUG_INSN_P (through_insn)) | |
2114 return MOVEUP_EXPR_NULL; | 2166 return MOVEUP_EXPR_NULL; |
2115 } | 2167 } |
2116 | 2168 |
2117 /* Don't move what we can't move. */ | 2169 /* Don't move what we can't move. */ |
2118 if (EXPR_CANT_MOVE (expr) | 2170 if (EXPR_CANT_MOVE (expr) |
2129 return MOVEUP_EXPR_NULL; | 2181 return MOVEUP_EXPR_NULL; |
2130 } | 2182 } |
2131 else | 2183 else |
2132 gcc_assert (!control_flow_insn_p (insn)); | 2184 gcc_assert (!control_flow_insn_p (insn)); |
2133 | 2185 |
2186 /* Don't move debug insns if this would require bookkeeping. */ | |
2187 if (DEBUG_INSN_P (insn) | |
2188 && BLOCK_FOR_INSN (through_insn) != BLOCK_FOR_INSN (insn) | |
2189 && moving_insn_creates_bookkeeping_block_p (insn, through_insn)) | |
2190 return MOVEUP_EXPR_NULL; | |
2191 | |
2134 /* Deal with data dependencies. */ | 2192 /* Deal with data dependencies. */ |
2135 was_target_conflict = false; | 2193 was_target_conflict = false; |
2136 full_ds = has_dependence_p (expr, through_insn, &has_dep_p); | 2194 full_ds = has_dependence_p (expr, through_insn, &has_dep_p); |
2137 if (full_ds == 0) | 2195 if (full_ds == 0) |
2138 { | 2196 { |
2139 if (!CANT_MOVE_TRAPPING (expr, through_insn)) | 2197 if (!CANT_MOVE_TRAPPING (expr, through_insn)) |
2140 return MOVEUP_EXPR_SAME; | 2198 return MOVEUP_EXPR_SAME; |
2141 } | 2199 } |
2142 else | 2200 else |
2143 { | 2201 { |
2144 /* We can move UNIQUE insn up only as a whole and unchanged, | 2202 /* We can move UNIQUE insn up only as a whole and unchanged, |
2145 so it shouldn't have any dependencies. */ | 2203 so it shouldn't have any dependencies. */ |
2146 if (VINSN_UNIQUE_P (vi)) | 2204 if (VINSN_UNIQUE_P (vi)) |
2147 return MOVEUP_EXPR_NULL; | 2205 return MOVEUP_EXPR_NULL; |
2148 } | 2206 } |
2149 | 2207 |
2168 if (has_dep_p[DEPS_IN_INSN]) | 2226 if (has_dep_p[DEPS_IN_INSN]) |
2169 /* We have some dependency that cannot be discarded. */ | 2227 /* We have some dependency that cannot be discarded. */ |
2170 return MOVEUP_EXPR_NULL; | 2228 return MOVEUP_EXPR_NULL; |
2171 | 2229 |
2172 if (has_dep_p[DEPS_IN_LHS]) | 2230 if (has_dep_p[DEPS_IN_LHS]) |
2173 { | 2231 { |
2174 /* Only separable insns can be moved up with the new register. | 2232 /* Only separable insns can be moved up with the new register. |
2175 Anyways, we should mark that the original register is | 2233 Anyways, we should mark that the original register is |
2176 unavailable. */ | 2234 unavailable. */ |
2177 if (!enable_schedule_as_rhs_p || !EXPR_SEPARABLE_P (expr)) | 2235 if (!enable_schedule_as_rhs_p || !EXPR_SEPARABLE_P (expr)) |
2178 return MOVEUP_EXPR_NULL; | 2236 return MOVEUP_EXPR_NULL; |
2179 | 2237 |
2180 EXPR_TARGET_AVAILABLE (expr) = false; | 2238 EXPR_TARGET_AVAILABLE (expr) = false; |
2189 | 2247 |
2190 Ex. 1: Ex.2 | 2248 Ex. 1: Ex.2 |
2191 y = x; y = x; | 2249 y = x; y = x; |
2192 z = y*2; y = y*2; | 2250 z = y*2; y = y*2; |
2193 | 2251 |
2194 In Ex.1 y*2 can be substituted for x*2 and the whole operation can be | 2252 In Ex.1 y*2 can be substituted for x*2 and the whole operation can be |
2195 moved above y=x assignment as z=x*2. | 2253 moved above y=x assignment as z=x*2. |
2196 | 2254 |
2197 In Ex.2 y*2 also can be substituted for x*2, but only the right hand | 2255 In Ex.2 y*2 also can be substituted for x*2, but only the right hand |
2198 side can be moved because of the output dependency. The operation was | 2256 side can be moved because of the output dependency. The operation was |
2199 cropped to its rhs above. */ | 2257 cropped to its rhs above. */ |
2200 if (has_dep_p[DEPS_IN_RHS]) | 2258 if (has_dep_p[DEPS_IN_RHS]) |
2201 { | 2259 { |
2202 ds_t *rhs_dsp = &has_dep_p[DEPS_IN_RHS]; | 2260 ds_t *rhs_dsp = &has_dep_p[DEPS_IN_RHS]; |
2205 gcc_assert (!VINSN_UNIQUE_P (vi)); | 2263 gcc_assert (!VINSN_UNIQUE_P (vi)); |
2206 | 2264 |
2207 if (can_speculate_dep_p (*rhs_dsp)) | 2265 if (can_speculate_dep_p (*rhs_dsp)) |
2208 { | 2266 { |
2209 int res; | 2267 int res; |
2210 | 2268 |
2211 res = speculate_expr (expr, *rhs_dsp); | 2269 res = speculate_expr (expr, *rhs_dsp); |
2212 if (res >= 0) | 2270 if (res >= 0) |
2213 { | 2271 { |
2214 /* Speculation was successful. */ | 2272 /* Speculation was successful. */ |
2215 *rhs_dsp = 0; | 2273 *rhs_dsp = 0; |
2242 This check should be at the end to give a chance to control speculation | 2300 This check should be at the end to give a chance to control speculation |
2243 to perform its duties. */ | 2301 to perform its duties. */ |
2244 if (CANT_MOVE_TRAPPING (expr, through_insn)) | 2302 if (CANT_MOVE_TRAPPING (expr, through_insn)) |
2245 return MOVEUP_EXPR_NULL; | 2303 return MOVEUP_EXPR_NULL; |
2246 | 2304 |
2247 return (was_changed | 2305 return (was_changed |
2248 ? MOVEUP_EXPR_CHANGED | 2306 ? MOVEUP_EXPR_CHANGED |
2249 : (as_rhs | 2307 : (as_rhs |
2250 ? MOVEUP_EXPR_AS_RHS | 2308 ? MOVEUP_EXPR_AS_RHS |
2251 : MOVEUP_EXPR_SAME)); | 2309 : MOVEUP_EXPR_SAME)); |
2252 } | 2310 } |
2253 | 2311 |
2254 /* Try to look at bitmap caches for EXPR and INSN pair, return true | 2312 /* Try to look at bitmap caches for EXPR and INSN pair, return true |
2255 if successful. When INSIDE_INSN_GROUP, also try ignore dependencies | 2313 if successful. When INSIDE_INSN_GROUP, also try ignore dependencies |
2256 that can exist within a parallel group. Write to RES the resulting | 2314 that can exist within a parallel group. Write to RES the resulting |
2257 code for moveup_expr. */ | 2315 code for moveup_expr. */ |
2258 static bool | 2316 static bool |
2259 try_bitmap_cache (expr_t expr, insn_t insn, | 2317 try_bitmap_cache (expr_t expr, insn_t insn, |
2260 bool inside_insn_group, | 2318 bool inside_insn_group, |
2261 enum MOVEUP_EXPR_CODE *res) | 2319 enum MOVEUP_EXPR_CODE *res) |
2262 { | 2320 { |
2263 int expr_uid = INSN_UID (EXPR_INSN_RTX (expr)); | 2321 int expr_uid = INSN_UID (EXPR_INSN_RTX (expr)); |
2264 | 2322 |
2265 /* First check whether we've analyzed this situation already. */ | 2323 /* First check whether we've analyzed this situation already. */ |
2266 if (bitmap_bit_p (INSN_ANALYZED_DEPS (insn), expr_uid)) | 2324 if (bitmap_bit_p (INSN_ANALYZED_DEPS (insn), expr_uid)) |
2267 { | 2325 { |
2268 if (bitmap_bit_p (INSN_FOUND_DEPS (insn), expr_uid)) | 2326 if (bitmap_bit_p (INSN_FOUND_DEPS (insn), expr_uid)) |
2269 { | 2327 { |
2286 { | 2344 { |
2287 if (sched_verbose >= 6) | 2345 if (sched_verbose >= 6) |
2288 sel_print ("unchanged (as RHS, cached, inside insn group)\n"); | 2346 sel_print ("unchanged (as RHS, cached, inside insn group)\n"); |
2289 *res = MOVEUP_EXPR_SAME; | 2347 *res = MOVEUP_EXPR_SAME; |
2290 return true; | 2348 return true; |
2291 | 2349 |
2292 } | 2350 } |
2293 else | 2351 else |
2294 EXPR_TARGET_AVAILABLE (expr) = false; | 2352 EXPR_TARGET_AVAILABLE (expr) = false; |
2295 | 2353 |
2296 /* This is the only case when propagation result can change over time, | 2354 /* This is the only case when propagation result can change over time, |
2297 as we can dynamically switch off scheduling as RHS. In this case, | 2355 as we can dynamically switch off scheduling as RHS. In this case, |
2298 just check the flag to reach the correct decision. */ | 2356 just check the flag to reach the correct decision. */ |
2299 if (enable_schedule_as_rhs_p) | 2357 if (enable_schedule_as_rhs_p) |
2300 { | 2358 { |
2301 if (sched_verbose >= 6) | 2359 if (sched_verbose >= 6) |
2302 sel_print ("unchanged (as RHS, cached)\n"); | 2360 sel_print ("unchanged (as RHS, cached)\n"); |
2314 } | 2372 } |
2315 | 2373 |
2316 return false; | 2374 return false; |
2317 } | 2375 } |
2318 | 2376 |
2319 /* Try to look at bitmap caches for EXPR and INSN pair, return true | 2377 /* Try to look at bitmap caches for EXPR and INSN pair, return true |
2320 if successful. Write to RES the resulting code for moveup_expr. */ | 2378 if successful. Write to RES the resulting code for moveup_expr. */ |
2321 static bool | 2379 static bool |
2322 try_transformation_cache (expr_t expr, insn_t insn, | 2380 try_transformation_cache (expr_t expr, insn_t insn, |
2323 enum MOVEUP_EXPR_CODE *res) | 2381 enum MOVEUP_EXPR_CODE *res) |
2324 { | 2382 { |
2325 struct transformed_insns *pti | 2383 struct transformed_insns *pti |
2326 = (struct transformed_insns *) | 2384 = (struct transformed_insns *) |
2327 htab_find_with_hash (INSN_TRANSFORMED_INSNS (insn), | 2385 htab_find_with_hash (INSN_TRANSFORMED_INSNS (insn), |
2328 &EXPR_VINSN (expr), | 2386 &EXPR_VINSN (expr), |
2329 VINSN_HASH_RTX (EXPR_VINSN (expr))); | 2387 VINSN_HASH_RTX (EXPR_VINSN (expr))); |
2330 if (pti) | 2388 if (pti) |
2331 { | 2389 { |
2332 /* This EXPR was already moved through this insn and was | 2390 /* This EXPR was already moved through this insn and was |
2333 changed as a result. Fetch the proper data from | 2391 changed as a result. Fetch the proper data from |
2334 the hashtable. */ | 2392 the hashtable. */ |
2335 insert_in_history_vect (&EXPR_HISTORY_OF_CHANGES (expr), | 2393 insert_in_history_vect (&EXPR_HISTORY_OF_CHANGES (expr), |
2336 INSN_UID (insn), pti->type, | 2394 INSN_UID (insn), pti->type, |
2337 pti->vinsn_old, pti->vinsn_new, | 2395 pti->vinsn_old, pti->vinsn_new, |
2338 EXPR_SPEC_DONE_DS (expr)); | 2396 EXPR_SPEC_DONE_DS (expr)); |
2339 | 2397 |
2340 if (INSN_IN_STREAM_P (VINSN_INSN_RTX (pti->vinsn_new))) | 2398 if (INSN_IN_STREAM_P (VINSN_INSN_RTX (pti->vinsn_new))) |
2341 pti->vinsn_new = vinsn_copy (pti->vinsn_new, true); | 2399 pti->vinsn_new = vinsn_copy (pti->vinsn_new, true); |
2342 change_vinsn_in_expr (expr, pti->vinsn_new); | 2400 change_vinsn_in_expr (expr, pti->vinsn_new); |
2343 if (pti->was_target_conflict) | 2401 if (pti->was_target_conflict) |
2344 EXPR_TARGET_AVAILABLE (expr) = false; | 2402 EXPR_TARGET_AVAILABLE (expr) = false; |
2345 if (pti->type == TRANS_SPECULATION) | 2403 if (pti->type == TRANS_SPECULATION) |
2346 { | 2404 { |
2347 ds_t ds; | 2405 ds_t ds; |
2348 | 2406 |
2349 ds = EXPR_SPEC_DONE_DS (expr); | 2407 ds = EXPR_SPEC_DONE_DS (expr); |
2350 | 2408 |
2351 EXPR_SPEC_DONE_DS (expr) = pti->ds; | 2409 EXPR_SPEC_DONE_DS (expr) = pti->ds; |
2352 EXPR_NEEDS_SPEC_CHECK_P (expr) |= pti->needs_check; | 2410 EXPR_NEEDS_SPEC_CHECK_P (expr) |= pti->needs_check; |
2353 } | 2411 } |
2354 | 2412 |
2355 if (sched_verbose >= 6) | 2413 if (sched_verbose >= 6) |
2366 return false; | 2424 return false; |
2367 } | 2425 } |
2368 | 2426 |
2369 /* Update bitmap caches on INSN with result RES of propagating EXPR. */ | 2427 /* Update bitmap caches on INSN with result RES of propagating EXPR. */ |
2370 static void | 2428 static void |
2371 update_bitmap_cache (expr_t expr, insn_t insn, bool inside_insn_group, | 2429 update_bitmap_cache (expr_t expr, insn_t insn, bool inside_insn_group, |
2372 enum MOVEUP_EXPR_CODE res) | 2430 enum MOVEUP_EXPR_CODE res) |
2373 { | 2431 { |
2374 int expr_uid = INSN_UID (EXPR_INSN_RTX (expr)); | 2432 int expr_uid = INSN_UID (EXPR_INSN_RTX (expr)); |
2375 | 2433 |
2376 /* Do not cache result of propagating jumps through an insn group, | 2434 /* Do not cache result of propagating jumps through an insn group, |
2377 as it is always true, which is not useful outside the group. */ | 2435 as it is always true, which is not useful outside the group. */ |
2378 if (inside_insn_group) | 2436 if (inside_insn_group) |
2379 return; | 2437 return; |
2380 | 2438 |
2381 if (res == MOVEUP_EXPR_NULL) | 2439 if (res == MOVEUP_EXPR_NULL) |
2382 { | 2440 { |
2383 bitmap_set_bit (INSN_ANALYZED_DEPS (insn), expr_uid); | 2441 bitmap_set_bit (INSN_ANALYZED_DEPS (insn), expr_uid); |
2384 bitmap_set_bit (INSN_FOUND_DEPS (insn), expr_uid); | 2442 bitmap_set_bit (INSN_FOUND_DEPS (insn), expr_uid); |
2385 } | 2443 } |
2398 } | 2456 } |
2399 | 2457 |
2400 /* Update hashtable on INSN with changed EXPR, old EXPR_OLD_VINSN | 2458 /* Update hashtable on INSN with changed EXPR, old EXPR_OLD_VINSN |
2401 and transformation type TRANS_TYPE. */ | 2459 and transformation type TRANS_TYPE. */ |
2402 static void | 2460 static void |
2403 update_transformation_cache (expr_t expr, insn_t insn, | 2461 update_transformation_cache (expr_t expr, insn_t insn, |
2404 bool inside_insn_group, | 2462 bool inside_insn_group, |
2405 enum local_trans_type trans_type, | 2463 enum local_trans_type trans_type, |
2406 vinsn_t expr_old_vinsn) | 2464 vinsn_t expr_old_vinsn) |
2407 { | 2465 { |
2408 struct transformed_insns *pti; | 2466 struct transformed_insns *pti; |
2409 | 2467 |
2410 if (inside_insn_group) | 2468 if (inside_insn_group) |
2411 return; | 2469 return; |
2412 | 2470 |
2413 pti = XNEW (struct transformed_insns); | 2471 pti = XNEW (struct transformed_insns); |
2414 pti->vinsn_old = expr_old_vinsn; | 2472 pti->vinsn_old = expr_old_vinsn; |
2415 pti->vinsn_new = EXPR_VINSN (expr); | 2473 pti->vinsn_new = EXPR_VINSN (expr); |
2416 pti->type = trans_type; | 2474 pti->type = trans_type; |
2417 pti->was_target_conflict = was_target_conflict; | 2475 pti->was_target_conflict = was_target_conflict; |
2418 pti->ds = EXPR_SPEC_DONE_DS (expr); | 2476 pti->ds = EXPR_SPEC_DONE_DS (expr); |
2419 pti->needs_check = EXPR_NEEDS_SPEC_CHECK_P (expr); | 2477 pti->needs_check = EXPR_NEEDS_SPEC_CHECK_P (expr); |
2420 vinsn_attach (pti->vinsn_old); | 2478 vinsn_attach (pti->vinsn_old); |
2421 vinsn_attach (pti->vinsn_new); | 2479 vinsn_attach (pti->vinsn_new); |
2422 *((struct transformed_insns **) | 2480 *((struct transformed_insns **) |
2423 htab_find_slot_with_hash (INSN_TRANSFORMED_INSNS (insn), | 2481 htab_find_slot_with_hash (INSN_TRANSFORMED_INSNS (insn), |
2424 pti, VINSN_HASH_RTX (expr_old_vinsn), | 2482 pti, VINSN_HASH_RTX (expr_old_vinsn), |
2425 INSERT)) = pti; | 2483 INSERT)) = pti; |
2426 } | 2484 } |
2427 | 2485 |
2428 /* Same as moveup_expr, but first looks up the result of | 2486 /* Same as moveup_expr, but first looks up the result of |
2429 transformation in caches. */ | 2487 transformation in caches. */ |
2430 static enum MOVEUP_EXPR_CODE | 2488 static enum MOVEUP_EXPR_CODE |
2431 moveup_expr_cached (expr_t expr, insn_t insn, bool inside_insn_group) | 2489 moveup_expr_cached (expr_t expr, insn_t insn, bool inside_insn_group) |
2432 { | 2490 { |
2433 enum MOVEUP_EXPR_CODE res; | 2491 enum MOVEUP_EXPR_CODE res; |
2434 bool got_answer = false; | 2492 bool got_answer = false; |
2435 | 2493 |
2436 if (sched_verbose >= 6) | 2494 if (sched_verbose >= 6) |
2437 { | 2495 { |
2438 sel_print ("Moving "); | 2496 sel_print ("Moving "); |
2439 dump_expr (expr); | 2497 dump_expr (expr); |
2440 sel_print (" through %d: ", INSN_UID (insn)); | 2498 sel_print (" through %d: ", INSN_UID (insn)); |
2441 } | 2499 } |
2442 | 2500 |
2443 if (try_bitmap_cache (expr, insn, inside_insn_group, &res)) | 2501 if (DEBUG_INSN_P (EXPR_INSN_RTX (expr)) |
2502 && (sel_bb_head (BLOCK_FOR_INSN (EXPR_INSN_RTX (expr))) | |
2503 == EXPR_INSN_RTX (expr))) | |
2504 /* Don't use cached information for debug insns that are heads of | |
2505 basic blocks. */; | |
2506 else if (try_bitmap_cache (expr, insn, inside_insn_group, &res)) | |
2444 /* When inside insn group, we do not want remove stores conflicting | 2507 /* When inside insn group, we do not want remove stores conflicting |
2445 with previosly issued loads. */ | 2508 with previosly issued loads. */ |
2446 got_answer = ! inside_insn_group || res != MOVEUP_EXPR_NULL; | 2509 got_answer = ! inside_insn_group || res != MOVEUP_EXPR_NULL; |
2447 else if (try_transformation_cache (expr, insn, &res)) | 2510 else if (try_transformation_cache (expr, insn, &res)) |
2448 got_answer = true; | 2511 got_answer = true; |
2454 ds_t expr_old_spec_ds = EXPR_SPEC_DONE_DS (expr); | 2517 ds_t expr_old_spec_ds = EXPR_SPEC_DONE_DS (expr); |
2455 int expr_uid = INSN_UID (VINSN_INSN_RTX (expr_old_vinsn)); | 2518 int expr_uid = INSN_UID (VINSN_INSN_RTX (expr_old_vinsn)); |
2456 bool unique_p = VINSN_UNIQUE_P (expr_old_vinsn); | 2519 bool unique_p = VINSN_UNIQUE_P (expr_old_vinsn); |
2457 enum local_trans_type trans_type = TRANS_SUBSTITUTION; | 2520 enum local_trans_type trans_type = TRANS_SUBSTITUTION; |
2458 | 2521 |
2459 /* ??? Invent something better than this. We can't allow old_vinsn | 2522 /* ??? Invent something better than this. We can't allow old_vinsn |
2460 to go, we need it for the history vector. */ | 2523 to go, we need it for the history vector. */ |
2461 vinsn_attach (expr_old_vinsn); | 2524 vinsn_attach (expr_old_vinsn); |
2462 | 2525 |
2463 res = moveup_expr (expr, insn, inside_insn_group, | 2526 res = moveup_expr (expr, insn, inside_insn_group, |
2464 &trans_type); | 2527 &trans_type); |
2484 break; | 2547 break; |
2485 | 2548 |
2486 case MOVEUP_EXPR_CHANGED: | 2549 case MOVEUP_EXPR_CHANGED: |
2487 gcc_assert (INSN_UID (EXPR_INSN_RTX (expr)) != expr_uid | 2550 gcc_assert (INSN_UID (EXPR_INSN_RTX (expr)) != expr_uid |
2488 || EXPR_SPEC_DONE_DS (expr) != expr_old_spec_ds); | 2551 || EXPR_SPEC_DONE_DS (expr) != expr_old_spec_ds); |
2489 insert_in_history_vect (&EXPR_HISTORY_OF_CHANGES (expr), | 2552 insert_in_history_vect (&EXPR_HISTORY_OF_CHANGES (expr), |
2490 INSN_UID (insn), trans_type, | 2553 INSN_UID (insn), trans_type, |
2491 expr_old_vinsn, EXPR_VINSN (expr), | 2554 expr_old_vinsn, EXPR_VINSN (expr), |
2492 expr_old_spec_ds); | 2555 expr_old_spec_ds); |
2493 update_transformation_cache (expr, insn, inside_insn_group, | 2556 update_transformation_cache (expr, insn, inside_insn_group, |
2494 trans_type, expr_old_vinsn); | 2557 trans_type, expr_old_vinsn); |
2495 if (sched_verbose >= 6) | 2558 if (sched_verbose >= 6) |
2496 { | 2559 { |
2507 } | 2570 } |
2508 | 2571 |
2509 return res; | 2572 return res; |
2510 } | 2573 } |
2511 | 2574 |
2512 /* Moves an av set AVP up through INSN, performing necessary | 2575 /* Moves an av set AVP up through INSN, performing necessary |
2513 transformations. */ | 2576 transformations. */ |
2514 static void | 2577 static void |
2515 moveup_set_expr (av_set_t *avp, insn_t insn, bool inside_insn_group) | 2578 moveup_set_expr (av_set_t *avp, insn_t insn, bool inside_insn_group) |
2516 { | 2579 { |
2517 av_set_iterator i; | 2580 av_set_iterator i; |
2518 expr_t expr; | 2581 expr_t expr; |
2519 | 2582 |
2520 FOR_EACH_EXPR_1 (expr, i, avp) | 2583 FOR_EACH_EXPR_1 (expr, i, avp) |
2521 { | 2584 { |
2522 | 2585 |
2523 switch (moveup_expr_cached (expr, insn, inside_insn_group)) | 2586 switch (moveup_expr_cached (expr, insn, inside_insn_group)) |
2524 { | 2587 { |
2525 case MOVEUP_EXPR_SAME: | 2588 case MOVEUP_EXPR_SAME: |
2526 case MOVEUP_EXPR_AS_RHS: | 2589 case MOVEUP_EXPR_AS_RHS: |
2527 break; | 2590 break; |
2531 break; | 2594 break; |
2532 | 2595 |
2533 case MOVEUP_EXPR_CHANGED: | 2596 case MOVEUP_EXPR_CHANGED: |
2534 expr = merge_with_other_exprs (avp, &i, expr); | 2597 expr = merge_with_other_exprs (avp, &i, expr); |
2535 break; | 2598 break; |
2536 | 2599 |
2537 default: | 2600 default: |
2538 gcc_unreachable (); | 2601 gcc_unreachable (); |
2539 } | 2602 } |
2540 } | 2603 } |
2541 } | 2604 } |
2543 /* Moves AVP set along PATH. */ | 2606 /* Moves AVP set along PATH. */ |
2544 static void | 2607 static void |
2545 moveup_set_inside_insn_group (av_set_t *avp, ilist_t path) | 2608 moveup_set_inside_insn_group (av_set_t *avp, ilist_t path) |
2546 { | 2609 { |
2547 int last_cycle; | 2610 int last_cycle; |
2548 | 2611 |
2549 if (sched_verbose >= 6) | 2612 if (sched_verbose >= 6) |
2550 sel_print ("Moving expressions up in the insn group...\n"); | 2613 sel_print ("Moving expressions up in the insn group...\n"); |
2551 if (! path) | 2614 if (! path) |
2552 return; | 2615 return; |
2553 last_cycle = INSN_SCHED_CYCLE (ILIST_INSN (path)); | 2616 last_cycle = INSN_SCHED_CYCLE (ILIST_INSN (path)); |
2554 while (path | 2617 while (path |
2555 && INSN_SCHED_CYCLE (ILIST_INSN (path)) == last_cycle) | 2618 && INSN_SCHED_CYCLE (ILIST_INSN (path)) == last_cycle) |
2556 { | 2619 { |
2557 moveup_set_expr (avp, ILIST_INSN (path), true); | 2620 moveup_set_expr (avp, ILIST_INSN (path), true); |
2558 path = ILIST_NEXT (path); | 2621 path = ILIST_NEXT (path); |
2559 } | 2622 } |
2567 int last_cycle; | 2630 int last_cycle; |
2568 bool res = true; | 2631 bool res = true; |
2569 | 2632 |
2570 copy_expr_onside (tmp, expr); | 2633 copy_expr_onside (tmp, expr); |
2571 last_cycle = path ? INSN_SCHED_CYCLE (ILIST_INSN (path)) : 0; | 2634 last_cycle = path ? INSN_SCHED_CYCLE (ILIST_INSN (path)) : 0; |
2572 while (path | 2635 while (path |
2573 && res | 2636 && res |
2574 && INSN_SCHED_CYCLE (ILIST_INSN (path)) == last_cycle) | 2637 && INSN_SCHED_CYCLE (ILIST_INSN (path)) == last_cycle) |
2575 { | 2638 { |
2576 res = (moveup_expr_cached (tmp, ILIST_INSN (path), true) | 2639 res = (moveup_expr_cached (tmp, ILIST_INSN (path), true) |
2577 != MOVEUP_EXPR_NULL); | 2640 != MOVEUP_EXPR_NULL); |
2578 path = ILIST_NEXT (path); | 2641 path = ILIST_NEXT (path); |
2579 } | 2642 } |
2580 | 2643 |
2581 if (res) | 2644 if (res) |
2592 } | 2655 } |
2593 | 2656 |
2594 | 2657 |
2595 /* Functions that compute av and lv sets. */ | 2658 /* Functions that compute av and lv sets. */ |
2596 | 2659 |
2597 /* Returns true if INSN is not a downward continuation of the given path P in | 2660 /* Returns true if INSN is not a downward continuation of the given path P in |
2598 the current stage. */ | 2661 the current stage. */ |
2599 static bool | 2662 static bool |
2600 is_ineligible_successor (insn_t insn, ilist_t p) | 2663 is_ineligible_successor (insn_t insn, ilist_t p) |
2601 { | 2664 { |
2602 insn_t prev_insn; | 2665 insn_t prev_insn; |
2616 if (/* a backward edge. */ | 2679 if (/* a backward edge. */ |
2617 INSN_SEQNO (insn) < INSN_SEQNO (prev_insn) | 2680 INSN_SEQNO (insn) < INSN_SEQNO (prev_insn) |
2618 /* is already visited. */ | 2681 /* is already visited. */ |
2619 || (INSN_SEQNO (insn) == INSN_SEQNO (prev_insn) | 2682 || (INSN_SEQNO (insn) == INSN_SEQNO (prev_insn) |
2620 && (ilist_is_in_p (p, insn) | 2683 && (ilist_is_in_p (p, insn) |
2621 /* We can reach another fence here and still seqno of insn | 2684 /* We can reach another fence here and still seqno of insn |
2622 would be equal to seqno of prev_insn. This is possible | 2685 would be equal to seqno of prev_insn. This is possible |
2623 when prev_insn is a previously created bookkeeping copy. | 2686 when prev_insn is a previously created bookkeeping copy. |
2624 In that case it'd get a seqno of insn. Thus, check here | 2687 In that case it'd get a seqno of insn. Thus, check here |
2625 whether insn is in current fence too. */ | 2688 whether insn is in current fence too. */ |
2626 || IN_CURRENT_FENCE_P (insn))) | 2689 || IN_CURRENT_FENCE_P (insn))) |
2627 /* Was already scheduled on this round. */ | 2690 /* Was already scheduled on this round. */ |
2628 || (INSN_SEQNO (insn) > INSN_SEQNO (prev_insn) | 2691 || (INSN_SEQNO (insn) > INSN_SEQNO (prev_insn) |
2629 && IN_CURRENT_FENCE_P (insn)) | 2692 && IN_CURRENT_FENCE_P (insn)) |
2630 /* An insn from another fence could also be | 2693 /* An insn from another fence could also be |
2631 scheduled earlier even if this insn is not in | 2694 scheduled earlier even if this insn is not in |
2632 a fence list right now. Check INSN_SCHED_CYCLE instead. */ | 2695 a fence list right now. Check INSN_SCHED_CYCLE instead. */ |
2633 || (!pipelining_p | 2696 || (!pipelining_p |
2634 && INSN_SCHED_TIMES (insn) > 0)) | 2697 && INSN_SCHED_TIMES (insn) > 0)) |
2635 return true; | 2698 return true; |
2636 else | 2699 else |
2637 return false; | 2700 return false; |
2638 } | 2701 } |
2639 | 2702 |
2640 /* Computes the av_set below the last bb insn INSN, doing all the 'dirty work' | 2703 /* Computes the av_set below the last bb insn INSN, doing all the 'dirty work' |
2641 of handling multiple successors and properly merging its av_sets. P is | 2704 of handling multiple successors and properly merging its av_sets. P is |
2642 the current path traversed. WS is the size of lookahead window. | 2705 the current path traversed. WS is the size of lookahead window. |
2643 Return the av set computed. */ | 2706 Return the av set computed. */ |
2644 static av_set_t | 2707 static av_set_t |
2645 compute_av_set_at_bb_end (insn_t insn, ilist_t p, int ws) | 2708 compute_av_set_at_bb_end (insn_t insn, ilist_t p, int ws) |
2646 { | 2709 { |
2647 struct succs_info *sinfo; | 2710 struct succs_info *sinfo; |
2650 insn_t succ, zero_succ = NULL; | 2713 insn_t succ, zero_succ = NULL; |
2651 av_set_t av1 = NULL; | 2714 av_set_t av1 = NULL; |
2652 | 2715 |
2653 gcc_assert (sel_bb_end_p (insn)); | 2716 gcc_assert (sel_bb_end_p (insn)); |
2654 | 2717 |
2655 /* Find different kind of successors needed for correct computing of | 2718 /* Find different kind of successors needed for correct computing of |
2656 SPEC and TARGET_AVAILABLE attributes. */ | 2719 SPEC and TARGET_AVAILABLE attributes. */ |
2657 sinfo = compute_succs_info (insn, SUCCS_NORMAL); | 2720 sinfo = compute_succs_info (insn, SUCCS_NORMAL); |
2658 | 2721 |
2659 /* Debug output. */ | 2722 /* Debug output. */ |
2660 if (sched_verbose >= 6) | 2723 if (sched_verbose >= 6) |
2674 av_set_t succ_set; | 2737 av_set_t succ_set; |
2675 | 2738 |
2676 /* We will edit SUCC_SET and EXPR_SPEC field of its elements. */ | 2739 /* We will edit SUCC_SET and EXPR_SPEC field of its elements. */ |
2677 succ_set = compute_av_set_inside_bb (succ, p, ws, true); | 2740 succ_set = compute_av_set_inside_bb (succ, p, ws, true); |
2678 | 2741 |
2679 av_set_split_usefulness (succ_set, | 2742 av_set_split_usefulness (succ_set, |
2680 VEC_index (int, sinfo->probs_ok, is), | 2743 VEC_index (int, sinfo->probs_ok, is), |
2681 sinfo->all_prob); | 2744 sinfo->all_prob); |
2682 | 2745 |
2683 if (sinfo->all_succs_n > 1 | 2746 if (sinfo->all_succs_n > 1 |
2684 && sinfo->all_succs_n == sinfo->succs_ok_n) | 2747 && sinfo->all_succs_n == sinfo->succs_ok_n) |
2685 { | 2748 { |
2686 /* Find EXPR'es that came from *all* successors and save them | 2749 /* Find EXPR'es that came from *all* successors and save them |
2687 into expr_in_all_succ_branches. This set will be used later | 2750 into expr_in_all_succ_branches. This set will be used later |
2688 for calculating speculation attributes of EXPR'es. */ | 2751 for calculating speculation attributes of EXPR'es. */ |
2689 if (is == 0) | 2752 if (is == 0) |
2690 { | 2753 { |
2691 expr_in_all_succ_branches = av_set_copy (succ_set); | 2754 expr_in_all_succ_branches = av_set_copy (succ_set); |
2695 } | 2758 } |
2696 else | 2759 else |
2697 { | 2760 { |
2698 av_set_iterator i; | 2761 av_set_iterator i; |
2699 expr_t expr; | 2762 expr_t expr; |
2700 | 2763 |
2701 FOR_EACH_EXPR_1 (expr, i, &expr_in_all_succ_branches) | 2764 FOR_EACH_EXPR_1 (expr, i, &expr_in_all_succ_branches) |
2702 if (!av_set_is_in_p (succ_set, EXPR_VINSN (expr))) | 2765 if (!av_set_is_in_p (succ_set, EXPR_VINSN (expr))) |
2703 av_set_iter_remove (&i); | 2766 av_set_iter_remove (&i); |
2704 } | 2767 } |
2705 } | 2768 } |
2710 { | 2773 { |
2711 basic_block bb0 = BLOCK_FOR_INSN (zero_succ); | 2774 basic_block bb0 = BLOCK_FOR_INSN (zero_succ); |
2712 basic_block bb1 = BLOCK_FOR_INSN (succ); | 2775 basic_block bb1 = BLOCK_FOR_INSN (succ); |
2713 | 2776 |
2714 gcc_assert (BB_LV_SET_VALID_P (bb0) && BB_LV_SET_VALID_P (bb1)); | 2777 gcc_assert (BB_LV_SET_VALID_P (bb0) && BB_LV_SET_VALID_P (bb1)); |
2715 av_set_union_and_live (&av1, &succ_set, | 2778 av_set_union_and_live (&av1, &succ_set, |
2716 BB_LV_SET (bb0), | 2779 BB_LV_SET (bb0), |
2717 BB_LV_SET (bb1), | 2780 BB_LV_SET (bb1), |
2718 insn); | 2781 insn); |
2719 } | 2782 } |
2720 else | 2783 else |
2721 av_set_union_and_clear (&av1, &succ_set, insn); | 2784 av_set_union_and_clear (&av1, &succ_set, insn); |
2722 } | 2785 } |
2723 | 2786 |
2724 /* Check liveness restrictions via hard way when there are more than | 2787 /* Check liveness restrictions via hard way when there are more than |
2725 two successors. */ | 2788 two successors. */ |
2726 if (sinfo->succs_ok_n > 2) | 2789 if (sinfo->succs_ok_n > 2) |
2727 for (is = 0; VEC_iterate (rtx, sinfo->succs_ok, is, succ); is++) | 2790 for (is = 0; VEC_iterate (rtx, sinfo->succs_ok, is, succ); is++) |
2728 { | 2791 { |
2729 basic_block succ_bb = BLOCK_FOR_INSN (succ); | 2792 basic_block succ_bb = BLOCK_FOR_INSN (succ); |
2730 | 2793 |
2731 gcc_assert (BB_LV_SET_VALID_P (succ_bb)); | 2794 gcc_assert (BB_LV_SET_VALID_P (succ_bb)); |
2732 mark_unavailable_targets (av1, BB_AV_SET (succ_bb), | 2795 mark_unavailable_targets (av1, BB_AV_SET (succ_bb), |
2733 BB_LV_SET (succ_bb)); | 2796 BB_LV_SET (succ_bb)); |
2734 } | 2797 } |
2735 | 2798 |
2736 /* Finally, check liveness restrictions on paths leaving the region. */ | 2799 /* Finally, check liveness restrictions on paths leaving the region. */ |
2737 if (sinfo->all_succs_n > sinfo->succs_ok_n) | 2800 if (sinfo->all_succs_n > sinfo->succs_ok_n) |
2738 for (is = 0; VEC_iterate (rtx, sinfo->succs_other, is, succ); is++) | 2801 for (is = 0; VEC_iterate (rtx, sinfo->succs_other, is, succ); is++) |
2739 mark_unavailable_targets | 2802 mark_unavailable_targets |
2740 (av1, NULL, BB_LV_SET (BLOCK_FOR_INSN (succ))); | 2803 (av1, NULL, BB_LV_SET (BLOCK_FOR_INSN (succ))); |
2741 | 2804 |
2742 if (sinfo->all_succs_n > 1) | 2805 if (sinfo->all_succs_n > 1) |
2743 { | 2806 { |
2744 av_set_iterator i; | 2807 av_set_iterator i; |
2745 expr_t expr; | 2808 expr_t expr; |
2746 | 2809 |
2747 /* Increase the spec attribute of all EXPR'es that didn't come | 2810 /* Increase the spec attribute of all EXPR'es that didn't come |
2748 from all successors. */ | 2811 from all successors. */ |
2749 FOR_EACH_EXPR (expr, i, av1) | 2812 FOR_EACH_EXPR (expr, i, av1) |
2750 if (!av_set_is_in_p (expr_in_all_succ_branches, EXPR_VINSN (expr))) | 2813 if (!av_set_is_in_p (expr_in_all_succ_branches, EXPR_VINSN (expr))) |
2751 EXPR_SPEC (expr)++; | 2814 EXPR_SPEC (expr)++; |
2752 | 2815 |
2753 av_set_clear (&expr_in_all_succ_branches); | 2816 av_set_clear (&expr_in_all_succ_branches); |
2754 | 2817 |
2755 /* Do not move conditional branches through other | 2818 /* Do not move conditional branches through other |
2756 conditional branches. So, remove all conditional | 2819 conditional branches. So, remove all conditional |
2757 branches from av_set if current operator is a conditional | 2820 branches from av_set if current operator is a conditional |
2758 branch. */ | 2821 branch. */ |
2759 av_set_substract_cond_branches (&av1); | 2822 av_set_substract_cond_branches (&av1); |
2760 } | 2823 } |
2761 | 2824 |
2762 ilist_remove (&p); | 2825 ilist_remove (&p); |
2763 free_succs_info (sinfo); | 2826 free_succs_info (sinfo); |
2764 | 2827 |
2765 if (sched_verbose >= 6) | 2828 if (sched_verbose >= 6) |
2766 { | 2829 { |
2770 } | 2833 } |
2771 | 2834 |
2772 return av1; | 2835 return av1; |
2773 } | 2836 } |
2774 | 2837 |
2775 /* This function computes av_set for the FIRST_INSN by dragging valid | 2838 /* This function computes av_set for the FIRST_INSN by dragging valid |
2776 av_set through all basic block insns either from the end of basic block | 2839 av_set through all basic block insns either from the end of basic block |
2777 (computed using compute_av_set_at_bb_end) or from the insn on which | 2840 (computed using compute_av_set_at_bb_end) or from the insn on which |
2778 MAX_WS was exceeded. It uses compute_av_set_at_bb_end to compute av_set | 2841 MAX_WS was exceeded. It uses compute_av_set_at_bb_end to compute av_set |
2779 below the basic block and handling conditional branches. | 2842 below the basic block and handling conditional branches. |
2780 FIRST_INSN - the basic block head, P - path consisting of the insns | 2843 FIRST_INSN - the basic block head, P - path consisting of the insns |
2781 traversed on the way to the FIRST_INSN (the path is sparse, only bb heads | 2844 traversed on the way to the FIRST_INSN (the path is sparse, only bb heads |
2782 and bb ends are added to the path), WS - current window size, | 2845 and bb ends are added to the path), WS - current window size, |
2783 NEED_COPY_P - true if we'll make a copy of av_set before returning it. */ | 2846 NEED_COPY_P - true if we'll make a copy of av_set before returning it. */ |
2784 static av_set_t | 2847 static av_set_t |
2785 compute_av_set_inside_bb (insn_t first_insn, ilist_t p, int ws, | 2848 compute_av_set_inside_bb (insn_t first_insn, ilist_t p, int ws, |
2786 bool need_copy_p) | 2849 bool need_copy_p) |
2787 { | 2850 { |
2788 insn_t cur_insn; | 2851 insn_t cur_insn; |
2789 int end_ws = ws; | 2852 int end_ws = ws; |
2790 insn_t bb_end = sel_bb_end (BLOCK_FOR_INSN (first_insn)); | 2853 insn_t bb_end = sel_bb_end (BLOCK_FOR_INSN (first_insn)); |
2800 sel_print ("Insn %d is ineligible_successor\n", INSN_UID (first_insn)); | 2863 sel_print ("Insn %d is ineligible_successor\n", INSN_UID (first_insn)); |
2801 | 2864 |
2802 return NULL; | 2865 return NULL; |
2803 } | 2866 } |
2804 | 2867 |
2805 /* If insn already has valid av(insn) computed, just return it. */ | 2868 /* If insn already has valid av(insn) computed, just return it. */ |
2806 if (AV_SET_VALID_P (first_insn)) | 2869 if (AV_SET_VALID_P (first_insn)) |
2807 { | 2870 { |
2808 av_set_t av_set; | 2871 av_set_t av_set; |
2809 | 2872 |
2810 if (sel_bb_head_p (first_insn)) | 2873 if (sel_bb_head_p (first_insn)) |
2823 } | 2886 } |
2824 | 2887 |
2825 ilist_add (&p, first_insn); | 2888 ilist_add (&p, first_insn); |
2826 | 2889 |
2827 /* As the result after this loop have completed, in LAST_INSN we'll | 2890 /* As the result after this loop have completed, in LAST_INSN we'll |
2828 have the insn which has valid av_set to start backward computation | 2891 have the insn which has valid av_set to start backward computation |
2829 from: it either will be NULL because on it the window size was exceeded | 2892 from: it either will be NULL because on it the window size was exceeded |
2830 or other valid av_set as returned by compute_av_set for the last insn | 2893 or other valid av_set as returned by compute_av_set for the last insn |
2831 of the basic block. */ | 2894 of the basic block. */ |
2832 for (last_insn = first_insn; last_insn != after_bb_end; | 2895 for (last_insn = first_insn; last_insn != after_bb_end; |
2833 last_insn = NEXT_INSN (last_insn)) | 2896 last_insn = NEXT_INSN (last_insn)) |
2834 { | 2897 { |
2835 /* We may encounter valid av_set not only on bb_head, but also on | 2898 /* We may encounter valid av_set not only on bb_head, but also on |
2842 } | 2905 } |
2843 | 2906 |
2844 /* The special case: the last insn of the BB may be an | 2907 /* The special case: the last insn of the BB may be an |
2845 ineligible_successor due to its SEQ_NO that was set on | 2908 ineligible_successor due to its SEQ_NO that was set on |
2846 it as a bookkeeping. */ | 2909 it as a bookkeeping. */ |
2847 if (last_insn != first_insn | 2910 if (last_insn != first_insn |
2848 && is_ineligible_successor (last_insn, p)) | 2911 && is_ineligible_successor (last_insn, p)) |
2849 { | 2912 { |
2850 if (sched_verbose >= 6) | 2913 if (sched_verbose >= 6) |
2851 sel_print ("Insn %d is ineligible_successor\n", INSN_UID (last_insn)); | 2914 sel_print ("Insn %d is ineligible_successor\n", INSN_UID (last_insn)); |
2852 break; | 2915 break; |
2853 } | 2916 } |
2917 | |
2918 if (DEBUG_INSN_P (last_insn)) | |
2919 continue; | |
2854 | 2920 |
2855 if (end_ws > max_ws) | 2921 if (end_ws > max_ws) |
2856 { | 2922 { |
2857 /* We can reach max lookahead size at bb_header, so clean av_set | 2923 /* We can reach max lookahead size at bb_header, so clean av_set |
2858 first. */ | 2924 first. */ |
2859 INSN_WS_LEVEL (last_insn) = global_level; | 2925 INSN_WS_LEVEL (last_insn) = global_level; |
2860 | 2926 |
2861 if (sched_verbose >= 6) | 2927 if (sched_verbose >= 6) |
2862 sel_print ("Insn %d is beyond the software lookahead window size\n", | 2928 sel_print ("Insn %d is beyond the software lookahead window size\n", |
2872 the successors on the last insn of the current bb. */ | 2938 the successors on the last insn of the current bb. */ |
2873 if (last_insn != after_bb_end) | 2939 if (last_insn != after_bb_end) |
2874 { | 2940 { |
2875 av = NULL; | 2941 av = NULL; |
2876 | 2942 |
2877 /* This is needed only to obtain av_sets that are identical to | 2943 /* This is needed only to obtain av_sets that are identical to |
2878 those computed by the old compute_av_set version. */ | 2944 those computed by the old compute_av_set version. */ |
2879 if (last_insn == first_insn && !INSN_NOP_P (last_insn)) | 2945 if (last_insn == first_insn && !INSN_NOP_P (last_insn)) |
2880 av_set_add (&av, INSN_EXPR (last_insn)); | 2946 av_set_add (&av, INSN_EXPR (last_insn)); |
2881 } | 2947 } |
2882 else | 2948 else |
2884 av = compute_av_set_at_bb_end (bb_end, p, end_ws); | 2950 av = compute_av_set_at_bb_end (bb_end, p, end_ws); |
2885 | 2951 |
2886 /* Compute av_set in AV starting from below the LAST_INSN up to | 2952 /* Compute av_set in AV starting from below the LAST_INSN up to |
2887 location above the FIRST_INSN. */ | 2953 location above the FIRST_INSN. */ |
2888 for (cur_insn = PREV_INSN (last_insn); cur_insn != PREV_INSN (first_insn); | 2954 for (cur_insn = PREV_INSN (last_insn); cur_insn != PREV_INSN (first_insn); |
2889 cur_insn = PREV_INSN (cur_insn)) | 2955 cur_insn = PREV_INSN (cur_insn)) |
2890 if (!INSN_NOP_P (cur_insn)) | 2956 if (!INSN_NOP_P (cur_insn)) |
2891 { | 2957 { |
2892 expr_t expr; | 2958 expr_t expr; |
2893 | 2959 |
2894 moveup_set_expr (&av, cur_insn, false); | 2960 moveup_set_expr (&av, cur_insn, false); |
2895 | 2961 |
2896 /* If the expression for CUR_INSN is already in the set, | 2962 /* If the expression for CUR_INSN is already in the set, |
2897 replace it by the new one. */ | 2963 replace it by the new one. */ |
2898 expr = av_set_lookup (av, INSN_VINSN (cur_insn)); | 2964 expr = av_set_lookup (av, INSN_VINSN (cur_insn)); |
2899 if (expr != NULL) | 2965 if (expr != NULL) |
2900 { | 2966 { |
2901 clear_expr (expr); | 2967 clear_expr (expr); |
2902 copy_expr (expr, INSN_EXPR (cur_insn)); | 2968 copy_expr (expr, INSN_EXPR (cur_insn)); |
2903 } | 2969 } |
2990 | 3056 |
2991 /* Return the valid set if we're already on it. */ | 3057 /* Return the valid set if we're already on it. */ |
2992 if (!ignore_first) | 3058 if (!ignore_first) |
2993 { | 3059 { |
2994 regset src = NULL; | 3060 regset src = NULL; |
2995 | 3061 |
2996 if (sel_bb_head_p (insn) && BB_LV_SET_VALID_P (bb)) | 3062 if (sel_bb_head_p (insn) && BB_LV_SET_VALID_P (bb)) |
2997 src = BB_LV_SET (bb); | 3063 src = BB_LV_SET (bb); |
2998 else | 3064 else |
2999 { | 3065 { |
3000 gcc_assert (in_current_region_p (bb)); | 3066 gcc_assert (in_current_region_p (bb)); |
3001 if (INSN_LIVE_VALID_P (insn)) | 3067 if (INSN_LIVE_VALID_P (insn)) |
3002 src = INSN_LIVE (insn); | 3068 src = INSN_LIVE (insn); |
3003 } | 3069 } |
3004 | 3070 |
3005 if (src) | 3071 if (src) |
3006 { | 3072 { |
3007 lv = get_regset_from_pool (); | 3073 lv = get_regset_from_pool (); |
3008 COPY_REG_SET (lv, src); | 3074 COPY_REG_SET (lv, src); |
3009 | 3075 |
3010 if (sel_bb_head_p (insn) && ! BB_LV_SET_VALID_P (bb)) | 3076 if (sel_bb_head_p (insn) && ! BB_LV_SET_VALID_P (bb)) |
3011 { | 3077 { |
3012 COPY_REG_SET (BB_LV_SET (bb), lv); | 3078 COPY_REG_SET (BB_LV_SET (bb), lv); |
3013 BB_LV_SET_VALID_P (bb) = true; | 3079 BB_LV_SET_VALID_P (bb) = true; |
3014 } | 3080 } |
3015 | 3081 |
3016 return_regset_to_pool (lv); | 3082 return_regset_to_pool (lv); |
3017 return lv; | 3083 return lv; |
3018 } | 3084 } |
3019 } | 3085 } |
3020 | 3086 |
3021 /* We've skipped the wrong lv_set. Don't skip the right one. */ | 3087 /* We've skipped the wrong lv_set. Don't skip the right one. */ |
3022 ignore_first = false; | 3088 ignore_first = false; |
3023 gcc_assert (in_current_region_p (bb)); | 3089 gcc_assert (in_current_region_p (bb)); |
3024 | 3090 |
3025 /* Find a valid LV set in this block or below, if needed. | 3091 /* Find a valid LV set in this block or below, if needed. |
3026 Start searching from the next insn: either ignore_first is true, or | 3092 Start searching from the next insn: either ignore_first is true, or |
3027 INSN doesn't have a correct live set. */ | 3093 INSN doesn't have a correct live set. */ |
3028 temp = NEXT_INSN (insn); | 3094 temp = NEXT_INSN (insn); |
3029 final = NEXT_INSN (BB_END (bb)); | 3095 final = NEXT_INSN (BB_END (bb)); |
3030 while (temp != final && ! INSN_LIVE_VALID_P (temp)) | 3096 while (temp != final && ! INSN_LIVE_VALID_P (temp)) |
3031 temp = NEXT_INSN (temp); | 3097 temp = NEXT_INSN (temp); |
3052 | 3118 |
3053 /* Also put it in a BB. */ | 3119 /* Also put it in a BB. */ |
3054 if (sel_bb_head_p (insn)) | 3120 if (sel_bb_head_p (insn)) |
3055 { | 3121 { |
3056 basic_block bb = BLOCK_FOR_INSN (insn); | 3122 basic_block bb = BLOCK_FOR_INSN (insn); |
3057 | 3123 |
3058 COPY_REG_SET (BB_LV_SET (bb), lv); | 3124 COPY_REG_SET (BB_LV_SET (bb), lv); |
3059 BB_LV_SET_VALID_P (bb) = true; | 3125 BB_LV_SET_VALID_P (bb) = true; |
3060 } | 3126 } |
3061 | 3127 |
3062 /* We return LV to the pool, but will not clear it there. Thus we can | 3128 /* We return LV to the pool, but will not clear it there. Thus we can |
3063 legimatelly use LV till the next use of regset_pool_get (). */ | 3129 legimatelly use LV till the next use of regset_pool_get (). */ |
3064 return_regset_to_pool (lv); | 3130 return_regset_to_pool (lv); |
3065 return lv; | 3131 return lv; |
3066 } | 3132 } |
3077 static inline void | 3143 static inline void |
3078 compute_live_below_insn (rtx insn, regset regs) | 3144 compute_live_below_insn (rtx insn, regset regs) |
3079 { | 3145 { |
3080 rtx succ; | 3146 rtx succ; |
3081 succ_iterator si; | 3147 succ_iterator si; |
3082 | 3148 |
3083 FOR_EACH_SUCC_1 (succ, si, insn, SUCCS_ALL) | 3149 FOR_EACH_SUCC_1 (succ, si, insn, SUCCS_ALL) |
3084 IOR_REG_SET (regs, compute_live (succ)); | 3150 IOR_REG_SET (regs, compute_live (succ)); |
3085 } | 3151 } |
3086 | 3152 |
3087 /* Update the data gathered in av and lv sets starting from INSN. */ | 3153 /* Update the data gathered in av and lv sets starting from INSN. */ |
3088 static void | 3154 static void |
3121 to_check_ds &= ~already_checked_ds; | 3187 to_check_ds &= ~already_checked_ds; |
3122 | 3188 |
3123 return to_check_ds; | 3189 return to_check_ds; |
3124 } | 3190 } |
3125 | 3191 |
3126 /* Find the set of registers that are unavailable for storing expres | 3192 /* Find the set of registers that are unavailable for storing expres |
3127 while moving ORIG_OPS up on the path starting from INSN due to | 3193 while moving ORIG_OPS up on the path starting from INSN due to |
3128 liveness (USED_REGS) or hardware restrictions (REG_RENAME_P). | 3194 liveness (USED_REGS) or hardware restrictions (REG_RENAME_P). |
3129 | 3195 |
3130 All the original operations found during the traversal are saved in the | 3196 All the original operations found during the traversal are saved in the |
3131 ORIGINAL_INSNS list. | 3197 ORIGINAL_INSNS list. |
3132 | 3198 |
3133 REG_RENAME_P denotes the set of hardware registers that | 3199 REG_RENAME_P denotes the set of hardware registers that |
3134 can not be used with renaming due to the register class restrictions, | 3200 can not be used with renaming due to the register class restrictions, |
3135 mode restrictions and other (the register we'll choose should be | 3201 mode restrictions and other (the register we'll choose should be |
3136 compatible class with the original uses, shouldn't be in call_used_regs, | 3202 compatible class with the original uses, shouldn't be in call_used_regs, |
3137 should be HARD_REGNO_RENAME_OK etc). | 3203 should be HARD_REGNO_RENAME_OK etc). |
3138 | 3204 |
3139 Returns TRUE if we've found all original insns, FALSE otherwise. | 3205 Returns TRUE if we've found all original insns, FALSE otherwise. |
3140 | 3206 |
3141 This function utilizes code_motion_path_driver (formerly find_used_regs_1) | 3207 This function utilizes code_motion_path_driver (formerly find_used_regs_1) |
3142 to traverse the code motion paths. This helper function finds registers | 3208 to traverse the code motion paths. This helper function finds registers |
3143 that are not available for storing expres while moving ORIG_OPS up on the | 3209 that are not available for storing expres while moving ORIG_OPS up on the |
3144 path starting from INSN. A register considered as used on the moving path, | 3210 path starting from INSN. A register considered as used on the moving path, |
3145 if one of the following conditions is not satisfied: | 3211 if one of the following conditions is not satisfied: |
3146 | 3212 |
3147 (1) a register not set or read on any path from xi to an instance of | 3213 (1) a register not set or read on any path from xi to an instance of |
3148 the original operation, | 3214 the original operation, |
3149 (2) not among the live registers of the point immediately following the | 3215 (2) not among the live registers of the point immediately following the |
3150 first original operation on a given downward path, except for the | 3216 first original operation on a given downward path, except for the |
3151 original target register of the operation, | 3217 original target register of the operation, |
3152 (3) not live on the other path of any conditional branch that is passed | 3218 (3) not live on the other path of any conditional branch that is passed |
3153 by the operation, in case original operations are not present on | 3219 by the operation, in case original operations are not present on |
3154 both paths of the conditional branch. | 3220 both paths of the conditional branch. |
3155 | 3221 |
3156 All the original operations found during the traversal are saved in the | 3222 All the original operations found during the traversal are saved in the |
3157 ORIGINAL_INSNS list. | 3223 ORIGINAL_INSNS list. |
3158 | 3224 |
3159 REG_RENAME_P->CROSSES_CALL is true, if there is a call insn on the path | 3225 REG_RENAME_P->CROSSES_CALL is true, if there is a call insn on the path |
3160 from INSN to original insn. In this case CALL_USED_REG_SET will be added | 3226 from INSN to original insn. In this case CALL_USED_REG_SET will be added |
3161 to unavailable hard regs at the point original operation is found. */ | 3227 to unavailable hard regs at the point original operation is found. */ |
3162 | 3228 |
3163 static bool | 3229 static bool |
3164 find_used_regs (insn_t insn, av_set_t orig_ops, regset used_regs, | 3230 find_used_regs (insn_t insn, av_set_t orig_ops, regset used_regs, |
3165 struct reg_rename *reg_rename_p, def_list_t *original_insns) | 3231 struct reg_rename *reg_rename_p, def_list_t *original_insns) |
3178 | 3244 |
3179 /* Init parameters for code_motion_path_driver. */ | 3245 /* Init parameters for code_motion_path_driver. */ |
3180 sparams.crosses_call = false; | 3246 sparams.crosses_call = false; |
3181 sparams.original_insns = original_insns; | 3247 sparams.original_insns = original_insns; |
3182 sparams.used_regs = used_regs; | 3248 sparams.used_regs = used_regs; |
3183 | 3249 |
3184 /* Set the appropriate hooks and data. */ | 3250 /* Set the appropriate hooks and data. */ |
3185 code_motion_path_driver_info = &fur_hooks; | 3251 code_motion_path_driver_info = &fur_hooks; |
3186 | 3252 |
3187 res = code_motion_path_driver (insn, orig_ops, NULL, &lparams, &sparams); | 3253 res = code_motion_path_driver (insn, orig_ops, NULL, &lparams, &sparams); |
3188 | 3254 |
3189 reg_rename_p->crosses_call |= sparams.crosses_call; | 3255 reg_rename_p->crosses_call |= sparams.crosses_call; |
3190 | 3256 |
3191 gcc_assert (res == 1); | 3257 gcc_assert (res == 1); |
3192 gcc_assert (original_insns && *original_insns); | 3258 gcc_assert (original_insns && *original_insns); |
3193 | 3259 |
3194 /* ??? We calculate whether an expression needs a check when computing | 3260 /* ??? We calculate whether an expression needs a check when computing |
3195 av sets. This information is not as precise as it could be due to | 3261 av sets. This information is not as precise as it could be due to |
3196 merging this bit in merge_expr. We can do better in find_used_regs, | 3262 merging this bit in merge_expr. We can do better in find_used_regs, |
3197 but we want to avoid multiple traversals of the same code motion | 3263 but we want to avoid multiple traversals of the same code motion |
3198 paths. */ | 3264 paths. */ |
3199 FOR_EACH_EXPR (expr, expr_iter, orig_ops) | 3265 FOR_EACH_EXPR (expr, expr_iter, orig_ops) |
3200 needs_spec_check_p |= EXPR_NEEDS_SPEC_CHECK_P (expr); | 3266 needs_spec_check_p |= EXPR_NEEDS_SPEC_CHECK_P (expr); |
3201 | 3267 |
3202 /* Mark hardware regs in REG_RENAME_P that are not suitable | 3268 /* Mark hardware regs in REG_RENAME_P that are not suitable |
3203 for renaming expr in INSN due to hardware restrictions (register class, | 3269 for renaming expr in INSN due to hardware restrictions (register class, |
3204 modes compatibility etc). */ | 3270 modes compatibility etc). */ |
3205 FOR_EACH_DEF (def, i, *original_insns) | 3271 FOR_EACH_DEF (def, i, *original_insns) |
3206 { | 3272 { |
3207 vinsn_t vinsn = INSN_VINSN (def->orig_insn); | 3273 vinsn_t vinsn = INSN_VINSN (def->orig_insn); |
3208 | 3274 |
3209 if (VINSN_SEPARABLE_P (vinsn)) | 3275 if (VINSN_SEPARABLE_P (vinsn)) |
3210 mark_unavailable_hard_regs (def, reg_rename_p, used_regs); | 3276 mark_unavailable_hard_regs (def, reg_rename_p, used_regs); |
3211 | 3277 |
3212 /* Do not allow clobbering of ld.[sa] address in case some of the | 3278 /* Do not allow clobbering of ld.[sa] address in case some of the |
3213 original operations need a check. */ | 3279 original operations need a check. */ |
3214 if (needs_spec_check_p) | 3280 if (needs_spec_check_p) |
3215 IOR_REG_SET (used_regs, VINSN_REG_USES (vinsn)); | 3281 IOR_REG_SET (used_regs, VINSN_REG_USES (vinsn)); |
3216 } | 3282 } |
3217 | 3283 |
3237 EXPR_PRIORITY_ADJ (expr) = new_priority - EXPR_PRIORITY (expr); | 3303 EXPR_PRIORITY_ADJ (expr) = new_priority - EXPR_PRIORITY (expr); |
3238 | 3304 |
3239 gcc_assert (EXPR_PRIORITY_ADJ (expr) >= 0); | 3305 gcc_assert (EXPR_PRIORITY_ADJ (expr) >= 0); |
3240 | 3306 |
3241 if (sched_verbose >= 2) | 3307 if (sched_verbose >= 2) |
3242 sel_print ("sel_target_adjust_priority: insn %d, %d +%d = %d.\n", | 3308 sel_print ("sel_target_adjust_priority: insn %d, %d +%d = %d.\n", |
3243 INSN_UID (EXPR_INSN_RTX (expr)), EXPR_PRIORITY (expr), | 3309 INSN_UID (EXPR_INSN_RTX (expr)), EXPR_PRIORITY (expr), |
3244 EXPR_PRIORITY_ADJ (expr), new_priority); | 3310 EXPR_PRIORITY_ADJ (expr), new_priority); |
3245 | 3311 |
3246 return new_priority; | 3312 return new_priority; |
3247 } | 3313 } |
3248 | 3314 |
3249 /* Rank two available exprs for schedule. Never return 0 here. */ | 3315 /* Rank two available exprs for schedule. Never return 0 here. */ |
3250 static int | 3316 static int |
3251 sel_rank_for_schedule (const void *x, const void *y) | 3317 sel_rank_for_schedule (const void *x, const void *y) |
3252 { | 3318 { |
3253 expr_t tmp = *(const expr_t *) y; | 3319 expr_t tmp = *(const expr_t *) y; |
3254 expr_t tmp2 = *(const expr_t *) x; | 3320 expr_t tmp2 = *(const expr_t *) x; |
3255 insn_t tmp_insn, tmp2_insn; | 3321 insn_t tmp_insn, tmp2_insn; |
3258 | 3324 |
3259 tmp_vinsn = EXPR_VINSN (tmp); | 3325 tmp_vinsn = EXPR_VINSN (tmp); |
3260 tmp2_vinsn = EXPR_VINSN (tmp2); | 3326 tmp2_vinsn = EXPR_VINSN (tmp2); |
3261 tmp_insn = EXPR_INSN_RTX (tmp); | 3327 tmp_insn = EXPR_INSN_RTX (tmp); |
3262 tmp2_insn = EXPR_INSN_RTX (tmp2); | 3328 tmp2_insn = EXPR_INSN_RTX (tmp2); |
3263 | 3329 |
3330 /* Schedule debug insns as early as possible. */ | |
3331 if (DEBUG_INSN_P (tmp_insn) && !DEBUG_INSN_P (tmp2_insn)) | |
3332 return -1; | |
3333 else if (DEBUG_INSN_P (tmp2_insn)) | |
3334 return 1; | |
3335 | |
3264 /* Prefer SCHED_GROUP_P insns to any others. */ | 3336 /* Prefer SCHED_GROUP_P insns to any others. */ |
3265 if (SCHED_GROUP_P (tmp_insn) != SCHED_GROUP_P (tmp2_insn)) | 3337 if (SCHED_GROUP_P (tmp_insn) != SCHED_GROUP_P (tmp2_insn)) |
3266 { | 3338 { |
3267 if (VINSN_UNIQUE_P (tmp_vinsn) && VINSN_UNIQUE_P (tmp2_vinsn)) | 3339 if (VINSN_UNIQUE_P (tmp_vinsn) && VINSN_UNIQUE_P (tmp2_vinsn)) |
3268 return SCHED_GROUP_P (tmp2_insn) ? 1 : -1; | 3340 return SCHED_GROUP_P (tmp2_insn) ? 1 : -1; |
3269 | 3341 |
3270 /* Now uniqueness means SCHED_GROUP_P is set, because schedule groups | 3342 /* Now uniqueness means SCHED_GROUP_P is set, because schedule groups |
3271 cannot be cloned. */ | 3343 cannot be cloned. */ |
3272 if (VINSN_UNIQUE_P (tmp2_vinsn)) | 3344 if (VINSN_UNIQUE_P (tmp2_vinsn)) |
3301 p1 = EXPR_PRIORITY (tmp) + EXPR_PRIORITY_ADJ (tmp); | 3373 p1 = EXPR_PRIORITY (tmp) + EXPR_PRIORITY_ADJ (tmp); |
3302 | 3374 |
3303 val = p2 * EXPR_USEFULNESS (tmp2) - p1 * EXPR_USEFULNESS (tmp); | 3375 val = p2 * EXPR_USEFULNESS (tmp2) - p1 * EXPR_USEFULNESS (tmp); |
3304 } | 3376 } |
3305 else | 3377 else |
3306 val = EXPR_PRIORITY (tmp2) - EXPR_PRIORITY (tmp) | 3378 val = EXPR_PRIORITY (tmp2) - EXPR_PRIORITY (tmp) |
3307 + EXPR_PRIORITY_ADJ (tmp2) - EXPR_PRIORITY_ADJ (tmp); | 3379 + EXPR_PRIORITY_ADJ (tmp2) - EXPR_PRIORITY_ADJ (tmp); |
3308 if (val) | 3380 if (val) |
3309 return val; | 3381 return val; |
3310 | 3382 |
3311 if (spec_info != NULL && spec_info->mask != 0) | 3383 if (spec_info != NULL && spec_info->mask != 0) |
3330 dw = dw2 - dw1; | 3402 dw = dw2 - dw1; |
3331 if (dw > (NO_DEP_WEAK / 8) || dw < -(NO_DEP_WEAK / 8)) | 3403 if (dw > (NO_DEP_WEAK / 8) || dw < -(NO_DEP_WEAK / 8)) |
3332 return dw; | 3404 return dw; |
3333 } | 3405 } |
3334 | 3406 |
3335 tmp_insn = EXPR_INSN_RTX (tmp); | |
3336 tmp2_insn = EXPR_INSN_RTX (tmp2); | |
3337 | |
3338 /* Prefer an old insn to a bookkeeping insn. */ | 3407 /* Prefer an old insn to a bookkeeping insn. */ |
3339 if (INSN_UID (tmp_insn) < first_emitted_uid | 3408 if (INSN_UID (tmp_insn) < first_emitted_uid |
3340 && INSN_UID (tmp2_insn) >= first_emitted_uid) | 3409 && INSN_UID (tmp2_insn) >= first_emitted_uid) |
3341 return -1; | 3410 return -1; |
3342 if (INSN_UID (tmp_insn) >= first_emitted_uid | 3411 if (INSN_UID (tmp_insn) >= first_emitted_uid |
3343 && INSN_UID (tmp2_insn) < first_emitted_uid) | 3412 && INSN_UID (tmp2_insn) < first_emitted_uid) |
3344 return 1; | 3413 return 1; |
3345 | 3414 |
3346 /* Prefer an insn with smaller UID, as a last resort. | 3415 /* Prefer an insn with smaller UID, as a last resort. |
3347 We can't safely use INSN_LUID as it is defined only for those insns | 3416 We can't safely use INSN_LUID as it is defined only for those insns |
3348 that are in the stream. */ | 3417 that are in the stream. */ |
3349 return INSN_UID (tmp_insn) - INSN_UID (tmp2_insn); | 3418 return INSN_UID (tmp_insn) - INSN_UID (tmp2_insn); |
3350 } | 3419 } |
3351 | 3420 |
3352 /* Filter out expressions from av set pointed to by AV_PTR | 3421 /* Filter out expressions from av set pointed to by AV_PTR |
3353 that are pipelined too many times. */ | 3422 that are pipelined too many times. */ |
3354 static void | 3423 static void |
3355 process_pipelined_exprs (av_set_t *av_ptr) | 3424 process_pipelined_exprs (av_set_t *av_ptr) |
3356 { | 3425 { |
3357 expr_t expr; | 3426 expr_t expr; |
3358 av_set_iterator si; | 3427 av_set_iterator si; |
3359 | 3428 |
3360 /* Don't pipeline already pipelined code as that would increase | 3429 /* Don't pipeline already pipelined code as that would increase |
3361 number of unnecessary register moves. */ | 3430 number of unnecessary register moves. */ |
3362 FOR_EACH_EXPR_1 (expr, si, av_ptr) | 3431 FOR_EACH_EXPR_1 (expr, si, av_ptr) |
3363 { | 3432 { |
3364 if (EXPR_SCHED_TIMES (expr) | 3433 if (EXPR_SCHED_TIMES (expr) |
3365 >= PARAM_VALUE (PARAM_SELSCHED_MAX_SCHED_TIMES)) | 3434 >= PARAM_VALUE (PARAM_SELSCHED_MAX_SCHED_TIMES)) |
3366 av_set_iter_remove (&si); | 3435 av_set_iter_remove (&si); |
3427 av_set_iter_remove (&si); | 3496 av_set_iter_remove (&si); |
3428 } | 3497 } |
3429 } | 3498 } |
3430 } | 3499 } |
3431 | 3500 |
3432 /* Search for any use-like insns in AV_PTR and decide on scheduling | 3501 /* Search for any use-like insns in AV_PTR and decide on scheduling |
3433 them. Return one when found, and NULL otherwise. | 3502 them. Return one when found, and NULL otherwise. |
3434 Note that we check here whether a USE could be scheduled to avoid | 3503 Note that we check here whether a USE could be scheduled to avoid |
3435 an infinite loop later. */ | 3504 an infinite loop later. */ |
3436 static expr_t | 3505 static expr_t |
3437 process_use_exprs (av_set_t *av_ptr) | 3506 process_use_exprs (av_set_t *av_ptr) |
3438 { | 3507 { |
3510 { | 3579 { |
3511 /* For non-separable instructions, the blocking insn can have | 3580 /* For non-separable instructions, the blocking insn can have |
3512 another pattern due to substitution, and we can't choose | 3581 another pattern due to substitution, and we can't choose |
3513 different register as in the above case. Check all registers | 3582 different register as in the above case. Check all registers |
3514 being written instead. */ | 3583 being written instead. */ |
3515 if (bitmap_intersect_p (VINSN_REG_SETS (vinsn), | 3584 if (bitmap_intersect_p (VINSN_REG_SETS (vinsn), |
3516 VINSN_REG_SETS (EXPR_VINSN (expr)))) | 3585 VINSN_REG_SETS (EXPR_VINSN (expr)))) |
3517 return true; | 3586 return true; |
3518 } | 3587 } |
3519 | 3588 |
3520 return false; | 3589 return false; |
3571 unsigned len = VEC_length (vinsn_t, *vinsn_vec); | 3640 unsigned len = VEC_length (vinsn_t, *vinsn_vec); |
3572 if (len > 0) | 3641 if (len > 0) |
3573 { | 3642 { |
3574 vinsn_t vinsn; | 3643 vinsn_t vinsn; |
3575 int n; | 3644 int n; |
3576 | 3645 |
3577 for (n = 0; VEC_iterate (vinsn_t, *vinsn_vec, n, vinsn); n++) | 3646 for (n = 0; VEC_iterate (vinsn_t, *vinsn_vec, n, vinsn); n++) |
3578 vinsn_detach (vinsn); | 3647 vinsn_detach (vinsn); |
3579 VEC_block_remove (vinsn_t, *vinsn_vec, 0, len); | 3648 VEC_block_remove (vinsn_t, *vinsn_vec, 0, len); |
3580 } | 3649 } |
3581 } | 3650 } |
3586 { | 3655 { |
3587 vinsn_attach (EXPR_VINSN (expr)); | 3656 vinsn_attach (EXPR_VINSN (expr)); |
3588 VEC_safe_push (vinsn_t, heap, *vinsn_vec, EXPR_VINSN (expr)); | 3657 VEC_safe_push (vinsn_t, heap, *vinsn_vec, EXPR_VINSN (expr)); |
3589 } | 3658 } |
3590 | 3659 |
3591 /* Free the vector representing blocked expressions. */ | 3660 /* Free the vector representing blocked expressions. */ |
3592 static void | 3661 static void |
3593 vinsn_vec_free (vinsn_vec_t *vinsn_vec) | 3662 vinsn_vec_free (vinsn_vec_t *vinsn_vec) |
3594 { | 3663 { |
3595 if (*vinsn_vec) | 3664 if (*vinsn_vec) |
3596 VEC_free (vinsn_t, heap, *vinsn_vec); | 3665 VEC_free (vinsn_t, heap, *vinsn_vec); |
3601 void sel_add_to_insn_priority (rtx insn, int amount) | 3670 void sel_add_to_insn_priority (rtx insn, int amount) |
3602 { | 3671 { |
3603 EXPR_PRIORITY_ADJ (INSN_EXPR (insn)) += amount; | 3672 EXPR_PRIORITY_ADJ (INSN_EXPR (insn)) += amount; |
3604 | 3673 |
3605 if (sched_verbose >= 2) | 3674 if (sched_verbose >= 2) |
3606 sel_print ("sel_add_to_insn_priority: insn %d, by %d (now %d+%d).\n", | 3675 sel_print ("sel_add_to_insn_priority: insn %d, by %d (now %d+%d).\n", |
3607 INSN_UID (insn), amount, EXPR_PRIORITY (INSN_EXPR (insn)), | 3676 INSN_UID (insn), amount, EXPR_PRIORITY (INSN_EXPR (insn)), |
3608 EXPR_PRIORITY_ADJ (INSN_EXPR (insn))); | 3677 EXPR_PRIORITY_ADJ (INSN_EXPR (insn))); |
3609 } | 3678 } |
3610 | 3679 |
3611 /* Turn AV into a vector, filter inappropriate insns and sort it. Return | 3680 /* Turn AV into a vector, filter inappropriate insns and sort it. Return |
3612 true if there is something to schedule. BNDS and FENCE are current | 3681 true if there is something to schedule. BNDS and FENCE are current |
3613 boundaries and fence, respectively. If we need to stall for some cycles | 3682 boundaries and fence, respectively. If we need to stall for some cycles |
3614 before an expr from AV would become available, write this number to | 3683 before an expr from AV would become available, write this number to |
3615 *PNEED_STALL. */ | 3684 *PNEED_STALL. */ |
3616 static bool | 3685 static bool |
3617 fill_vec_av_set (av_set_t av, blist_t bnds, fence_t fence, | 3686 fill_vec_av_set (av_set_t av, blist_t bnds, fence_t fence, |
3618 int *pneed_stall) | 3687 int *pneed_stall) |
3619 { | 3688 { |
3635 | 3704 |
3636 /* Turn the set into a vector for sorting and call sel_target_adjust_priority | 3705 /* Turn the set into a vector for sorting and call sel_target_adjust_priority |
3637 for each insn. */ | 3706 for each insn. */ |
3638 gcc_assert (VEC_empty (expr_t, vec_av_set)); | 3707 gcc_assert (VEC_empty (expr_t, vec_av_set)); |
3639 FOR_EACH_EXPR (expr, si, av) | 3708 FOR_EACH_EXPR (expr, si, av) |
3640 { | 3709 { |
3641 VEC_safe_push (expr_t, heap, vec_av_set, expr); | 3710 VEC_safe_push (expr_t, heap, vec_av_set, expr); |
3642 | 3711 |
3643 gcc_assert (EXPR_PRIORITY_ADJ (expr) == 0 || *pneed_stall); | 3712 gcc_assert (EXPR_PRIORITY_ADJ (expr) == 0 || *pneed_stall); |
3644 | 3713 |
3645 /* Adjust priority using target backend hook. */ | 3714 /* Adjust priority using target backend hook. */ |
3671 { | 3740 { |
3672 VEC_unordered_remove (expr_t, vec_av_set, n); | 3741 VEC_unordered_remove (expr_t, vec_av_set, n); |
3673 continue; | 3742 continue; |
3674 } | 3743 } |
3675 | 3744 |
3676 /* Set number of sched_next insns (just in case there | 3745 /* Set number of sched_next insns (just in case there |
3677 could be several). */ | 3746 could be several). */ |
3678 if (FENCE_SCHED_NEXT (fence)) | 3747 if (FENCE_SCHED_NEXT (fence)) |
3679 sched_next_worked++; | 3748 sched_next_worked++; |
3680 | 3749 |
3681 /* Check all liveness requirements and try renaming. | 3750 /* Check all liveness requirements and try renaming. |
3682 FIXME: try to minimize calls to this. */ | 3751 FIXME: try to minimize calls to this. */ |
3683 target_available = EXPR_TARGET_AVAILABLE (expr); | 3752 target_available = EXPR_TARGET_AVAILABLE (expr); |
3684 | 3753 |
3685 /* If insn was already scheduled on the current fence, | 3754 /* If insn was already scheduled on the current fence, |
3686 set TARGET_AVAILABLE to -1 no matter what expr's attribute says. */ | 3755 set TARGET_AVAILABLE to -1 no matter what expr's attribute says. */ |
3697 if (sched_verbose >= 4) | 3766 if (sched_verbose >= 4) |
3698 sel_print ("Expr %d is blocked by bookkeeping inserted earlier\n", | 3767 sel_print ("Expr %d is blocked by bookkeeping inserted earlier\n", |
3699 INSN_UID (insn)); | 3768 INSN_UID (insn)); |
3700 continue; | 3769 continue; |
3701 } | 3770 } |
3702 | 3771 |
3703 if (target_available == true) | 3772 if (target_available == true) |
3704 { | 3773 { |
3705 /* Do nothing -- we can use an existing register. */ | 3774 /* Do nothing -- we can use an existing register. */ |
3706 is_orig_reg_p = EXPR_SEPARABLE_P (expr); | 3775 is_orig_reg_p = EXPR_SEPARABLE_P (expr); |
3707 } | 3776 } |
3708 else if (/* Non-separable instruction will never | 3777 else if (/* Non-separable instruction will never |
3709 get another register. */ | 3778 get another register. */ |
3710 (target_available == false | 3779 (target_available == false |
3711 && !EXPR_SEPARABLE_P (expr)) | 3780 && !EXPR_SEPARABLE_P (expr)) |
3712 /* Don't try to find a register for low-priority expression. */ | 3781 /* Don't try to find a register for low-priority expression. */ |
3713 || (int) VEC_length (expr_t, vec_av_set) - 1 - n >= max_insns_to_rename | 3782 || (int) VEC_length (expr_t, vec_av_set) - 1 - n >= max_insns_to_rename |
3715 || (EXPR_SPEC_DONE_DS (expr) & BEGIN_DATA) | 3784 || (EXPR_SPEC_DONE_DS (expr) & BEGIN_DATA) |
3716 || ! find_best_reg_for_expr (expr, bnds, &is_orig_reg_p)) | 3785 || ! find_best_reg_for_expr (expr, bnds, &is_orig_reg_p)) |
3717 { | 3786 { |
3718 VEC_unordered_remove (expr_t, vec_av_set, n); | 3787 VEC_unordered_remove (expr_t, vec_av_set, n); |
3719 if (sched_verbose >= 4) | 3788 if (sched_verbose >= 4) |
3720 sel_print ("Expr %d has no suitable target register\n", | 3789 sel_print ("Expr %d has no suitable target register\n", |
3721 INSN_UID (insn)); | 3790 INSN_UID (insn)); |
3722 continue; | 3791 continue; |
3723 } | 3792 } |
3724 | 3793 |
3725 /* Filter expressions that need to be renamed or speculated when | 3794 /* Filter expressions that need to be renamed or speculated when |
3783 EXPR_PRIORITY (expr) + need_cycles); | 3852 EXPR_PRIORITY (expr) + need_cycles); |
3784 | 3853 |
3785 if (need_cycles > 0) | 3854 if (need_cycles > 0) |
3786 { | 3855 { |
3787 stalled++; | 3856 stalled++; |
3788 min_need_stall = (min_need_stall < 0 | 3857 min_need_stall = (min_need_stall < 0 |
3789 ? need_cycles | 3858 ? need_cycles |
3790 : MIN (min_need_stall, need_cycles)); | 3859 : MIN (min_need_stall, need_cycles)); |
3791 VEC_unordered_remove (expr_t, vec_av_set, n); | 3860 VEC_unordered_remove (expr_t, vec_av_set, n); |
3792 | 3861 |
3793 if (sched_verbose >= 4) | 3862 if (sched_verbose >= 4) |
3794 sel_print ("Expr %d is not ready until cycle %d (cached)\n", | 3863 sel_print ("Expr %d is not ready until cycle %d (cached)\n", |
3795 INSN_UID (insn), | 3864 INSN_UID (insn), |
3796 FENCE_READY_TICKS (fence)[INSN_UID (insn)]); | 3865 FENCE_READY_TICKS (fence)[INSN_UID (insn)]); |
3797 continue; | 3866 continue; |
3798 } | 3867 } |
3799 } | 3868 } |
3800 | 3869 |
3801 /* Now resort to dependence analysis to find whether EXPR might be | 3870 /* Now resort to dependence analysis to find whether EXPR might be |
3802 stalled due to dependencies from FENCE's context. */ | 3871 stalled due to dependencies from FENCE's context. */ |
3803 need_cycles = tick_check_p (expr, dc, fence); | 3872 need_cycles = tick_check_p (expr, dc, fence); |
3804 new_prio = EXPR_PRIORITY (expr) + EXPR_PRIORITY_ADJ (expr) + need_cycles; | 3873 new_prio = EXPR_PRIORITY (expr) + EXPR_PRIORITY_ADJ (expr) + need_cycles; |
3805 | 3874 |
3806 if (EXPR_ORIG_SCHED_CYCLE (expr) <= 0) | 3875 if (EXPR_ORIG_SCHED_CYCLE (expr) <= 0) |
3810 if (need_cycles > 0) | 3879 if (need_cycles > 0) |
3811 { | 3880 { |
3812 if (INSN_UID (insn) >= FENCE_READY_TICKS_SIZE (fence)) | 3881 if (INSN_UID (insn) >= FENCE_READY_TICKS_SIZE (fence)) |
3813 { | 3882 { |
3814 int new_size = INSN_UID (insn) * 3 / 2; | 3883 int new_size = INSN_UID (insn) * 3 / 2; |
3815 | 3884 |
3816 FENCE_READY_TICKS (fence) | 3885 FENCE_READY_TICKS (fence) |
3817 = (int *) xrecalloc (FENCE_READY_TICKS (fence), | 3886 = (int *) xrecalloc (FENCE_READY_TICKS (fence), |
3818 new_size, FENCE_READY_TICKS_SIZE (fence), | 3887 new_size, FENCE_READY_TICKS_SIZE (fence), |
3819 sizeof (int)); | 3888 sizeof (int)); |
3820 } | 3889 } |
3821 FENCE_READY_TICKS (fence)[INSN_UID (insn)] | 3890 FENCE_READY_TICKS (fence)[INSN_UID (insn)] |
3822 = FENCE_CYCLE (fence) + need_cycles; | 3891 = FENCE_CYCLE (fence) + need_cycles; |
3823 | 3892 |
3824 stalled++; | 3893 stalled++; |
3825 min_need_stall = (min_need_stall < 0 | 3894 min_need_stall = (min_need_stall < 0 |
3826 ? need_cycles | 3895 ? need_cycles |
3827 : MIN (min_need_stall, need_cycles)); | 3896 : MIN (min_need_stall, need_cycles)); |
3828 | 3897 |
3829 VEC_unordered_remove (expr_t, vec_av_set, n); | 3898 VEC_unordered_remove (expr_t, vec_av_set, n); |
3830 | 3899 |
3831 if (sched_verbose >= 4) | 3900 if (sched_verbose >= 4) |
3832 sel_print ("Expr %d is not ready yet until cycle %d\n", | 3901 sel_print ("Expr %d is not ready yet until cycle %d\n", |
3833 INSN_UID (insn), | 3902 INSN_UID (insn), |
3834 FENCE_READY_TICKS (fence)[INSN_UID (insn)]); | 3903 FENCE_READY_TICKS (fence)[INSN_UID (insn)]); |
3835 continue; | 3904 continue; |
3836 } | 3905 } |
3837 | 3906 |
3862 gcc_assert (min_need_stall == 0); | 3931 gcc_assert (min_need_stall == 0); |
3863 | 3932 |
3864 /* Sort the vector. */ | 3933 /* Sort the vector. */ |
3865 qsort (VEC_address (expr_t, vec_av_set), VEC_length (expr_t, vec_av_set), | 3934 qsort (VEC_address (expr_t, vec_av_set), VEC_length (expr_t, vec_av_set), |
3866 sizeof (expr_t), sel_rank_for_schedule); | 3935 sizeof (expr_t), sel_rank_for_schedule); |
3867 | 3936 |
3868 if (sched_verbose >= 4) | 3937 if (sched_verbose >= 4) |
3869 { | 3938 { |
3870 sel_print ("Total ready exprs: %d, stalled: %d\n", | 3939 sel_print ("Total ready exprs: %d, stalled: %d\n", |
3871 VEC_length (expr_t, vec_av_set), stalled); | 3940 VEC_length (expr_t, vec_av_set), stalled); |
3872 sel_print ("Sorted av set (%d): ", VEC_length (expr_t, vec_av_set)); | 3941 sel_print ("Sorted av set (%d): ", VEC_length (expr_t, vec_av_set)); |
3873 for (n = 0; VEC_iterate (expr_t, vec_av_set, n, expr); n++) | 3942 for (n = 0; VEC_iterate (expr_t, vec_av_set, n, expr); n++) |
3874 dump_expr (expr); | 3943 dump_expr (expr); |
3875 sel_print ("\n"); | 3944 sel_print ("\n"); |
3888 expr_t expr; | 3957 expr_t expr; |
3889 | 3958 |
3890 /* Allocate and fill the ready list from the sorted vector. */ | 3959 /* Allocate and fill the ready list from the sorted vector. */ |
3891 ready.n_ready = VEC_length (expr_t, vec_av_set); | 3960 ready.n_ready = VEC_length (expr_t, vec_av_set); |
3892 ready.first = ready.n_ready - 1; | 3961 ready.first = ready.n_ready - 1; |
3893 | 3962 |
3894 gcc_assert (ready.n_ready > 0); | 3963 gcc_assert (ready.n_ready > 0); |
3895 | 3964 |
3896 if (ready.n_ready > max_issue_size) | 3965 if (ready.n_ready > max_issue_size) |
3897 { | 3966 { |
3898 max_issue_size = ready.n_ready; | 3967 max_issue_size = ready.n_ready; |
3899 sched_extend_ready_list (ready.n_ready); | 3968 sched_extend_ready_list (ready.n_ready); |
3900 } | 3969 } |
3901 | 3970 |
3902 for (n = 0; VEC_iterate (expr_t, vec_av_set, n, expr); n++) | 3971 for (n = 0; VEC_iterate (expr_t, vec_av_set, n, expr); n++) |
3903 { | 3972 { |
3904 vinsn_t vi = EXPR_VINSN (expr); | 3973 vinsn_t vi = EXPR_VINSN (expr); |
3905 insn_t insn = VINSN_INSN_RTX (vi); | 3974 insn_t insn = VINSN_INSN_RTX (vi); |
3906 | 3975 |
3909 } | 3978 } |
3910 } | 3979 } |
3911 | 3980 |
3912 /* Initialize ready list from *AV_PTR for the max_issue () call. | 3981 /* Initialize ready list from *AV_PTR for the max_issue () call. |
3913 If any unrecognizable insn found in *AV_PTR, return it (and skip | 3982 If any unrecognizable insn found in *AV_PTR, return it (and skip |
3914 max_issue). BND and FENCE are current boundary and fence, | 3983 max_issue). BND and FENCE are current boundary and fence, |
3915 respectively. If we need to stall for some cycles before an expr | 3984 respectively. If we need to stall for some cycles before an expr |
3916 from *AV_PTR would become available, write this number to *PNEED_STALL. */ | 3985 from *AV_PTR would become available, write this number to *PNEED_STALL. */ |
3917 static expr_t | 3986 static expr_t |
3918 fill_ready_list (av_set_t *av_ptr, blist_t bnds, fence_t fence, | 3987 fill_ready_list (av_set_t *av_ptr, blist_t bnds, fence_t fence, |
3919 int *pneed_stall) | 3988 int *pneed_stall) |
3920 { | 3989 { |
3921 expr_t expr; | 3990 expr_t expr; |
3922 | 3991 |
3923 /* We do not support multiple boundaries per fence. */ | 3992 /* We do not support multiple boundaries per fence. */ |
3924 gcc_assert (BLIST_NEXT (bnds) == NULL); | 3993 gcc_assert (BLIST_NEXT (bnds) == NULL); |
3925 | 3994 |
3926 /* Process expressions required special handling, i.e. pipelined, | 3995 /* Process expressions required special handling, i.e. pipelined, |
3927 speculative and recog() < 0 expressions first. */ | 3996 speculative and recog() < 0 expressions first. */ |
3928 process_pipelined_exprs (av_ptr); | 3997 process_pipelined_exprs (av_ptr); |
3929 process_spec_exprs (av_ptr); | 3998 process_spec_exprs (av_ptr); |
3930 | 3999 |
3931 /* A USE could be scheduled immediately. */ | 4000 /* A USE could be scheduled immediately. */ |
3950 | 4019 |
3951 /* Wrapper for dfa_new_cycle (). Returns TRUE if cycle was advanced. */ | 4020 /* Wrapper for dfa_new_cycle (). Returns TRUE if cycle was advanced. */ |
3952 static bool | 4021 static bool |
3953 sel_dfa_new_cycle (insn_t insn, fence_t fence) | 4022 sel_dfa_new_cycle (insn_t insn, fence_t fence) |
3954 { | 4023 { |
3955 int last_scheduled_cycle = FENCE_LAST_SCHEDULED_INSN (fence) | 4024 int last_scheduled_cycle = FENCE_LAST_SCHEDULED_INSN (fence) |
3956 ? INSN_SCHED_CYCLE (FENCE_LAST_SCHEDULED_INSN (fence)) | 4025 ? INSN_SCHED_CYCLE (FENCE_LAST_SCHEDULED_INSN (fence)) |
3957 : FENCE_CYCLE (fence) - 1; | 4026 : FENCE_CYCLE (fence) - 1; |
3958 bool res = false; | 4027 bool res = false; |
3959 int sort_p = 0; | 4028 int sort_p = 0; |
3960 | 4029 |
3961 if (!targetm.sched.dfa_new_cycle) | 4030 if (!targetm.sched.dfa_new_cycle) |
4013 } | 4082 } |
4014 else if (targetm.sched.reorder2 | 4083 else if (targetm.sched.reorder2 |
4015 && !SCHED_GROUP_P (ready_element (&ready, 0))) | 4084 && !SCHED_GROUP_P (ready_element (&ready, 0))) |
4016 { | 4085 { |
4017 if (ready.n_ready == 1) | 4086 if (ready.n_ready == 1) |
4018 issue_more = | 4087 issue_more = |
4019 targetm.sched.reorder2 (sched_dump, sched_verbose, | 4088 targetm.sched.reorder2 (sched_dump, sched_verbose, |
4020 ready_lastpos (&ready), | 4089 ready_lastpos (&ready), |
4021 &ready.n_ready, FENCE_CYCLE (fence)); | 4090 &ready.n_ready, FENCE_CYCLE (fence)); |
4022 else | 4091 else |
4023 { | 4092 { |
4034 ++ready.n_ready; | 4103 ++ready.n_ready; |
4035 } | 4104 } |
4036 | 4105 |
4037 ran_hook = true; | 4106 ran_hook = true; |
4038 } | 4107 } |
4039 else | 4108 else |
4040 issue_more = issue_rate; | 4109 issue_more = issue_rate; |
4041 | 4110 |
4042 /* Ensure that ready list and vec_av_set are in line with each other, | 4111 /* Ensure that ready list and vec_av_set are in line with each other, |
4043 i.e. vec_av_set[i] == ready_element (&ready, i). */ | 4112 i.e. vec_av_set[i] == ready_element (&ready, i). */ |
4044 if (issue_more && ran_hook) | 4113 if (issue_more && ran_hook) |
4055 for (j = i; j < n; j++) | 4124 for (j = i; j < n; j++) |
4056 if (EXPR_INSN_RTX (vec[j]) == arr[i]) | 4125 if (EXPR_INSN_RTX (vec[j]) == arr[i]) |
4057 break; | 4126 break; |
4058 gcc_assert (j < n); | 4127 gcc_assert (j < n); |
4059 | 4128 |
4060 tmp = vec[i]; | 4129 tmp = vec[i]; |
4061 vec[i] = vec[j]; | 4130 vec[i] = vec[j]; |
4062 vec[j] = tmp; | 4131 vec[j] = tmp; |
4063 } | 4132 } |
4064 } | 4133 } |
4065 | 4134 |
4066 return issue_more; | 4135 return issue_more; |
4067 } | 4136 } |
4068 | 4137 |
4069 /* Return an EXPR correponding to INDEX element of ready list, if | 4138 /* Return an EXPR correponding to INDEX element of ready list, if |
4070 FOLLOW_READY_ELEMENT is true (i.e., an expr of | 4139 FOLLOW_READY_ELEMENT is true (i.e., an expr of |
4071 ready_element (&ready, INDEX) will be returned), and to INDEX element of | 4140 ready_element (&ready, INDEX) will be returned), and to INDEX element of |
4072 ready.vec otherwise. */ | 4141 ready.vec otherwise. */ |
4073 static inline expr_t | 4142 static inline expr_t |
4074 find_expr_for_ready (int index, bool follow_ready_element) | 4143 find_expr_for_ready (int index, bool follow_ready_element) |
4075 { | 4144 { |
4076 expr_t expr; | 4145 expr_t expr; |
4088 of such insns found. */ | 4157 of such insns found. */ |
4089 static int | 4158 static int |
4090 invoke_dfa_lookahead_guard (void) | 4159 invoke_dfa_lookahead_guard (void) |
4091 { | 4160 { |
4092 int i, n; | 4161 int i, n; |
4093 bool have_hook | 4162 bool have_hook |
4094 = targetm.sched.first_cycle_multipass_dfa_lookahead_guard != NULL; | 4163 = targetm.sched.first_cycle_multipass_dfa_lookahead_guard != NULL; |
4095 | 4164 |
4096 if (sched_verbose >= 2) | 4165 if (sched_verbose >= 2) |
4097 sel_print ("ready after reorder: "); | 4166 sel_print ("ready after reorder: "); |
4098 | 4167 |
4100 { | 4169 { |
4101 expr_t expr; | 4170 expr_t expr; |
4102 insn_t insn; | 4171 insn_t insn; |
4103 int r; | 4172 int r; |
4104 | 4173 |
4105 /* In this loop insn is Ith element of the ready list given by | 4174 /* In this loop insn is Ith element of the ready list given by |
4106 ready_element, not Ith element of ready.vec. */ | 4175 ready_element, not Ith element of ready.vec. */ |
4107 insn = ready_element (&ready, i); | 4176 insn = ready_element (&ready, i); |
4108 | 4177 |
4109 if (! have_hook || i == 0) | 4178 if (! have_hook || i == 0) |
4110 r = 0; | 4179 r = 0; |
4111 else | 4180 else |
4112 r = !targetm.sched.first_cycle_multipass_dfa_lookahead_guard (insn); | 4181 r = !targetm.sched.first_cycle_multipass_dfa_lookahead_guard (insn); |
4113 | 4182 |
4114 gcc_assert (INSN_CODE (insn) >= 0); | 4183 gcc_assert (INSN_CODE (insn) >= 0); |
4115 | 4184 |
4116 /* Only insns with ready_try = 0 can get here | 4185 /* Only insns with ready_try = 0 can get here |
4117 from fill_ready_list. */ | 4186 from fill_ready_list. */ |
4118 gcc_assert (ready_try [i] == 0); | 4187 gcc_assert (ready_try [i] == 0); |
4119 ready_try[i] = r; | 4188 ready_try[i] = r; |
4120 if (!r) | 4189 if (!r) |
4121 n++; | 4190 n++; |
4122 | 4191 |
4123 expr = find_expr_for_ready (i, true); | 4192 expr = find_expr_for_ready (i, true); |
4124 | 4193 |
4125 if (sched_verbose >= 2) | 4194 if (sched_verbose >= 2) |
4126 { | 4195 { |
4127 dump_vinsn (EXPR_VINSN (expr)); | 4196 dump_vinsn (EXPR_VINSN (expr)); |
4128 sel_print (":%d; ", ready_try[i]); | 4197 sel_print (":%d; ", ready_try[i]); |
4129 } | 4198 } |
4150 if (! min_spec_expr) | 4219 if (! min_spec_expr) |
4151 { | 4220 { |
4152 min_spec_insn = ready_element (&ready, i); | 4221 min_spec_insn = ready_element (&ready, i); |
4153 min_spec_expr = find_expr_for_ready (i, true); | 4222 min_spec_expr = find_expr_for_ready (i, true); |
4154 } | 4223 } |
4155 | 4224 |
4156 cur_insn = ready_element (&ready, i); | 4225 cur_insn = ready_element (&ready, i); |
4157 cur_expr = find_expr_for_ready (i, true); | 4226 cur_expr = find_expr_for_ready (i, true); |
4158 | 4227 |
4159 if (EXPR_SPEC (cur_expr) > EXPR_SPEC (min_spec_expr)) | 4228 if (EXPR_SPEC (cur_expr) > EXPR_SPEC (min_spec_expr)) |
4160 break; | 4229 break; |
4169 sel_print ("privileged_n: %d insns with SPEC %d\n", | 4238 sel_print ("privileged_n: %d insns with SPEC %d\n", |
4170 privileged_n, privileged_n ? EXPR_SPEC (min_spec_expr) : -1); | 4239 privileged_n, privileged_n ? EXPR_SPEC (min_spec_expr) : -1); |
4171 return privileged_n; | 4240 return privileged_n; |
4172 } | 4241 } |
4173 | 4242 |
4174 /* Call the rest of the hooks after the choice was made. Return | 4243 /* Call the rest of the hooks after the choice was made. Return |
4175 the number of insns that still can be issued given that the current | 4244 the number of insns that still can be issued given that the current |
4176 number is ISSUE_MORE. FENCE and BEST_INSN are the current fence | 4245 number is ISSUE_MORE. FENCE and BEST_INSN are the current fence |
4177 and the insn chosen for scheduling, respectively. */ | 4246 and the insn chosen for scheduling, respectively. */ |
4178 static int | 4247 static int |
4179 invoke_aftermath_hooks (fence_t fence, rtx best_insn, int issue_more) | 4248 invoke_aftermath_hooks (fence_t fence, rtx best_insn, int issue_more) |
4180 { | 4249 { |
4181 gcc_assert (INSN_P (best_insn)); | 4250 gcc_assert (INSN_P (best_insn)); |
4182 | 4251 |
4183 /* First, call dfa_new_cycle, and then variable_issue, if available. */ | 4252 /* First, call dfa_new_cycle, and then variable_issue, if available. */ |
4184 sel_dfa_new_cycle (best_insn, fence); | 4253 sel_dfa_new_cycle (best_insn, fence); |
4185 | 4254 |
4186 if (targetm.sched.variable_issue) | 4255 if (targetm.sched.variable_issue) |
4187 { | 4256 { |
4188 memcpy (curr_state, FENCE_STATE (fence), dfa_state_size); | 4257 memcpy (curr_state, FENCE_STATE (fence), dfa_state_size); |
4189 issue_more = | 4258 issue_more = |
4190 targetm.sched.variable_issue (sched_dump, sched_verbose, best_insn, | 4259 targetm.sched.variable_issue (sched_dump, sched_verbose, best_insn, |
4191 issue_more); | 4260 issue_more); |
4192 memcpy (FENCE_STATE (fence), curr_state, dfa_state_size); | 4261 memcpy (FENCE_STATE (fence), curr_state, dfa_state_size); |
4193 } | 4262 } |
4194 else if (GET_CODE (PATTERN (best_insn)) != USE | 4263 else if (GET_CODE (PATTERN (best_insn)) != USE |
4216 else if (cost == 0) | 4285 else if (cost == 0) |
4217 return 1; | 4286 return 1; |
4218 return cost; | 4287 return cost; |
4219 } | 4288 } |
4220 | 4289 |
4221 /* Return the cost of issuing EXPR on the FENCE as estimated by DFA. | 4290 /* Return the cost of issuing EXPR on the FENCE as estimated by DFA. |
4222 This function properly handles ASMs, USEs etc. */ | 4291 This function properly handles ASMs, USEs etc. */ |
4223 static int | 4292 static int |
4224 get_expr_cost (expr_t expr, fence_t fence) | 4293 get_expr_cost (expr_t expr, fence_t fence) |
4225 { | 4294 { |
4226 rtx insn = EXPR_INSN_RTX (expr); | 4295 rtx insn = EXPR_INSN_RTX (expr); |
4227 | 4296 |
4228 if (recog_memoized (insn) < 0) | 4297 if (recog_memoized (insn) < 0) |
4229 { | 4298 { |
4230 if (!FENCE_STARTS_CYCLE_P (fence) | 4299 if (!FENCE_STARTS_CYCLE_P (fence) |
4231 /* FIXME: Is this condition necessary? */ | 4300 /* FIXME: Is this condition necessary? */ |
4232 && VINSN_UNIQUE_P (EXPR_VINSN (expr)) | 4301 && VINSN_UNIQUE_P (EXPR_VINSN (expr)) |
4233 && INSN_ASM_P (insn)) | 4302 && INSN_ASM_P (insn)) |
4234 /* This is asm insn which is tryed to be issued on the | 4303 /* This is asm insn which is tryed to be issued on the |
4235 cycle not first. Issue it on the next cycle. */ | 4304 cycle not first. Issue it on the next cycle. */ |
4243 } | 4312 } |
4244 else | 4313 else |
4245 return estimate_insn_cost (insn, FENCE_STATE (fence)); | 4314 return estimate_insn_cost (insn, FENCE_STATE (fence)); |
4246 } | 4315 } |
4247 | 4316 |
4248 /* Find the best insn for scheduling, either via max_issue or just take | 4317 /* Find the best insn for scheduling, either via max_issue or just take |
4249 the most prioritized available. */ | 4318 the most prioritized available. */ |
4250 static int | 4319 static int |
4251 choose_best_insn (fence_t fence, int privileged_n, int *index) | 4320 choose_best_insn (fence_t fence, int privileged_n, int *index) |
4252 { | 4321 { |
4253 int can_issue = 0; | 4322 int can_issue = 0; |
4290 } | 4359 } |
4291 | 4360 |
4292 return can_issue; | 4361 return can_issue; |
4293 } | 4362 } |
4294 | 4363 |
4295 /* Choose the best expr from *AV_VLIW_PTR and a suitable register for it. | 4364 /* Choose the best expr from *AV_VLIW_PTR and a suitable register for it. |
4296 BNDS and FENCE are current boundaries and scheduling fence respectively. | 4365 BNDS and FENCE are current boundaries and scheduling fence respectively. |
4297 Return the expr found and NULL if nothing can be issued atm. | 4366 Return the expr found and NULL if nothing can be issued atm. |
4298 Write to PNEED_STALL the number of cycles to stall if no expr was found. */ | 4367 Write to PNEED_STALL the number of cycles to stall if no expr was found. */ |
4299 static expr_t | 4368 static expr_t |
4300 find_best_expr (av_set_t *av_vliw_ptr, blist_t bnds, fence_t fence, | 4369 find_best_expr (av_set_t *av_vliw_ptr, blist_t bnds, fence_t fence, |
4301 int *pneed_stall) | 4370 int *pneed_stall) |
4302 { | 4371 { |
4303 expr_t best; | 4372 expr_t best; |
4304 | 4373 |
4305 /* Choose the best insn for scheduling via: | 4374 /* Choose the best insn for scheduling via: |
4306 1) sorting the ready list based on priority; | 4375 1) sorting the ready list based on priority; |
4307 2) calling the reorder hook; | 4376 2) calling the reorder hook; |
4308 3) calling max_issue. */ | 4377 3) calling max_issue. */ |
4309 best = fill_ready_list (av_vliw_ptr, bnds, fence, pneed_stall); | 4378 best = fill_ready_list (av_vliw_ptr, bnds, fence, pneed_stall); |
4312 int privileged_n, index, avail_n; | 4381 int privileged_n, index, avail_n; |
4313 | 4382 |
4314 can_issue_more = invoke_reorder_hooks (fence); | 4383 can_issue_more = invoke_reorder_hooks (fence); |
4315 if (can_issue_more > 0) | 4384 if (can_issue_more > 0) |
4316 { | 4385 { |
4317 /* Try choosing the best insn until we find one that is could be | 4386 /* Try choosing the best insn until we find one that is could be |
4318 scheduled due to liveness restrictions on its destination register. | 4387 scheduled due to liveness restrictions on its destination register. |
4319 In the future, we'd like to choose once and then just probe insns | 4388 In the future, we'd like to choose once and then just probe insns |
4320 in the order of their priority. */ | 4389 in the order of their priority. */ |
4321 avail_n = invoke_dfa_lookahead_guard (); | 4390 avail_n = invoke_dfa_lookahead_guard (); |
4322 privileged_n = calculate_privileged_insns (); | 4391 privileged_n = calculate_privileged_insns (); |
4323 can_issue_more = choose_best_insn (fence, privileged_n, &index); | 4392 can_issue_more = choose_best_insn (fence, privileged_n, &index); |
4324 if (can_issue_more) | 4393 if (can_issue_more) |
4325 best = find_expr_for_ready (index, true); | 4394 best = find_expr_for_ready (index, true); |
4326 } | 4395 } |
4327 /* We had some available insns, so if we can't issue them, | 4396 /* We had some available insns, so if we can't issue them, |
4328 we have a stall. */ | 4397 we have a stall. */ |
4329 if (can_issue_more == 0) | 4398 if (can_issue_more == 0) |
4330 { | 4399 { |
4331 best = NULL; | 4400 best = NULL; |
4332 *pneed_stall = 1; | 4401 *pneed_stall = 1; |
4338 can_issue_more = invoke_aftermath_hooks (fence, EXPR_INSN_RTX (best), | 4407 can_issue_more = invoke_aftermath_hooks (fence, EXPR_INSN_RTX (best), |
4339 can_issue_more); | 4408 can_issue_more); |
4340 if (can_issue_more == 0) | 4409 if (can_issue_more == 0) |
4341 *pneed_stall = 1; | 4410 *pneed_stall = 1; |
4342 } | 4411 } |
4343 | 4412 |
4344 if (sched_verbose >= 2) | 4413 if (sched_verbose >= 2) |
4345 { | 4414 { |
4346 if (best != NULL) | 4415 if (best != NULL) |
4347 { | 4416 { |
4348 sel_print ("Best expression (vliw form): "); | 4417 sel_print ("Best expression (vliw form): "); |
4358 | 4427 |
4359 | 4428 |
4360 /* Functions that implement the core of the scheduler. */ | 4429 /* Functions that implement the core of the scheduler. */ |
4361 | 4430 |
4362 | 4431 |
4363 /* Emit an instruction from EXPR with SEQNO and VINSN after | 4432 /* Emit an instruction from EXPR with SEQNO and VINSN after |
4364 PLACE_TO_INSERT. */ | 4433 PLACE_TO_INSERT. */ |
4365 static insn_t | 4434 static insn_t |
4366 emit_insn_from_expr_after (expr_t expr, vinsn_t vinsn, int seqno, | 4435 emit_insn_from_expr_after (expr_t expr, vinsn_t vinsn, int seqno, |
4367 insn_t place_to_insert) | 4436 insn_t place_to_insert) |
4368 { | 4437 { |
4369 /* This assert fails when we have identical instructions | 4438 /* This assert fails when we have identical instructions |
4370 one of which dominates the other. In this case move_op () | 4439 one of which dominates the other. In this case move_op () |
4371 finds the first instruction and doesn't search for second one. | 4440 finds the first instruction and doesn't search for second one. |
4375 gcc_assert (!INSN_IN_STREAM_P (EXPR_INSN_RTX (expr))); | 4444 gcc_assert (!INSN_IN_STREAM_P (EXPR_INSN_RTX (expr))); |
4376 | 4445 |
4377 if (EXPR_WAS_RENAMED (expr)) | 4446 if (EXPR_WAS_RENAMED (expr)) |
4378 { | 4447 { |
4379 unsigned regno = expr_dest_regno (expr); | 4448 unsigned regno = expr_dest_regno (expr); |
4380 | 4449 |
4381 if (HARD_REGISTER_NUM_P (regno)) | 4450 if (HARD_REGISTER_NUM_P (regno)) |
4382 { | 4451 { |
4383 df_set_regs_ever_live (regno, true); | 4452 df_set_regs_ever_live (regno, true); |
4384 reg_rename_tick[regno] = ++reg_rename_this_tick; | 4453 reg_rename_tick[regno] = ++reg_rename_this_tick; |
4385 } | 4454 } |
4386 } | 4455 } |
4387 | 4456 |
4388 return sel_gen_insn_from_expr_after (expr, vinsn, seqno, | 4457 return sel_gen_insn_from_expr_after (expr, vinsn, seqno, |
4389 place_to_insert); | 4458 place_to_insert); |
4390 } | 4459 } |
4391 | 4460 |
4392 /* Return TRUE if BB can hold bookkeeping code. */ | 4461 /* Return TRUE if BB can hold bookkeeping code. */ |
4393 static bool | 4462 static bool |
4410 } | 4479 } |
4411 | 4480 |
4412 /* Attempt to find a block that can hold bookkeeping code for path(s) incoming | 4481 /* Attempt to find a block that can hold bookkeeping code for path(s) incoming |
4413 into E2->dest, except from E1->src (there may be a sequence of empty basic | 4482 into E2->dest, except from E1->src (there may be a sequence of empty basic |
4414 blocks between E1->src and E2->dest). Return found block, or NULL if new | 4483 blocks between E1->src and E2->dest). Return found block, or NULL if new |
4415 one must be created. */ | 4484 one must be created. If LAX holds, don't assume there is a simple path |
4485 from E1->src to E2->dest. */ | |
4416 static basic_block | 4486 static basic_block |
4417 find_block_for_bookkeeping (edge e1, edge e2) | 4487 find_block_for_bookkeeping (edge e1, edge e2, bool lax) |
4418 { | 4488 { |
4419 basic_block candidate_block = NULL; | 4489 basic_block candidate_block = NULL; |
4420 edge e; | 4490 edge e; |
4421 | 4491 |
4422 /* Loop over edges from E1 to E2, inclusive. */ | 4492 /* Loop over edges from E1 to E2, inclusive. */ |
4423 for (e = e1; ; e = EDGE_SUCC (e->dest, 0)) | 4493 for (e = e1; !lax || e->dest != EXIT_BLOCK_PTR; e = EDGE_SUCC (e->dest, 0)) |
4424 { | 4494 { |
4425 if (EDGE_COUNT (e->dest->preds) == 2) | 4495 if (EDGE_COUNT (e->dest->preds) == 2) |
4426 { | 4496 { |
4427 if (candidate_block == NULL) | 4497 if (candidate_block == NULL) |
4428 candidate_block = (EDGE_PRED (e->dest, 0) == e | 4498 candidate_block = (EDGE_PRED (e->dest, 0) == e |
4436 else if (EDGE_COUNT (e->dest->preds) > 2) | 4506 else if (EDGE_COUNT (e->dest->preds) > 2) |
4437 /* Several edges leading to path from e1 to e2 from aside. */ | 4507 /* Several edges leading to path from e1 to e2 from aside. */ |
4438 return NULL; | 4508 return NULL; |
4439 | 4509 |
4440 if (e == e2) | 4510 if (e == e2) |
4441 return (block_valid_for_bookkeeping_p (candidate_block) | 4511 return ((!lax || candidate_block) |
4512 && block_valid_for_bookkeeping_p (candidate_block) | |
4442 ? candidate_block | 4513 ? candidate_block |
4443 : NULL); | 4514 : NULL); |
4444 } | 4515 |
4516 if (lax && EDGE_COUNT (e->dest->succs) != 1) | |
4517 return NULL; | |
4518 } | |
4519 | |
4520 if (lax) | |
4521 return NULL; | |
4522 | |
4445 gcc_unreachable (); | 4523 gcc_unreachable (); |
4446 } | 4524 } |
4447 | 4525 |
4448 /* Create new basic block for bookkeeping code for path(s) incoming into | 4526 /* Create new basic block for bookkeeping code for path(s) incoming into |
4449 E2->dest, except from E1->src. Return created block. */ | 4527 E2->dest, except from E1->src. Return created block. */ |
4483 sel_redirect_edge_and_branch (e1, new_bb); | 4561 sel_redirect_edge_and_branch (e1, new_bb); |
4484 | 4562 |
4485 gcc_assert (e1->dest == new_bb); | 4563 gcc_assert (e1->dest == new_bb); |
4486 gcc_assert (sel_bb_empty_p (bb)); | 4564 gcc_assert (sel_bb_empty_p (bb)); |
4487 | 4565 |
4566 /* To keep basic block numbers in sync between debug and non-debug | |
4567 compilations, we have to rotate blocks here. Consider that we | |
4568 started from (a,b)->d, (c,d)->e, and d contained only debug | |
4569 insns. It would have been removed before if the debug insns | |
4570 weren't there, so we'd have split e rather than d. So what we do | |
4571 now is to swap the block numbers of new_bb and | |
4572 single_succ(new_bb) == e, so that the insns that were in e before | |
4573 get the new block number. */ | |
4574 | |
4575 if (MAY_HAVE_DEBUG_INSNS) | |
4576 { | |
4577 basic_block succ; | |
4578 insn_t insn = sel_bb_head (new_bb); | |
4579 insn_t last; | |
4580 | |
4581 if (DEBUG_INSN_P (insn) | |
4582 && single_succ_p (new_bb) | |
4583 && (succ = single_succ (new_bb)) | |
4584 && succ != EXIT_BLOCK_PTR | |
4585 && DEBUG_INSN_P ((last = sel_bb_end (new_bb)))) | |
4586 { | |
4587 while (insn != last && (DEBUG_INSN_P (insn) || NOTE_P (insn))) | |
4588 insn = NEXT_INSN (insn); | |
4589 | |
4590 if (insn == last) | |
4591 { | |
4592 sel_global_bb_info_def gbi; | |
4593 sel_region_bb_info_def rbi; | |
4594 int i; | |
4595 | |
4596 if (sched_verbose >= 2) | |
4597 sel_print ("Swapping block ids %i and %i\n", | |
4598 new_bb->index, succ->index); | |
4599 | |
4600 i = new_bb->index; | |
4601 new_bb->index = succ->index; | |
4602 succ->index = i; | |
4603 | |
4604 SET_BASIC_BLOCK (new_bb->index, new_bb); | |
4605 SET_BASIC_BLOCK (succ->index, succ); | |
4606 | |
4607 memcpy (&gbi, SEL_GLOBAL_BB_INFO (new_bb), sizeof (gbi)); | |
4608 memcpy (SEL_GLOBAL_BB_INFO (new_bb), SEL_GLOBAL_BB_INFO (succ), | |
4609 sizeof (gbi)); | |
4610 memcpy (SEL_GLOBAL_BB_INFO (succ), &gbi, sizeof (gbi)); | |
4611 | |
4612 memcpy (&rbi, SEL_REGION_BB_INFO (new_bb), sizeof (rbi)); | |
4613 memcpy (SEL_REGION_BB_INFO (new_bb), SEL_REGION_BB_INFO (succ), | |
4614 sizeof (rbi)); | |
4615 memcpy (SEL_REGION_BB_INFO (succ), &rbi, sizeof (rbi)); | |
4616 | |
4617 i = BLOCK_TO_BB (new_bb->index); | |
4618 BLOCK_TO_BB (new_bb->index) = BLOCK_TO_BB (succ->index); | |
4619 BLOCK_TO_BB (succ->index) = i; | |
4620 | |
4621 i = CONTAINING_RGN (new_bb->index); | |
4622 CONTAINING_RGN (new_bb->index) = CONTAINING_RGN (succ->index); | |
4623 CONTAINING_RGN (succ->index) = i; | |
4624 | |
4625 for (i = 0; i < current_nr_blocks; i++) | |
4626 if (BB_TO_BLOCK (i) == succ->index) | |
4627 BB_TO_BLOCK (i) = new_bb->index; | |
4628 else if (BB_TO_BLOCK (i) == new_bb->index) | |
4629 BB_TO_BLOCK (i) = succ->index; | |
4630 | |
4631 FOR_BB_INSNS (new_bb, insn) | |
4632 if (INSN_P (insn)) | |
4633 EXPR_ORIG_BB_INDEX (INSN_EXPR (insn)) = new_bb->index; | |
4634 | |
4635 FOR_BB_INSNS (succ, insn) | |
4636 if (INSN_P (insn)) | |
4637 EXPR_ORIG_BB_INDEX (INSN_EXPR (insn)) = succ->index; | |
4638 | |
4639 if (bitmap_bit_p (code_motion_visited_blocks, new_bb->index)) | |
4640 { | |
4641 bitmap_set_bit (code_motion_visited_blocks, succ->index); | |
4642 bitmap_clear_bit (code_motion_visited_blocks, new_bb->index); | |
4643 } | |
4644 | |
4645 gcc_assert (LABEL_P (BB_HEAD (new_bb)) | |
4646 && LABEL_P (BB_HEAD (succ))); | |
4647 | |
4648 if (sched_verbose >= 4) | |
4649 sel_print ("Swapping code labels %i and %i\n", | |
4650 CODE_LABEL_NUMBER (BB_HEAD (new_bb)), | |
4651 CODE_LABEL_NUMBER (BB_HEAD (succ))); | |
4652 | |
4653 i = CODE_LABEL_NUMBER (BB_HEAD (new_bb)); | |
4654 CODE_LABEL_NUMBER (BB_HEAD (new_bb)) | |
4655 = CODE_LABEL_NUMBER (BB_HEAD (succ)); | |
4656 CODE_LABEL_NUMBER (BB_HEAD (succ)) = i; | |
4657 } | |
4658 } | |
4659 } | |
4660 | |
4488 return bb; | 4661 return bb; |
4489 } | 4662 } |
4490 | 4663 |
4491 /* Return insn after which we must insert bookkeeping code for path(s) incoming | 4664 /* Return insn after which we must insert bookkeeping code for path(s) incoming |
4492 into E2->dest, except from E1->src. */ | 4665 into E2->dest, except from E1->src. */ |
4494 find_place_for_bookkeeping (edge e1, edge e2) | 4667 find_place_for_bookkeeping (edge e1, edge e2) |
4495 { | 4668 { |
4496 insn_t place_to_insert; | 4669 insn_t place_to_insert; |
4497 /* Find a basic block that can hold bookkeeping. If it can be found, do not | 4670 /* Find a basic block that can hold bookkeeping. If it can be found, do not |
4498 create new basic block, but insert bookkeeping there. */ | 4671 create new basic block, but insert bookkeeping there. */ |
4499 basic_block book_block = find_block_for_bookkeeping (e1, e2); | 4672 basic_block book_block = find_block_for_bookkeeping (e1, e2, FALSE); |
4673 | |
4674 if (book_block) | |
4675 { | |
4676 place_to_insert = BB_END (book_block); | |
4677 | |
4678 /* Don't use a block containing only debug insns for | |
4679 bookkeeping, this causes scheduling differences between debug | |
4680 and non-debug compilations, for the block would have been | |
4681 removed already. */ | |
4682 if (DEBUG_INSN_P (place_to_insert)) | |
4683 { | |
4684 rtx insn = sel_bb_head (book_block); | |
4685 | |
4686 while (insn != place_to_insert && | |
4687 (DEBUG_INSN_P (insn) || NOTE_P (insn))) | |
4688 insn = NEXT_INSN (insn); | |
4689 | |
4690 if (insn == place_to_insert) | |
4691 book_block = NULL; | |
4692 } | |
4693 } | |
4500 | 4694 |
4501 if (!book_block) | 4695 if (!book_block) |
4502 book_block = create_block_for_bookkeeping (e1, e2); | 4696 { |
4503 | 4697 book_block = create_block_for_bookkeeping (e1, e2); |
4504 place_to_insert = BB_END (book_block); | 4698 place_to_insert = BB_END (book_block); |
4699 if (sched_verbose >= 9) | |
4700 sel_print ("New block is %i, split from bookkeeping block %i\n", | |
4701 EDGE_SUCC (book_block, 0)->dest->index, book_block->index); | |
4702 } | |
4703 else | |
4704 { | |
4705 if (sched_verbose >= 9) | |
4706 sel_print ("Pre-existing bookkeeping block is %i\n", book_block->index); | |
4707 } | |
4505 | 4708 |
4506 /* If basic block ends with a jump, insert bookkeeping code right before it. */ | 4709 /* If basic block ends with a jump, insert bookkeeping code right before it. */ |
4507 if (INSN_P (place_to_insert) && control_flow_insn_p (place_to_insert)) | 4710 if (INSN_P (place_to_insert) && control_flow_insn_p (place_to_insert)) |
4508 place_to_insert = PREV_INSN (place_to_insert); | 4711 place_to_insert = PREV_INSN (place_to_insert); |
4509 | 4712 |
4519 rtx next; | 4722 rtx next; |
4520 | 4723 |
4521 /* Check if we are about to insert bookkeeping copy before a jump, and use | 4724 /* Check if we are about to insert bookkeeping copy before a jump, and use |
4522 jump's seqno for the copy; otherwise, use JOIN_POINT's seqno. */ | 4725 jump's seqno for the copy; otherwise, use JOIN_POINT's seqno. */ |
4523 next = NEXT_INSN (place_to_insert); | 4726 next = NEXT_INSN (place_to_insert); |
4524 if (INSN_P (next) | 4727 if (INSN_P (next) |
4525 && JUMP_P (next) | 4728 && JUMP_P (next) |
4526 && BLOCK_FOR_INSN (next) == BLOCK_FOR_INSN (place_to_insert)) | 4729 && BLOCK_FOR_INSN (next) == BLOCK_FOR_INSN (place_to_insert)) |
4527 seqno = INSN_SEQNO (next); | 4730 { |
4731 gcc_assert (INSN_SCHED_TIMES (next) == 0); | |
4732 seqno = INSN_SEQNO (next); | |
4733 } | |
4528 else if (INSN_SEQNO (join_point) > 0) | 4734 else if (INSN_SEQNO (join_point) > 0) |
4529 seqno = INSN_SEQNO (join_point); | 4735 seqno = INSN_SEQNO (join_point); |
4530 else | 4736 else |
4531 seqno = get_seqno_by_preds (place_to_insert); | 4737 { |
4532 | 4738 seqno = get_seqno_by_preds (place_to_insert); |
4739 | |
4740 /* Sometimes the fences can move in such a way that there will be | |
4741 no instructions with positive seqno around this bookkeeping. | |
4742 This means that there will be no way to get to it by a regular | |
4743 fence movement. Never mind because we pick up such pieces for | |
4744 rescheduling anyways, so any positive value will do for now. */ | |
4745 if (seqno < 0) | |
4746 { | |
4747 gcc_assert (pipelining_p); | |
4748 seqno = 1; | |
4749 } | |
4750 } | |
4751 | |
4533 gcc_assert (seqno > 0); | 4752 gcc_assert (seqno > 0); |
4534 return seqno; | 4753 return seqno; |
4535 } | 4754 } |
4536 | 4755 |
4537 /* Insert bookkeeping copy of C_EXPS's insn after PLACE_TO_INSERT, assigning | 4756 /* Insert bookkeeping copy of C_EXPS's insn after PLACE_TO_INSERT, assigning |
4569 sel_print ("Generating bookkeeping insn (%d->%d)\n", e1->src->index, | 4788 sel_print ("Generating bookkeeping insn (%d->%d)\n", e1->src->index, |
4570 e2->dest->index); | 4789 e2->dest->index); |
4571 | 4790 |
4572 join_point = sel_bb_head (e2->dest); | 4791 join_point = sel_bb_head (e2->dest); |
4573 place_to_insert = find_place_for_bookkeeping (e1, e2); | 4792 place_to_insert = find_place_for_bookkeeping (e1, e2); |
4793 if (!place_to_insert) | |
4794 return NULL; | |
4574 new_seqno = find_seqno_for_bookkeeping (place_to_insert, join_point); | 4795 new_seqno = find_seqno_for_bookkeeping (place_to_insert, join_point); |
4575 need_to_exchange_data_sets | 4796 need_to_exchange_data_sets |
4576 = sel_bb_empty_p (BLOCK_FOR_INSN (place_to_insert)); | 4797 = sel_bb_empty_p (BLOCK_FOR_INSN (place_to_insert)); |
4577 | 4798 |
4578 new_insn = emit_bookkeeping_insn (place_to_insert, c_expr, new_seqno); | 4799 new_insn = emit_bookkeeping_insn (place_to_insert, c_expr, new_seqno); |
4591 | 4812 |
4592 stat_bookkeeping_copies++; | 4813 stat_bookkeeping_copies++; |
4593 return BLOCK_FOR_INSN (new_insn); | 4814 return BLOCK_FOR_INSN (new_insn); |
4594 } | 4815 } |
4595 | 4816 |
4596 /* Remove from AV_PTR all insns that may need bookkeeping when scheduling | 4817 /* Remove from AV_PTR all insns that may need bookkeeping when scheduling |
4597 on FENCE, but we are unable to copy them. */ | 4818 on FENCE, but we are unable to copy them. */ |
4598 static void | 4819 static void |
4599 remove_insns_that_need_bookkeeping (fence_t fence, av_set_t *av_ptr) | 4820 remove_insns_that_need_bookkeeping (fence_t fence, av_set_t *av_ptr) |
4600 { | 4821 { |
4601 expr_t expr; | 4822 expr_t expr; |
4602 av_set_iterator i; | 4823 av_set_iterator i; |
4603 | 4824 |
4604 /* An expression does not need bookkeeping if it is available on all paths | 4825 /* An expression does not need bookkeeping if it is available on all paths |
4605 from current block to original block and current block dominates | 4826 from current block to original block and current block dominates |
4606 original block. We check availability on all paths by examining | 4827 original block. We check availability on all paths by examining |
4607 EXPR_SPEC; this is not equivalent, because it may be positive even | 4828 EXPR_SPEC; this is not equivalent, because it may be positive even |
4608 if expr is available on all paths (but if expr is not available on | 4829 if expr is available on all paths (but if expr is not available on |
4609 any path, EXPR_SPEC will be positive). */ | 4830 any path, EXPR_SPEC will be positive). */ |
4610 | 4831 |
4611 FOR_EACH_EXPR_1 (expr, i, av_ptr) | 4832 FOR_EACH_EXPR_1 (expr, i, av_ptr) |
4612 { | 4833 { |
4613 if (!control_flow_insn_p (EXPR_INSN_RTX (expr)) | 4834 if (!control_flow_insn_p (EXPR_INSN_RTX (expr)) |
4636 (p8) mov [r14]=r23 | 4857 (p8) mov [r14]=r23 |
4637 (!p8) jump L1;; | 4858 (!p8) jump L1;; |
4638 NOTE BASIC BLOCK: | 4859 NOTE BASIC BLOCK: |
4639 ... | 4860 ... |
4640 | 4861 |
4641 We can schedule jump one cycle earlier, than mov, because they cannot be | 4862 We can schedule jump one cycle earlier, than mov, because they cannot be |
4642 executed together as their predicates are mutually exclusive. | 4863 executed together as their predicates are mutually exclusive. |
4643 | 4864 |
4644 This is done in this way: first, new fallthrough basic block is created | 4865 This is done in this way: first, new fallthrough basic block is created |
4645 after jump (it is always can be done, because there already should be a | 4866 after jump (it is always can be done, because there already should be a |
4646 fallthrough block, where control flow goes in case of predicate being true - | 4867 fallthrough block, where control flow goes in case of predicate being true - |
4647 in our example; otherwise there should be a dependence between those | 4868 in our example; otherwise there should be a dependence between those |
4648 instructions and jump and we cannot schedule jump right now); | 4869 instructions and jump and we cannot schedule jump right now); |
4649 next, all instructions between jump and current scheduling point are moved | 4870 next, all instructions between jump and current scheduling point are moved |
4650 to this new block. And the result is this: | 4871 to this new block. And the result is this: |
4651 | 4872 |
4652 NOTE BASIC BLOCK: | 4873 NOTE BASIC BLOCK: |
4653 (!p8) jump L1 <- current scheduling point | 4874 (!p8) jump L1 <- current scheduling point |
4654 NOTE BASIC BLOCK: <- bb header | 4875 NOTE BASIC BLOCK: <- bb header |
4730 } | 4951 } |
4731 | 4952 |
4732 /* Remove nops generated during move_op for preventing removal of empty | 4953 /* Remove nops generated during move_op for preventing removal of empty |
4733 basic blocks. */ | 4954 basic blocks. */ |
4734 static void | 4955 static void |
4735 remove_temp_moveop_nops (void) | 4956 remove_temp_moveop_nops (bool full_tidying) |
4736 { | 4957 { |
4737 int i; | 4958 int i; |
4738 insn_t insn; | 4959 insn_t insn; |
4739 | 4960 |
4740 for (i = 0; VEC_iterate (insn_t, vec_temp_moveop_nops, i, insn); i++) | 4961 for (i = 0; VEC_iterate (insn_t, vec_temp_moveop_nops, i, insn); i++) |
4741 { | 4962 { |
4742 gcc_assert (INSN_NOP_P (insn)); | 4963 gcc_assert (INSN_NOP_P (insn)); |
4743 return_nop_to_pool (insn); | 4964 return_nop_to_pool (insn, full_tidying); |
4744 } | 4965 } |
4745 | 4966 |
4746 /* Empty the vector. */ | 4967 /* Empty the vector. */ |
4747 if (VEC_length (insn_t, vec_temp_moveop_nops) > 0) | 4968 if (VEC_length (insn_t, vec_temp_moveop_nops) > 0) |
4748 VEC_block_remove (insn_t, vec_temp_moveop_nops, 0, | 4969 VEC_block_remove (insn_t, vec_temp_moveop_nops, 0, |
4749 VEC_length (insn_t, vec_temp_moveop_nops)); | 4970 VEC_length (insn_t, vec_temp_moveop_nops)); |
4750 } | 4971 } |
4751 | 4972 |
4752 /* Records the maximal UID before moving up an instruction. Used for | 4973 /* Records the maximal UID before moving up an instruction. Used for |
4753 distinguishing between bookkeeping copies and original insns. */ | 4974 distinguishing between bookkeeping copies and original insns. */ |
4772 if (EXPR_INSN_RTX (expr) != next) | 4993 if (EXPR_INSN_RTX (expr) != next) |
4773 av_set_iter_remove (&av_it); | 4994 av_set_iter_remove (&av_it); |
4774 } | 4995 } |
4775 } | 4996 } |
4776 | 4997 |
4777 /* Compute available instructions on BNDS. FENCE is the current fence. Write | 4998 /* Compute available instructions on BNDS. FENCE is the current fence. Write |
4778 the computed set to *AV_VLIW_P. */ | 4999 the computed set to *AV_VLIW_P. */ |
4779 static void | 5000 static void |
4780 compute_av_set_on_boundaries (fence_t fence, blist_t bnds, av_set_t *av_vliw_p) | 5001 compute_av_set_on_boundaries (fence_t fence, blist_t bnds, av_set_t *av_vliw_p) |
4781 { | 5002 { |
4782 if (sched_verbose >= 2) | 5003 if (sched_verbose >= 2) |
4817 | 5038 |
4818 av_set_clear (&BND_AV1 (bnd)); | 5039 av_set_clear (&BND_AV1 (bnd)); |
4819 BND_AV1 (bnd) = av_set_copy (BND_AV (bnd)); | 5040 BND_AV1 (bnd) = av_set_copy (BND_AV (bnd)); |
4820 | 5041 |
4821 moveup_set_inside_insn_group (&BND_AV1 (bnd), NULL); | 5042 moveup_set_inside_insn_group (&BND_AV1 (bnd), NULL); |
4822 | 5043 |
4823 av1_copy = av_set_copy (BND_AV1 (bnd)); | 5044 av1_copy = av_set_copy (BND_AV1 (bnd)); |
4824 av_set_union_and_clear (av_vliw_p, &av1_copy, NULL); | 5045 av_set_union_and_clear (av_vliw_p, &av1_copy, NULL); |
4825 } | 5046 } |
4826 | 5047 |
4827 if (sched_verbose >= 2) | 5048 if (sched_verbose >= 2) |
4830 dump_av_set (*av_vliw_p); | 5051 dump_av_set (*av_vliw_p); |
4831 sel_print ("\n"); | 5052 sel_print ("\n"); |
4832 } | 5053 } |
4833 } | 5054 } |
4834 | 5055 |
4835 /* Calculate the sequential av set on BND corresponding to the EXPR_VLIW | 5056 /* Calculate the sequential av set on BND corresponding to the EXPR_VLIW |
4836 expression. When FOR_MOVEOP is true, also replace the register of | 5057 expression. When FOR_MOVEOP is true, also replace the register of |
4837 expressions found with the register from EXPR_VLIW. */ | 5058 expressions found with the register from EXPR_VLIW. */ |
4838 static av_set_t | 5059 static av_set_t |
4839 find_sequential_best_exprs (bnd_t bnd, expr_t expr_vliw, bool for_moveop) | 5060 find_sequential_best_exprs (bnd_t bnd, expr_t expr_vliw, bool for_moveop) |
4840 { | 5061 { |
4841 av_set_t expr_seq = NULL; | 5062 av_set_t expr_seq = NULL; |
4842 expr_t expr; | 5063 expr_t expr; |
4843 av_set_iterator i; | 5064 av_set_iterator i; |
4844 | 5065 |
4845 FOR_EACH_EXPR (expr, i, BND_AV (bnd)) | 5066 FOR_EACH_EXPR (expr, i, BND_AV (bnd)) |
4846 { | 5067 { |
4847 if (equal_after_moveup_path_p (expr, NULL, expr_vliw)) | 5068 if (equal_after_moveup_path_p (expr, NULL, expr_vliw)) |
4848 { | 5069 { |
4849 if (for_moveop) | 5070 if (for_moveop) |
4850 { | 5071 { |
4851 /* The sequential expression has the right form to pass | 5072 /* The sequential expression has the right form to pass |
4852 to move_op except when renaming happened. Put the | 5073 to move_op except when renaming happened. Put the |
4853 correct register in EXPR then. */ | 5074 correct register in EXPR then. */ |
4854 if (EXPR_SEPARABLE_P (expr) && REG_P (EXPR_LHS (expr))) | 5075 if (EXPR_SEPARABLE_P (expr) && REG_P (EXPR_LHS (expr))) |
4855 { | 5076 { |
4856 if (expr_dest_regno (expr) != expr_dest_regno (expr_vliw)) | 5077 if (expr_dest_regno (expr) != expr_dest_regno (expr_vliw)) |
4857 { | 5078 { |
4858 replace_dest_with_reg_in_expr (expr, EXPR_LHS (expr_vliw)); | 5079 replace_dest_with_reg_in_expr (expr, EXPR_LHS (expr_vliw)); |
4859 stat_renamed_scheduled++; | 5080 stat_renamed_scheduled++; |
4860 } | 5081 } |
4861 /* Also put the correct TARGET_AVAILABLE bit on the expr. | 5082 /* Also put the correct TARGET_AVAILABLE bit on the expr. |
4862 This is needed when renaming came up with original | 5083 This is needed when renaming came up with original |
4863 register. */ | 5084 register. */ |
4864 else if (EXPR_TARGET_AVAILABLE (expr) | 5085 else if (EXPR_TARGET_AVAILABLE (expr) |
4865 != EXPR_TARGET_AVAILABLE (expr_vliw)) | 5086 != EXPR_TARGET_AVAILABLE (expr_vliw)) |
4866 { | 5087 { |
4867 gcc_assert (EXPR_TARGET_AVAILABLE (expr_vliw) == 1); | 5088 gcc_assert (EXPR_TARGET_AVAILABLE (expr_vliw) == 1); |
4868 EXPR_TARGET_AVAILABLE (expr) = 1; | 5089 EXPR_TARGET_AVAILABLE (expr) = 1; |
4869 } | 5090 } |
4871 if (EXPR_WAS_SUBSTITUTED (expr)) | 5092 if (EXPR_WAS_SUBSTITUTED (expr)) |
4872 stat_substitutions_total++; | 5093 stat_substitutions_total++; |
4873 } | 5094 } |
4874 | 5095 |
4875 av_set_add (&expr_seq, expr); | 5096 av_set_add (&expr_seq, expr); |
4876 | 5097 |
4877 /* With substitution inside insn group, it is possible | 5098 /* With substitution inside insn group, it is possible |
4878 that more than one expression in expr_seq will correspond | 5099 that more than one expression in expr_seq will correspond |
4879 to expr_vliw. In this case, choose one as the attempt to | 5100 to expr_vliw. In this case, choose one as the attempt to |
4880 move both leads to miscompiles. */ | 5101 move both leads to miscompiles. */ |
4881 break; | 5102 break; |
4882 } | 5103 } |
4883 } | 5104 } |
4884 | 5105 |
4886 { | 5107 { |
4887 sel_print ("Best expression(s) (sequential form): "); | 5108 sel_print ("Best expression(s) (sequential form): "); |
4888 dump_av_set (expr_seq); | 5109 dump_av_set (expr_seq); |
4889 sel_print ("\n"); | 5110 sel_print ("\n"); |
4890 } | 5111 } |
4891 | 5112 |
4892 return expr_seq; | 5113 return expr_seq; |
4893 } | 5114 } |
4894 | 5115 |
4895 | 5116 |
4896 /* Move nop to previous block. */ | 5117 /* Move nop to previous block. */ |
4897 static void ATTRIBUTE_UNUSED | 5118 static void ATTRIBUTE_UNUSED |
4898 move_nop_to_previous_block (insn_t nop, basic_block prev_bb) | 5119 move_nop_to_previous_block (insn_t nop, basic_block prev_bb) |
4899 { | 5120 { |
4900 insn_t prev_insn, next_insn, note; | 5121 insn_t prev_insn, next_insn, note; |
4901 | 5122 |
4902 gcc_assert (sel_bb_head_p (nop) | 5123 gcc_assert (sel_bb_head_p (nop) |
4903 && prev_bb == BLOCK_FOR_INSN (nop)->prev_bb); | 5124 && prev_bb == BLOCK_FOR_INSN (nop)->prev_bb); |
4904 note = bb_note (BLOCK_FOR_INSN (nop)); | 5125 note = bb_note (BLOCK_FOR_INSN (nop)); |
4905 prev_insn = sel_bb_end (prev_bb); | 5126 prev_insn = sel_bb_end (prev_bb); |
4906 next_insn = NEXT_INSN (nop); | 5127 next_insn = NEXT_INSN (nop); |
4907 gcc_assert (prev_insn != NULL_RTX | 5128 gcc_assert (prev_insn != NULL_RTX |
4931 if (/* If this is not the first insn scheduled. */ | 5152 if (/* If this is not the first insn scheduled. */ |
4932 BND_PTR (bnd)) | 5153 BND_PTR (bnd)) |
4933 { | 5154 { |
4934 /* Add it after last scheduled. */ | 5155 /* Add it after last scheduled. */ |
4935 place_to_insert = ILIST_INSN (BND_PTR (bnd)); | 5156 place_to_insert = ILIST_INSN (BND_PTR (bnd)); |
5157 if (DEBUG_INSN_P (place_to_insert)) | |
5158 { | |
5159 ilist_t l = BND_PTR (bnd); | |
5160 while ((l = ILIST_NEXT (l)) && | |
5161 DEBUG_INSN_P (ILIST_INSN (l))) | |
5162 ; | |
5163 if (!l) | |
5164 place_to_insert = NULL; | |
5165 } | |
4936 } | 5166 } |
4937 else | 5167 else |
5168 place_to_insert = NULL; | |
5169 | |
5170 if (!place_to_insert) | |
4938 { | 5171 { |
4939 /* Add it before BND_TO. The difference is in the | 5172 /* Add it before BND_TO. The difference is in the |
4940 basic block, where INSN will be added. */ | 5173 basic block, where INSN will be added. */ |
4941 place_to_insert = get_nop_from_pool (BND_TO (bnd)); | 5174 place_to_insert = get_nop_from_pool (BND_TO (bnd)); |
4942 gcc_assert (BLOCK_FOR_INSN (place_to_insert) | 5175 gcc_assert (BLOCK_FOR_INSN (place_to_insert) |
4944 } | 5177 } |
4945 | 5178 |
4946 return place_to_insert; | 5179 return place_to_insert; |
4947 } | 5180 } |
4948 | 5181 |
4949 /* Find original instructions for EXPR_SEQ and move it to BND boundary. | 5182 /* Find original instructions for EXPR_SEQ and move it to BND boundary. |
4950 Return the expression to emit in C_EXPR. */ | 5183 Return the expression to emit in C_EXPR. */ |
4951 static bool | 5184 static bool |
4952 move_exprs_to_boundary (bnd_t bnd, expr_t expr_vliw, | 5185 move_exprs_to_boundary (bnd_t bnd, expr_t expr_vliw, |
4953 av_set_t expr_seq, expr_t c_expr) | 5186 av_set_t expr_seq, expr_t c_expr) |
4954 { | 5187 { |
4955 bool b, should_move; | 5188 bool b, should_move; |
4956 unsigned book_uid; | 5189 unsigned book_uid; |
4957 bitmap_iterator bi; | 5190 bitmap_iterator bi; |
4964 n_bookkeeping_copies_before_moveop = stat_bookkeeping_copies; | 5197 n_bookkeeping_copies_before_moveop = stat_bookkeeping_copies; |
4965 max_uid_before_move_op = get_max_uid (); | 5198 max_uid_before_move_op = get_max_uid (); |
4966 bitmap_clear (current_copies); | 5199 bitmap_clear (current_copies); |
4967 bitmap_clear (current_originators); | 5200 bitmap_clear (current_originators); |
4968 | 5201 |
4969 b = move_op (BND_TO (bnd), expr_seq, expr_vliw, | 5202 b = move_op (BND_TO (bnd), expr_seq, expr_vliw, |
4970 get_dest_from_orig_ops (expr_seq), c_expr, &should_move); | 5203 get_dest_from_orig_ops (expr_seq), c_expr, &should_move); |
4971 | 5204 |
4972 /* We should be able to find the expression we've chosen for | 5205 /* We should be able to find the expression we've chosen for |
4973 scheduling. */ | 5206 scheduling. */ |
4974 gcc_assert (b); | 5207 gcc_assert (b); |
4975 | 5208 |
4976 if (stat_bookkeeping_copies > n_bookkeeping_copies_before_moveop) | 5209 if (stat_bookkeeping_copies > n_bookkeeping_copies_before_moveop) |
4977 stat_insns_needed_bookkeeping++; | 5210 stat_insns_needed_bookkeeping++; |
4978 | 5211 |
4979 EXECUTE_IF_SET_IN_BITMAP (current_copies, 0, book_uid, bi) | 5212 EXECUTE_IF_SET_IN_BITMAP (current_copies, 0, book_uid, bi) |
4980 { | 5213 { |
4981 /* We allocate these bitmaps lazily. */ | 5214 /* We allocate these bitmaps lazily. */ |
4982 if (! INSN_ORIGINATORS_BY_UID (book_uid)) | 5215 if (! INSN_ORIGINATORS_BY_UID (book_uid)) |
4983 INSN_ORIGINATORS_BY_UID (book_uid) = BITMAP_ALLOC (NULL); | 5216 INSN_ORIGINATORS_BY_UID (book_uid) = BITMAP_ALLOC (NULL); |
4984 | 5217 |
4985 bitmap_copy (INSN_ORIGINATORS_BY_UID (book_uid), | 5218 bitmap_copy (INSN_ORIGINATORS_BY_UID (book_uid), |
4986 current_originators); | 5219 current_originators); |
4987 } | 5220 } |
4988 | 5221 |
4989 return should_move; | 5222 return should_move; |
4990 } | 5223 } |
5001 for (i = 0, p = (unsigned char *) state; i < size; i++) | 5234 for (i = 0, p = (unsigned char *) state; i < size; i++) |
5002 sel_print (" %d", p[i]); | 5235 sel_print (" %d", p[i]); |
5003 sel_print ("\n"); | 5236 sel_print ("\n"); |
5004 } | 5237 } |
5005 | 5238 |
5006 /* Advance state on FENCE with INSN. Return true if INSN is | 5239 /* Advance state on FENCE with INSN. Return true if INSN is |
5007 an ASM, and we should advance state once more. */ | 5240 an ASM, and we should advance state once more. */ |
5008 static bool | 5241 static bool |
5009 advance_state_on_fence (fence_t fence, insn_t insn) | 5242 advance_state_on_fence (fence_t fence, insn_t insn) |
5010 { | 5243 { |
5011 bool asm_p; | 5244 bool asm_p; |
5012 | 5245 |
5013 if (recog_memoized (insn) >= 0) | 5246 if (recog_memoized (insn) >= 0) |
5014 { | 5247 { |
5015 int res; | 5248 int res; |
5016 state_t temp_state = alloca (dfa_state_size); | 5249 state_t temp_state = alloca (dfa_state_size); |
5017 | 5250 |
5018 gcc_assert (!INSN_ASM_P (insn)); | 5251 gcc_assert (!INSN_ASM_P (insn)); |
5019 asm_p = false; | 5252 asm_p = false; |
5020 | 5253 |
5021 memcpy (temp_state, FENCE_STATE (fence), dfa_state_size); | 5254 memcpy (temp_state, FENCE_STATE (fence), dfa_state_size); |
5022 res = state_transition (FENCE_STATE (fence), insn); | 5255 res = state_transition (FENCE_STATE (fence), insn); |
5028 | 5261 |
5029 /* We should never issue more than issue_rate insns. */ | 5262 /* We should never issue more than issue_rate insns. */ |
5030 if (FENCE_ISSUED_INSNS (fence) > issue_rate) | 5263 if (FENCE_ISSUED_INSNS (fence) > issue_rate) |
5031 gcc_unreachable (); | 5264 gcc_unreachable (); |
5032 } | 5265 } |
5033 } | 5266 } |
5034 else | 5267 else |
5035 { | 5268 { |
5036 /* This could be an ASM insn which we'd like to schedule | 5269 /* This could be an ASM insn which we'd like to schedule |
5037 on the next cycle. */ | 5270 on the next cycle. */ |
5038 asm_p = INSN_ASM_P (insn); | 5271 asm_p = INSN_ASM_P (insn); |
5039 if (!FENCE_STARTS_CYCLE_P (fence) && asm_p) | 5272 if (!FENCE_STARTS_CYCLE_P (fence) && asm_p) |
5040 advance_one_cycle (fence); | 5273 advance_one_cycle (fence); |
5041 } | 5274 } |
5042 | 5275 |
5043 if (sched_verbose >= 2) | 5276 if (sched_verbose >= 2) |
5044 debug_state (FENCE_STATE (fence)); | 5277 debug_state (FENCE_STATE (fence)); |
5045 FENCE_STARTS_CYCLE_P (fence) = 0; | 5278 if (!DEBUG_INSN_P (insn)) |
5279 FENCE_STARTS_CYCLE_P (fence) = 0; | |
5046 return asm_p; | 5280 return asm_p; |
5047 } | 5281 } |
5048 | 5282 |
5049 /* Update FENCE on which INSN was scheduled and this INSN, too. NEED_STALL | 5283 /* Update FENCE on which INSN was scheduled and this INSN, too. NEED_STALL |
5050 is nonzero if we need to stall after issuing INSN. */ | 5284 is nonzero if we need to stall after issuing INSN. */ |
5051 static void | 5285 static void |
5052 update_fence_and_insn (fence_t fence, insn_t insn, int need_stall) | 5286 update_fence_and_insn (fence_t fence, insn_t insn, int need_stall) |
5053 { | 5287 { |
5054 bool asm_p; | 5288 bool asm_p; |
5055 | 5289 |
5056 /* First, reflect that something is scheduled on this fence. */ | 5290 /* First, reflect that something is scheduled on this fence. */ |
5057 asm_p = advance_state_on_fence (fence, insn); | 5291 asm_p = advance_state_on_fence (fence, insn); |
5058 FENCE_LAST_SCHEDULED_INSN (fence) = insn; | 5292 FENCE_LAST_SCHEDULED_INSN (fence) = insn; |
5059 VEC_safe_push (rtx, gc, FENCE_EXECUTING_INSNS (fence), insn); | 5293 VEC_safe_push (rtx, gc, FENCE_EXECUTING_INSNS (fence), insn); |
5060 if (SCHED_GROUP_P (insn)) | 5294 if (SCHED_GROUP_P (insn)) |
5074 EXPR_ORIG_SCHED_CYCLE (INSN_EXPR (insn)) = FENCE_CYCLE (fence); | 5308 EXPR_ORIG_SCHED_CYCLE (INSN_EXPR (insn)) = FENCE_CYCLE (fence); |
5075 INSN_AFTER_STALL_P (insn) = FENCE_AFTER_STALL_P (fence); | 5309 INSN_AFTER_STALL_P (insn) = FENCE_AFTER_STALL_P (fence); |
5076 INSN_SCHED_CYCLE (insn) = FENCE_CYCLE (fence); | 5310 INSN_SCHED_CYCLE (insn) = FENCE_CYCLE (fence); |
5077 | 5311 |
5078 /* This does not account for adjust_cost hooks, just add the biggest | 5312 /* This does not account for adjust_cost hooks, just add the biggest |
5079 constant the hook may add to the latency. TODO: make this | 5313 constant the hook may add to the latency. TODO: make this |
5080 a target dependent constant. */ | 5314 a target dependent constant. */ |
5081 INSN_READY_CYCLE (insn) | 5315 INSN_READY_CYCLE (insn) |
5082 = INSN_SCHED_CYCLE (insn) + (INSN_CODE (insn) < 0 | 5316 = INSN_SCHED_CYCLE (insn) + (INSN_CODE (insn) < 0 |
5083 ? 1 | 5317 ? 1 |
5084 : maximal_insn_latency (insn) + 1); | 5318 : maximal_insn_latency (insn) + 1); |
5085 | 5319 |
5086 /* Change these fields last, as they're used above. */ | 5320 /* Change these fields last, as they're used above. */ |
5087 FENCE_AFTER_STALL_P (fence) = 0; | 5321 FENCE_AFTER_STALL_P (fence) = 0; |
5088 if (asm_p || need_stall) | 5322 if (asm_p || need_stall) |
5089 advance_one_cycle (fence); | 5323 advance_one_cycle (fence); |
5090 | 5324 |
5091 /* Indicate that we've scheduled something on this fence. */ | 5325 /* Indicate that we've scheduled something on this fence. */ |
5092 FENCE_SCHEDULED_P (fence) = true; | 5326 FENCE_SCHEDULED_P (fence) = true; |
5093 scheduled_something_on_previous_fence = true; | 5327 scheduled_something_on_previous_fence = true; |
5094 | 5328 |
5095 /* Print debug information when insn's fields are updated. */ | 5329 /* Print debug information when insn's fields are updated. */ |
5099 dump_insn_1 (insn, 1); | 5333 dump_insn_1 (insn, 1); |
5100 sel_print ("\n"); | 5334 sel_print ("\n"); |
5101 } | 5335 } |
5102 } | 5336 } |
5103 | 5337 |
5104 /* Update boundary BND with INSN, remove the old boundary from | 5338 /* Update boundary BND (and, if needed, FENCE) with INSN, remove the |
5105 BNDSP, add new boundaries to BNDS_TAIL_P and return it. */ | 5339 old boundary from BNDSP, add new boundaries to BNDS_TAIL_P and |
5340 return it. */ | |
5106 static blist_t * | 5341 static blist_t * |
5107 update_boundaries (bnd_t bnd, insn_t insn, blist_t *bndsp, | 5342 update_boundaries (fence_t fence, bnd_t bnd, insn_t insn, blist_t *bndsp, |
5108 blist_t *bnds_tailp) | 5343 blist_t *bnds_tailp) |
5109 { | 5344 { |
5110 succ_iterator si; | 5345 succ_iterator si; |
5111 insn_t succ; | 5346 insn_t succ; |
5112 | 5347 |
5113 advance_deps_context (BND_DC (bnd), insn); | 5348 advance_deps_context (BND_DC (bnd), insn); |
5114 FOR_EACH_SUCC_1 (succ, si, insn, | 5349 FOR_EACH_SUCC_1 (succ, si, insn, |
5115 SUCCS_NORMAL | SUCCS_SKIP_TO_LOOP_EXITS) | 5350 SUCCS_NORMAL | SUCCS_SKIP_TO_LOOP_EXITS) |
5116 { | 5351 { |
5117 ilist_t ptr = ilist_copy (BND_PTR (bnd)); | 5352 ilist_t ptr = ilist_copy (BND_PTR (bnd)); |
5118 | 5353 |
5119 ilist_add (&ptr, insn); | 5354 ilist_add (&ptr, insn); |
5355 | |
5356 if (DEBUG_INSN_P (insn) && sel_bb_end_p (insn) | |
5357 && is_ineligible_successor (succ, ptr)) | |
5358 { | |
5359 ilist_clear (&ptr); | |
5360 continue; | |
5361 } | |
5362 | |
5363 if (FENCE_INSN (fence) == insn && !sel_bb_end_p (insn)) | |
5364 { | |
5365 if (sched_verbose >= 9) | |
5366 sel_print ("Updating fence insn from %i to %i\n", | |
5367 INSN_UID (insn), INSN_UID (succ)); | |
5368 FENCE_INSN (fence) = succ; | |
5369 } | |
5120 blist_add (bnds_tailp, succ, ptr, BND_DC (bnd)); | 5370 blist_add (bnds_tailp, succ, ptr, BND_DC (bnd)); |
5121 bnds_tailp = &BLIST_NEXT (*bnds_tailp); | 5371 bnds_tailp = &BLIST_NEXT (*bnds_tailp); |
5122 } | 5372 } |
5123 | 5373 |
5124 blist_remove (bndsp); | 5374 blist_remove (bndsp); |
5125 return bnds_tailp; | 5375 return bnds_tailp; |
5126 } | 5376 } |
5127 | 5377 |
5128 /* Schedule EXPR_VLIW on BND. Return the insn emitted. */ | 5378 /* Schedule EXPR_VLIW on BND. Return the insn emitted. */ |
5136 bool should_move; | 5386 bool should_move; |
5137 | 5387 |
5138 expr_seq = find_sequential_best_exprs (bnd, expr_vliw, true); | 5388 expr_seq = find_sequential_best_exprs (bnd, expr_vliw, true); |
5139 | 5389 |
5140 /* In case of scheduling a jump skipping some other instructions, | 5390 /* In case of scheduling a jump skipping some other instructions, |
5141 prepare CFG. After this, jump is at the boundary and can be | 5391 prepare CFG. After this, jump is at the boundary and can be |
5142 scheduled as usual insn by MOVE_OP. */ | 5392 scheduled as usual insn by MOVE_OP. */ |
5143 if (vinsn_cond_branch_p (EXPR_VINSN (expr_vliw))) | 5393 if (vinsn_cond_branch_p (EXPR_VINSN (expr_vliw))) |
5144 { | 5394 { |
5145 insn = EXPR_INSN_RTX (expr_vliw); | 5395 insn = EXPR_INSN_RTX (expr_vliw); |
5146 | 5396 |
5147 /* Speculative jumps are not handled. */ | 5397 /* Speculative jumps are not handled. */ |
5148 if (insn != BND_TO (bnd) | 5398 if (insn != BND_TO (bnd) |
5149 && !sel_insn_is_speculation_check (insn)) | 5399 && !sel_insn_is_speculation_check (insn)) |
5150 move_cond_jump (insn, bnd); | 5400 move_cond_jump (insn, bnd); |
5151 } | 5401 } |
5152 | 5402 |
5153 /* Find a place for C_EXPR to schedule. */ | 5403 /* Find a place for C_EXPR to schedule. */ |
5154 place_to_insert = prepare_place_to_insert (bnd); | 5404 place_to_insert = prepare_place_to_insert (bnd); |
5155 should_move = move_exprs_to_boundary (bnd, expr_vliw, expr_seq, c_expr); | 5405 should_move = move_exprs_to_boundary (bnd, expr_vliw, expr_seq, c_expr); |
5156 clear_expr (c_expr); | 5406 clear_expr (c_expr); |
5157 | 5407 |
5158 /* Add the instruction. The corner case to care about is when | 5408 /* Add the instruction. The corner case to care about is when |
5159 the expr_seq set has more than one expr, and we chose the one that | 5409 the expr_seq set has more than one expr, and we chose the one that |
5160 is not equal to expr_vliw. Then expr_vliw may be insn in stream, and | 5410 is not equal to expr_vliw. Then expr_vliw may be insn in stream, and |
5161 we can't use it. Generate the new vinsn. */ | 5411 we can't use it. Generate the new vinsn. */ |
5162 if (INSN_IN_STREAM_P (EXPR_INSN_RTX (expr_vliw))) | 5412 if (INSN_IN_STREAM_P (EXPR_INSN_RTX (expr_vliw))) |
5163 { | 5413 { |
5164 vinsn_t vinsn_new; | 5414 vinsn_t vinsn_new; |
5165 | 5415 |
5166 vinsn_new = vinsn_copy (EXPR_VINSN (expr_vliw), false); | 5416 vinsn_new = vinsn_copy (EXPR_VINSN (expr_vliw), false); |
5167 change_vinsn_in_expr (expr_vliw, vinsn_new); | 5417 change_vinsn_in_expr (expr_vliw, vinsn_new); |
5168 should_move = false; | 5418 should_move = false; |
5169 } | 5419 } |
5170 if (should_move) | 5420 if (should_move) |
5171 insn = sel_move_insn (expr_vliw, seqno, place_to_insert); | 5421 insn = sel_move_insn (expr_vliw, seqno, place_to_insert); |
5172 else | 5422 else |
5173 insn = emit_insn_from_expr_after (expr_vliw, NULL, seqno, | 5423 insn = emit_insn_from_expr_after (expr_vliw, NULL, seqno, |
5174 place_to_insert); | 5424 place_to_insert); |
5175 | 5425 |
5176 /* Return the nops generated for preserving of data sets back | 5426 /* Return the nops generated for preserving of data sets back |
5177 into pool. */ | 5427 into pool. */ |
5178 if (INSN_NOP_P (place_to_insert)) | 5428 if (INSN_NOP_P (place_to_insert)) |
5179 return_nop_to_pool (place_to_insert); | 5429 return_nop_to_pool (place_to_insert, !DEBUG_INSN_P (insn)); |
5180 remove_temp_moveop_nops (); | 5430 remove_temp_moveop_nops (!DEBUG_INSN_P (insn)); |
5181 | 5431 |
5182 av_set_clear (&expr_seq); | 5432 av_set_clear (&expr_seq); |
5183 | 5433 |
5184 /* Save the expression scheduled so to reset target availability if we'll | 5434 /* Save the expression scheduled so to reset target availability if we'll |
5185 meet it later on the same fence. */ | 5435 meet it later on the same fence. */ |
5186 if (EXPR_WAS_RENAMED (expr_vliw)) | 5436 if (EXPR_WAS_RENAMED (expr_vliw)) |
5187 vinsn_vec_add (&vec_target_unavailable_vinsns, INSN_EXPR (insn)); | 5437 vinsn_vec_add (&vec_target_unavailable_vinsns, INSN_EXPR (insn)); |
5188 | 5438 |
5189 /* Check that the recent movement didn't destroyed loop | 5439 /* Check that the recent movement didn't destroyed loop |
5197 /* Stall for N cycles on FENCE. */ | 5447 /* Stall for N cycles on FENCE. */ |
5198 static void | 5448 static void |
5199 stall_for_cycles (fence_t fence, int n) | 5449 stall_for_cycles (fence_t fence, int n) |
5200 { | 5450 { |
5201 int could_more; | 5451 int could_more; |
5202 | 5452 |
5203 could_more = n > 1 || FENCE_ISSUED_INSNS (fence) < issue_rate; | 5453 could_more = n > 1 || FENCE_ISSUED_INSNS (fence) < issue_rate; |
5204 while (n--) | 5454 while (n--) |
5205 advance_one_cycle (fence); | 5455 advance_one_cycle (fence); |
5206 if (could_more) | 5456 if (could_more) |
5207 FENCE_AFTER_STALL_P (fence) = 1; | 5457 FENCE_AFTER_STALL_P (fence) = 1; |
5208 } | 5458 } |
5209 | 5459 |
5210 /* Gather a parallel group of insns at FENCE and assign their seqno | 5460 /* Gather a parallel group of insns at FENCE and assign their seqno |
5211 to SEQNO. All scheduled insns are gathered in SCHEDULED_INSNS_TAILPP | 5461 to SEQNO. All scheduled insns are gathered in SCHEDULED_INSNS_TAILPP |
5212 list for later recalculation of seqnos. */ | 5462 list for later recalculation of seqnos. */ |
5213 static void | 5463 static void |
5214 fill_insns (fence_t fence, int seqno, ilist_t **scheduled_insns_tailpp) | 5464 fill_insns (fence_t fence, int seqno, ilist_t **scheduled_insns_tailpp) |
5215 { | 5465 { |
5216 blist_t bnds = NULL, *bnds_tailp; | 5466 blist_t bnds = NULL, *bnds_tailp; |
5217 av_set_t av_vliw = NULL; | 5467 av_set_t av_vliw = NULL; |
5218 insn_t insn = FENCE_INSN (fence); | 5468 insn_t insn = FENCE_INSN (fence); |
5219 | 5469 |
5220 if (sched_verbose >= 2) | 5470 if (sched_verbose >= 2) |
5221 sel_print ("Starting fill_insns for insn %d, cycle %d\n", | 5471 sel_print ("Starting fill_insns for insn %d, cycle %d\n", |
5222 INSN_UID (insn), FENCE_CYCLE (fence)); | 5472 INSN_UID (insn), FENCE_CYCLE (fence)); |
5223 | 5473 |
5224 blist_add (&bnds, insn, NULL, FENCE_DC (fence)); | 5474 blist_add (&bnds, insn, NULL, FENCE_DC (fence)); |
5225 bnds_tailp = &BLIST_NEXT (bnds); | 5475 bnds_tailp = &BLIST_NEXT (bnds); |
5226 set_target_context (FENCE_TC (fence)); | 5476 set_target_context (FENCE_TC (fence)); |
5233 expr_t expr_vliw; | 5483 expr_t expr_vliw; |
5234 int need_stall; | 5484 int need_stall; |
5235 int was_stall = 0, scheduled_insns = 0, stall_iterations = 0; | 5485 int was_stall = 0, scheduled_insns = 0, stall_iterations = 0; |
5236 int max_insns = pipelining_p ? issue_rate : 2 * issue_rate; | 5486 int max_insns = pipelining_p ? issue_rate : 2 * issue_rate; |
5237 int max_stall = pipelining_p ? 1 : 3; | 5487 int max_stall = pipelining_p ? 1 : 3; |
5238 | 5488 bool last_insn_was_debug = false; |
5489 bool was_debug_bb_end_p = false; | |
5490 | |
5239 compute_av_set_on_boundaries (fence, bnds, &av_vliw); | 5491 compute_av_set_on_boundaries (fence, bnds, &av_vliw); |
5240 remove_insns_that_need_bookkeeping (fence, &av_vliw); | 5492 remove_insns_that_need_bookkeeping (fence, &av_vliw); |
5241 remove_insns_for_debug (bnds, &av_vliw); | 5493 remove_insns_for_debug (bnds, &av_vliw); |
5242 | 5494 |
5243 /* Return early if we have nothing to schedule. */ | 5495 /* Return early if we have nothing to schedule. */ |
5267 break; | 5519 break; |
5268 } | 5520 } |
5269 } | 5521 } |
5270 } | 5522 } |
5271 while (! expr_vliw && need_stall); | 5523 while (! expr_vliw && need_stall); |
5272 | 5524 |
5273 /* Now either we've selected expr_vliw or we have nothing to schedule. */ | 5525 /* Now either we've selected expr_vliw or we have nothing to schedule. */ |
5274 if (!expr_vliw) | 5526 if (!expr_vliw) |
5275 { | 5527 { |
5276 av_set_clear (&av_vliw); | 5528 av_set_clear (&av_vliw); |
5277 break; | 5529 break; |
5279 | 5531 |
5280 bndsp = &bnds; | 5532 bndsp = &bnds; |
5281 bnds_tailp1 = bnds_tailp; | 5533 bnds_tailp1 = bnds_tailp; |
5282 | 5534 |
5283 do | 5535 do |
5284 /* This code will be executed only once until we'd have several | 5536 /* This code will be executed only once until we'd have several |
5285 boundaries per fence. */ | 5537 boundaries per fence. */ |
5286 { | 5538 { |
5287 bnd_t bnd = BLIST_BND (*bndsp); | 5539 bnd_t bnd = BLIST_BND (*bndsp); |
5288 | 5540 |
5289 if (!av_set_is_in_p (BND_AV1 (bnd), EXPR_VINSN (expr_vliw))) | 5541 if (!av_set_is_in_p (BND_AV1 (bnd), EXPR_VINSN (expr_vliw))) |
5290 { | 5542 { |
5291 bndsp = &BLIST_NEXT (*bndsp); | 5543 bndsp = &BLIST_NEXT (*bndsp); |
5292 continue; | 5544 continue; |
5293 } | 5545 } |
5294 | 5546 |
5295 insn = schedule_expr_on_boundary (bnd, expr_vliw, seqno); | 5547 insn = schedule_expr_on_boundary (bnd, expr_vliw, seqno); |
5548 last_insn_was_debug = DEBUG_INSN_P (insn); | |
5549 if (last_insn_was_debug) | |
5550 was_debug_bb_end_p = (insn == BND_TO (bnd) && sel_bb_end_p (insn)); | |
5296 update_fence_and_insn (fence, insn, need_stall); | 5551 update_fence_and_insn (fence, insn, need_stall); |
5297 bnds_tailp = update_boundaries (bnd, insn, bndsp, bnds_tailp); | 5552 bnds_tailp = update_boundaries (fence, bnd, insn, bndsp, bnds_tailp); |
5298 | 5553 |
5299 /* Add insn to the list of scheduled on this cycle instructions. */ | 5554 /* Add insn to the list of scheduled on this cycle instructions. */ |
5300 ilist_add (*scheduled_insns_tailpp, insn); | 5555 ilist_add (*scheduled_insns_tailpp, insn); |
5301 *scheduled_insns_tailpp = &ILIST_NEXT (**scheduled_insns_tailpp); | 5556 *scheduled_insns_tailpp = &ILIST_NEXT (**scheduled_insns_tailpp); |
5302 } | 5557 } |
5303 while (*bndsp != *bnds_tailp1); | 5558 while (*bndsp != *bnds_tailp1); |
5304 | 5559 |
5305 av_set_clear (&av_vliw); | 5560 av_set_clear (&av_vliw); |
5306 scheduled_insns++; | 5561 if (!last_insn_was_debug) |
5562 scheduled_insns++; | |
5307 | 5563 |
5308 /* We currently support information about candidate blocks only for | 5564 /* We currently support information about candidate blocks only for |
5309 one 'target_bb' block. Hence we can't schedule after jump insn, | 5565 one 'target_bb' block. Hence we can't schedule after jump insn, |
5310 as this will bring two boundaries and, hence, necessity to handle | 5566 as this will bring two boundaries and, hence, necessity to handle |
5311 information for two or more blocks concurrently. */ | 5567 information for two or more blocks concurrently. */ |
5312 if (sel_bb_end_p (insn) | 5568 if ((last_insn_was_debug ? was_debug_bb_end_p : sel_bb_end_p (insn)) |
5313 || (was_stall | 5569 || (was_stall |
5314 && (was_stall >= max_stall | 5570 && (was_stall >= max_stall |
5315 || scheduled_insns >= max_insns))) | 5571 || scheduled_insns >= max_insns))) |
5316 break; | 5572 break; |
5317 } | 5573 } |
5318 while (bnds); | 5574 while (bnds); |
5319 | 5575 |
5320 gcc_assert (!FENCE_BNDS (fence)); | 5576 gcc_assert (!FENCE_BNDS (fence)); |
5321 | 5577 |
5322 /* Update boundaries of the FENCE. */ | 5578 /* Update boundaries of the FENCE. */ |
5323 while (bnds) | 5579 while (bnds) |
5324 { | 5580 { |
5325 ilist_t ptr = BND_PTR (BLIST_BND (bnds)); | 5581 ilist_t ptr = BND_PTR (BLIST_BND (bnds)); |
5326 | 5582 |
5329 insn = ILIST_INSN (ptr); | 5585 insn = ILIST_INSN (ptr); |
5330 | 5586 |
5331 if (!ilist_is_in_p (FENCE_BNDS (fence), insn)) | 5587 if (!ilist_is_in_p (FENCE_BNDS (fence), insn)) |
5332 ilist_add (&FENCE_BNDS (fence), insn); | 5588 ilist_add (&FENCE_BNDS (fence), insn); |
5333 } | 5589 } |
5334 | 5590 |
5335 blist_remove (&bnds); | 5591 blist_remove (&bnds); |
5336 } | 5592 } |
5337 | 5593 |
5338 /* Update target context on the fence. */ | 5594 /* Update target context on the fence. */ |
5339 reset_target_context (FENCE_TC (fence), false); | 5595 reset_target_context (FENCE_TC (fence), false); |
5375 av_set_iterator i; | 5631 av_set_iterator i; |
5376 av_set_t old_av_set = NULL; | 5632 av_set_t old_av_set = NULL; |
5377 expr_t cur_expr; | 5633 expr_t cur_expr; |
5378 rtx bb_end = sel_bb_end (book_block); | 5634 rtx bb_end = sel_bb_end (book_block); |
5379 | 5635 |
5380 /* First, get correct liveness in the bookkeeping block. The problem is | 5636 /* First, get correct liveness in the bookkeeping block. The problem is |
5381 the range between the bookeeping insn and the end of block. */ | 5637 the range between the bookeeping insn and the end of block. */ |
5382 update_liveness_on_insn (bb_end); | 5638 update_liveness_on_insn (bb_end); |
5383 if (control_flow_insn_p (bb_end)) | 5639 if (control_flow_insn_p (bb_end)) |
5384 update_liveness_on_insn (PREV_INSN (bb_end)); | 5640 update_liveness_on_insn (PREV_INSN (bb_end)); |
5385 | 5641 |
5388 actually blocked from moving up with the bookkeeping we create here. */ | 5644 actually blocked from moving up with the bookkeeping we create here. */ |
5389 if (AV_SET_VALID_P (sel_bb_head (book_block))) | 5645 if (AV_SET_VALID_P (sel_bb_head (book_block))) |
5390 { | 5646 { |
5391 old_av_set = av_set_copy (BB_AV_SET (book_block)); | 5647 old_av_set = av_set_copy (BB_AV_SET (book_block)); |
5392 update_data_sets (sel_bb_head (book_block)); | 5648 update_data_sets (sel_bb_head (book_block)); |
5393 | 5649 |
5394 /* Traverse all the expressions in the old av_set and check whether | 5650 /* Traverse all the expressions in the old av_set and check whether |
5395 CUR_EXPR is in new AV_SET. */ | 5651 CUR_EXPR is in new AV_SET. */ |
5396 FOR_EACH_EXPR (cur_expr, i, old_av_set) | 5652 FOR_EACH_EXPR (cur_expr, i, old_av_set) |
5397 { | 5653 { |
5398 expr_t new_expr = av_set_lookup (BB_AV_SET (book_block), | 5654 expr_t new_expr = av_set_lookup (BB_AV_SET (book_block), |
5399 EXPR_VINSN (cur_expr)); | 5655 EXPR_VINSN (cur_expr)); |
5400 | 5656 |
5401 if (! new_expr | 5657 if (! new_expr |
5402 /* In this case, we can just turn off the E_T_A bit, but we can't | 5658 /* In this case, we can just turn off the E_T_A bit, but we can't |
5403 represent this information with the current vector. */ | 5659 represent this information with the current vector. */ |
5404 || EXPR_TARGET_AVAILABLE (new_expr) | 5660 || EXPR_TARGET_AVAILABLE (new_expr) |
5405 != EXPR_TARGET_AVAILABLE (cur_expr)) | 5661 != EXPR_TARGET_AVAILABLE (cur_expr)) |
5406 /* Unfortunately, the below code could be also fired up on | 5662 /* Unfortunately, the below code could be also fired up on |
5407 separable insns. | 5663 separable insns. |
5408 FIXME: add an example of how this could happen. */ | 5664 FIXME: add an example of how this could happen. */ |
5409 vinsn_vec_add (&vec_bookkeeping_blocked_vinsns, cur_expr); | 5665 vinsn_vec_add (&vec_bookkeeping_blocked_vinsns, cur_expr); |
5411 | 5667 |
5412 av_set_clear (&old_av_set); | 5668 av_set_clear (&old_av_set); |
5413 } | 5669 } |
5414 } | 5670 } |
5415 | 5671 |
5416 /* The main effect of this function is that sparams->c_expr is merged | 5672 /* The main effect of this function is that sparams->c_expr is merged |
5417 with (or copied to) lparams->c_expr_merged. If there's only one successor, | 5673 with (or copied to) lparams->c_expr_merged. If there's only one successor, |
5418 we avoid merging anything by copying sparams->c_expr to lparams->c_expr_merged. | 5674 we avoid merging anything by copying sparams->c_expr to lparams->c_expr_merged. |
5419 lparams->c_expr_merged is copied back to sparams->c_expr after all | 5675 lparams->c_expr_merged is copied back to sparams->c_expr after all |
5420 successors has been traversed. lparams->c_expr_local is an expr allocated | 5676 successors has been traversed. lparams->c_expr_local is an expr allocated |
5421 on stack in the caller function, and is used if there is more than one | 5677 on stack in the caller function, and is used if there is more than one |
5422 successor. | 5678 successor. |
5423 | 5679 |
5424 SUCC is one of the SUCCS_NORMAL successors of INSN, | 5680 SUCC is one of the SUCCS_NORMAL successors of INSN, |
5425 MOVEOP_DRV_CALL_RES is the result of call code_motion_path_driver on succ, | 5681 MOVEOP_DRV_CALL_RES is the result of call code_motion_path_driver on succ, |
5426 LPARAMS and STATIC_PARAMS contain the parameters described above. */ | 5682 LPARAMS and STATIC_PARAMS contain the parameters described above. */ |
5427 static void | 5683 static void |
5428 move_op_merge_succs (insn_t insn ATTRIBUTE_UNUSED, | 5684 move_op_merge_succs (insn_t insn ATTRIBUTE_UNUSED, |
5429 insn_t succ ATTRIBUTE_UNUSED, | 5685 insn_t succ ATTRIBUTE_UNUSED, |
5430 int moveop_drv_call_res, | 5686 int moveop_drv_call_res, |
5431 cmpd_local_params_p lparams, void *static_params) | 5687 cmpd_local_params_p lparams, void *static_params) |
5432 { | 5688 { |
5433 moveop_static_params_p sparams = (moveop_static_params_p) static_params; | 5689 moveop_static_params_p sparams = (moveop_static_params_p) static_params; |
5434 | 5690 |
5435 /* Nothing to do, if original expr wasn't found below. */ | 5691 /* Nothing to do, if original expr wasn't found below. */ |
5450 speculation success probability and only then with the | 5706 speculation success probability and only then with the |
5451 good probability. As a result the insn will get epsilon | 5707 good probability. As a result the insn will get epsilon |
5452 probability and will never be scheduled because of | 5708 probability and will never be scheduled because of |
5453 weakness_cutoff in find_best_expr. | 5709 weakness_cutoff in find_best_expr. |
5454 | 5710 |
5455 We call merge_expr_data here instead of merge_expr | 5711 We call merge_expr_data here instead of merge_expr |
5456 because due to speculation C_EXPR and X may have the | 5712 because due to speculation C_EXPR and X may have the |
5457 same insns with different speculation types. And as of | 5713 same insns with different speculation types. And as of |
5458 now such insns are considered non-equal. | 5714 now such insns are considered non-equal. |
5459 | 5715 |
5460 However, EXPR_SCHED_TIMES is different -- we must get | 5716 However, EXPR_SCHED_TIMES is different -- we must get |
5461 SCHED_TIMES from a real insn, not a bookkeeping copy. | 5717 SCHED_TIMES from a real insn, not a bookkeeping copy. |
5462 We force this here. Instead, we may consider merging | 5718 We force this here. Instead, we may consider merging |
5463 SCHED_TIMES to the maximum instead of minimum in the | 5719 SCHED_TIMES to the maximum instead of minimum in the |
5464 below function. */ | 5720 below function. */ |
5465 int old_times = EXPR_SCHED_TIMES (lparams->c_expr_merged); | 5721 int old_times = EXPR_SCHED_TIMES (lparams->c_expr_merged); |
5466 | 5722 |
5467 merge_expr_data (lparams->c_expr_merged, sparams->c_expr, NULL); | 5723 merge_expr_data (lparams->c_expr_merged, sparams->c_expr, NULL); |
5468 if (EXPR_SCHED_TIMES (sparams->c_expr) == 0) | 5724 if (EXPR_SCHED_TIMES (sparams->c_expr) == 0) |
5477 SUCC is one of the SUCCS_NORMAL successors of INSN, | 5733 SUCC is one of the SUCCS_NORMAL successors of INSN, |
5478 MOVEOP_DRV_CALL_RES is the result of call code_motion_path_driver on succ or 0, | 5734 MOVEOP_DRV_CALL_RES is the result of call code_motion_path_driver on succ or 0, |
5479 if SUCC is one of SUCCS_BACK or SUCCS_OUT. | 5735 if SUCC is one of SUCCS_BACK or SUCCS_OUT. |
5480 STATIC_PARAMS contain USED_REGS set. */ | 5736 STATIC_PARAMS contain USED_REGS set. */ |
5481 static void | 5737 static void |
5482 fur_merge_succs (insn_t insn ATTRIBUTE_UNUSED, insn_t succ, | 5738 fur_merge_succs (insn_t insn ATTRIBUTE_UNUSED, insn_t succ, |
5483 int moveop_drv_call_res, | 5739 int moveop_drv_call_res, |
5484 cmpd_local_params_p lparams ATTRIBUTE_UNUSED, | 5740 cmpd_local_params_p lparams ATTRIBUTE_UNUSED, |
5485 void *static_params) | 5741 void *static_params) |
5486 { | 5742 { |
5487 regset succ_live; | 5743 regset succ_live; |
5488 fur_static_params_p sparams = (fur_static_params_p) static_params; | 5744 fur_static_params_p sparams = (fur_static_params_p) static_params; |
5489 | 5745 |
5490 /* Here we compute live regsets only for branches that do not lie | 5746 /* Here we compute live regsets only for branches that do not lie |
5491 on the code motion paths. These branches correspond to value | 5747 on the code motion paths. These branches correspond to value |
5492 MOVEOP_DRV_CALL_RES==0 and include SUCCS_BACK and SUCCS_OUT, though | 5748 MOVEOP_DRV_CALL_RES==0 and include SUCCS_BACK and SUCCS_OUT, though |
5493 for such branches code_motion_path_driver is not called. */ | 5749 for such branches code_motion_path_driver is not called. */ |
5494 if (moveop_drv_call_res != 0) | 5750 if (moveop_drv_call_res != 0) |
5495 return; | 5751 return; |
5496 | 5752 |
5505 | 5761 |
5506 /* This function is called after the last successor. Copies LP->C_EXPR_MERGED | 5762 /* This function is called after the last successor. Copies LP->C_EXPR_MERGED |
5507 into SP->CEXPR. */ | 5763 into SP->CEXPR. */ |
5508 static void | 5764 static void |
5509 move_op_after_merge_succs (cmpd_local_params_p lp, void *sparams) | 5765 move_op_after_merge_succs (cmpd_local_params_p lp, void *sparams) |
5510 { | 5766 { |
5511 moveop_static_params_p sp = (moveop_static_params_p) sparams; | 5767 moveop_static_params_p sp = (moveop_static_params_p) sparams; |
5512 | 5768 |
5513 sp->c_expr = lp->c_expr_merged; | 5769 sp->c_expr = lp->c_expr_merged; |
5514 } | 5770 } |
5515 | 5771 |
5526 { | 5782 { |
5527 /* Note that original block needs to be rescheduled, as we pulled an | 5783 /* Note that original block needs to be rescheduled, as we pulled an |
5528 instruction out of it. */ | 5784 instruction out of it. */ |
5529 if (INSN_SCHED_TIMES (insn) > 0) | 5785 if (INSN_SCHED_TIMES (insn) > 0) |
5530 bitmap_set_bit (blocks_to_reschedule, BLOCK_FOR_INSN (insn)->index); | 5786 bitmap_set_bit (blocks_to_reschedule, BLOCK_FOR_INSN (insn)->index); |
5531 else if (INSN_UID (insn) < first_emitted_uid) | 5787 else if (INSN_UID (insn) < first_emitted_uid && !DEBUG_INSN_P (insn)) |
5532 num_insns_scheduled++; | 5788 num_insns_scheduled++; |
5533 } | 5789 } |
5534 else | 5790 else |
5535 bitmap_clear_bit (current_copies, INSN_UID (insn)); | 5791 bitmap_clear_bit (current_copies, INSN_UID (insn)); |
5536 | 5792 |
5540 For expr we must make insn look like "INSN_REG (insn) := c_expr". */ | 5796 For expr we must make insn look like "INSN_REG (insn) := c_expr". */ |
5541 if (INSN_UID (insn) > max_uid_before_move_op) | 5797 if (INSN_UID (insn) > max_uid_before_move_op) |
5542 stat_bookkeeping_copies--; | 5798 stat_bookkeeping_copies--; |
5543 } | 5799 } |
5544 | 5800 |
5545 /* Emit a register-register copy for INSN if needed. Return true if | 5801 /* Emit a register-register copy for INSN if needed. Return true if |
5546 emitted one. PARAMS is the move_op static parameters. */ | 5802 emitted one. PARAMS is the move_op static parameters. */ |
5547 static bool | 5803 static bool |
5548 maybe_emit_renaming_copy (rtx insn, | 5804 maybe_emit_renaming_copy (rtx insn, |
5549 moveop_static_params_p params) | 5805 moveop_static_params_p params) |
5550 { | 5806 { |
5551 bool insn_emitted = false; | 5807 bool insn_emitted = false; |
5552 rtx cur_reg = expr_dest_reg (params->c_expr); | 5808 rtx cur_reg = expr_dest_reg (params->c_expr); |
5553 | 5809 |
5557 that expr is not original operation's dest reg, substitute | 5813 that expr is not original operation's dest reg, substitute |
5558 operation's right hand side with the register chosen. */ | 5814 operation's right hand side with the register chosen. */ |
5559 if (cur_reg != NULL_RTX && REGNO (params->dest) != REGNO (cur_reg)) | 5815 if (cur_reg != NULL_RTX && REGNO (params->dest) != REGNO (cur_reg)) |
5560 { | 5816 { |
5561 insn_t reg_move_insn, reg_move_insn_rtx; | 5817 insn_t reg_move_insn, reg_move_insn_rtx; |
5562 | 5818 |
5563 reg_move_insn_rtx = create_insn_rtx_with_rhs (INSN_VINSN (insn), | 5819 reg_move_insn_rtx = create_insn_rtx_with_rhs (INSN_VINSN (insn), |
5564 params->dest); | 5820 params->dest); |
5565 reg_move_insn = sel_gen_insn_from_rtx_after (reg_move_insn_rtx, | 5821 reg_move_insn = sel_gen_insn_from_rtx_after (reg_move_insn_rtx, |
5566 INSN_EXPR (insn), | 5822 INSN_EXPR (insn), |
5567 INSN_SEQNO (insn), | 5823 INSN_SEQNO (insn), |
5568 insn); | 5824 insn); |
5569 EXPR_SPEC_DONE_DS (INSN_EXPR (reg_move_insn)) = 0; | 5825 EXPR_SPEC_DONE_DS (INSN_EXPR (reg_move_insn)) = 0; |
5570 replace_dest_with_reg_in_expr (params->c_expr, params->dest); | 5826 replace_dest_with_reg_in_expr (params->c_expr, params->dest); |
5571 | 5827 |
5572 insn_emitted = true; | 5828 insn_emitted = true; |
5573 params->was_renamed = true; | 5829 params->was_renamed = true; |
5574 } | 5830 } |
5575 | 5831 |
5576 return insn_emitted; | 5832 return insn_emitted; |
5577 } | 5833 } |
5578 | 5834 |
5579 /* Emit a speculative check for INSN speculated as EXPR if needed. | 5835 /* Emit a speculative check for INSN speculated as EXPR if needed. |
5580 Return true if we've emitted one. PARAMS is the move_op static | 5836 Return true if we've emitted one. PARAMS is the move_op static |
5581 parameters. */ | 5837 parameters. */ |
5582 static bool | 5838 static bool |
5583 maybe_emit_speculative_check (rtx insn, expr_t expr, | 5839 maybe_emit_speculative_check (rtx insn, expr_t expr, |
5584 moveop_static_params_p params) | 5840 moveop_static_params_p params) |
5585 { | 5841 { |
5597 else | 5853 else |
5598 { | 5854 { |
5599 EXPR_SPEC_DONE_DS (INSN_EXPR (insn)) = 0; | 5855 EXPR_SPEC_DONE_DS (INSN_EXPR (insn)) = 0; |
5600 x = insn; | 5856 x = insn; |
5601 } | 5857 } |
5602 | 5858 |
5603 gcc_assert (EXPR_SPEC_DONE_DS (INSN_EXPR (x)) == 0 | 5859 gcc_assert (EXPR_SPEC_DONE_DS (INSN_EXPR (x)) == 0 |
5604 && EXPR_SPEC_TO_CHECK_DS (INSN_EXPR (x)) == 0); | 5860 && EXPR_SPEC_TO_CHECK_DS (INSN_EXPR (x)) == 0); |
5605 return insn_emitted; | 5861 return insn_emitted; |
5606 } | 5862 } |
5607 | 5863 |
5608 /* Handle transformations that leave an insn in place of original | 5864 /* Handle transformations that leave an insn in place of original |
5609 insn such as renaming/speculation. Return true if one of such | 5865 insn such as renaming/speculation. Return true if one of such |
5610 transformations actually happened, and we have emitted this insn. */ | 5866 transformations actually happened, and we have emitted this insn. */ |
5611 static bool | 5867 static bool |
5612 handle_emitting_transformations (rtx insn, expr_t expr, | 5868 handle_emitting_transformations (rtx insn, expr_t expr, |
5613 moveop_static_params_p params) | 5869 moveop_static_params_p params) |
5614 { | 5870 { |
5615 bool insn_emitted = false; | 5871 bool insn_emitted = false; |
5616 | 5872 |
5617 insn_emitted = maybe_emit_renaming_copy (insn, params); | 5873 insn_emitted = maybe_emit_renaming_copy (insn, params); |
5618 insn_emitted |= maybe_emit_speculative_check (insn, expr, params); | 5874 insn_emitted |= maybe_emit_speculative_check (insn, expr, params); |
5619 | 5875 |
5620 return insn_emitted; | 5876 return insn_emitted; |
5621 } | 5877 } |
5622 | 5878 |
5623 /* Remove INSN from stream. When ONLY_DISCONNECT is true, its data | 5879 /* If INSN is the only insn in the basic block (not counting JUMP, |
5880 which may be a jump to next insn, and DEBUG_INSNs), we want to | |
5881 leave a NOP there till the return to fill_insns. */ | |
5882 | |
5883 static bool | |
5884 need_nop_to_preserve_insn_bb (rtx insn) | |
5885 { | |
5886 insn_t bb_head, bb_end, bb_next, in_next; | |
5887 basic_block bb = BLOCK_FOR_INSN (insn); | |
5888 | |
5889 bb_head = sel_bb_head (bb); | |
5890 bb_end = sel_bb_end (bb); | |
5891 | |
5892 if (bb_head == bb_end) | |
5893 return true; | |
5894 | |
5895 while (bb_head != bb_end && DEBUG_INSN_P (bb_head)) | |
5896 bb_head = NEXT_INSN (bb_head); | |
5897 | |
5898 if (bb_head == bb_end) | |
5899 return true; | |
5900 | |
5901 while (bb_head != bb_end && DEBUG_INSN_P (bb_end)) | |
5902 bb_end = PREV_INSN (bb_end); | |
5903 | |
5904 if (bb_head == bb_end) | |
5905 return true; | |
5906 | |
5907 bb_next = NEXT_INSN (bb_head); | |
5908 while (bb_next != bb_end && DEBUG_INSN_P (bb_next)) | |
5909 bb_next = NEXT_INSN (bb_next); | |
5910 | |
5911 if (bb_next == bb_end && JUMP_P (bb_end)) | |
5912 return true; | |
5913 | |
5914 in_next = NEXT_INSN (insn); | |
5915 while (DEBUG_INSN_P (in_next)) | |
5916 in_next = NEXT_INSN (in_next); | |
5917 | |
5918 if (IN_CURRENT_FENCE_P (in_next)) | |
5919 return true; | |
5920 | |
5921 return false; | |
5922 } | |
5923 | |
5924 /* Remove INSN from stream. When ONLY_DISCONNECT is true, its data | |
5624 is not removed but reused when INSN is re-emitted. */ | 5925 is not removed but reused when INSN is re-emitted. */ |
5625 static void | 5926 static void |
5626 remove_insn_from_stream (rtx insn, bool only_disconnect) | 5927 remove_insn_from_stream (rtx insn, bool only_disconnect) |
5627 { | 5928 { |
5628 insn_t nop, bb_head, bb_end; | |
5629 bool need_nop_to_preserve_bb; | |
5630 basic_block bb = BLOCK_FOR_INSN (insn); | |
5631 | |
5632 /* If INSN is the only insn in the basic block (not counting JUMP, | |
5633 which may be a jump to next insn), leave NOP there till the | |
5634 return to fill_insns. */ | |
5635 bb_head = sel_bb_head (bb); | |
5636 bb_end = sel_bb_end (bb); | |
5637 need_nop_to_preserve_bb = ((bb_head == bb_end) | |
5638 || (NEXT_INSN (bb_head) == bb_end | |
5639 && JUMP_P (bb_end)) | |
5640 || IN_CURRENT_FENCE_P (NEXT_INSN (insn))); | |
5641 | |
5642 /* If there's only one insn in the BB, make sure that a nop is | 5929 /* If there's only one insn in the BB, make sure that a nop is |
5643 inserted into it, so the basic block won't disappear when we'll | 5930 inserted into it, so the basic block won't disappear when we'll |
5644 delete INSN below with sel_remove_insn. It should also survive | 5931 delete INSN below with sel_remove_insn. It should also survive |
5645 till the return to fill_insns. */ | 5932 till the return to fill_insns. */ |
5646 if (need_nop_to_preserve_bb) | 5933 if (need_nop_to_preserve_insn_bb (insn)) |
5647 { | 5934 { |
5648 nop = get_nop_from_pool (insn); | 5935 insn_t nop = get_nop_from_pool (insn); |
5649 gcc_assert (INSN_NOP_P (nop)); | 5936 gcc_assert (INSN_NOP_P (nop)); |
5650 VEC_safe_push (insn_t, heap, vec_temp_moveop_nops, nop); | 5937 VEC_safe_push (insn_t, heap, vec_temp_moveop_nops, nop); |
5651 } | 5938 } |
5652 | 5939 |
5653 sel_remove_insn (insn, only_disconnect, false); | 5940 sel_remove_insn (insn, only_disconnect, false); |
5654 } | 5941 } |
5655 | 5942 |
5656 /* This function is called when original expr is found. | 5943 /* This function is called when original expr is found. |
5657 INSN - current insn traversed, EXPR - the corresponding expr found. | 5944 INSN - current insn traversed, EXPR - the corresponding expr found. |
5658 LPARAMS is the local parameters of code modion driver, STATIC_PARAMS | 5945 LPARAMS is the local parameters of code modion driver, STATIC_PARAMS |
5659 is static parameters of move_op. */ | 5946 is static parameters of move_op. */ |
5660 static void | 5947 static void |
5661 move_op_orig_expr_found (insn_t insn, expr_t expr, | 5948 move_op_orig_expr_found (insn_t insn, expr_t expr, |
5662 cmpd_local_params_p lparams ATTRIBUTE_UNUSED, | 5949 cmpd_local_params_p lparams ATTRIBUTE_UNUSED, |
5663 void *static_params) | 5950 void *static_params) |
5664 { | 5951 { |
5665 bool only_disconnect, insn_emitted; | 5952 bool only_disconnect, insn_emitted; |
5666 moveop_static_params_p params = (moveop_static_params_p) static_params; | 5953 moveop_static_params_p params = (moveop_static_params_p) static_params; |
5667 | 5954 |
5668 copy_expr_onside (params->c_expr, INSN_EXPR (insn)); | 5955 copy_expr_onside (params->c_expr, INSN_EXPR (insn)); |
5669 track_scheduled_insns_and_blocks (insn); | 5956 track_scheduled_insns_and_blocks (insn); |
5670 insn_emitted = handle_emitting_transformations (insn, expr, params); | 5957 insn_emitted = handle_emitting_transformations (insn, expr, params); |
5671 only_disconnect = (params->uid == INSN_UID (insn) | 5958 only_disconnect = (params->uid == INSN_UID (insn) |
5672 && ! insn_emitted && ! EXPR_WAS_CHANGED (expr)); | 5959 && ! insn_emitted && ! EXPR_WAS_CHANGED (expr)); |
5692 params->crosses_call = true; | 5979 params->crosses_call = true; |
5693 | 5980 |
5694 def_list_add (params->original_insns, insn, params->crosses_call); | 5981 def_list_add (params->original_insns, insn, params->crosses_call); |
5695 | 5982 |
5696 /* Mark the registers that do not meet the following condition: | 5983 /* Mark the registers that do not meet the following condition: |
5697 (2) not among the live registers of the point | 5984 (2) not among the live registers of the point |
5698 immediately following the first original operation on | 5985 immediately following the first original operation on |
5699 a given downward path, except for the original target | 5986 a given downward path, except for the original target |
5700 register of the operation. */ | 5987 register of the operation. */ |
5701 tmp = get_clear_regset_from_pool (); | 5988 tmp = get_clear_regset_from_pool (); |
5702 compute_live_below_insn (insn, tmp); | 5989 compute_live_below_insn (insn, tmp); |
5703 AND_COMPL_REG_SET (tmp, INSN_REG_SETS (insn)); | 5990 AND_COMPL_REG_SET (tmp, INSN_REG_SETS (insn)); |
5714 427: [ax+dx+0x1]=ax | 6001 427: [ax+dx+0x1]=ax |
5715 REG_DEAD: ax | 6002 REG_DEAD: ax |
5716 168: di=dx | 6003 168: di=dx |
5717 REG_DEAD: dx | 6004 REG_DEAD: dx |
5718 */ | 6005 */ |
5719 /* FIXME: see comment above and enable MEM_P | 6006 /* FIXME: see comment above and enable MEM_P |
5720 in vinsn_separable_p. */ | 6007 in vinsn_separable_p. */ |
5721 gcc_assert (!VINSN_SEPARABLE_P (INSN_VINSN (insn)) | 6008 gcc_assert (!VINSN_SEPARABLE_P (INSN_VINSN (insn)) |
5722 || !MEM_P (INSN_LHS (insn))); | 6009 || !MEM_P (INSN_LHS (insn))); |
5723 } | 6010 } |
5724 | 6011 |
5725 /* This function is called on the ascending pass, before returning from | 6012 /* This function is called on the ascending pass, before returning from |
5726 current basic block. */ | 6013 current basic block. */ |
5727 static void | 6014 static void |
5728 move_op_at_first_insn (insn_t insn, cmpd_local_params_p lparams, | 6015 move_op_at_first_insn (insn_t insn, cmpd_local_params_p lparams, |
5729 void *static_params) | 6016 void *static_params) |
5730 { | 6017 { |
5731 moveop_static_params_p sparams = (moveop_static_params_p) static_params; | 6018 moveop_static_params_p sparams = (moveop_static_params_p) static_params; |
5732 basic_block book_block = NULL; | 6019 basic_block book_block = NULL; |
5733 | 6020 |
5734 /* When we have removed the boundary insn for scheduling, which also | 6021 /* When we have removed the boundary insn for scheduling, which also |
5735 happened to be the end insn in its bb, we don't need to update sets. */ | 6022 happened to be the end insn in its bb, we don't need to update sets. */ |
5736 if (!lparams->removed_last_insn | 6023 if (!lparams->removed_last_insn |
5737 && lparams->e1 | 6024 && lparams->e1 |
5738 && sel_bb_head_p (insn)) | 6025 && sel_bb_head_p (insn)) |
5739 { | 6026 { |
5740 /* We should generate bookkeeping code only if we are not at the | 6027 /* We should generate bookkeeping code only if we are not at the |
5741 top level of the move_op. */ | 6028 top level of the move_op. */ |
5743 book_block = generate_bookkeeping_insn (sparams->c_expr, | 6030 book_block = generate_bookkeeping_insn (sparams->c_expr, |
5744 lparams->e1, lparams->e2); | 6031 lparams->e1, lparams->e2); |
5745 /* Update data sets for the current insn. */ | 6032 /* Update data sets for the current insn. */ |
5746 update_data_sets (insn); | 6033 update_data_sets (insn); |
5747 } | 6034 } |
5748 | 6035 |
5749 /* If bookkeeping code was inserted, we need to update av sets of basic | 6036 /* If bookkeeping code was inserted, we need to update av sets of basic |
5750 block that received bookkeeping. After generation of bookkeeping insn, | 6037 block that received bookkeeping. After generation of bookkeeping insn, |
5751 bookkeeping block does not contain valid av set because we are not following | 6038 bookkeeping block does not contain valid av set because we are not following |
5752 the original algorithm in every detail with regards to e.g. renaming | 6039 the original algorithm in every detail with regards to e.g. renaming |
5753 simple reg-reg copies. Consider example: | 6040 simple reg-reg copies. Consider example: |
5754 | 6041 |
5755 bookkeeping block scheduling fence | 6042 bookkeeping block scheduling fence |
5756 \ / | 6043 \ / |
5757 \ join / | 6044 \ join / |
5758 ---------- | 6045 ---------- |
5759 | | | 6046 | | |
5760 ---------- | 6047 ---------- |
5761 / \ | 6048 / \ |
5762 / \ | 6049 / \ |
5763 r1 := r2 r1 := r3 | 6050 r1 := r2 r1 := r3 |
5764 | 6051 |
5765 We try to schedule insn "r1 := r3" on the current | 6052 We try to schedule insn "r1 := r3" on the current |
5766 scheduling fence. Also, note that av set of bookkeeping block | 6053 scheduling fence. Also, note that av set of bookkeeping block |
5767 contain both insns "r1 := r2" and "r1 := r3". When the insn has | 6054 contain both insns "r1 := r2" and "r1 := r3". When the insn has |
5768 been scheduled, the CFG is as follows: | 6055 been scheduled, the CFG is as follows: |
5769 | 6056 |
5770 r1 := r3 r1 := r3 | 6057 r1 := r3 r1 := r3 |
5780 | 6067 |
5781 Here, insn "r1 := r3" was scheduled at the current scheduling point | 6068 Here, insn "r1 := r3" was scheduled at the current scheduling point |
5782 and bookkeeping code was generated at the bookeeping block. This | 6069 and bookkeeping code was generated at the bookeeping block. This |
5783 way insn "r1 := r2" is no longer available as a whole instruction | 6070 way insn "r1 := r2" is no longer available as a whole instruction |
5784 (but only as expr) ahead of insn "r1 := r3" in bookkeeping block. | 6071 (but only as expr) ahead of insn "r1 := r3" in bookkeeping block. |
5785 This situation is handled by calling update_data_sets. | 6072 This situation is handled by calling update_data_sets. |
5786 | 6073 |
5787 Since update_data_sets is called only on the bookkeeping block, and | 6074 Since update_data_sets is called only on the bookkeeping block, and |
5788 it also may have predecessors with av_sets, containing instructions that | 6075 it also may have predecessors with av_sets, containing instructions that |
5789 are no longer available, we save all such expressions that become | 6076 are no longer available, we save all such expressions that become |
5790 unavailable during data sets update on the bookkeeping block in | 6077 unavailable during data sets update on the bookkeeping block in |
5791 VEC_BOOKKEEPING_BLOCKED_VINSNS. Later we avoid selecting such | 6078 VEC_BOOKKEEPING_BLOCKED_VINSNS. Later we avoid selecting such |
5792 expressions for scheduling. This allows us to avoid recomputation of | 6079 expressions for scheduling. This allows us to avoid recomputation of |
5793 av_sets outside the code motion path. */ | 6080 av_sets outside the code motion path. */ |
5794 | 6081 |
5795 if (book_block) | 6082 if (book_block) |
5796 update_and_record_unavailable_insns (book_block); | 6083 update_and_record_unavailable_insns (book_block); |
5797 | 6084 |
5798 /* If INSN was previously marked for deletion, it's time to do it. */ | 6085 /* If INSN was previously marked for deletion, it's time to do it. */ |
5799 if (lparams->removed_last_insn) | 6086 if (lparams->removed_last_insn) |
5800 insn = PREV_INSN (insn); | 6087 insn = PREV_INSN (insn); |
5801 | 6088 |
5802 /* Do not tidy control flow at the topmost moveop, as we can erroneously | 6089 /* Do not tidy control flow at the topmost moveop, as we can erroneously |
5803 kill a block with a single nop in which the insn should be emitted. */ | 6090 kill a block with a single nop in which the insn should be emitted. */ |
5804 if (lparams->e1) | 6091 if (lparams->e1) |
5805 tidy_control_flow (BLOCK_FOR_INSN (insn), true); | 6092 tidy_control_flow (BLOCK_FOR_INSN (insn), true); |
5806 } | 6093 } |
5807 | 6094 |
5808 /* This function is called on the ascending pass, before returning from the | 6095 /* This function is called on the ascending pass, before returning from the |
5809 current basic block. */ | 6096 current basic block. */ |
5810 static void | 6097 static void |
5811 fur_at_first_insn (insn_t insn, | 6098 fur_at_first_insn (insn_t insn, |
5812 cmpd_local_params_p lparams ATTRIBUTE_UNUSED, | 6099 cmpd_local_params_p lparams ATTRIBUTE_UNUSED, |
5813 void *static_params ATTRIBUTE_UNUSED) | 6100 void *static_params ATTRIBUTE_UNUSED) |
5814 { | 6101 { |
5815 gcc_assert (!sel_bb_head_p (insn) || AV_SET_VALID_P (insn) | 6102 gcc_assert (!sel_bb_head_p (insn) || AV_SET_VALID_P (insn) |
5816 || AV_LEVEL (insn) == -1); | 6103 || AV_LEVEL (insn) == -1); |
5817 } | 6104 } |
5832 | 6119 |
5833 /* Update liveness for this insn as it was invalidated. */ | 6120 /* Update liveness for this insn as it was invalidated. */ |
5834 update_liveness_on_insn (insn); | 6121 update_liveness_on_insn (insn); |
5835 } | 6122 } |
5836 | 6123 |
5837 /* This function is called on enter to the basic block. | 6124 /* This function is called on enter to the basic block. |
5838 Returns TRUE if this block already have been visited and | 6125 Returns TRUE if this block already have been visited and |
5839 code_motion_path_driver should return 1, FALSE otherwise. */ | 6126 code_motion_path_driver should return 1, FALSE otherwise. */ |
5840 static int | 6127 static int |
5841 fur_on_enter (insn_t insn ATTRIBUTE_UNUSED, cmpd_local_params_p local_params, | 6128 fur_on_enter (insn_t insn ATTRIBUTE_UNUSED, cmpd_local_params_p local_params, |
5842 void *static_params, bool visited_p) | 6129 void *static_params, bool visited_p) |
5843 { | 6130 { |
5844 fur_static_params_p sparams = (fur_static_params_p) static_params; | 6131 fur_static_params_p sparams = (fur_static_params_p) static_params; |
5845 | 6132 |
5846 if (visited_p) | 6133 if (visited_p) |
5860 return 1; | 6147 return 1; |
5861 } | 6148 } |
5862 | 6149 |
5863 /* Same as above but for move_op. */ | 6150 /* Same as above but for move_op. */ |
5864 static int | 6151 static int |
5865 move_op_on_enter (insn_t insn ATTRIBUTE_UNUSED, | 6152 move_op_on_enter (insn_t insn ATTRIBUTE_UNUSED, |
5866 cmpd_local_params_p local_params ATTRIBUTE_UNUSED, | 6153 cmpd_local_params_p local_params ATTRIBUTE_UNUSED, |
5867 void *static_params ATTRIBUTE_UNUSED, bool visited_p) | 6154 void *static_params ATTRIBUTE_UNUSED, bool visited_p) |
5868 { | 6155 { |
5869 if (visited_p) | 6156 if (visited_p) |
5870 return -1; | 6157 return -1; |
5871 return 1; | 6158 return 1; |
5872 } | 6159 } |
5873 | 6160 |
5874 /* This function is called while descending current basic block if current | 6161 /* This function is called while descending current basic block if current |
5875 insn is not the original EXPR we're searching for. | 6162 insn is not the original EXPR we're searching for. |
5876 | 6163 |
5877 Return value: FALSE, if code_motion_path_driver should perform a local | 6164 Return value: FALSE, if code_motion_path_driver should perform a local |
5878 cleanup and return 0 itself; | 6165 cleanup and return 0 itself; |
5879 TRUE, if code_motion_path_driver should continue. */ | 6166 TRUE, if code_motion_path_driver should continue. */ |
5880 static bool | 6167 static bool |
5881 move_op_orig_expr_not_found (insn_t insn, av_set_t orig_ops ATTRIBUTE_UNUSED, | 6168 move_op_orig_expr_not_found (insn_t insn, av_set_t orig_ops ATTRIBUTE_UNUSED, |
5882 void *static_params) | 6169 void *static_params) |
5886 #ifdef ENABLE_CHECKING | 6173 #ifdef ENABLE_CHECKING |
5887 sparams->failed_insn = insn; | 6174 sparams->failed_insn = insn; |
5888 #endif | 6175 #endif |
5889 | 6176 |
5890 /* If we're scheduling separate expr, in order to generate correct code | 6177 /* If we're scheduling separate expr, in order to generate correct code |
5891 we need to stop the search at bookkeeping code generated with the | 6178 we need to stop the search at bookkeeping code generated with the |
5892 same destination register or memory. */ | 6179 same destination register or memory. */ |
5893 if (lhs_of_insn_equals_to_dest_p (insn, sparams->dest)) | 6180 if (lhs_of_insn_equals_to_dest_p (insn, sparams->dest)) |
5894 return false; | 6181 return false; |
5895 return true; | 6182 return true; |
5896 } | 6183 } |
5897 | 6184 |
5898 /* This function is called while descending current basic block if current | 6185 /* This function is called while descending current basic block if current |
5899 insn is not the original EXPR we're searching for. | 6186 insn is not the original EXPR we're searching for. |
5900 | 6187 |
5901 Return value: TRUE (code_motion_path_driver should continue). */ | 6188 Return value: TRUE (code_motion_path_driver should continue). */ |
5902 static bool | 6189 static bool |
5903 fur_orig_expr_not_found (insn_t insn, av_set_t orig_ops, void *static_params) | 6190 fur_orig_expr_not_found (insn_t insn, av_set_t orig_ops, void *static_params) |
5907 av_set_iterator avi; | 6194 av_set_iterator avi; |
5908 fur_static_params_p sparams = (fur_static_params_p) static_params; | 6195 fur_static_params_p sparams = (fur_static_params_p) static_params; |
5909 | 6196 |
5910 if (CALL_P (insn)) | 6197 if (CALL_P (insn)) |
5911 sparams->crosses_call = true; | 6198 sparams->crosses_call = true; |
6199 else if (DEBUG_INSN_P (insn)) | |
6200 return true; | |
5912 | 6201 |
5913 /* If current insn we are looking at cannot be executed together | 6202 /* If current insn we are looking at cannot be executed together |
5914 with original insn, then we can skip it safely. | 6203 with original insn, then we can skip it safely. |
5915 | 6204 |
5916 Example: ORIG_OPS = { (p6) r14 = sign_extend (r15); } | 6205 Example: ORIG_OPS = { (p6) r14 = sign_extend (r15); } |
5953 move_op_at_first_insn, | 6242 move_op_at_first_insn, |
5954 SUCCS_NORMAL, | 6243 SUCCS_NORMAL, |
5955 "move_op" | 6244 "move_op" |
5956 }; | 6245 }; |
5957 | 6246 |
5958 /* Hooks and data to perform find_used_regs operations | 6247 /* Hooks and data to perform find_used_regs operations |
5959 with code_motion_path_driver. */ | 6248 with code_motion_path_driver. */ |
5960 struct code_motion_path_driver_info_def fur_hooks = { | 6249 struct code_motion_path_driver_info_def fur_hooks = { |
5961 fur_on_enter, | 6250 fur_on_enter, |
5962 fur_orig_expr_found, | 6251 fur_orig_expr_found, |
5963 fur_orig_expr_not_found, | 6252 fur_orig_expr_not_found, |
5968 SUCCS_ALL, | 6257 SUCCS_ALL, |
5969 "find_used_regs" | 6258 "find_used_regs" |
5970 }; | 6259 }; |
5971 | 6260 |
5972 /* Traverse all successors of INSN. For each successor that is SUCCS_NORMAL | 6261 /* Traverse all successors of INSN. For each successor that is SUCCS_NORMAL |
5973 code_motion_path_driver is called recursively. Original operation | 6262 code_motion_path_driver is called recursively. Original operation |
5974 was found at least on one path that is starting with one of INSN's | 6263 was found at least on one path that is starting with one of INSN's |
5975 successors (this fact is asserted). ORIG_OPS is expressions we're looking | 6264 successors (this fact is asserted). ORIG_OPS is expressions we're looking |
5976 for, PATH is the path we've traversed, STATIC_PARAMS is the parameters | 6265 for, PATH is the path we've traversed, STATIC_PARAMS is the parameters |
5977 of either move_op or find_used_regs depending on the caller. | 6266 of either move_op or find_used_regs depending on the caller. |
5978 | 6267 |
5979 Return 0 if we haven't found expression, 1 if we found it, -1 if we don't | 6268 Return 0 if we haven't found expression, 1 if we found it, -1 if we don't |
5980 know for sure at this point. */ | 6269 know for sure at this point. */ |
5981 static int | 6270 static int |
5982 code_motion_process_successors (insn_t insn, av_set_t orig_ops, | 6271 code_motion_process_successors (insn_t insn, av_set_t orig_ops, |
5983 ilist_t path, void *static_params) | 6272 ilist_t path, void *static_params) |
5984 { | 6273 { |
5985 int res = 0; | 6274 int res = 0; |
5986 succ_iterator succ_i; | 6275 succ_iterator succ_i; |
5987 rtx succ; | 6276 rtx succ; |
5994 | 6283 |
5995 lparams.c_expr_local = &_x; | 6284 lparams.c_expr_local = &_x; |
5996 lparams.c_expr_merged = NULL; | 6285 lparams.c_expr_merged = NULL; |
5997 | 6286 |
5998 /* We need to process only NORMAL succs for move_op, and collect live | 6287 /* We need to process only NORMAL succs for move_op, and collect live |
5999 registers from ALL branches (including those leading out of the | 6288 registers from ALL branches (including those leading out of the |
6000 region) for find_used_regs. | 6289 region) for find_used_regs. |
6001 | 6290 |
6002 In move_op, there can be a case when insn's bb number has changed | 6291 In move_op, there can be a case when insn's bb number has changed |
6003 due to created bookkeeping. This happens very rare, as we need to | 6292 due to created bookkeeping. This happens very rare, as we need to |
6004 move expression from the beginning to the end of the same block. | 6293 move expression from the beginning to the end of the same block. |
6005 Rescan successors in this case. */ | 6294 Rescan successors in this case. */ |
6006 | 6295 |
6007 rescan: | 6296 rescan: |
6008 bb = BLOCK_FOR_INSN (insn); | 6297 bb = BLOCK_FOR_INSN (insn); |
6009 old_index = bb->index; | 6298 old_index = bb->index; |
6010 old_succs = EDGE_COUNT (bb->succs); | 6299 old_succs = EDGE_COUNT (bb->succs); |
6011 | 6300 |
6012 FOR_EACH_SUCC_1 (succ, succ_i, insn, code_motion_path_driver_info->succ_flags) | 6301 FOR_EACH_SUCC_1 (succ, succ_i, insn, code_motion_path_driver_info->succ_flags) |
6013 { | 6302 { |
6014 int b; | 6303 int b; |
6015 | 6304 |
6016 lparams.e1 = succ_i.e1; | 6305 lparams.e1 = succ_i.e1; |
6017 lparams.e2 = succ_i.e2; | 6306 lparams.e2 = succ_i.e2; |
6018 | 6307 |
6019 /* Go deep into recursion only for NORMAL edges (non-backedges within the | 6308 /* Go deep into recursion only for NORMAL edges (non-backedges within the |
6020 current region). */ | 6309 current region). */ |
6021 if (succ_i.current_flags == SUCCS_NORMAL) | 6310 if (succ_i.current_flags == SUCCS_NORMAL) |
6022 b = code_motion_path_driver (succ, orig_ops, path, &lparams, | 6311 b = code_motion_path_driver (succ, orig_ops, path, &lparams, |
6023 static_params); | 6312 static_params); |
6024 else | 6313 else |
6025 b = 0; | 6314 b = 0; |
6026 | 6315 |
6027 /* Merge c_expres found or unify live register sets from different | 6316 /* Merge c_expres found or unify live register sets from different |
6039 || EDGE_COUNT (bb->succs) != old_succs) | 6328 || EDGE_COUNT (bb->succs) != old_succs) |
6040 goto rescan; | 6329 goto rescan; |
6041 } | 6330 } |
6042 | 6331 |
6043 #ifdef ENABLE_CHECKING | 6332 #ifdef ENABLE_CHECKING |
6044 /* Here, RES==1 if original expr was found at least for one of the | 6333 /* Here, RES==1 if original expr was found at least for one of the |
6045 successors. After the loop, RES may happen to have zero value | 6334 successors. After the loop, RES may happen to have zero value |
6046 only if at some point the expr searched is present in av_set, but is | 6335 only if at some point the expr searched is present in av_set, but is |
6047 not found below. In most cases, this situation is an error. | 6336 not found below. In most cases, this situation is an error. |
6048 The exception is when the original operation is blocked by | 6337 The exception is when the original operation is blocked by |
6049 bookkeeping generated for another fence or for another path in current | 6338 bookkeeping generated for another fence or for another path in current |
6050 move_op. */ | 6339 move_op. */ |
6051 gcc_assert (res == 1 | 6340 gcc_assert (res == 1 |
6052 || (res == 0 | 6341 || (res == 0 |
6053 && av_set_could_be_blocked_by_bookkeeping_p (orig_ops, | 6342 && av_set_could_be_blocked_by_bookkeeping_p (orig_ops, |
6054 static_params)) | 6343 static_params)) |
6055 || res == -1); | 6344 || res == -1); |
6056 #endif | 6345 #endif |
6057 | 6346 |
6058 /* Merge data, clean up, etc. */ | 6347 /* Merge data, clean up, etc. */ |
6059 if (res != -1 && code_motion_path_driver_info->after_merge_succs) | 6348 if (res != -1 && code_motion_path_driver_info->after_merge_succs) |
6060 code_motion_path_driver_info->after_merge_succs (&lparams, static_params); | 6349 code_motion_path_driver_info->after_merge_succs (&lparams, static_params); |
6061 | 6350 |
6062 return res; | 6351 return res; |
6063 } | 6352 } |
6064 | 6353 |
6065 | 6354 |
6066 /* Perform a cleanup when the driver is about to terminate. ORIG_OPS_P | 6355 /* Perform a cleanup when the driver is about to terminate. ORIG_OPS_P |
6067 is the pointer to the av set with expressions we were looking for, | 6356 is the pointer to the av set with expressions we were looking for, |
6068 PATH_P is the pointer to the traversed path. */ | 6357 PATH_P is the pointer to the traversed path. */ |
6069 static inline void | 6358 static inline void |
6070 code_motion_path_driver_cleanup (av_set_t *orig_ops_p, ilist_t *path_p) | 6359 code_motion_path_driver_cleanup (av_set_t *orig_ops_p, ilist_t *path_p) |
6071 { | 6360 { |
6072 ilist_remove (path_p); | 6361 ilist_remove (path_p); |
6073 av_set_clear (orig_ops_p); | 6362 av_set_clear (orig_ops_p); |
6074 } | 6363 } |
6075 | 6364 |
6076 /* The driver function that implements move_op or find_used_regs | 6365 /* The driver function that implements move_op or find_used_regs |
6077 functionality dependent whether code_motion_path_driver_INFO is set to | 6366 functionality dependent whether code_motion_path_driver_INFO is set to |
6078 &MOVE_OP_HOOKS or &FUR_HOOKS. This function implements the common parts | 6367 &MOVE_OP_HOOKS or &FUR_HOOKS. This function implements the common parts |
6079 of code (CFG traversal etc) that are shared among both functions. INSN | 6368 of code (CFG traversal etc) that are shared among both functions. INSN |
6080 is the insn we're starting the search from, ORIG_OPS are the expressions | 6369 is the insn we're starting the search from, ORIG_OPS are the expressions |
6081 we're searching for, PATH is traversed path, LOCAL_PARAMS_IN are local | 6370 we're searching for, PATH is traversed path, LOCAL_PARAMS_IN are local |
6082 parameters of the driver, and STATIC_PARAMS are static parameters of | 6371 parameters of the driver, and STATIC_PARAMS are static parameters of |
6083 the caller. | 6372 the caller. |
6084 | 6373 |
6085 Returns whether original instructions were found. Note that top-level | 6374 Returns whether original instructions were found. Note that top-level |
6086 code_motion_path_driver always returns true. */ | 6375 code_motion_path_driver always returns true. */ |
6087 static int | 6376 static int |
6088 code_motion_path_driver (insn_t insn, av_set_t orig_ops, ilist_t path, | 6377 code_motion_path_driver (insn_t insn, av_set_t orig_ops, ilist_t path, |
6089 cmpd_local_params_p local_params_in, | 6378 cmpd_local_params_p local_params_in, |
6090 void *static_params) | 6379 void *static_params) |
6091 { | 6380 { |
6092 expr_t expr = NULL; | 6381 expr_t expr = NULL; |
6093 basic_block bb = BLOCK_FOR_INSN (insn); | 6382 basic_block bb = BLOCK_FOR_INSN (insn); |
6094 insn_t first_insn, bb_tail, before_first; | 6383 insn_t first_insn, bb_tail, before_first; |
6110 { | 6399 { |
6111 if (sched_verbose >= 6) | 6400 if (sched_verbose >= 6) |
6112 sel_print ("Insn %d is ineligible successor\n", INSN_UID (insn)); | 6401 sel_print ("Insn %d is ineligible successor\n", INSN_UID (insn)); |
6113 return false; | 6402 return false; |
6114 } | 6403 } |
6115 | 6404 |
6116 /* The block can have invalid av set, in which case it was created earlier | 6405 /* The block can have invalid av set, in which case it was created earlier |
6117 during move_op. Return immediately. */ | 6406 during move_op. Return immediately. */ |
6118 if (sel_bb_head_p (insn)) | 6407 if (sel_bb_head_p (insn)) |
6119 { | 6408 { |
6120 if (! AV_SET_VALID_P (insn)) | 6409 if (! AV_SET_VALID_P (insn)) |
6127 | 6416 |
6128 if (bitmap_bit_p (code_motion_visited_blocks, bb->index)) | 6417 if (bitmap_bit_p (code_motion_visited_blocks, bb->index)) |
6129 { | 6418 { |
6130 /* We have already found an original operation on this branch, do not | 6419 /* We have already found an original operation on this branch, do not |
6131 go any further and just return TRUE here. If we don't stop here, | 6420 go any further and just return TRUE here. If we don't stop here, |
6132 function can have exponential behaviour even on the small code | 6421 function can have exponential behaviour even on the small code |
6133 with many different paths (e.g. with data speculation and | 6422 with many different paths (e.g. with data speculation and |
6134 recovery blocks). */ | 6423 recovery blocks). */ |
6135 if (sched_verbose >= 6) | 6424 if (sched_verbose >= 6) |
6136 sel_print ("Block %d already visited in this traversal\n", bb->index); | 6425 sel_print ("Block %d already visited in this traversal\n", bb->index); |
6137 if (code_motion_path_driver_info->on_enter) | 6426 if (code_motion_path_driver_info->on_enter) |
6138 return code_motion_path_driver_info->on_enter (insn, | 6427 return code_motion_path_driver_info->on_enter (insn, |
6139 local_params_in, | 6428 local_params_in, |
6140 static_params, | 6429 static_params, |
6141 true); | 6430 true); |
6142 } | 6431 } |
6143 } | 6432 } |
6144 | 6433 |
6145 if (code_motion_path_driver_info->on_enter) | 6434 if (code_motion_path_driver_info->on_enter) |
6146 code_motion_path_driver_info->on_enter (insn, local_params_in, | 6435 code_motion_path_driver_info->on_enter (insn, local_params_in, |
6147 static_params, false); | 6436 static_params, false); |
6148 orig_ops = av_set_copy (orig_ops); | 6437 orig_ops = av_set_copy (orig_ops); |
6149 | 6438 |
6158 sel_print ("No intersection with av set of block %d\n", bb->index); | 6447 sel_print ("No intersection with av set of block %d\n", bb->index); |
6159 return false; | 6448 return false; |
6160 } | 6449 } |
6161 | 6450 |
6162 /* For non-speculative insns we have to leave only one form of the | 6451 /* For non-speculative insns we have to leave only one form of the |
6163 original operation, because if we don't, we may end up with | 6452 original operation, because if we don't, we may end up with |
6164 different C_EXPRes and, consequently, with bookkeepings for different | 6453 different C_EXPRes and, consequently, with bookkeepings for different |
6165 expression forms along the same code motion path. That may lead to | 6454 expression forms along the same code motion path. That may lead to |
6166 generation of incorrect code. So for each code motion we stick to | 6455 generation of incorrect code. So for each code motion we stick to |
6167 the single form of the instruction, except for speculative insns | 6456 the single form of the instruction, except for speculative insns |
6168 which we need to keep in different forms with all speculation | 6457 which we need to keep in different forms with all speculation |
6169 types. */ | 6458 types. */ |
6170 av_set_leave_one_nonspec (&orig_ops); | 6459 av_set_leave_one_nonspec (&orig_ops); |
6171 | 6460 |
6172 /* It is not possible that all ORIG_OPS are filtered out. */ | 6461 /* It is not possible that all ORIG_OPS are filtered out. */ |
6173 gcc_assert (orig_ops); | 6462 gcc_assert (orig_ops); |
6177 ilist_add (&path, insn); | 6466 ilist_add (&path, insn); |
6178 first_insn = insn; | 6467 first_insn = insn; |
6179 bb_tail = sel_bb_end (bb); | 6468 bb_tail = sel_bb_end (bb); |
6180 | 6469 |
6181 /* Descend the basic block in search of the original expr; this part | 6470 /* Descend the basic block in search of the original expr; this part |
6182 corresponds to the part of the original move_op procedure executed | 6471 corresponds to the part of the original move_op procedure executed |
6183 before the recursive call. */ | 6472 before the recursive call. */ |
6184 for (;;) | 6473 for (;;) |
6185 { | 6474 { |
6186 /* Look at the insn and decide if it could be an ancestor of currently | 6475 /* Look at the insn and decide if it could be an ancestor of currently |
6187 scheduling operation. If it is so, then the insn "dest = op" could | 6476 scheduling operation. If it is so, then the insn "dest = op" could |
6188 either be replaced with "dest = reg", because REG now holds the result | 6477 either be replaced with "dest = reg", because REG now holds the result |
6189 of OP, or just removed, if we've scheduled the insn as a whole. | 6478 of OP, or just removed, if we've scheduled the insn as a whole. |
6190 | 6479 |
6191 If this insn doesn't contain currently scheduling OP, then proceed | 6480 If this insn doesn't contain currently scheduling OP, then proceed |
6192 with searching and look at its successors. Operations we're searching | 6481 with searching and look at its successors. Operations we're searching |
6193 for could have changed when moving up through this insn via | 6482 for could have changed when moving up through this insn via |
6194 substituting. In this case, perform unsubstitution on them first. | 6483 substituting. In this case, perform unsubstitution on them first. |
6195 | 6484 |
6196 When traversing the DAG below this insn is finished, insert | 6485 When traversing the DAG below this insn is finished, insert |
6197 bookkeeping code, if the insn is a joint point, and remove | 6486 bookkeeping code, if the insn is a joint point, and remove |
6198 leftovers. */ | 6487 leftovers. */ |
6204 | 6493 |
6205 /* We have found the original operation. */ | 6494 /* We have found the original operation. */ |
6206 if (sched_verbose >= 6) | 6495 if (sched_verbose >= 6) |
6207 sel_print ("Found original operation at insn %d\n", INSN_UID (insn)); | 6496 sel_print ("Found original operation at insn %d\n", INSN_UID (insn)); |
6208 | 6497 |
6209 code_motion_path_driver_info->orig_expr_found | 6498 code_motion_path_driver_info->orig_expr_found |
6210 (insn, expr, local_params_in, static_params); | 6499 (insn, expr, local_params_in, static_params); |
6211 | 6500 |
6212 /* Step back, so on the way back we'll start traversing from the | 6501 /* Step back, so on the way back we'll start traversing from the |
6213 previous insn (or we'll see that it's bb_note and skip that | 6502 previous insn (or we'll see that it's bb_note and skip that |
6214 loop). */ | 6503 loop). */ |
6215 if (insn == first_insn) | 6504 if (insn == first_insn) |
6216 { | 6505 { |
6217 first_insn = NEXT_INSN (last_insn); | 6506 first_insn = NEXT_INSN (last_insn); |
6218 removed_last_insn = sel_bb_end_p (last_insn); | 6507 removed_last_insn = sel_bb_end_p (last_insn); |
6222 } | 6511 } |
6223 else | 6512 else |
6224 { | 6513 { |
6225 /* We haven't found the original expr, continue descending the basic | 6514 /* We haven't found the original expr, continue descending the basic |
6226 block. */ | 6515 block. */ |
6227 if (code_motion_path_driver_info->orig_expr_not_found | 6516 if (code_motion_path_driver_info->orig_expr_not_found |
6228 (insn, orig_ops, static_params)) | 6517 (insn, orig_ops, static_params)) |
6229 { | 6518 { |
6230 /* Av set ops could have been changed when moving through this | 6519 /* Av set ops could have been changed when moving through this |
6231 insn. To find them below it, we have to un-substitute them. */ | 6520 insn. To find them below it, we have to un-substitute them. */ |
6232 undo_transformations (&orig_ops, insn); | 6521 undo_transformations (&orig_ops, insn); |
6233 } | 6522 } |
6234 else | 6523 else |
6235 { | 6524 { |
6236 /* Clean up and return, if the hook tells us to do so. It may | 6525 /* Clean up and return, if the hook tells us to do so. It may |
6237 happen if we've encountered the previously created | 6526 happen if we've encountered the previously created |
6238 bookkeeping. */ | 6527 bookkeeping. */ |
6239 code_motion_path_driver_cleanup (&orig_ops, &path); | 6528 code_motion_path_driver_cleanup (&orig_ops, &path); |
6240 return -1; | 6529 return -1; |
6241 } | 6530 } |
6242 | 6531 |
6248 break; | 6537 break; |
6249 | 6538 |
6250 insn = NEXT_INSN (insn); | 6539 insn = NEXT_INSN (insn); |
6251 } | 6540 } |
6252 | 6541 |
6253 /* Here INSN either points to the insn before the original insn (may be | 6542 /* Here INSN either points to the insn before the original insn (may be |
6254 bb_note, if original insn was a bb_head) or to the bb_end. */ | 6543 bb_note, if original insn was a bb_head) or to the bb_end. */ |
6255 if (!expr) | 6544 if (!expr) |
6256 { | 6545 { |
6257 int res; | 6546 int res; |
6258 | 6547 |
6261 /* Add bb tail to PATH (but it doesn't make any sense if it's a bb_head - | 6550 /* Add bb tail to PATH (but it doesn't make any sense if it's a bb_head - |
6262 it's already in PATH then). */ | 6551 it's already in PATH then). */ |
6263 if (insn != first_insn) | 6552 if (insn != first_insn) |
6264 ilist_add (&path, insn); | 6553 ilist_add (&path, insn); |
6265 | 6554 |
6266 /* Process_successors should be able to find at least one | 6555 /* Process_successors should be able to find at least one |
6267 successor for which code_motion_path_driver returns TRUE. */ | 6556 successor for which code_motion_path_driver returns TRUE. */ |
6268 res = code_motion_process_successors (insn, orig_ops, | 6557 res = code_motion_process_successors (insn, orig_ops, |
6269 path, static_params); | 6558 path, static_params); |
6270 | 6559 |
6271 /* Remove bb tail from path. */ | 6560 /* Remove bb tail from path. */ |
6272 if (insn != first_insn) | 6561 if (insn != first_insn) |
6273 ilist_remove (&path); | 6562 ilist_remove (&path); |
6274 | 6563 |
6275 if (res != 1) | 6564 if (res != 1) |
6276 { | 6565 { |
6277 /* This is the case when one of the original expr is no longer available | 6566 /* This is the case when one of the original expr is no longer available |
6278 due to bookkeeping created on this branch with the same register. | 6567 due to bookkeeping created on this branch with the same register. |
6279 In the original algorithm, which doesn't have update_data_sets call | 6568 In the original algorithm, which doesn't have update_data_sets call |
6280 on a bookkeeping block, it would simply result in returning | 6569 on a bookkeeping block, it would simply result in returning |
6281 FALSE when we've encountered a previously generated bookkeeping | 6570 FALSE when we've encountered a previously generated bookkeeping |
6282 insn in moveop_orig_expr_not_found. */ | 6571 insn in moveop_orig_expr_not_found. */ |
6283 code_motion_path_driver_cleanup (&orig_ops, &path); | 6572 code_motion_path_driver_cleanup (&orig_ops, &path); |
6284 return res; | 6573 return res; |
6285 } | 6574 } |
6286 } | 6575 } |
6287 | 6576 |
6288 /* Don't need it any more. */ | 6577 /* Don't need it any more. */ |
6289 av_set_clear (&orig_ops); | 6578 av_set_clear (&orig_ops); |
6290 | 6579 |
6291 /* Backward pass: now, when we have C_EXPR computed, we'll drag it to | 6580 /* Backward pass: now, when we have C_EXPR computed, we'll drag it to |
6292 the beginning of the basic block. */ | 6581 the beginning of the basic block. */ |
6293 before_first = PREV_INSN (first_insn); | 6582 before_first = PREV_INSN (first_insn); |
6294 while (insn != before_first) | 6583 while (insn != before_first) |
6295 { | 6584 { |
6296 if (code_motion_path_driver_info->ascend) | 6585 if (code_motion_path_driver_info->ascend) |
6297 code_motion_path_driver_info->ascend (insn, static_params); | 6586 code_motion_path_driver_info->ascend (insn, static_params); |
6298 | 6587 |
6299 insn = PREV_INSN (insn); | 6588 insn = PREV_INSN (insn); |
6300 } | 6589 } |
6301 | 6590 |
6302 /* Now we're at the bb head. */ | 6591 /* Now we're at the bb head. */ |
6303 insn = first_insn; | 6592 insn = first_insn; |
6304 ilist_remove (&path); | 6593 ilist_remove (&path); |
6305 local_params_in->removed_last_insn = removed_last_insn; | 6594 local_params_in->removed_last_insn = removed_last_insn; |
6306 code_motion_path_driver_info->at_first_insn (insn, local_params_in, static_params); | 6595 code_motion_path_driver_info->at_first_insn (insn, local_params_in, static_params); |
6307 | 6596 |
6308 /* This should be the very last operation as at bb head we could change | 6597 /* This should be the very last operation as at bb head we could change |
6309 the numbering by creating bookkeeping blocks. */ | 6598 the numbering by creating bookkeeping blocks. */ |
6310 if (removed_last_insn) | 6599 if (removed_last_insn) |
6311 insn = PREV_INSN (insn); | 6600 insn = PREV_INSN (insn); |
6312 bitmap_set_bit (code_motion_visited_blocks, BLOCK_FOR_INSN (insn)->index); | 6601 bitmap_set_bit (code_motion_visited_blocks, BLOCK_FOR_INSN (insn)->index); |
6313 return true; | 6602 return true; |
6314 } | 6603 } |
6315 | 6604 |
6316 /* Move up the operations from ORIG_OPS set traversing the dag starting | 6605 /* Move up the operations from ORIG_OPS set traversing the dag starting |
6317 from INSN. PATH represents the edges traversed so far. | 6606 from INSN. PATH represents the edges traversed so far. |
6318 DEST is the register chosen for scheduling the current expr. Insert | 6607 DEST is the register chosen for scheduling the current expr. Insert |
6319 bookkeeping code in the join points. EXPR_VLIW is the chosen expression, | 6608 bookkeeping code in the join points. EXPR_VLIW is the chosen expression, |
6320 C_EXPR is how it looks like at the given cfg point. | 6609 C_EXPR is how it looks like at the given cfg point. |
6321 Set *SHOULD_MOVE to indicate whether we have only disconnected | 6610 Set *SHOULD_MOVE to indicate whether we have only disconnected |
6322 one of the insns found. | 6611 one of the insns found. |
6323 | 6612 |
6324 Returns whether original instructions were found, which is asserted | 6613 Returns whether original instructions were found, which is asserted |
6325 to be true in the caller. */ | 6614 to be true in the caller. */ |
6326 static bool | 6615 static bool |
6327 move_op (insn_t insn, av_set_t orig_ops, expr_t expr_vliw, | 6616 move_op (insn_t insn, av_set_t orig_ops, expr_t expr_vliw, |
6328 rtx dest, expr_t c_expr, bool *should_move) | 6617 rtx dest, expr_t c_expr, bool *should_move) |
6329 { | 6618 { |
6330 struct moveop_static_params sparams; | 6619 struct moveop_static_params sparams; |
6331 struct cmpd_local_params lparams; | 6620 struct cmpd_local_params lparams; |
6332 bool res; | 6621 bool res; |
6333 | 6622 |
6334 /* Init params for code_motion_path_driver. */ | 6623 /* Init params for code_motion_path_driver. */ |
6335 sparams.dest = dest; | 6624 sparams.dest = dest; |
6336 sparams.c_expr = c_expr; | 6625 sparams.c_expr = c_expr; |
6337 sparams.uid = INSN_UID (EXPR_INSN_RTX (expr_vliw)); | 6626 sparams.uid = INSN_UID (EXPR_INSN_RTX (expr_vliw)); |
6338 #ifdef ENABLE_CHECKING | 6627 #ifdef ENABLE_CHECKING |
6339 sparams.failed_insn = NULL; | 6628 sparams.failed_insn = NULL; |
6341 sparams.was_renamed = false; | 6630 sparams.was_renamed = false; |
6342 lparams.e1 = NULL; | 6631 lparams.e1 = NULL; |
6343 | 6632 |
6344 /* We haven't visited any blocks yet. */ | 6633 /* We haven't visited any blocks yet. */ |
6345 bitmap_clear (code_motion_visited_blocks); | 6634 bitmap_clear (code_motion_visited_blocks); |
6346 | 6635 |
6347 /* Set appropriate hooks and data. */ | 6636 /* Set appropriate hooks and data. */ |
6348 code_motion_path_driver_info = &move_op_hooks; | 6637 code_motion_path_driver_info = &move_op_hooks; |
6349 res = code_motion_path_driver (insn, orig_ops, NULL, &lparams, &sparams); | 6638 res = code_motion_path_driver (insn, orig_ops, NULL, &lparams, &sparams); |
6350 | 6639 |
6351 if (sparams.was_renamed) | 6640 if (sparams.was_renamed) |
6360 /* Functions that work with regions. */ | 6649 /* Functions that work with regions. */ |
6361 | 6650 |
6362 /* Current number of seqno used in init_seqno and init_seqno_1. */ | 6651 /* Current number of seqno used in init_seqno and init_seqno_1. */ |
6363 static int cur_seqno; | 6652 static int cur_seqno; |
6364 | 6653 |
6365 /* A helper for init_seqno. Traverse the region starting from BB and | 6654 /* A helper for init_seqno. Traverse the region starting from BB and |
6366 compute seqnos for visited insns, marking visited bbs in VISITED_BBS. | 6655 compute seqnos for visited insns, marking visited bbs in VISITED_BBS. |
6367 Clear visited blocks from BLOCKS_TO_RESCHEDULE. */ | 6656 Clear visited blocks from BLOCKS_TO_RESCHEDULE. */ |
6368 static void | 6657 static void |
6369 init_seqno_1 (basic_block bb, sbitmap visited_bbs, bitmap blocks_to_reschedule) | 6658 init_seqno_1 (basic_block bb, sbitmap visited_bbs, bitmap blocks_to_reschedule) |
6370 { | 6659 { |
6371 int bbi = BLOCK_TO_BB (bb->index); | 6660 int bbi = BLOCK_TO_BB (bb->index); |
6375 | 6664 |
6376 SET_BIT (visited_bbs, bbi); | 6665 SET_BIT (visited_bbs, bbi); |
6377 if (blocks_to_reschedule) | 6666 if (blocks_to_reschedule) |
6378 bitmap_clear_bit (blocks_to_reschedule, bb->index); | 6667 bitmap_clear_bit (blocks_to_reschedule, bb->index); |
6379 | 6668 |
6380 FOR_EACH_SUCC_1 (succ_insn, si, BB_END (bb), | 6669 FOR_EACH_SUCC_1 (succ_insn, si, BB_END (bb), |
6381 SUCCS_NORMAL | SUCCS_SKIP_TO_LOOP_EXITS) | 6670 SUCCS_NORMAL | SUCCS_SKIP_TO_LOOP_EXITS) |
6382 { | 6671 { |
6383 basic_block succ = BLOCK_FOR_INSN (succ_insn); | 6672 basic_block succ = BLOCK_FOR_INSN (succ_insn); |
6384 int succ_bbi = BLOCK_TO_BB (succ->index); | 6673 int succ_bbi = BLOCK_TO_BB (succ->index); |
6385 | 6674 |
6396 for (insn = BB_END (bb); insn != note; insn = PREV_INSN (insn)) | 6685 for (insn = BB_END (bb); insn != note; insn = PREV_INSN (insn)) |
6397 INSN_SEQNO (insn) = cur_seqno--; | 6686 INSN_SEQNO (insn) = cur_seqno--; |
6398 } | 6687 } |
6399 | 6688 |
6400 /* Initialize seqnos for the current region. NUMBER_OF_INSNS is the number | 6689 /* Initialize seqnos for the current region. NUMBER_OF_INSNS is the number |
6401 of instructions in the region, BLOCKS_TO_RESCHEDULE contains blocks on | 6690 of instructions in the region, BLOCKS_TO_RESCHEDULE contains blocks on |
6402 which we're rescheduling when pipelining, FROM is the block where | 6691 which we're rescheduling when pipelining, FROM is the block where |
6403 traversing region begins (it may not be the head of the region when | 6692 traversing region begins (it may not be the head of the region when |
6404 pipelining, but the head of the loop instead). | 6693 pipelining, but the head of the loop instead). |
6405 | 6694 |
6406 Returns the maximal seqno found. */ | 6695 Returns the maximal seqno found. */ |
6407 static int | 6696 static int |
6408 init_seqno (int number_of_insns, bitmap blocks_to_reschedule, basic_block from) | 6697 init_seqno (int number_of_insns, bitmap blocks_to_reschedule, basic_block from) |
6409 { | 6698 { |
6440 static void | 6729 static void |
6441 sel_setup_region_sched_flags (void) | 6730 sel_setup_region_sched_flags (void) |
6442 { | 6731 { |
6443 enable_schedule_as_rhs_p = 1; | 6732 enable_schedule_as_rhs_p = 1; |
6444 bookkeeping_p = 1; | 6733 bookkeeping_p = 1; |
6445 pipelining_p = (bookkeeping_p | 6734 pipelining_p = (bookkeeping_p |
6446 && (flag_sel_sched_pipelining != 0) | 6735 && (flag_sel_sched_pipelining != 0) |
6447 && current_loop_nest != NULL); | 6736 && current_loop_nest != NULL); |
6448 max_insns_to_rename = PARAM_VALUE (PARAM_SELSCHED_INSNS_TO_RENAME); | 6737 max_insns_to_rename = PARAM_VALUE (PARAM_SELSCHED_INSNS_TO_RENAME); |
6449 max_ws = MAX_WS; | 6738 max_ws = MAX_WS; |
6450 } | 6739 } |
6481 | 6770 |
6482 /* Purge meaningless empty blocks in the middle of a region. */ | 6771 /* Purge meaningless empty blocks in the middle of a region. */ |
6483 static void | 6772 static void |
6484 purge_empty_blocks (void) | 6773 purge_empty_blocks (void) |
6485 { | 6774 { |
6486 int i ; | 6775 /* Do not attempt to delete preheader. */ |
6487 | 6776 int i = sel_is_loop_preheader_p (BASIC_BLOCK (BB_TO_BLOCK (0))) ? 1 : 0; |
6488 for (i = 1; i < current_nr_blocks; ) | 6777 |
6778 while (i < current_nr_blocks) | |
6489 { | 6779 { |
6490 basic_block b = BASIC_BLOCK (BB_TO_BLOCK (i)); | 6780 basic_block b = BASIC_BLOCK (BB_TO_BLOCK (i)); |
6491 | 6781 |
6492 if (maybe_tidy_empty_bb (b)) | 6782 if (maybe_tidy_empty_bb (b)) |
6493 continue; | 6783 continue; |
6520 int i; | 6810 int i; |
6521 bb_vec_t bbs; | 6811 bb_vec_t bbs; |
6522 | 6812 |
6523 rgn_setup_region (rgn); | 6813 rgn_setup_region (rgn); |
6524 | 6814 |
6525 /* Even if sched_is_disabled_for_current_region_p() is true, we still | 6815 /* Even if sched_is_disabled_for_current_region_p() is true, we still |
6526 do region initialization here so the region can be bundled correctly, | 6816 do region initialization here so the region can be bundled correctly, |
6527 but we'll skip the scheduling in sel_sched_region (). */ | 6817 but we'll skip the scheduling in sel_sched_region (). */ |
6528 if (current_region_empty_p ()) | 6818 if (current_region_empty_p ()) |
6529 return true; | 6819 return true; |
6530 | 6820 |
6569 int header = (sel_is_loop_preheader_p (BASIC_BLOCK (BB_TO_BLOCK (0))) | 6859 int header = (sel_is_loop_preheader_p (BASIC_BLOCK (BB_TO_BLOCK (0))) |
6570 ? 1 | 6860 ? 1 |
6571 : 0); | 6861 : 0); |
6572 | 6862 |
6573 if (current_nr_blocks == header + 1) | 6863 if (current_nr_blocks == header + 1) |
6574 update_liveness_on_insn | 6864 update_liveness_on_insn |
6575 (sel_bb_head (BASIC_BLOCK (BB_TO_BLOCK (header)))); | 6865 (sel_bb_head (BASIC_BLOCK (BB_TO_BLOCK (header)))); |
6576 } | 6866 } |
6577 | 6867 |
6578 /* Set hooks so that no newly generated insn will go out unnoticed. */ | 6868 /* Set hooks so that no newly generated insn will go out unnoticed. */ |
6579 sel_register_cfg_hooks (); | 6869 sel_register_cfg_hooks (); |
6580 | 6870 |
6581 /* !!! We call target.sched.md_init () for the whole region, but we invoke | 6871 /* !!! We call target.sched.md_init () for the whole region, but we invoke |
6582 targetm.sched.md_finish () for every ebb. */ | 6872 targetm.sched.md_finish () for every ebb. */ |
6616 FOR_BB_INSNS (bb, insn) | 6906 FOR_BB_INSNS (bb, insn) |
6617 if (INSN_P (insn)) | 6907 if (INSN_P (insn)) |
6618 { | 6908 { |
6619 expr_t expr = INSN_EXPR (insn); | 6909 expr_t expr = INSN_EXPR (insn); |
6620 | 6910 |
6621 if (EXPR_WAS_SUBSTITUTED (expr)) | 6911 if (EXPR_WAS_SUBSTITUTED (expr)) |
6622 validate_simplify_insn (insn); | 6912 validate_simplify_insn (insn); |
6623 } | 6913 } |
6624 } | 6914 } |
6625 } | 6915 } |
6626 | 6916 |
6671 targetm.sched.md_init (sched_dump, sched_verbose, -1); | 6961 targetm.sched.md_init (sched_dump, sched_verbose, -1); |
6672 } | 6962 } |
6673 | 6963 |
6674 state_reset (curr_state); | 6964 state_reset (curr_state); |
6675 advance_state (curr_state); | 6965 advance_state (curr_state); |
6676 | 6966 |
6677 for (insn = current_sched_info->head; | 6967 for (insn = current_sched_info->head; |
6678 insn != current_sched_info->next_tail; | 6968 insn != current_sched_info->next_tail; |
6679 insn = NEXT_INSN (insn)) | 6969 insn = NEXT_INSN (insn)) |
6680 { | 6970 { |
6681 int cost, haifa_cost; | 6971 int cost, haifa_cost; |
6700 if (asm_p) | 6990 if (asm_p) |
6701 /* This is asm insn which *had* to be scheduled first | 6991 /* This is asm insn which *had* to be scheduled first |
6702 on the cycle. */ | 6992 on the cycle. */ |
6703 haifa_cost = 1; | 6993 haifa_cost = 1; |
6704 else | 6994 else |
6705 /* This is a use/clobber insn. It should not change | 6995 /* This is a use/clobber insn. It should not change |
6706 cost. */ | 6996 cost. */ |
6707 haifa_cost = 0; | 6997 haifa_cost = 0; |
6708 } | 6998 } |
6709 else | 6999 else |
6710 haifa_cost = estimate_insn_cost (insn, curr_state); | 7000 haifa_cost = estimate_insn_cost (insn, curr_state); |
6730 { | 7020 { |
6731 sel_print ("advance_state (state_transition)\n"); | 7021 sel_print ("advance_state (state_transition)\n"); |
6732 debug_state (curr_state); | 7022 debug_state (curr_state); |
6733 } | 7023 } |
6734 | 7024 |
6735 /* The DFA may report that e.g. insn requires 2 cycles to be | 7025 /* The DFA may report that e.g. insn requires 2 cycles to be |
6736 issued, but on the next cycle it says that insn is ready | 7026 issued, but on the next cycle it says that insn is ready |
6737 to go. Check this here. */ | 7027 to go. Check this here. */ |
6738 if (!after_stall | 7028 if (!after_stall |
6739 && real_insn | 7029 && real_insn |
6740 && haifa_cost > 0 | 7030 && haifa_cost > 0 |
6741 && estimate_insn_cost (insn, curr_state) == 0) | 7031 && estimate_insn_cost (insn, curr_state) == 0) |
6742 break; | 7032 break; |
6743 } | 7033 } |
6744 | 7034 |
6817 if (sched_verbose >= 2) | 7107 if (sched_verbose >= 2) |
6818 sel_print ("Cost for insn %d is %d\n", INSN_UID (insn), cost); | 7108 sel_print ("Cost for insn %d is %d\n", INSN_UID (insn), cost); |
6819 } | 7109 } |
6820 } | 7110 } |
6821 | 7111 |
6822 /* Perform MD_FINISH on EBBs comprising current region. When | 7112 /* Perform MD_FINISH on EBBs comprising current region. When |
6823 RESET_SCHED_CYCLES_P is true, run a pass emulating the scheduler | 7113 RESET_SCHED_CYCLES_P is true, run a pass emulating the scheduler |
6824 to produce correct sched cycles on insns. */ | 7114 to produce correct sched cycles on insns. */ |
6825 static void | 7115 static void |
6826 sel_region_target_finish (bool reset_sched_cycles_p) | 7116 sel_region_target_finish (bool reset_sched_cycles_p) |
6827 { | 7117 { |
6863 | 7153 |
6864 BITMAP_FREE (scheduled_blocks); | 7154 BITMAP_FREE (scheduled_blocks); |
6865 } | 7155 } |
6866 | 7156 |
6867 /* Free the scheduling data for the current region. When RESET_SCHED_CYCLES_P | 7157 /* Free the scheduling data for the current region. When RESET_SCHED_CYCLES_P |
6868 is true, make an additional pass emulating scheduler to get correct insn | 7158 is true, make an additional pass emulating scheduler to get correct insn |
6869 cycles for md_finish calls. */ | 7159 cycles for md_finish calls. */ |
6870 static void | 7160 static void |
6871 sel_region_finish (bool reset_sched_cycles_p) | 7161 sel_region_finish (bool reset_sched_cycles_p) |
6872 { | 7162 { |
6873 simplify_changed_insns (); | 7163 simplify_changed_insns (); |
6944 { | 7234 { |
6945 fence_t fence = NULL; | 7235 fence_t fence = NULL; |
6946 int seqno = 0; | 7236 int seqno = 0; |
6947 flist_t fences2; | 7237 flist_t fences2; |
6948 bool first_p = true; | 7238 bool first_p = true; |
6949 | 7239 |
6950 /* Choose the next fence group to schedule. | 7240 /* Choose the next fence group to schedule. |
6951 The fact that insn can be scheduled only once | 7241 The fact that insn can be scheduled only once |
6952 on the cycle is guaranteed by two properties: | 7242 on the cycle is guaranteed by two properties: |
6953 1. seqnos of parallel groups decrease with each iteration. | 7243 1. seqnos of parallel groups decrease with each iteration. |
6954 2. If is_ineligible_successor () sees the larger seqno, it | 7244 2. If is_ineligible_successor () sees the larger seqno, it |
6980 fill_insns (fence, seqno, scheduled_insns_tailpp); | 7270 fill_insns (fence, seqno, scheduled_insns_tailpp); |
6981 FENCE_PROCESSED_P (fence) = true; | 7271 FENCE_PROCESSED_P (fence) = true; |
6982 } | 7272 } |
6983 | 7273 |
6984 /* All av_sets are invalidated by GLOBAL_LEVEL increase, thus we | 7274 /* All av_sets are invalidated by GLOBAL_LEVEL increase, thus we |
6985 don't need to keep bookkeeping-invalidated and target-unavailable | 7275 don't need to keep bookkeeping-invalidated and target-unavailable |
6986 vinsns any more. */ | 7276 vinsns any more. */ |
6987 vinsn_vec_clear (&vec_bookkeeping_blocked_vinsns); | 7277 vinsn_vec_clear (&vec_bookkeeping_blocked_vinsns); |
6988 vinsn_vec_clear (&vec_target_unavailable_vinsns); | 7278 vinsn_vec_clear (&vec_target_unavailable_vinsns); |
6989 } | 7279 } |
6990 | 7280 |
6996 | 7286 |
6997 /* The first element is already processed. */ | 7287 /* The first element is already processed. */ |
6998 while ((fences = FLIST_NEXT (fences))) | 7288 while ((fences = FLIST_NEXT (fences))) |
6999 { | 7289 { |
7000 int seqno = INSN_SEQNO (FENCE_INSN (FLIST_FENCE (fences))); | 7290 int seqno = INSN_SEQNO (FENCE_INSN (FLIST_FENCE (fences))); |
7001 | 7291 |
7002 if (*min_seqno > seqno) | 7292 if (*min_seqno > seqno) |
7003 *min_seqno = seqno; | 7293 *min_seqno = seqno; |
7004 else if (*max_seqno < seqno) | 7294 else if (*max_seqno < seqno) |
7005 *max_seqno = seqno; | 7295 *max_seqno = seqno; |
7006 } | 7296 } |
7007 } | 7297 } |
7008 | 7298 |
7009 /* Calculate new fences from FENCES. */ | 7299 /* Calculate new fences from FENCES. */ |
7010 static flist_t | 7300 static flist_t |
7011 calculate_new_fences (flist_t fences, int orig_max_seqno) | 7301 calculate_new_fences (flist_t fences, int orig_max_seqno) |
7012 { | 7302 { |
7013 flist_t old_fences = fences; | 7303 flist_t old_fences = fences; |
7014 struct flist_tail_def _new_fences, *new_fences = &_new_fences; | 7304 struct flist_tail_def _new_fences, *new_fences = &_new_fences; |
7015 | 7305 |
7016 flist_tail_init (new_fences); | 7306 flist_tail_init (new_fences); |
7017 for (; fences; fences = FLIST_NEXT (fences)) | 7307 for (; fences; fences = FLIST_NEXT (fences)) |
7018 { | 7308 { |
7019 fence_t fence = FLIST_FENCE (fences); | 7309 fence_t fence = FLIST_FENCE (fences); |
7020 insn_t insn; | 7310 insn_t insn; |
7021 | 7311 |
7022 if (!FENCE_BNDS (fence)) | 7312 if (!FENCE_BNDS (fence)) |
7023 { | 7313 { |
7024 /* This fence doesn't have any successors. */ | 7314 /* This fence doesn't have any successors. */ |
7025 if (!FENCE_SCHEDULED_P (fence)) | 7315 if (!FENCE_SCHEDULED_P (fence)) |
7026 { | 7316 { |
7030 insn = FENCE_INSN (fence); | 7320 insn = FENCE_INSN (fence); |
7031 seqno = INSN_SEQNO (insn); | 7321 seqno = INSN_SEQNO (insn); |
7032 gcc_assert (seqno > 0 && seqno <= orig_max_seqno); | 7322 gcc_assert (seqno > 0 && seqno <= orig_max_seqno); |
7033 | 7323 |
7034 if (sched_verbose >= 1) | 7324 if (sched_verbose >= 1) |
7035 sel_print ("Fence %d[%d] has not changed\n", | 7325 sel_print ("Fence %d[%d] has not changed\n", |
7036 INSN_UID (insn), | 7326 INSN_UID (insn), |
7037 BLOCK_NUM (insn)); | 7327 BLOCK_NUM (insn)); |
7038 move_fence_to_fences (fences, new_fences); | 7328 move_fence_to_fences (fences, new_fences); |
7039 } | 7329 } |
7040 } | 7330 } |
7048 | 7338 |
7049 /* Update seqnos of insns given by PSCHEDULED_INSNS. MIN_SEQNO and MAX_SEQNO | 7339 /* Update seqnos of insns given by PSCHEDULED_INSNS. MIN_SEQNO and MAX_SEQNO |
7050 are the miminum and maximum seqnos of the group, HIGHEST_SEQNO_IN_USE is | 7340 are the miminum and maximum seqnos of the group, HIGHEST_SEQNO_IN_USE is |
7051 the highest seqno used in a region. Return the updated highest seqno. */ | 7341 the highest seqno used in a region. Return the updated highest seqno. */ |
7052 static int | 7342 static int |
7053 update_seqnos_and_stage (int min_seqno, int max_seqno, | 7343 update_seqnos_and_stage (int min_seqno, int max_seqno, |
7054 int highest_seqno_in_use, | 7344 int highest_seqno_in_use, |
7055 ilist_t *pscheduled_insns) | 7345 ilist_t *pscheduled_insns) |
7056 { | 7346 { |
7057 int new_hs; | 7347 int new_hs; |
7058 ilist_iterator ii; | 7348 ilist_iterator ii; |
7059 insn_t insn; | 7349 insn_t insn; |
7060 | 7350 |
7061 /* Actually, new_hs is the seqno of the instruction, that was | 7351 /* Actually, new_hs is the seqno of the instruction, that was |
7062 scheduled first (i.e. it is the first one in SCHEDULED_INSNS). */ | 7352 scheduled first (i.e. it is the first one in SCHEDULED_INSNS). */ |
7063 if (*pscheduled_insns) | 7353 if (*pscheduled_insns) |
7064 { | 7354 { |
7065 new_hs = (INSN_SEQNO (ILIST_INSN (*pscheduled_insns)) | 7355 new_hs = (INSN_SEQNO (ILIST_INSN (*pscheduled_insns)) |
7072 FOR_EACH_INSN (insn, ii, *pscheduled_insns) | 7362 FOR_EACH_INSN (insn, ii, *pscheduled_insns) |
7073 { | 7363 { |
7074 gcc_assert (INSN_SEQNO (insn) < 0); | 7364 gcc_assert (INSN_SEQNO (insn) < 0); |
7075 INSN_SEQNO (insn) += highest_seqno_in_use + max_seqno - min_seqno + 2; | 7365 INSN_SEQNO (insn) += highest_seqno_in_use + max_seqno - min_seqno + 2; |
7076 gcc_assert (INSN_SEQNO (insn) <= new_hs); | 7366 gcc_assert (INSN_SEQNO (insn) <= new_hs); |
7367 | |
7368 /* When not pipelining, purge unneeded insn info on the scheduled insns. | |
7369 For example, having reg_last array of INSN_DEPS_CONTEXT in memory may | |
7370 require > 1GB of memory e.g. on limit-fnargs.c. */ | |
7371 if (! pipelining_p) | |
7372 free_data_for_scheduled_insn (insn); | |
7077 } | 7373 } |
7078 | 7374 |
7079 ilist_clear (pscheduled_insns); | 7375 ilist_clear (pscheduled_insns); |
7080 global_level++; | 7376 global_level++; |
7081 | 7377 |
7082 return new_hs; | 7378 return new_hs; |
7083 } | 7379 } |
7084 | 7380 |
7085 /* The main driver for scheduling a region. This function is responsible | 7381 /* The main driver for scheduling a region. This function is responsible |
7086 for correct propagation of fences (i.e. scheduling points) and creating | 7382 for correct propagation of fences (i.e. scheduling points) and creating |
7087 a group of parallel insns at each of them. It also supports | 7383 a group of parallel insns at each of them. It also supports |
7088 pipelining. ORIG_MAX_SEQNO is the maximal seqno before this pass | 7384 pipelining. ORIG_MAX_SEQNO is the maximal seqno before this pass |
7089 of scheduling. */ | 7385 of scheduling. */ |
7090 static void | 7386 static void |
7091 sel_sched_region_2 (int orig_max_seqno) | 7387 sel_sched_region_2 (int orig_max_seqno) |
7092 { | 7388 { |
7119 stat_insns_needed_bookkeeping, | 7415 stat_insns_needed_bookkeeping, |
7120 stat_renamed_scheduled, | 7416 stat_renamed_scheduled, |
7121 stat_substitutions_total); | 7417 stat_substitutions_total); |
7122 } | 7418 } |
7123 | 7419 |
7124 /* Schedule a region. When pipelining, search for possibly never scheduled | 7420 /* Schedule a region. When pipelining, search for possibly never scheduled |
7125 bookkeeping code and schedule it. Reschedule pipelined code without | 7421 bookkeeping code and schedule it. Reschedule pipelined code without |
7126 pipelining after. */ | 7422 pipelining after. */ |
7127 static void | 7423 static void |
7128 sel_sched_region_1 (void) | 7424 sel_sched_region_1 (void) |
7129 { | 7425 { |
7130 int number_of_insns; | 7426 int number_of_insns; |
7131 int orig_max_seqno; | 7427 int orig_max_seqno; |
7132 | 7428 |
7133 /* Remove empty blocks that might be in the region from the beginning. | 7429 /* Remove empty blocks that might be in the region from the beginning. |
7134 We need to do save sched_max_luid before that, as it actually shows | 7430 We need to do save sched_max_luid before that, as it actually shows |
7135 the number of insns in the region, and purge_empty_blocks can | 7431 the number of insns in the region, and purge_empty_blocks can |
7136 alter it. */ | 7432 alter it. */ |
7137 number_of_insns = sched_max_luid - 1; | 7433 number_of_insns = sched_max_luid - 1; |
7138 purge_empty_blocks (); | 7434 purge_empty_blocks (); |
7139 | 7435 |
7140 orig_max_seqno = init_seqno (number_of_insns, NULL, NULL); | 7436 orig_max_seqno = init_seqno (number_of_insns, NULL, NULL); |
7197 | 7493 |
7198 for (i = 0; i < current_nr_blocks; i++) | 7494 for (i = 0; i < current_nr_blocks; i++) |
7199 { | 7495 { |
7200 bb = EBB_FIRST_BB (i); | 7496 bb = EBB_FIRST_BB (i); |
7201 | 7497 |
7202 /* While pipelining outer loops, skip bundling for loop | 7498 /* While pipelining outer loops, skip bundling for loop |
7203 preheaders. Those will be rescheduled in the outer | 7499 preheaders. Those will be rescheduled in the outer |
7204 loop. */ | 7500 loop. */ |
7205 if (sel_is_loop_preheader_p (bb)) | 7501 if (sel_is_loop_preheader_p (bb)) |
7206 { | 7502 { |
7207 clear_outdated_rtx_info (bb); | 7503 clear_outdated_rtx_info (bb); |
7208 continue; | 7504 continue; |
7209 } | 7505 } |
7210 | 7506 |
7211 if (bitmap_bit_p (blocks_to_reschedule, bb->index)) | 7507 if (bitmap_bit_p (blocks_to_reschedule, bb->index)) |
7212 { | 7508 { |
7213 flist_tail_init (new_fences); | 7509 flist_tail_init (new_fences); |
7214 | 7510 |
7215 orig_max_seqno = init_seqno (0, blocks_to_reschedule, bb); | 7511 orig_max_seqno = init_seqno (0, blocks_to_reschedule, bb); |
7216 | 7512 |
7217 /* Mark BB as head of the new ebb. */ | 7513 /* Mark BB as head of the new ebb. */ |
7218 bitmap_set_bit (forced_ebb_heads, bb->index); | 7514 bitmap_set_bit (forced_ebb_heads, bb->index); |
7219 | 7515 |
7220 bitmap_clear_bit (blocks_to_reschedule, bb->index); | 7516 bitmap_clear_bit (blocks_to_reschedule, bb->index); |
7221 | 7517 |
7222 gcc_assert (fences == NULL); | 7518 gcc_assert (fences == NULL); |
7223 | 7519 |
7224 init_fences (bb_note (bb)); | 7520 init_fences (bb_note (bb)); |
7225 | 7521 |
7226 sel_sched_region_2 (orig_max_seqno); | 7522 sel_sched_region_2 (orig_max_seqno); |
7227 | 7523 |
7228 do_p = true; | 7524 do_p = true; |
7229 break; | 7525 break; |
7230 } | 7526 } |
7231 } | 7527 } |
7232 } | 7528 } |
7252 if (schedule_p) | 7548 if (schedule_p) |
7253 sel_sched_region_1 (); | 7549 sel_sched_region_1 (); |
7254 else | 7550 else |
7255 /* Force initialization of INSN_SCHED_CYCLEs for correct bundling. */ | 7551 /* Force initialization of INSN_SCHED_CYCLEs for correct bundling. */ |
7256 reset_sched_cycles_p = true; | 7552 reset_sched_cycles_p = true; |
7257 | 7553 |
7258 sel_region_finish (reset_sched_cycles_p); | 7554 sel_region_finish (reset_sched_cycles_p); |
7259 } | 7555 } |
7260 | 7556 |
7261 /* Perform global init for the scheduler. */ | 7557 /* Perform global init for the scheduler. */ |
7262 static void | 7558 static void |
7273 sched_init (); | 7569 sched_init (); |
7274 | 7570 |
7275 sched_init_bbs (); | 7571 sched_init_bbs (); |
7276 /* Reset AFTER_RECOVERY if it has been set by the 1st scheduler pass. */ | 7572 /* Reset AFTER_RECOVERY if it has been set by the 1st scheduler pass. */ |
7277 after_recovery = 0; | 7573 after_recovery = 0; |
7278 can_issue_more = issue_rate; | 7574 can_issue_more = issue_rate; |
7279 | 7575 |
7280 sched_extend_target (); | 7576 sched_extend_target (); |
7281 sched_deps_init (true); | 7577 sched_deps_init (true); |
7282 setup_nop_and_exit_insns (); | 7578 setup_nop_and_exit_insns (); |
7283 sel_extend_global_bb_info (); | 7579 sel_extend_global_bb_info (); |