|
111
|
1 This file contains a list of notes for those writing testcases and those
|
|
|
2 writing expect scripts. At present, they're in random order.
|
|
|
3
|
|
|
4 Verbosity Levels
|
|
|
5
|
|
|
6 - each level adds verbosity so level 2 prints all level 1 and level 2 stuff.
|
|
|
7
|
|
|
8 1) Print a one-liner indicating the testcase being run (and maybe special
|
|
|
9 compiler options).
|
|
|
10
|
|
|
11 2) Print compiler and program invocation including arguments and their output.
|
|
|
12 Proc's gcc_load and gcc_start handle the latter two.
|
|
|
13
|
|
|
14 3) Print detailed testcase analysis like "Looking for pattern ...", etc.
|
|
|
15
|
|
|
16 4) Maximum verbosity. Print anything else of interest.
|
|
|
17
|
|
|
18 send_log conventions
|
|
|
19
|
|
|
20 Various messages are stored in gcc.log by the testing framework and we
|
|
|
21 try to augment them with some of our own. The format of the framework
|
|
|
22 messages are:
|
|
|
23
|
|
|
24 PASS: blah blah ...
|
|
|
25 FAIL: blah blah ...
|
|
|
26
|
|
|
27 so we use
|
|
|
28
|
|
|
29 XXXX: blah blah ...
|
|
|
30
|
|
|
31 Current messages are:
|
|
|
32
|
|
|
33 EXEC: program being executed (so compiler path and args are recorded)
|
|
|
34 STAT: intermediate pass/fail statistics
|
|
|
35
|
|
|
36 Test scripts must ignore the compiler messages "path prefix never used"
|
|
|
37 and "linker input file unused". Don't let their appearance cause a testcase
|
|
|
38 to fail. See lib/dg.exp for the exact regsub to use.
|
|
|
39
|
|
|
40 If you're unclear about which directory a testcase should be installed in,
|
|
|
41 ask gcc-local.
|
|
|
42
|
|
|
43 Have the text of a fail message be the same as that for pass.
|
|
|
44 IE: have
|
|
|
45
|
|
|
46 if ...success...
|
|
|
47 pass "pr 1234"
|
|
|
48 else
|
|
|
49 fail "pr 1234"
|
|
|
50
|
|
|
51 not
|
|
|
52
|
|
|
53 if ...success...
|
|
|
54 pass "pr 1234 passed"
|
|
|
55 else
|
|
|
56 fail "pr 1234 failed"
|
|
|
57
|
|
|
58
|
|
|
59 This lets test-tool (which drives the nightly tests) do a better job
|
|
|
60 at tracking which tests have digressed or been fixed.
|
|
|
61
|
|
|
62 DO NOT PUT NON-PORTABLE TESTCASES IN gcc.c-torture.
|
|
|
63
|
|
|
64 ANY TARGET SPECIFIC TESTCASE MUST HAVE APPROPRIATE CODE TO PREVENT IT FROM
|
|
|
65 CAUSING A `FAILURE' ON UNSUPPORTED PLATFORMS.
|
|
|
66
|
|
|
67 The "torture" tests are meant to be generic tests that can run on any
|
|
|
68 target. So you have to be careful about endianness, assumptions about
|
|
|
69 sizes of datatypes, etc etc.
|
|
|
70
|
|
|
71 For tests that merely need to compile, put them in the "compile" directory.
|
|
|
72
|
|
|
73 For tests which should give an error, put them in the "noncompile" directory
|
|
|
74 and update noncompile.exp appropriately (see examples in noncompile.exp).
|
|
|
75
|
|
|
76 For IEEE FP specific tests, put them in execute/ieee.
|
|
|
77
|
|
|
78 For execution tests, put them in execute.
|
|
|
79
|
|
|
80 If a test does not fit into the torture framework, use the dg framework.
|
|
|
81
|
|
|
82
|
|
131
|
83 Copyright (C) 1997-2018 Free Software Foundation, Inc.
|
|
111
|
84
|
|
|
85 Copying and distribution of this file, with or without modification,
|
|
|
86 are permitted in any medium without royalty provided the copyright
|
|
|
87 notice and this notice are preserved.
|
