Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 189 lines (130 sloc) 5.501 kb
6db42a0 @coke #39217 - copyright cleanup. (docs/)
coke authored
1 # Copyright (C) 2001-2006, The Perl Foundation.
9d8cce4 a bit more tidying up
Michael Scott authored
2 # $Id$
9b4e5d0 Fixups to make parrotcode.org's autogenerated pages happier
Dan Sugalski authored
3
9d8cce4 a bit more tidying up
Michael Scott authored
4 =head1 NAME
5
6 docs/tests.pod - Testing Parrot
8c4a44c @quidity This is a basic quide to writing tests with assembler.
quidity authored
7
e48bb2e @chromatic Patch #37887 from Alberto Simoes (ambs@cpan.org), plus spelling correcti...
chromatic authored
8 =head1 A basic guide to writing and running tests for Parrot
8c4a44c @quidity This is a basic quide to writing tests with assembler.
quidity authored
9
aab5022 @bschmalhofer docs: Going over docs/tests.pod.
bschmalhofer authored
10 This is quick and dirty pointer to how the Parrot test suite is executed and
11 to how new tests for Parrot should be written.
12 The testing system is liable to change in the future, but tests written
13 following the guidelines below should be easy to port into a new test suite.
8c4a44c @quidity This is a basic quide to writing tests with assembler.
quidity authored
14
e48bb2e @chromatic Patch #37887 from Alberto Simoes (ambs@cpan.org), plus spelling correcti...
chromatic authored
15 =head1 How to test parrot
16
17 The easy way to test parrot is running C<make test>. If you have
18 updated your code recently and tests began failing, go for a C<make
19 realclean> and recompile parrot before complaining.
20
21 If your architecture supports JIT, you can test parrot JIT engine using C<make
22 testj>. It works just like C<make test>, but uses the JIT engine when possible.
23
c8ca9f0 @bschmalhofer Mention the make targets 'languages-test' and 'languages-smoke'
bschmalhofer authored
24 C<make languages-test> runs the test suite for most language implementations
25 in the languages directory.
26
e48bb2e @chromatic Patch #37887 from Alberto Simoes (ambs@cpan.org), plus spelling correcti...
chromatic authored
27 =head2 Submitting smoke test results
28
29 Parrot has a status page with smoke test results
38d4ce0 @coke Fix the URL to the smoke server.
coke authored
30 L<http://smoke.parrotcode.org/smoke/>. You can supply new tests
c138099 @bschmalhofer Move Parrot_Docs.t from t/docs to t/perl, as
bschmalhofer authored
31 results by just running C<make smoke>. It will run the same tests as
32 C<make test> would, but will additionally create a HTML table with the test
e48bb2e @chromatic Patch #37887 from Alberto Simoes (ambs@cpan.org), plus spelling correcti...
chromatic authored
33 results. At the end, it will try to upload the test results to the
34 smoke server.
35
36 It is also possible to run a smoke test on JIT. For that, try running
8710563 @jhoblitt add smokej jit smoke test target
jhoblitt authored
37 C<make smokej>.
e48bb2e @chromatic Patch #37887 from Alberto Simoes (ambs@cpan.org), plus spelling correcti...
chromatic authored
38
c8ca9f0 @bschmalhofer Mention the make targets 'languages-test' and 'languages-smoke'
bschmalhofer authored
39 C<make languages-smoke> does smoke testing for most language implementations
40 in the languages directory.
41
aab5022 @bschmalhofer docs: Going over docs/tests.pod.
bschmalhofer authored
42 =head1 Location of the test files
43
44 The parrot test files, the F<*.t> files, can be found in the F<t> directory.
45 A quick overview over the subdirs in F<t> can be found in F<t/README>.
1acea25 update info on pir tests in main; don't generate packfile-perl.pod
Leopold Toetsch authored
46
aab5022 @bschmalhofer docs: Going over docs/tests.pod.
bschmalhofer authored
47 The language implementations usually have their test files in F<languages/*/t>.
48
49 New tests should be added to an existing F<*.t> file.
50 If a previously untested feature is tested,
a754c8c @bschmalhofer Mention 'pir_output_is', 'language_output_is' and TODO in docs/tests.pod...
bschmalhofer authored
51 it might also make sense to create a new F<*.t> file.
8c4a44c @quidity This is a basic quide to writing tests with assembler.
quidity authored
52
aab5022 @bschmalhofer docs: Going over docs/tests.pod.
bschmalhofer authored
53 =head1 How to write a test
54
05a7e7b @bschmalhofer This patch is huge because of all the whitespace reformatting. This was
bschmalhofer authored
55 The testing framework needs to know how many tests it should expect. So the
c138099 @bschmalhofer Move Parrot_Docs.t from t/docs to t/perl, as
bschmalhofer authored
56 number of planned tests needs to be incremented when adding a new test. This
05a7e7b @bschmalhofer This patch is huge because of all the whitespace reformatting. This was
bschmalhofer authored
57 is done near the top of a test file, in a line that looks like:
553e254 Patch from Simon Glover <scog@amnh.org>:
Nicholas Clark authored
58
59 use Parrot::Test tests => 8;
60
a754c8c @bschmalhofer Mention 'pir_output_is', 'language_output_is' and TODO in docs/tests.pod...
bschmalhofer authored
61 =head2 Parrot Assembler
c543d6f @boemmels Add C test info to tests.pod (Courtesy of Michael Scott <michael_scott@m...
boemmels authored
62
e48bb2e @chromatic Patch #37887 from Alberto Simoes (ambs@cpan.org), plus spelling correcti...
chromatic authored
63 PASM tests are mostly used for testing ops. Appropriate test files for basic
05a7e7b @bschmalhofer This patch is huge because of all the whitespace reformatting. This was
bschmalhofer authored
64 ops are F<t/op/*.t>. Perl Magic Cookies are tested in F<t/pmc/*.t>. Add the
65 new test like this:
c543d6f @boemmels Add C test info to tests.pod (Courtesy of Michael Scott <michael_scott@m...
boemmels authored
66
b38c8aa @bschmalhofer Some fiddling with test suite:
bschmalhofer authored
67 pasm_output_is(<<'CODE', <<'OUTPUT', "name for test");
c543d6f @boemmels Add C test info to tests.pod (Courtesy of Michael Scott <michael_scott@m...
boemmels authored
68 *** a big chunk of assembler, eg:
69 print 1
70 print "\n" # you can even comment it if it's obscure
71 end # don't forget this...!
72 CODE
73 *** what you expect the output of the chunk to be, eg.
74 1
75 OUTPUT
76
a754c8c @bschmalhofer Mention 'pir_output_is', 'language_output_is' and TODO in docs/tests.pod...
bschmalhofer authored
77 =head2 Parrot Intermediate Representation
78
67c929f @ambs While discussion continues on p2, pod files should be at most
ambs authored
79 Writing tests in B<PIR> is more convenient. This is done with
80 C<pir_output_is> and friends.
a754c8c @bschmalhofer Mention 'pir_output_is', 'language_output_is' and TODO in docs/tests.pod...
bschmalhofer authored
81
82 pir_output_is(<<'CODE',<<'OUT','nothing useful');
eb9e7dd @particle #37520: [TODO] rename library files from .imc to .pir
particle authored
83 .include 'library/config.pir'
a754c8c @bschmalhofer Mention 'pir_output_is', 'language_output_is' and TODO in docs/tests.pod...
bschmalhofer authored
84
3721ccc [PATCH] @directive -> :directive - part 2 src, docs, tools
Leopold Toetsch authored
85 .sub main :main
a754c8c @bschmalhofer Mention 'pir_output_is', 'language_output_is' and TODO in docs/tests.pod...
bschmalhofer authored
86 print "hi\n"
87 .end
88 CODE
89 hi
90 OUT
1acea25 update info on pir tests in main; don't generate packfile-perl.pod
Leopold Toetsch authored
91
92 =head2 C source tests
c543d6f @boemmels Add C test info to tests.pod (Courtesy of Michael Scott <michael_scott@m...
boemmels authored
93
ea0630c @jhoblitt reformat all Pod files under docs with podtidy (modified to also remove ...
jhoblitt authored
94 C source tests are usually located in F<t/src/*.t>. A simple test looks like:
280b750 @jhoblitt fix verbatim block in docs/tests.pod
jhoblitt authored
95
c543d6f @boemmels Add C test info to tests.pod (Courtesy of Michael Scott <michael_scott@m...
boemmels authored
96 c_output_is(<<'CODE', <<'OUTPUT', "name for test");
97 #include <stdio.h>
98 #include "parrot/parrot.h"
99 #include "parrot/embed.h"
58c0aa8 update info on src tests
Leopold Toetsch authored
100
101 static opcode_t *the_test(Parrot_Interp, opcode_t *, opcode_t *);
102
c543d6f @boemmels Add C test info to tests.pod (Courtesy of Michael Scott <michael_scott@m...
boemmels authored
103 int main(int argc, char* argv[]) {
58c0aa8 update info on src tests
Leopold Toetsch authored
104 Parrot_Interp interpreter;
01f4e37 Updated doc and some code to reflect new Parrot_new() interface
Jens Rieks authored
105 interpreter = Parrot_new(NULL);
c543d6f @boemmels Add C test info to tests.pod (Courtesy of Michael Scott <michael_scott@m...
boemmels authored
106
58c0aa8 update info on src tests
Leopold Toetsch authored
107 if (!interpreter)
a25ee79 @bschmalhofer Replace a couple of calls to
bschmalhofer authored
108 return 1;
58c0aa8 update info on src tests
Leopold Toetsch authored
109
110 Parrot_run_native(interpreter, the_test);
c543d6f @boemmels Add C test info to tests.pod (Courtesy of Michael Scott <michael_scott@m...
boemmels authored
111 printf("done\n");
9d8cce4 a bit more tidying up
Michael Scott authored
112 fflush(stdout);
c543d6f @boemmels Add C test info to tests.pod (Courtesy of Michael Scott <michael_scott@m...
boemmels authored
113 return 0;
114 }
58c0aa8 update info on src tests
Leopold Toetsch authored
115
116 static opcode_t*
117 the_test(Parrot_Interp interpreter,
9d8cce4 a bit more tidying up
Michael Scott authored
118 opcode_t *cur_op, opcode_t *start)
58c0aa8 update info on src tests
Leopold Toetsch authored
119 {
120 /* Your test goes here. */
121
122 return NULL; /* always return NULL */
123 }
c543d6f @boemmels Add C test info to tests.pod (Courtesy of Michael Scott <michael_scott@m...
boemmels authored
124 CODE
125 # Anything that might be output prior to "done".
126 done
127 OUTPUT
128
a754c8c @bschmalhofer Mention 'pir_output_is', 'language_output_is' and TODO in docs/tests.pod...
bschmalhofer authored
129 Note that it's always a good idea to output "done" to confirm that the compiled
05a7e7b @bschmalhofer This patch is huge because of all the whitespace reformatting. This was
bschmalhofer authored
130 code executed completely. When mixing C<printf> and C<PIO_printf> always append
131 a C<fflush(stdout);> after the former.
8c4a44c @quidity This is a basic quide to writing tests with assembler.
quidity authored
132
b9beb9d @bschmalhofer Explain what t/perl is for.
bschmalhofer authored
133 =head2 Test Perl5 helpers
134
135 Perl5 unit tests are in F<t/perl/*.t>.
136
a754c8c @bschmalhofer Mention 'pir_output_is', 'language_output_is' and TODO in docs/tests.pod...
bschmalhofer authored
137 =head2 Testing language implementations
138
c138099 @bschmalhofer Move Parrot_Docs.t from t/docs to t/perl, as
bschmalhofer authored
139 Language implementations are usually tested with
140 C<language_output_is> and friends..
a754c8c @bschmalhofer Mention 'pir_output_is', 'language_output_is' and TODO in docs/tests.pod...
bschmalhofer authored
141
1acea25 update info on pir tests in main; don't generate packfile-perl.pod
Leopold Toetsch authored
142 =head1 Ideal tests:
8c4a44c @quidity This is a basic quide to writing tests with assembler.
quidity authored
143
144 =over 4
145
afc981d @chromatic Fixed some POD formatting errors.
chromatic authored
146 =item *
8c4a44c @quidity This is a basic quide to writing tests with assembler.
quidity authored
147
05a7e7b @bschmalhofer This patch is huge because of all the whitespace reformatting. This was
bschmalhofer authored
148 Probe the boundaries (including edge cases, errors thrown etc.) of whatever
149 code they're testing. These should include potentially out of band input
150 unless we decide that compilers should check for this themselves.
8c4a44c @quidity This is a basic quide to writing tests with assembler.
quidity authored
151
afc981d @chromatic Fixed some POD formatting errors.
chromatic authored
152 =item *
8c4a44c @quidity This is a basic quide to writing tests with assembler.
quidity authored
153
05a7e7b @bschmalhofer This patch is huge because of all the whitespace reformatting. This was
bschmalhofer authored
154 Are small and self contained, so that if the tested feature breaks we can
155 identify where and why quickly.
8c4a44c @quidity This is a basic quide to writing tests with assembler.
quidity authored
156
afc981d @chromatic Fixed some POD formatting errors.
chromatic authored
157 =item *
8c4a44c @quidity This is a basic quide to writing tests with assembler.
quidity authored
158
19b32a9 Minor edits
Simon Glover authored
159 Are valid. Essentially, they should conform to the additional documentation
05a7e7b @bschmalhofer This patch is huge because of all the whitespace reformatting. This was
bschmalhofer authored
160 that accompanies the feature (if any). [If there isn't any documentation, then
161 feel free to add some and/or complain to the mailing list].
8c4a44c @quidity This is a basic quide to writing tests with assembler.
quidity authored
162
afc981d @chromatic Fixed some POD formatting errors.
chromatic authored
163 =item *
8c4a44c @quidity This is a basic quide to writing tests with assembler.
quidity authored
164
165 Are a chunk of assembler and a chunk of expected output.
166
9b4e5d0 Fixups to make parrotcode.org's autogenerated pages happier
Dan Sugalski authored
167 =back
a754c8c @bschmalhofer Mention 'pir_output_is', 'language_output_is' and TODO in docs/tests.pod...
bschmalhofer authored
168
169 =head1 TODO tests
170
05a7e7b @bschmalhofer This patch is huge because of all the whitespace reformatting. This was
bschmalhofer authored
171 In test driven development, tests are implemented first. So the tests are
172 initially expected to fail. This can be expressed by marking the tests as
aab5022 @bschmalhofer docs: Going over docs/tests.pod.
bschmalhofer authored
173 TODO. See L<Test::More> on how to do that.
174
175 =head1 SKIP tests
176
177 TODO test actually executed, so that unexpected success can be detected.
178 In the case of missing requirements and in the case of serious breakdowns
179 the execution of tests can be skipped.
180 See L<Test::More> on how to do that.
a754c8c @bschmalhofer Mention 'pir_output_is', 'language_output_is' and TODO in docs/tests.pod...
bschmalhofer authored
181
182 =head1 SEE ALSO
183
e48bb2e @chromatic Patch #37887 from Alberto Simoes (ambs@cpan.org), plus spelling correcti...
chromatic authored
184 L<http://qa.perl.org/>
b9beb9d @bschmalhofer Explain what t/perl is for.
bschmalhofer authored
185 F<t/TESTS.STATUS.pod>
c138099 @bschmalhofer Move Parrot_Docs.t from t/docs to t/perl, as
bschmalhofer authored
186 F<t/README>
a754c8c @bschmalhofer Mention 'pir_output_is', 'language_output_is' and TODO in docs/tests.pod...
bschmalhofer authored
187
188 =cut
Something went wrong with that request. Please try again.