How do I write test cases for interactive commands?

[Yoshihiro Ota]

Hi,

I'm interested in writing some test cases for lldb and clang.

How can I write interactive command in/out in Kyua?
Does anyone have some examples?

I'm also interested in writing test programs to compile again clang to find compiler/linker bugs.
Where do we use for temporary files and how do we clean up?

Thanks,
Hiro

[Alan Somers]

Post by Yoshihiro Ota
Hi,
I'm interested in writing some test cases for lldb and clang.
How can I write interactive command in/out in Kyua?
Does anyone have some examples?
I'm also interested in writing test programs to compile again clang to
find compiler/linker bugs.
Where do we use for temporary files and how do we clean up?
Thanks,
Hiro

Interactive commands like those that print a prompt and wait for user
input? Use lang/expect from ports. As for temporary files, Kyua creates a
fresh temporary directory for every test. So you can write files to $PWD,
and Kyua will automatically clean it up.
-Alan

[Gleb Popov]

Post by Yoshihiro Ota
Hi,
I'm interested in writing some test cases for lldb and clang.
How can I write interactive command in/out in Kyua?
Does anyone have some examples?
I'm also interested in writing test programs to compile again clang to
find compiler/linker bugs.
Where do we use for temporary files and how do we clean up?
Thanks,
Hiro
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers

Take a look at LLVM testing infrastructure and specifically the llvm-lit
tool.

[Enji Cooper]




Post by Gleb Popov
Take a look at LLVM testing infrastructure and specifically the llvm-lit
tool.

Does their test infrastructure leverage GoogleTest? If so, then the bulk majority of the work would just be integrating it into Kyua.
Cheers,
-Enji

[Poul-Henning Kamp]

--------

Post by Gleb Popov

Post by Gleb Popov
Take a look at LLVM testing infrastructure and specifically the

llvm-lit

Post by Gleb Popov
tool.

Does their test infrastructure leverage GoogleTest? If so, then the bulk
majority of the work would just be integrating it into Kyua.
Cheers,
-Enji

Given that pretty much any software to come out of Google is usually
pretty overweight, in particular on dependencies:

In the V(arnish)Test tool we built a facility for interactive and
curses-based tests using teken, (which we got from FreeBSD):

https://github.com/vtest/VTest

The code for that sub-part lives in:

https://github.com/vtest/VTest/blob/master/src/vtc_process.c

Here is a typical test-case using it:

https://github.com/varnishcache/varnish-cache/blob/master/bin/varnishtest/tests/u00008.vtc

The verbose output from that test can be seen here:

http://phk.freebsd.dk/misc/u00008.txt

V(arnish)Test is a modular program, and if you leave out all the
HTTP related modules, and substitute 'sbuf' for 'vsb', and "TAILQ" for "VTAILQ"
it would fit nicely into FreeBSD.

[Alexander Richardson]

Post by Enji Cooper
…

Post by Gleb Popov
Take a look at LLVM testing infrastructure and specifically the llvm-lit
tool.

Does their test infrastructure leverage GoogleTest? If so, then the bulk majority of the work would just be integrating it into Kyua.
Cheers,
-Enji

If you are interested in writing tests for llvm/clang/lldb those
should be submitted upstream and not be part of the FreeBSD source
tree. Therefore, Kyua is almost certainly not the right solution.
https://www.llvm.org/docs/TestingGuide.html explains how to write
tests for LLVM, however, it is quite stale. I would suggest looking at
existing tests inside LLVM and adjusting those to match what you want
to test.

The FreeBSD src contrib/ import does not include the extensive LLVM
testsuite, so you could start by looking at
https://github.com/llvm/llvm-project/tree/main/clang/test for clang
and https://github.com/llvm/llvm-project/tree/main/lldb/test for LLDB.
I have never looked at the LLDB tests, but as far as I know there are
tests that use python to drive the interaction.

Alex

[Enji Cooper]

Post by Alexander Richardson

Post by Enji Cooper



Post by Gleb Popov
Take a look at LLVM testing infrastructure and specifically the llvm-lit
tool.

Does their test infrastructure leverage GoogleTest? If so, then the bulk majority of the work would just be integrating it into Kyua.
Cheers,
-Enji

If you are interested in writing tests for llvm/clang/lldb those
should be submitted upstream and not be part of the FreeBSD source
tree. Therefore, Kyua is almost certainly not the right solution.
https://www.llvm.org/docs/TestingGuide.html explains how to write
tests for LLVM, however, it is quite stale. I would suggest looking at
existing tests inside LLVM and adjusting those to match what you want
to test.
The FreeBSD src contrib/ import does not include the extensive LLVM
testsuite, so you could start by looking at
https://github.com/llvm/llvm-project/tree/main/clang/test for clang
and https://github.com/llvm/llvm-project/tree/main/lldb/test for LLDB.
I have never looked at the LLDB tests, but as far as I know there are
tests that use python to drive the interaction.

Hi Alex,
My suggestion was more that kyua should be used to run/aggregate the test information, not be the actual test framework/infrastructure/payload.
I really appreciate the references :).
Cheers!
-Enji