The QAPI_TEST_UPDATE env var can be set when running the QAPI
schema tests to regenerate the reference output. For consistent
naming with the tracetool test, change the env var name to
QEMU_TEST_REGENERATE.
The test is modified to provide a hint about use of the new
env var and it is also added to the developer documentation.document its usage.
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20250916081638.764020-8-berrange@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
When reviewing tracetool patches it is often very unclear what the
expected output will be for the generated backends. Compounding
this is that a default build will only enable the 'log' trace
backend, so developers won't see generated code for other backends
without making a special effort. Some backends are also platform
specific, so can't be enabled in QEMU builds, even though tracetool
could generate the code.
To address this, introduce a test suite for tracetool which is
conceptually similar to the qapi-schema test. It is a simple
python program that runs tracetool and compares the actual output
to historical reference output kept in git. The test directly
emits TAP format logs for ease of integration with meson.
This can be run with
make check-tracetool
to make it easier for developers changing generated output, the
sample expected content can be auto-recreated
QEMU_TEST_REGENERATE=1 make check-tracetool
and the changes reviewed and added to the commit. This will also
assist reviewers interpreting the change.
Developers are reminded of this in the test output on failure:
$ make check-tracetool
1/6 qemu:tracetool / dtrace OK 0.14s
2/6 qemu:tracetool / ftrace FAIL 0.06s exit status 1
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
1..2
ok 1 - ftrace.c
#
not ok 1 - ftrace.h (set QEMU_TEST_REGENERATE=1 to recreate reference output if tracetool generator was intentionally changed)
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
3/6 qemu:tracetool / log OK 0.06s
4/6 qemu:tracetool / simple OK 0.06s
5/6 qemu:tracetool / syslog OK 0.06s
6/6 qemu:tracetool / ust OK 0.11s
Summary of Failures:
2/6 qemu:tracetool / ftrace FAIL 0.06s exit status 1
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20250916081638.764020-6-berrange@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Sphinx supports the :kbd: role for notating keyboard input. They get
formatted as <kbd> HTML elements in the readthedocs theme we currently
use for Sphinx.
Besides the better visual formatting, it also helps with accessibility
as screen readers can announce the semantics of the <kbd> element to the
user.
Signed-off-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Message-ID: <20250709-docs_rst_improvements-v2-1-cb5096ad0022@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250710104531.3099313-7-alex.bennee@linaro.org>
Now that all Avocado tests have been converted to or been replaced by
other functional tests, we can delete the remainders of the Avocado
tests from the QEMU source tree.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-16-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This file was meant for defining the vocabulary for our testing
efforts, but it did not age well: First, the definitions are not
only about the CI part, but also about testing in general, so most
of the information should rather reside in main.rst instead.
Second, some vocabulary has never been really adopted by the QEMU
project, for example we never really use the word "system testing"
since "system" rather means the system emulator binaries in the
QEMU project (and we also don't do any testing with other components
like libvirt and virt-managers here). It also defines that the qtests
are the "functional" tests in QEMU, which is not really up to date
anymore after the "tests/functional" framework has been introduced
a couple of months ago (FWIW, the qtests could rather be seen as a
mix between unit testing and functional testing).
To solve this problem, move the useful parts of this file into
main.rst and directly into ci.rst, and drop the ones (like "system
testing") that we don't really need anymore.
Message-ID: <20250314085959.1585568-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Present the various parts of QEMU and organization of codebase.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Message-Id: <20241209183104.365796-6-pierrick.bouvier@linaro.org>
[AJB: tweak commit summary, update MAINTAINERS]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250116160306.1709518-36-alex.bennee@linaro.org>
Mention it's now possible to build with gcc, instead of clang, and
explain how to build a sanitized glib version.
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20240910174013.1433331-4-pierrick.bouvier@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20241023113406.1284676-6-alex.bennee@linaro.org>
I guess the same change came in via two patch series. Remove the
repetition.
Fixes: 2a851fca9f (docs/devel: remind developers to run CI container pipeline when updating images)
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20240910173900.4154726-5-alex.bennee@linaro.org>
Document the new functional testing framework. The text is originally
based on the Avocado documentation, but heavily modified to match the
new framework.
Message-ID: <20240830133841.142644-45-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The main testing documentation file got very overloaded already.
Thus let's split the Avocado information into a separate file.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20240830133841.142644-43-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Building and testing are two separate topics, so let's split the testing
into a separate category and move the related files into a separate folder.
Message-ID: <20240830133841.142644-42-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
