============= Release Notes ============= This page briefly summarizes the KUnit changes in each Linux release. For more specific documentation on KUnit features, see the in-kernel documentation: https://kernel.org/doc/html/latest/dev-tools/kunit. .. _v5.5: 5.5 === - KUnit introduced, including all the basic functionality. - ``kunitconfig`` was renamed to ``.kunitconfig`` later in 5.5 (`commit `__) .. _v5.6: 5.6 === - KUnit tests can be built as modules, `docs `__ .. _v5.7: 5.7 === - kunit.py switched to using ``.kunit/`` as the default build dir (and ``.kunitconfig`` => ``.kunit/.kunitconfig``). - ``CONFIG_KUNIT_DEBUGFS`` to expose per-suite test results in ``/sys/kernel/debug/kunit//results``, `docs `__ .. _v5.8: 5.8 === - ``kunit.py`` now supports separate ``config/build/exec/parse`` commands in addition to ``run``. - You can use these if you don't want to let ``kunit.py run`` do everything for you, e.g. if you want to run manually but parse the results using ``kunit.py``. - ``CONFIG_KUNIT_ALL_TESTS`` introduced - This is just a convention. New tests should use it like so: .. code-block:: none config FOO_KUNIT_TEST tristate "KUnit test for foo" if !KUNIT_ALL_TESTS depends on KUNIT default KUNIT_ALL_TESTS help This builds unit tests for foo. See `the upstream documentation `__ for the authoritative source on how to write Kconfig entries. .. _v5.9: 5.9 === - Named resources: tests can store and retrieve data via string keys. - This is a power feature intended to be a cleaner alternative to using global variables in tests. - See `test_kasan.c `__ for an example that uses ``"kasan_data"``. .. _v5.10: 5.10 ==== - KASAN violations cause the current KUnit test case to be marked FAILED. - Note: KASAN doesn't currently work on UML. .. _v5.11: 5.11 ==== - Parameterized testing: see the `docs `__. .. _v5.12: 5.12 ==== - You can pass in a path to a kunitconfig file instead of using ``.kunit/.kunitconfig``, e.g. .. code-block:: shell $ ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4/.kunitconfig - You can tell ``kunit.py`` to run a subset of built suites by passing in a glob argument, e.g. .. code-block:: shell # Only run "list" suites $ ./tools/testing/kunit/kunit.py run '*list*' .. _v5.13: 5.13 ==== - Can call ``kunit_fail_current_test()`` from anywhere to fail the current KUnit test (e.g. if an invariant is broken, a UBSAN violation is detected, etc.) - You can pass a directory ``--kunitconfig`` and it'll implicitly append ``.kunitconfig``, e.g. ``kunit.py run --kunitconfig=lib/kunit``. .. _v5.14: 5.14 ==== - ``kunit.py`` can run tests on non-UML architectures (using QEMU), `docs `__ - It's now possible to generate code coverage reports when using UML, `docs `__ - Tests can call ``kunit_skip()`` to bail out and mark a test as "SKIPPED" instead of "PASSED" or "FAILED" .. _v5.15: 5.15 ==== - UBSAN violations cause the current KUnit test case to be marked FAILED. - Can pass kernel commandline parameters from ``kunit.py`` via ``--kernel_args`` - KUnit prints a summary per suite of how many test cases passed/failed to dmesg, e.g. .. code-block:: shell # property-entry: pass:7 fail:0 skip:0 total:7 .. _v5.16: 5.16 ==== - You can filter on test names now in addition to suite names, e.g. .. code-block:: shell # Only run tests that contain "foo" $ ./tools/testing/kunit/kunit.py run '*.*foo*' # Only run tests in "mysuite" that contain "foo" $ ./tools/testing/kunit/kunit.py run 'mysuite.*foo*' - You can ask ``kunit.py`` to run each suite/test by itself. This can be useful to track down issues due to non-hermetic tests .. code-block:: shell $ ./tools/testing/kunit/kunit.py run --run_isolated=suite $ ./tools/testing/kunit/kunit.py run --run_isolated=test - ``kunit.py``'s parsing logic was largely rewritten, so the output looks slightly different - Parsed output is also now mostly printed out in real time. - ``kunit.py`` with ``--raw_output`` will print output in real time. - In-kernel documentation has been significantly revamped. .. _v5.17: 5.17 ==== - Print out parsed test results in real time. - Allow tweaking Kconfig options from command-line, e.g. .. code-block:: shell $ ./tools/testing/kunit/kunit.py run --kconfig_add=MY_CONFIG=y - Reconfigure kunit kernel when the used ``kunitconfig`` has changed - Before, if you removed an entry from ``.kunitconfig``, ``kunit.py`` would not reconfigure, so your edit would have no effect. - This now also works if you remove a ``--kconfig_add`` flag. - Print out parameters as sub-test cases (both in raw and parsed output), e.g.: .. code-block:: [12:37:14] =============== ext4_inode_test (1 subtest) ================ [12:37:14] ======= inode_test_xtimestamp_decoding (16 subtests) ======= [12:37:14] [PASSED] 1901-12-13 Lower bound of 32bit < 0 timestamp, no extra bits ... [12:37:14] [PASSED] 2446-05-10 Upper bound of 32bit >=0 timestamp. All extra sec bits on [12:37:14] ========= [PASSED] inode_test_xtimestamp_decoding ========== [12:37:14] ================= [PASSED] ext4_inode_test ================= [12:37:14] ============================================================ .. _v5.18: 5.18 ==== - Drastically decrease stack usage of ``KUNIT_EXPECT_`` macros. - Before, we largely relied on the compiler to optimize away some internal structs, but that didn't always work. - Now on UML (x86_64), ``KUNIT_EXPECT_EQ()`` uses 32 bytes instead of 88. .. _v5.19: 5.19 ==== - Add support for suite-level init/exit functions. - ``suite_init``/``suite_exit`` are like ``init`` and ``exit`` instead of once per test case. - Add ``KUNIT_EXPECT_NULL()``/``KUNIT_EXPECT_NOT_NULL()`` macros. - Split out KUnit resource API from ```` into ````. - Including ```` will still automatically ``#include `` for now. - ``kunit.py`` no longer colorizes output when stdout is not a tty. - Add a manually curated ``tools/testing/kunit/configs/all_tests_uml.config`` - This attempts to enable as many tests as possible on UML. - Change quoting requirements for qemu_config files (`commit `__) - E.g. instead of ``['-m 256']``, do ``['-m', '256']`` .. _v6.0: 6.0 === - Overhaul support for building tests as modules - Before, it wouldn't work in modules that already had their own ``module_init``/``module_exit``. Now this restriction is gone. - Add ``--qemu_args`` flag, e.g. use ``--qemu_args='-m 2048'`` to give tests more RAM. - Allow ``--kunitconfig`` to be repeated. - This will just concatenante the contents together. - One use case for this is appending the options for code coverage: .. code-block:: shell $ ./tools/testing/kunit/kunit.py run --kunitconfig=my/dir --kunitconfig=tools/testing/kunit/configs/coverage_uml.config - Taint the kernel if any KUnit tests run. - Add a new ``TAINT_TEST`` flag, shows up as ``'N'``. - Add support for KASAN to run on 64-bit UML. - By default, enable PCI support under UML. .. _v6.1: 6.1 === - ``kunit.py run --alltests`` now works. - Renamed ``all_tests_uml.config`` (added in :ref:`v5.19`) to ``all_tests.config``. - Changed the ``--alltests`` flag to use this manually curated file instead of trying to get ``allyesconfig`` to work. - Removed Virtio and PCI UML specific options from ``all_tests_uml.config``. - ``Documentation/`` updates and fixes. - Further reduction in the stack consumption of ``KUNIT_EXPECT_`` macros. - For context, see :ref:`v5.18`. - On UML, now ``KUNIT_FAIL()`` uses 0 bytes (was 8), and ``KUNIT_EXPECT_EQ()`` uses 24 (was 32). - In edge cases, ``kunit_kfree()`` matches ``kfree()``: - ``kunit_kfree(NULL)`` is a no-op. - ``kunit_kfree(invalid_ptr)`` will no longer crash, but it will now fail the test. - Introduce new ``kunit.enable`` kernel command-line argument. - This is a niche feature. If you don't want to run tests, prefer setting ``CONFIG_KUNIT=n``. - However, if you need to compile KUnit, you can use this to forcibly prevent tests from running. - You can set the default value of the option via ``CONFIG_KUNIT_DEFAULT_ENABLED``. .. _v6.2: 6.2 === - Added new ``KUNIT_EXPECT_MEMEQ(test, a, b, size)`` macro for comparing memory blocks. - This is an alternative to ``KUNIT_EXPECT_EQ(test, memcmp(a, b, size), 0)``. - On failure, KUnit will print out the bytes and highlight where the regions differ. - Failure logs are available in decimal and hexadecimal output. - Use a static key to check if tests are currently running. - This reduces the performance overhead of `kunit_get_current_test()` and `kunit_fail_current_test()` when no test is running. - For more details, see `kunit: Provide a static key to check if KUnit is actively running tests `__. - Make KUnit output and the kunit.py parser compliant with KTAP version 1 specification. - In KUnit output, subtests are now indicated using "KTAP version 1" headers. - Added two macros, `VISIBLE_IF_KUNIT` and `EXPORT_SYMBOL_IF_KUNIT`, that let tests to conditionally expose static symbols. - For an example of how to use these macros, see `apparmor: test: make static symbols visible during kunit testing `__. - Made the following KUnit documentation changes: - Updated the "architecture.rst" page for style and grammatical issues. - Made "usage.rst" a superset of "tips.rst" and removed "tips.rst" page. - Reworded assertion description. - Fix "How Do I Use This" / "Next Steps" section. - Updated the requirements for test result description in KTAP specification. .. _v6.3: 6.3 === - Added "hooks" to call into KUnit when it's built as a module. - These allow some KUnit features to be used in built-in code, or in other modules, without introducing a hard dependency on KUnit. - Introduced a "static_stub" feature to add redirection support for KUnit tests. Any function that wants to be intercepted adds a call to a macro which checks if a test has redirected calls to it, and then calls the corresponding replacement. .. _v6.4: 6.4 === - kunit.py now supports building / running tests via QEMU on the m68k and SuperH (sh) architectures. - Several fixes to debugfs log support, particularly with parameterized tests. - The default log size has been increased to 2048 bytes. .. _v6.5: 6.5 === - Added a new 'deferred actions' API, based on the devm\_/devres APIs, to call a function automatically on test exit. This should simplify resource management an cleanup. - See: https://docs.kernel.org/dev-tools/kunit/usage.html#registering-cleanup-actions - Test cleanup is now always run from a KUnit-managed test thread. - Previously, if a test aborted early, its cleanup (deferred actions, resources, and test / suite exit functions) would be run from the main KUnit executor thread. - This could cause problems if an assertion or crash occurred during cleanup. - Now, if the original test thred exits, a new cleanup thread is spawned. - Improved documentation and examples for test / suite init()/exit() functions. .. _v6.6: 6.6 === - Tests and suites can now have 'attributes', which can be filtered upon. - 'speed' denotes the expected runtime of a test: tests which are expected to take > 1 second to execute can be marked slow with ``KUNIT_CASE_SLOW()``. - 'module' lists the name of the module containing the test. - See: https://docs.kernel.org/dev-tools/kunit/running_tips.html#test-attributes-and-filtering - Rust 'doctests' are now automatically converted into and run as KUnit tests. - Additional architectural features are supported in the arm64 QEMU target. .. _v6.7: 6.7 === - Test logs are no longer limited to 2KiB in debugfs. The log will now grow as required. - KUnit will now warn if a test takes longer than 2 seconds to run, but is not marked as slow. .. _v6.8: 6.8 === - A new API to manage devices as KUnit resources. - These devices will be automatically unregistered on test exit. These helpers can either use a user-provided struct device_driver, or have one automatically created and managed by KUnit. In both cases, the device lives on a new ``kunit_bus``. - See: https://docs.kernel.org/dev-tools/kunit/usage.html#managing-fake-devices-and-drivers - See: https://docs.kernel.org/dev-tools/kunit/api/resource.html#managed-devices - Tests can now be run / re-run after boot from debugfs - Write to ``/sys/kernel/debugfs/kunit//run`` - Note that this doesn't work for tests in the init section, which are already discarded. - See: https://docs.kernel.org/dev-tools/kunit/run_manual.html#run-tests-after-kernel-has-booted - Deferred actions which take a non-void* parameter now need to be wrapped with KUNIT_DEFINE_ACTION_WRAPPER() - Casting a function pointer breaks Control Flow Integrity (CFI), so this macro creates a wrapper function with the correct type instead. - See the updated documentation at: https://docs.kernel.org/dev-tools/kunit/usage.html#registering-cleanup-actions - Tests in the init section now have the ``init`` boolean attribute. .. _v6.9: 6.9 === - Assertions' format strings are now verified at compile time. - kunit.py now prints the command used to run the kernel (and hence the tests) on UML as well as with QEMU. - There is improved documentation for running Rust tests: https://docs.kernel.org/rust/testing.html .. _v6.10: 6.10 ==== - KUnit will now print the file:line of the last KUnit macro invocation on a crash. - This can help identify the cause of a crash if reliable stacktraces are unavailable. - KUnit tests which deliberately trigger a fault (e.g., to test fault handling mechanisms) now live behind CONFIG_KUNIT_FAULT_TEST option. - This makes them easier to disable on architectures which cannot or do not trap faults. - Rust code can now be compiled (and therefore tested) on 32-bit x86 UML. - Pass ``--kconfig_add CONFIG_64BIT=n`` to ``kunit.py`` - You may also need ``--kconfig_add CONFIG_FORTIFY_SOURCE=n``, as this is not supported, and may be enabled by default. .. _v6.11: 6.11 ==== - KUnit can now manage per-test memory mappings. - The ``kunit_vm_mmap()`` function will map memory into the test's context. - See https://docs.kernel.org/dev-tools/kunit/api/test.html#c.kunit_vm_mmap - New assertion macros ``KUNIT_ASSERT_MEMEQ()`` and ``KUNIT_ASSERT_MEMNEQ()`` - These match the existing ``KUNIT_EXPECT_MEMEQ()`` and ``KUNIT_EXPECT_MEMNEQ()`` macros. - ``KUNIT_ASSERT_FAILURE()`` has been renamed to ``KUNIT_FAIL_AND_ABORT()`` - The name passed to ``kunit_driver_create()`` is now managed by KUnit. - If it goes out-of-scope, there will no longer be a crash, as KUnit keeps a copy behind the scenes. - New managed ``kunit_strdup()`` function to duplicate a string to KUnit-managed memory. - New ``kunit_kstrdup_const()`` and ``kunit_free_const()``, which will allocate (and free) memory only if the source string is not in kernel rodata. - This is useful when handling strings which may be constant, or may be temporary. .. _v6.12: 6.12 ==== - There are new best-practices for naming KUnit tests and test files: - In short, test modules should have the ``_kunit`` suffix, and their code should live in a ``tests/`` subdirectory. - See https://docs.kernel.org/dev-tools/kunit/style.html - KUnit can now manage platform devices - See https://docs.kernel.org/dev-tools/kunit/api/platformdevice.html - KUnit can now load DeviceTree overlays for tests. - See https://docs.kernel.org/dev-tools/kunit/api/of.html - There is now special support for testing clk devices: - See https://docs.kernel.org/dev-tools/kunit/api/clk.html - ``kunit.py`` now generates ``compile_commands.json`` by default, for use with ``clangd``-based language servers. - The generated file is in the build output directory, (``.kunit`` by default), so you'll need to pass ``-compile-commands-dir=.kunit`` to ``clangd``. .. _v6.13: 6.13 ==== - It is now possible to only show the summary of KUnit results, rather than the full list of tests. - Pass the ``--summary`` option to ``kunit.py run`` or ``kunit.py exec``. - (This option is mutually exclusive with ``--raw_output``) - It is now possible to only show results for failed tests. - Pass the ``--failed`` option to ``kunit.py run`` or ``kunit.py exec``. - (This option is mutually exclusive with ``--raw_output``) - ``kunit.py`` now supports the loongarch architecture under QEMU - Use ``--arch loongarch``, and set ``--cross_compile`` if cross-compiling. .. _v6.14: 6.14 ==== - The automatic running of tests on boot or module load can now be disabled via the ``kunit.autorun`` option. - This is a kernel command-line option, which can be passed via the bootloader (if KUnit is built-in), or modprobe (if it's a module). - ``CONFIG_KUNIT_AUTORUN_ENABLED`` can be used to set the default behaviour. - ``kunit.py`` will now use hardware-accelerated virtualisation on arm64, if available. - Both KVM and HVF are supported. This can reduce the time taken to run tests in QEMU by up to 90%. - ``kunit.py`` is now more compatible with macOS hosts. (Though this is not really supported, so good luck!) .. _v6.15: 6.15 ==== - ``kunit.py`` now uses qboot when running on x86_64 QEMU configurations to improve boot times. - ``kunit.py`` now generates gdb scripts by default, to help with kernel debugging. - ``kunit.py`` now uses the Zilog console in QEMU on the SPARC architecture. .. _v6.16: 6.16 ==== - Rust tests can now be built and run as KUnit tests. - Decorate individual tests with ``#[test]``, and the module with ``#[kunit_tests(suite_name)]``. - See https://docs.kernel.org/rust/testing.html#the-test-tests - It is now possible to list the supported architectures in ``kunit.py`` by passing ``--arch help``. - Several new architecture configurations are supported in ``kunit.py`` - RISC-V 32-bit, SPARC 64-bit, PowerPC 32-bit Big-Endian and 64-bit Little-Endian