============= Release Notes ============= This page briefly summarizes the KUnit changes in each Linux release. For more specific documentation on KUnit features, see the in-kernel documentation: https://kernel.org/doc/html/latest/dev-tools/kunit. .. _v5.5: 5.5 === - KUnit introduced, including all the basic functionality. - ``kunitconfig`` was renamed to ``.kunitconfig`` later in 5.5 (`commit `__) .. _v5.6: 5.6 === - KUnit tests can be built as modules, `docs `__ .. _v5.7: 5.7 === - kunit.py switched to using ``.kunit/`` as the default build dir (and ``.kunitconfig`` => ``.kunit/.kunitconfig``). - ``CONFIG_KUNIT_DEBUGFS`` to expose per-suite test results in ``/sys/kernel/debug/kunit//results``, `docs `__ .. _v5.8: 5.8 === - ``kunit.py`` now supports separate ``config/build/exec/parse`` commands in addition to ``run``. - You can use these if you don't want to let ``kunit.py run`` do everything for you, e.g. if you want to run manually but parse the results using ``kunit.py``. - ``CONFIG_KUNIT_ALL_TESTS`` introduced - This is just a convention. New tests should use it like so: .. code-block:: none config FOO_KUNIT_TEST tristate "KUnit test for foo" if !KUNIT_ALL_TESTS depends on KUNIT default KUNIT_ALL_TESTS help This builds unit tests for foo. See `the upstream documentation `__ for the authoritative source on how to write Kconfig entries. .. _v5.9: 5.9 === - Named resources: tests can store and retrieve data via string keys. - This is a power feature intended to be a cleaner alternative to using global variables in tests. - See `test_kasan.c `__ for an example that uses ``"kasan_data"``. .. _v5.10: 5.10 ==== - KASAN violations cause the current KUnit test case to be marked FAILED. - Note: KASAN doesn't currently work on UML. .. _v5.11: 5.11 ==== - Parameterized testing: see the `docs `__. .. _v5.12: 5.12 ==== - You can pass in a path to a kunitconfig file instead of using ``.kunit/.kunitconfig``, e.g. .. code-block:: shell $ ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4/.kunitconfig - You can tell ``kunit.py`` to run a subset of built suites by passing in a glob argument, e.g. .. code-block:: shell # Only run "list" suites $ ./tools/testing/kunit/kunit.py run '*list*' .. _v5.13: 5.13 ==== - Can call ``kunit_fail_current_test()`` from anywhere to fail the current KUnit test (e.g. if an invariant is broken, a UBSAN violation is detected, etc.) - You can pass a directory ``--kunitconfig`` and it'll implicitly append ``.kunitconfig``, e.g. ``kunit.py run --kunitconfig=lib/kunit``. .. _v5.14: 5.14 ==== - ``kunit.py`` can run tests on non-UML architectures (using QEMU), `docs `__ - It's now possible to generate code coverage reports when using UML, `docs `__ - Tests can call ``kunit_skip()`` to bail out and mark a test as "SKIPPED" instead of "PASSED" or "FAILED" .. _v5.15: 5.15 ==== - UBSAN violations cause the current KUnit test case to be marked FAILED. - Can pass kernel commandline parameters from ``kunit.py`` via ``--kernel_args`` - KUnit prints a summary per suite of how many test cases passed/failed to dmesg, e.g. .. code-block:: shell # property-entry: pass:7 fail:0 skip:0 total:7 .. _v5.16: 5.16 ==== - You can filter on test names now in addition to suite names, e.g. .. code-block:: shell # Only run tests that contain "foo" $ ./tools/testing/kunit/kunit.py run '*.*foo*' # Only run tests in "mysuite" that contain "foo" $ ./tools/testing/kunit/kunit.py run 'mysuite.*foo*' - You can ask ``kunit.py`` to run each suite/test by itself. This can be useful to track down issues due to non-hermetic tests .. code-block:: shell $ ./tools/testing/kunit/kunit.py run --run_isolated=suite $ ./tools/testing/kunit/kunit.py run --run_isolated=test - ``kunit.py``'s parsing logic was largely rewritten, so the output looks slightly different - Parsed output is also now mostly printed out in real time. - ``kunit.py`` with ``--raw_output`` will print output in real time. - In-kernel documentation has been significantly revamped. .. _v5.17: 5.17 ==== - Print out parsed test results in real time. - Allow tweaking Kconfig options from command-line, e.g. .. code-block:: shell $ ./tools/testing/kunit/kunit.py run --kconfig_add=MY_CONFIG=y - Reconfigure kunit kernel when the used ``kunitconfig`` has changed - Before, if you removed an entry from ``.kunitconfig``, ``kunit.py`` would not reconfigure, so your edit would have no effect. - This now also works if you remove a ``--kconfig_add`` flag. - Print out parameters as sub-test cases (both in raw and parsed output), e.g.: .. code-block:: [12:37:14] =============== ext4_inode_test (1 subtest) ================ [12:37:14] ======= inode_test_xtimestamp_decoding (16 subtests) ======= [12:37:14] [PASSED] 1901-12-13 Lower bound of 32bit < 0 timestamp, no extra bits ... [12:37:14] [PASSED] 2446-05-10 Upper bound of 32bit >=0 timestamp. All extra sec bits on [12:37:14] ========= [PASSED] inode_test_xtimestamp_decoding ========== [12:37:14] ================= [PASSED] ext4_inode_test ================= [12:37:14] ============================================================ .. _v5.18: 5.18 ==== - Drastically decrease stack usage of ``KUNIT_EXPECT_`` macros. - Before, we largely relied on the compiler to optimize away some internal structs, but that didn't always work. - Now on UML (x86_64), ``KUNIT_EXPECT_EQ()`` uses 32 bytes instead of 88. .. _v5.19: 5.19 ==== - Add support for suite-level init/exit functions. - ``suite_init``/``suite_exit`` are like ``init`` and ``exit`` instead of once per test case. - Add ``KUNIT_EXPECT_NULL()``/``KUNIT_EXPECT_NOT_NULL()`` macros. - Split out KUnit resource API from ```` into ````. - Including ```` will still automatically ``#include `` for now. - ``kunit.py`` no longer colorizes output when stdout is not a tty. - Add a manually curated ``tools/testing/kunit/configs/all_tests_uml.config`` - This attempts to enable as many tests as possible on UML. - Change quoting requirements for qemu_config files (`commit `__) - E.g. instead of ``['-m 256']``, do ``['-m', '256']`` .. _v6.0: 6.0 === - Overhaul support for building tests as modules - Before, it wouldn't work in modules that already had their own ``module_init``/``module_exit``. Now this restriction is gone. - Add ``--qemu_args`` flag, e.g. use ``--qemu_args='-m 2048'`` to give tests more RAM. - Allow ``--kunitconfig`` to be repeated. - This will just concatenante the contents together. - One use case for this is appending the options for code coverage: .. code-block:: shell $ ./tools/testing/kunit/kunit.py run --kunitconfig=my/dir --kunitconfig=tools/testing/kunit/configs/coverage_uml.config - Taint the kernel if any KUnit tests run. - Add a new ``TAINT_TEST`` flag, shows up as ``'N'``. - Add support for KASAN to run on 64-bit UML. - By default, enable PCI support under UML. .. _v6.1: 6.1 === - ``kunit.py run --alltests`` now works. - Renamed ``all_tests_uml.config`` (added in :ref:`v5.19`) to ``all_tests.config``. - Changed the ``--alltests`` flag to use this manually curated file instead of trying to get ``allyesconfig`` to work. - Removed Virtio and PCI UML specific options from ``all_tests_uml.config``. - ``Documentation/`` updates and fixes. - Further reduction in the stack consumption of ``KUNIT_EXPECT_`` macros. - For context, see :ref:`v5.18`. - On UML, now ``KUNIT_FAIL()`` uses 0 bytes (was 8), and ``KUNIT_EXPECT_EQ()`` uses 24 (was 32). - In edge cases, ``kunit_kfree()`` matches ``kfree()``: - ``kunit_kfree(NULL)`` is a no-op. - ``kunit_kfree(invalid_ptr)`` will no longer crash, but it will now fail the test. - Introduce new ``kunit.enable`` kernel command-line argument. - This is a niche feature. If you don't want to run tests, prefer setting ``CONFIG_KUNIT=n``. - However, if you need to compile KUnit, you can use this to forcibly prevent tests from running. - You can set the default value of the option via ``CONFIG_KUNIT_DEFAULT_ENABLED``. .. _v6.2: 6.2 === - Added new ``KUNIT_EXPECT_MEMEQ(test, a, b, size)`` macro for comparing memory blocks. - This is an alternative to ``KUNIT_EXPECT_EQ(test, memcmp(a, b, size), 0)``. - On failure, KUnit will print out the bytes and highlight where the regions differ. - Failure logs are available in decimal and hexadecimal output. - Use a static key to check if tests are currently running. - This reduces the performance overhead of `kunit_get_current_test()` and `kunit_fail_current_test()` when no test is running. - For more details, see `kunit: Provide a static key to check if KUnit is actively running tests `__. - Make KUnit output and the kunit.py parser compliant with KTAP version 1 specification. - In KUnit output, subtests are now indicated using "KTAP version 1" headers. - Added two macros, `VISIBLE_IF_KUNIT` and `EXPORT_SYMBOL_IF_KUNIT`, that let tests to conditionally expose static symbols. - For an example of how to use these macros, see `apparmor: test: make static symbols visible during kunit testing `__. - Made the following KUnit documentation changes: - Updated the "architecture.rst" page for style and grammatical issues. - Made "usage.rst" a superset of "tips.rst" and removed "tips.rst" page. - Reworded assertion description. - Fix "How Do I Use This" / "Next Steps" section. - Updated the requirements for test result description in KTAP specification. .. _v6.3: 6.3 === - Added "hooks" to call into KUnit when it's built as a module. - These allow some KUnit features to be used in built-in code, or in other modules, without introducing a hard dependency on KUnit. - Introduced a "static_stub" feature to add redirection support for KUnit tests. Any function that wants to be intercepted adds a call to a macro which checks if a test has redirected calls to it, and then calls the corresponding replacement. .. _v6.4: 6.4 === - kunit.py now supports building / running tests via QEMU on the m68k and SuperH (sh) architectures. - Several fixes to debugfs log support, particularly with parameterized tests. - The default log size has been increased to 2048 bytes. .. _v6.5: 6.5 === - Added a new 'deferred actions' API, based on the devm\_/devres APIs, to call a function automatically on test exit. This should simplify resource management an cleanup. - See: https://docs.kernel.org/dev-tools/kunit/usage.html#registering-cleanup-actions - Test cleanup is now always run from a KUnit-managed test thread. - Previously, if a test aborted early, its cleanup (deferred actions, resources, and test / suite exit functions) would be run from the main KUnit executor thread. - This could cause problems if an assertion or crash occurred during cleanup. - Now, if the original test thred exits, a new cleanup thread is spawned. - Improved documentation and examples for test / suite init()/exit() functions. .. _v6.6: 6.6 === - Tests and suites can now have 'attributes', which can be filtered upon. - 'speed' denotes the expected runtime of a test: tests which are expected to take > 1 second to execute can be marked slow with ``KUNIT_CASE_SLOW()``. - 'module' lists the name of the module containing the test. - See: https://docs.kernel.org/dev-tools/kunit/running_tips.html#test-attributes-and-filtering - Rust 'doctests' are now automatically converted into and run as KUnit tests. - Additional architectural features are supported in the arm64 QEMU target. .. _v6.7: 6.7 === - Test logs are no longer limited to 2KiB in debugfs. The log will now grow as required. - KUnit will now warn if a test takes longer than 2 seconds to run, but is not marked as slow. .. _v6.8: 6.8 === - A new API to manage devices as KUnit resources. - These devices will be automatically unregistered on test exit. These helpers can either use a user-provided struct device_driver, or have one automatically created and managed by KUnit. In both cases, the device lives on a new ``kunit_bus``. - See: https://docs.kernel.org/dev-tools/kunit/usage.html#managing-fake-devices-and-drivers - See: https://docs.kernel.org/dev-tools/kunit/api/resource.html#managed-devices - Tests can now be run / re-run after boot from debugfs - Write to ``/sys/kernel/debugfs/kunit//run`` - Note that this doesn't work for tests in the init section, which are already discarded. - See: https://docs.kernel.org/dev-tools/kunit/run_manual.html#run-tests-after-kernel-has-booted - Deferred actions which take a non-void* parameter now need to be wrapped with KUNIT_DEFINE_ACTION_WRAPPER() - Casting a function pointer breaks Control Flow Integrity (CFI), so this macro creates a wrapper function with the correct type instead. - See the updated documentation at: https://docs.kernel.org/dev-tools/kunit/usage.html#registering-cleanup-actions - Tests in the init section now have the ``init`` boolean attribute. .. _v6.9: 6.9 === - Assertions' format strings are now verified at compile time. - kunit.py now prints the command used to run the kernel (and hence the tests) on UML as well as with QEMU. - There is improved documentation for running Rust tests: https://docs.kernel.org/rust/testing.html .. _v6.10: 6.10 ==== - KUnit will now print the file:line of the last KUnit macro invocation on a crash. - This can help identify the cause of a crash if reliable stacktraces are unavailable. - KUnit tests which deliberately trigger a fault (e.g., to test fault handling mechanisms) now live behind CONFIG_KUNIT_FAULT_TEST option. - This makes them easier to disable on architectures which cannot or do not trap faults. - Rust code can now be compiled (and therefore tested) on 32-bit x86 UML. - Pass ``--kconfig_add CONFIG_64BIT=n`` to ``kunit.py`` - You may also need ``--kconfig_add CONFIG_FORTIFY_SOURCE=n``, as this is not supported, and may be enabled by default.