Release Notes

This page briefly summarizes the KUnit changes in each Linux release.

For more specific documentation on KUnit features, see the in-kernel documentation: https://kernel.org/doc/html/latest/dev-tools/kunit.

5.5

• KUnit introduced, including all the basic functionality.

• kunitconfig was renamed to .kunitconfig later in 5.5 (commit)

5.6

• KUnit tests can be built as modules, docs

5.7

• kunit.py switched to using .kunit/ as the default build dir (and .kunitconfig => .kunit/.kunitconfig).

• CONFIG_KUNIT_DEBUGFS to expose per-suite test results in /sys/kernel/debug/kunit/<suite>/results, docs

5.8

• kunit.py now supports separate config/build/exec/parse commands in addition to run.

  • You can use these if you don’t want to let kunit.py run do everything for you, e.g. if you want to run manually but parse the results using kunit.py.

• CONFIG_KUNIT_ALL_TESTS introduced

  • This is just a convention. New tests should use it like so:

config FOO_KUNIT_TEST
        tristate "KUnit test for foo" if !KUNIT_ALL_TESTS
        depends on KUNIT
        default KUNIT_ALL_TESTS
        help
          This builds unit tests for foo.

See the upstream documentation for the authoritative source on how to write Kconfig entries.

5.9

• Named resources: tests can store and retrieve data via string keys.

  • This is a power feature intended to be a cleaner alternative to using global variables in tests.

  • See test_kasan.c for an example that uses "kasan_data".

5.10

• KASAN violations cause the current KUnit test case to be marked FAILED.

  • Note: KASAN doesn’t currently work on UML.

5.11

• Parameterized testing: see the docs.

5.12

• You can pass in a path to a kunitconfig file instead of using .kunit/.kunitconfig, e.g.

$ ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4/.kunitconfig

• You can tell kunit.py to run a subset of built suites by passing in a glob argument, e.g.

# Only run "list" suites
$ ./tools/testing/kunit/kunit.py run '*list*'

5.13

• Can call kunit_fail_current_test() from anywhere to fail the current KUnit test (e.g. if an invariant is broken, a UBSAN violation is detected, etc.)

• You can pass a directory --kunitconfig and it’ll implicitly append .kunitconfig, e.g. kunit.py run --kunitconfig=lib/kunit.

5.14

• kunit.py can run tests on non-UML architectures (using QEMU), docs

• It’s now possible to generate code coverage reports when using UML, docs

• Tests can call kunit_skip() to bail out and mark a test as “SKIPPED” instead of “PASSED” or “FAILED”

5.15

• UBSAN violations cause the current KUnit test case to be marked FAILED.

• Can pass kernel commandline parameters from kunit.py via --kernel_args

• KUnit prints a summary per suite of how many test cases passed/failed to dmesg, e.g.

# property-entry: pass:7 fail:0 skip:0 total:7

5.16

• You can filter on test names now in addition to suite names, e.g.

# Only run tests that contain "foo"
$ ./tools/testing/kunit/kunit.py run '*.*foo*'
# Only run tests in "mysuite" that contain "foo"
$ ./tools/testing/kunit/kunit.py run 'mysuite.*foo*'

• You can ask kunit.py to run each suite/test by itself. This can be useful to track down issues due to non-hermetic tests

$ ./tools/testing/kunit/kunit.py run --run_isolated=suite
$ ./tools/testing/kunit/kunit.py run --run_isolated=test

• kunit.py’s parsing logic was largely rewritten, so the output looks slightly different

  • Parsed output is also now mostly printed out in real time.

• kunit.py with --raw_output will print output in real time.

• In-kernel documentation has been significantly revamped.

5.17

• Print out parsed test results in real time.

• Allow tweaking Kconfig options from command-line, e.g.

$ ./tools/testing/kunit/kunit.py run --kconfig_add=MY_CONFIG=y

• Reconfigure kunit kernel when the used kunitconfig has changed

  • Before, if you removed an entry from .kunitconfig, kunit.py would not reconfigure, so your edit would have no effect.

  • This now also works if you remove a --kconfig_add flag.

• Print out parameters as sub-test cases (both in raw and parsed output), e.g.:

[12:37:14] =============== ext4_inode_test (1 subtest) ================
[12:37:14] ======= inode_test_xtimestamp_decoding (16 subtests) =======
[12:37:14] [PASSED] 1901-12-13 Lower bound of 32bit < 0 timestamp, no extra bits
...
[12:37:14] [PASSED] 2446-05-10 Upper bound of 32bit >=0 timestamp. All extra sec bits on
[12:37:14] ========= [PASSED] inode_test_xtimestamp_decoding ==========
[12:37:14] ================= [PASSED] ext4_inode_test =================
[12:37:14] ============================================================

5.18

• Drastically decrease stack usage of KUNIT_EXPECT_ macros.

  • Before, we largely relied on the compiler to optimize away some internal structs, but that didn’t always work.

  • Now on UML (x86_64), KUNIT_EXPECT_EQ() uses 32 bytes instead of 88.

5.19

• Add support for suite-level init/exit functions.

  • suite_init/suite_exit are like init and exit instead of once per test case.

• Add KUNIT_EXPECT_NULL()/KUNIT_EXPECT_NOT_NULL() macros.

• Split out KUnit resource API from <kunit/test.h> into <kunit/resource.h>.

  • Including <kunit/test.h> will still automatically #include <kunit/resource.h> for now.

• kunit.py no longer colorizes output when stdout is not a tty.

• Add a manually curated tools/testing/kunit/configs/all_tests_uml.config

  • This attempts to enable as many tests as possible on UML.

• Change quoting requirements for qemu_config files (commit)

  • E.g. instead of ['-m 256'], do ['-m', '256']

6.0

• Overhaul support for building tests as modules

  • Before, it wouldn’t work in modules that already had their own module_init/module_exit. Now this restriction is gone.

• Add --qemu_args flag, e.g. use --qemu_args='-m 2048' to give tests more RAM.

• Allow --kunitconfig to be repeated.

  • This will just concatenante the contents together.

  • One use case for this is appending the options for code coverage:

$ ./tools/testing/kunit/kunit.py run --kunitconfig=my/dir --kunitconfig=tools/testing/kunit/configs/coverage_uml.config

• Taint the kernel if any KUnit tests run.

  • Add a new TAINT_TEST flag, shows up as 'N'.

• Add support for KASAN to run on 64-bit UML.

• By default, enable PCI support under UML.

6.1

• kunit.py run --alltests now works.

  • Renamed all_tests_uml.config (added in 5.19) to all_tests.config.

  • Changed the --alltests flag to use this manually curated file instead of trying to get allyesconfig to work.

  • Removed Virtio and PCI UML specific options from all_tests_uml.config.

• Documentation/ updates and fixes.

• Further reduction in the stack consumption of KUNIT_EXPECT_ macros.

  • For context, see 5.18.

  • On UML, now KUNIT_FAIL() uses 0 bytes (was 8), and KUNIT_EXPECT_EQ() uses 24 (was 32).

• In edge cases, kunit_kfree() matches kfree():

  • kunit_kfree(NULL) is a no-op.

  • kunit_kfree(invalid_ptr) will no longer crash, but it will now fail the test.

• Introduce new kunit.enable kernel command-line argument.

  • This is a niche feature. If you don’t want to run tests, prefer setting CONFIG_KUNIT=n.

  • However, if you need to compile KUnit, you can use this to forcibly prevent tests from running.

  • You can set the default value of the option via CONFIG_KUNIT_DEFAULT_ENABLED.

6.2

• Added new KUNIT_EXPECT_MEMEQ(test, a, b, size) macro for comparing memory blocks.

  • This is an alternative to KUNIT_EXPECT_EQ(test, memcmp(a, b, size), 0).

  • On failure, KUnit will print out the bytes and highlight where the regions differ.

• Failure logs are available in decimal and hexadecimal output.

• Use a static key to check if tests are currently running.

  • This reduces the performance overhead of kunit_get_current_test() and kunit_fail_current_test() when no test is running.

  • For more details, see kunit: Provide a static key to check if KUnit is actively running tests.

• Make KUnit output and the kunit.py parser compliant with KTAP version 1 specification.

  • In KUnit output, subtests are now indicated using “KTAP version 1” headers.

• Added two macros, VISIBLE_IF_KUNIT and EXPORT_SYMBOL_IF_KUNIT, that let tests to conditionally expose static symbols.

  • For an example of how to use these macros, see apparmor: test: make static symbols visible during kunit testing.

• Made the following KUnit documentation changes:

  • Updated the “architecture.rst” page for style and grammatical issues.

  • Made “usage.rst” a superset of “tips.rst” and removed “tips.rst” page.

  • Reworded assertion description.

  • Fix “How Do I Use This” / “Next Steps” section.

  • Updated the requirements for test result description in KTAP specification.

6.3

• Added “hooks” to call into KUnit when it’s built as a module.

  • These allow some KUnit features to be used in built-in code, or in other modules, without introducing a hard dependency on KUnit.

• Introduced a “static_stub” feature to add redirection support for KUnit tests. Any function that wants to be intercepted adds a call to a macro which checks if a test has redirected calls to it, and then calls the corresponding replacement.

6.4

• kunit.py now supports building / running tests via QEMU on the m68k and SuperH (sh) architectures.

• Several fixes to debugfs log support, particularly with parameterized tests.

• The default log size has been increased to 2048 bytes.

6.5

• Added a new ‘deferred actions’ API, based on the devm_/devres APIs, to call a function automatically on test exit. This should simplify resource management an cleanup.

  • See: https://docs.kernel.org/dev-tools/kunit/usage.html#registering-cleanup-actions

• Test cleanup is now always run from a KUnit-managed test thread.

  • Previously, if a test aborted early, its cleanup (deferred actions, resources, and test / suite exit functions) would be run from the main KUnit executor thread.

  • This could cause problems if an assertion or crash occurred during cleanup.

  • Now, if the original test thred exits, a new cleanup thread is spawned.

• Improved documentation and examples for test / suite init()/exit() functions.

6.6

• Tests and suites can now have ‘attributes’, which can be filtered upon.

  • ‘speed’ denotes the expected runtime of a test: tests which are expected to take > 1 second to execute can be marked slow with KUNIT_CASE_SLOW().

  • ‘module’ lists the name of the module containing the test.

  • See: https://docs.kernel.org/dev-tools/kunit/running_tips.html#test-attributes-and-filtering

• Rust ‘doctests’ are now automatically converted into and run as KUnit tests.

• Additional architectural features are supported in the arm64 QEMU target.

6.7

• Test logs are no longer limited to 2KiB in debugfs. The log will now grow as required.

• KUnit will now warn if a test takes longer than 2 seconds to run, but is not marked as slow.

6.8

• A new API to manage devices as KUnit resources.

  • These devices will be automatically unregistered on test exit. These helpers can either use a user-provided struct device_driver, or have one automatically created and managed by KUnit. In both cases, the device lives on a new kunit_bus.

  • See: https://docs.kernel.org/dev-tools/kunit/usage.html#managing-fake-devices-and-drivers

  • See: https://docs.kernel.org/dev-tools/kunit/api/resource.html#managed-devices

• Tests can now be run / re-run after boot from debugfs

  • Write to /sys/kernel/debugfs/kunit/<test_suite>/run

  • Note that this doesn’t work for tests in the init section, which are already discarded.

  • See: https://docs.kernel.org/dev-tools/kunit/run_manual.html#run-tests-after-kernel-has-booted

• Deferred actions which take a non-void* parameter now need to be wrapped with KUNIT_DEFINE_ACTION_WRAPPER()

  • Casting a function pointer breaks Control Flow Integrity (CFI), so this macro creates a wrapper function with the correct type instead.

  • See the updated documentation at: https://docs.kernel.org/dev-tools/kunit/usage.html#registering-cleanup-actions

• Tests in the init section now have the init boolean attribute.

6.9

• Assertions’ format strings are now verified at compile time.

• kunit.py now prints the command used to run the kernel (and hence the tests) on UML as well as with QEMU.

• There is improved documentation for running Rust tests: https://docs.kernel.org/rust/testing.html

6.10

• KUnit will now print the file:line of the last KUnit macro invocation on a crash.

  • This can help identify the cause of a crash if reliable stacktraces are unavailable.

• KUnit tests which deliberately trigger a fault (e.g., to test fault handling mechanisms) now live behind CONFIG_KUNIT_FAULT_TEST option.

  • This makes them easier to disable on architectures which cannot or do not trap faults.

• Rust code can now be compiled (and therefore tested) on 32-bit x86 UML.

  • Pass --kconfig_add CONFIG_64BIT=n to kunit.py

  • You may also need --kconfig_add CONFIG_FORTIFY_SOURCE=n, as this is not supported, and may be enabled by default.