Centos-kernel-stream-9/tools/testing/selftests/net/lib/py/utils.py

156 lines
4.3 KiB
Python
Raw Normal View History

# SPDX-License-Identifier: GPL-2.0
import errno
import json as _json
import random
import re
import socket
import subprocess
import time
selftests: drv-net: rss_ctx: add tests for RSS configuration and contexts JIRA: https://issues.redhat.com/browse/RHEL-89014 Conflict: - Minor context diff in Makefile caused by the absence of code hunks from commit: (selftests: drv-net-hw: add test for memory allocation failures with page pool) commit f898c16a0624e7f2dcb0b1cda6916c9be6489197 Author: Jakub Kicinski <kuba@kernel.org> Date: Tue Jun 25 18:24:56 2024 -0700 selftests: drv-net: rss_ctx: add tests for RSS configuration and contexts Add tests focusing on indirection table configuration and creating extra RSS contexts in drivers which support it. $ export NETIF=eth0 REMOTE_... $ ./drivers/net/hw/rss_ctx.py KTAP version 1 1..8 ok 1 rss_ctx.test_rss_key_indir ok 2 rss_ctx.test_rss_context ok 3 rss_ctx.test_rss_context4 # Increasing queue count 44 -> 66 # Failed to create context 32, trying to test what we got ok 4 rss_ctx.test_rss_context32 # SKIP Tested only 31 contexts, wanted 32 ok 5 rss_ctx.test_rss_context_overlap ok 6 rss_ctx.test_rss_context_overlap2 # .. sprays traffic like a headless chicken .. not ok 7 rss_ctx.test_rss_context_out_of_order ok 8 rss_ctx.test_rss_context4_create_with_cfg # Totals: pass:6 fail:1 xfail:0 xpass:0 skip:1 error:0 Note that rss_ctx.test_rss_context_out_of_order fails with the device I tested with, but it seems to be a device / driver bug. Reviewed-by: Petr Machata <petrm@nvidia.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://patch.msgid.link/20240626012456.2326192-5-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org> Signed-off-by: Mohammad Heib <mheib@redhat.com>
2025-06-04 09:58:06 +00:00
class CmdExitFailure(Exception):
pass
class cmd:
def __init__(self, comm, shell=True, fail=True, ns=None, background=False, host=None, timeout=5):
if ns:
comm = f'ip netns exec {ns} ' + comm
self.stdout = None
self.stderr = None
self.ret = None
self.comm = comm
selftests: drv-net: define endpoint structures JIRA: https://issues.redhat.com/browse/RHEL-57764 commit 1a20a9a0ddef17c0bd67eece34a7439b02a7b0ba Author: Jakub Kicinski <kuba@kernel.org> Date: Fri Apr 19 19:52:31 2024 -0700 selftests: drv-net: define endpoint structures Define the remote endpoint "model". To execute most meaningful device driver tests we need to be able to communicate with a remote system, and have it send traffic to the device under test. Various test environments will have different requirements. 0) "Local" netdevsim-based testing can simply use net namespaces. netdevsim supports connecting two devices now, to form a veth-like construct. 1) Similarly on hosts with multiple NICs, the NICs may be connected together with a loopback cable or internal device loopback. One interface may be placed into separate netns, and tests would proceed much like in the netdevsim case. Note that the loopback config or the moving of one interface into a netns is not expected to be part of selftest code. 2) Some systems may need to communicate with the remote endpoint via SSH. 3) Last but not least environment may have its own custom communication method. Fundamentally we only need two operations: - run a command remotely - deploy a binary (if some tool we need is built as part of kselftests) Wrap these two in a class. Use dynamic loading to load the Remote class. This will allow very easy definition of other communication methods without bothering upstream code base. Stick to the "simple" / "no unnecessary abstractions" model for referring to the remote endpoints. The host / remote object are passed as an argument to the usual cmd() or ip() invocation. For example: ip("link show", json=True, host=remote) Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://lore.kernel.org/r/20240420025237.3309296-2-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org> Signed-off-by: Ivan Vecera <ivecera@redhat.com>
2024-10-04 08:59:04 +00:00
if host:
self.proc = host.cmd(comm)
else:
self.proc = subprocess.Popen(comm, shell=shell, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
if not background:
self.process(terminate=False, fail=fail, timeout=timeout)
def process(self, terminate=True, fail=None, timeout=5):
if fail is None:
fail = not terminate
if terminate:
self.proc.terminate()
stdout, stderr = self.proc.communicate(timeout)
self.stdout = stdout.decode("utf-8")
self.stderr = stderr.decode("utf-8")
self.proc.stdout.close()
self.proc.stderr.close()
self.ret = self.proc.returncode
if self.proc.returncode != 0 and fail:
if len(stderr) > 0 and stderr[-1] == "\n":
stderr = stderr[:-1]
selftests: drv-net: rss_ctx: add tests for RSS configuration and contexts JIRA: https://issues.redhat.com/browse/RHEL-89014 Conflict: - Minor context diff in Makefile caused by the absence of code hunks from commit: (selftests: drv-net-hw: add test for memory allocation failures with page pool) commit f898c16a0624e7f2dcb0b1cda6916c9be6489197 Author: Jakub Kicinski <kuba@kernel.org> Date: Tue Jun 25 18:24:56 2024 -0700 selftests: drv-net: rss_ctx: add tests for RSS configuration and contexts Add tests focusing on indirection table configuration and creating extra RSS contexts in drivers which support it. $ export NETIF=eth0 REMOTE_... $ ./drivers/net/hw/rss_ctx.py KTAP version 1 1..8 ok 1 rss_ctx.test_rss_key_indir ok 2 rss_ctx.test_rss_context ok 3 rss_ctx.test_rss_context4 # Increasing queue count 44 -> 66 # Failed to create context 32, trying to test what we got ok 4 rss_ctx.test_rss_context32 # SKIP Tested only 31 contexts, wanted 32 ok 5 rss_ctx.test_rss_context_overlap ok 6 rss_ctx.test_rss_context_overlap2 # .. sprays traffic like a headless chicken .. not ok 7 rss_ctx.test_rss_context_out_of_order ok 8 rss_ctx.test_rss_context4_create_with_cfg # Totals: pass:6 fail:1 xfail:0 xpass:0 skip:1 error:0 Note that rss_ctx.test_rss_context_out_of_order fails with the device I tested with, but it seems to be a device / driver bug. Reviewed-by: Petr Machata <petrm@nvidia.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://patch.msgid.link/20240626012456.2326192-5-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org> Signed-off-by: Mohammad Heib <mheib@redhat.com>
2025-06-04 09:58:06 +00:00
raise CmdExitFailure("Command failed: %s\nSTDOUT: %s\nSTDERR: %s" %
(self.proc.args, stdout, stderr))
class bkg(cmd):
def __init__(self, comm, shell=True, fail=None, ns=None, host=None,
exit_wait=False):
super().__init__(comm, background=True,
shell=shell, fail=fail, ns=ns, host=host)
self.terminate = not exit_wait
self.check_fail = fail
def __enter__(self):
return self
def __exit__(self, ex_type, ex_value, ex_tb):
return self.process(terminate=self.terminate, fail=self.check_fail)
selftests: drv-net: add ability to schedule cleanup with defer() JIRA: https://issues.redhat.com/browse/RHEL-89014 Conflict: - Small context diff due to unrelated code change commit 8510801a9dbd9f0d64079d7061d3452efc752550 Author: Jakub Kicinski <kuba@kernel.org> Date: Thu Jun 27 11:55:01 2024 -0700 selftests: drv-net: add ability to schedule cleanup with defer() This implements what I was describing in [1]. When writing a test author can schedule cleanup / undo actions right after the creation completes, eg: cmd("touch /tmp/file") defer(cmd, "rm /tmp/file") defer() takes the function name as first argument, and the rest are arguments for that function. defer()red functions are called in inverse order after test exits. It's also possible to capture them and execute earlier (in which case they get automatically de-queued). undo = defer(cmd, "rm /tmp/file") # ... some unsafe code ... undo.exec() As a nice safety all exceptions from defer()ed calls are captured, printed, and ignored (they do make the test fail, however). This addresses the common problem of exceptions in cleanup paths often being unhandled, leading to potential leaks. There is a global action queue, flushed by ksft_run(). We could support function level defers too, I guess, but there's no immediate need.. Link: https://lore.kernel.org/all/877cedb2ki.fsf@nvidia.com/ # [1] Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com> Reviewed-by: Petr Machata <petrm@nvidia.com> Link: https://patch.msgid.link/20240627185502.3069139-3-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org> Signed-off-by: Mohammad Heib <mheib@redhat.com>
2025-06-04 10:19:33 +00:00
global_defer_queue = []
class defer:
def __init__(self, func, *args, **kwargs):
global global_defer_queue
if not callable(func):
raise Exception("defer created with un-callable object, did you call the function instead of passing its name?")
self.func = func
self.args = args
self.kwargs = kwargs
self._queue = global_defer_queue
self._queue.append(self)
def __enter__(self):
return self
def __exit__(self, ex_type, ex_value, ex_tb):
return self.exec()
def exec_only(self):
self.func(*self.args, **self.kwargs)
def cancel(self):
self._queue.remove(self)
def exec(self):
self.cancel()
self.exec_only()
def tool(name, args, json=None, ns=None, host=None):
cmd_str = name + ' '
if json:
cmd_str += '--json '
cmd_str += args
selftests: drv-net: define endpoint structures JIRA: https://issues.redhat.com/browse/RHEL-57764 commit 1a20a9a0ddef17c0bd67eece34a7439b02a7b0ba Author: Jakub Kicinski <kuba@kernel.org> Date: Fri Apr 19 19:52:31 2024 -0700 selftests: drv-net: define endpoint structures Define the remote endpoint "model". To execute most meaningful device driver tests we need to be able to communicate with a remote system, and have it send traffic to the device under test. Various test environments will have different requirements. 0) "Local" netdevsim-based testing can simply use net namespaces. netdevsim supports connecting two devices now, to form a veth-like construct. 1) Similarly on hosts with multiple NICs, the NICs may be connected together with a loopback cable or internal device loopback. One interface may be placed into separate netns, and tests would proceed much like in the netdevsim case. Note that the loopback config or the moving of one interface into a netns is not expected to be part of selftest code. 2) Some systems may need to communicate with the remote endpoint via SSH. 3) Last but not least environment may have its own custom communication method. Fundamentally we only need two operations: - run a command remotely - deploy a binary (if some tool we need is built as part of kselftests) Wrap these two in a class. Use dynamic loading to load the Remote class. This will allow very easy definition of other communication methods without bothering upstream code base. Stick to the "simple" / "no unnecessary abstractions" model for referring to the remote endpoints. The host / remote object are passed as an argument to the usual cmd() or ip() invocation. For example: ip("link show", json=True, host=remote) Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://lore.kernel.org/r/20240420025237.3309296-2-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org> Signed-off-by: Ivan Vecera <ivecera@redhat.com>
2024-10-04 08:59:04 +00:00
cmd_obj = cmd(cmd_str, ns=ns, host=host)
if json:
return _json.loads(cmd_obj.stdout)
return cmd_obj
def ip(args, json=None, ns=None, host=None):
if ns:
args = f'-netns {ns} ' + args
return tool('ip', args, json=json, host=host)
def ethtool(args, json=None, ns=None, host=None):
return tool('ethtool', args, json=json, ns=ns, host=host)
def rand_port():
"""
Get a random unprivileged port, try to make sure it's not already used.
"""
for _ in range(1000):
port = random.randint(10000, 65535)
try:
with socket.socket(socket.AF_INET6, socket.SOCK_STREAM) as s:
s.bind(("", port))
return port
except OSError as e:
if e.errno != errno.EADDRINUSE:
raise
raise Exception("Can't find any free unprivileged port")
def wait_port_listen(port, proto="tcp", ns=None, host=None, sleep=0.005, deadline=5):
end = time.monotonic() + deadline
pattern = f":{port:04X} .* "
if proto == "tcp": # for tcp protocol additionally check the socket state
pattern += "0A"
pattern = re.compile(pattern)
while True:
data = cmd(f'cat /proc/net/{proto}*', ns=ns, host=host, shell=True).stdout
for row in data.split("\n"):
if pattern.search(row):
return
if time.monotonic() > end:
raise Exception("Waiting for port listen timed out")
time.sleep(sleep)