pytest multihost plugin doc

pytest_multihost.config

Utilities for configuration of multi-master tests

class pytest_multihost.config.Config(**kwargs)

Container for global configuration and a list of Domains

See README for an overview of the core classes.

Methods

filter(descriptions) Destructively filters hosts and orders domains to fit description
from_dict(dct) Load a Config object from a dict
get_domain_class()
get_logger(name) Get a logger of the given name
host_by_name(name) Get a host from any domain by name
to_dict([_autosave_names]) Save this Config object to a dict compatible with from_dict
__init__(**kwargs)
filter(descriptions)

Destructively filters hosts and orders domains to fit description

Parameters:descriptions

List of dicts such as:

[
{
‘type’: ‘ipa’, ‘hosts’: {
‘master’: 1, ‘replica’: 2,

},

},

]

i.e. the “type” is a type of domain, and “hosts” a dict mapping host roles to the number of hosts of this role that are required.

classmethod from_dict(dct)

Load a Config object from a dict

The dict is usually loaded from an user-supplied YAML or JSON file.

In the base implementation, the dict is just passed to the constructor. If more arguments are needed, include them in the class’ extra_init_args set.

get_logger(name)

Get a logger of the given name

Override in subclasses to use a custom logging system

host_by_name(name)

Get a host from any domain by name

If multiple hosts have the same name, return the first one. Raise LookupError if no host is found.

See Domain.host_by_name for details on matching.

to_dict(_autosave_names=())

Save this Config object to a dict compatible with from_dict

Parameters:_autosave_names – To be used by subclasses only. Lists names that should be included in the dict. Values are taken from attributes of the same name. Usually this is a subset of the class’ extra_init_args
class pytest_multihost.config.Domain(config, name, domain_type)

Configuration for a domain

See README for an overview of the core classes.

Attributes

extra_roles Roles of this Domain’s hosts that aren’t included in static_roles
host_classes
roles All the roles of the hosts in this domain
static_roles Roles typical for this domain type

Methods

filter(host_counts) Destructively filter hosts in this domain
fits(description) Return True if the this fits the description
from_dict(dct, config) Load this Domain from a dict
get_host_class(host_dict)
host_by_name(name) Return a host with the given name
host_by_role(role) Return the first host of the given role
hosts_by_role(role) Return all hosts of the given role
to_dict() Export this Domain from a dict
__init__(config, name, domain_type)
extra_roles

Roles of this Domain’s hosts that aren’t included in static_roles

filter(host_counts)

Destructively filter hosts in this domain

Parameters:host_counts – Mapping of host role to number of hosts wanted for that role

All extra hosts are removed from this Domain.

fits(description)

Return True if the this fits the description

See Domain.filter for discussion of the description.

classmethod from_dict(dct, config)

Load this Domain from a dict

host_by_name(name)

Return a host with the given name

Checks all of: hostname, external_hostname, shortname.

If more hosts match, returns the first one. Raises LookupError if no host is found.

host_by_role(role)

Return the first host of the given role

hosts_by_role(role)

Return all hosts of the given role

roles

All the roles of the hosts in this domain

static_roles

Roles typical for this domain type

To be overridden in subclasses

to_dict()

Export this Domain from a dict

exception pytest_multihost.config.FilterError

Raised when domains description could not be satisfied

pytest_multihost.plugin

class pytest_multihost.plugin.MultihostFixture(config, request)

A fixture containing the multihost testing configuration

Contains the config; other attributes may be added to it for convenience.

Methods

install() Call install()/uninstall() for the class this fixture is used on
__init__(config, request)
install()

Call install()/uninstall() for the class this fixture is used on

This function is DEPRECATED.

class pytest_multihost.plugin.MultihostPlugin(confdict)

The Multihost plugin

The plugin is available as pluginmanager.getplugin(‘MultihostPlugin’), and its presence indicates that multihost testing has been configured.

__init__(confdict)
pytest_multihost.plugin.make_multihost_fixture(request, descriptions, config_class=<class 'pytest_multihost.config.Config'>, _config=None)

Create a MultihostFixture, or skip the test

Parameters:
  • request – The Pytest request object
  • descriptions – Descriptions of wanted domains (see README or Domain.filter)
  • config_class – Custom Config class to use
  • _config – Config to be used directly. Intended mostly for testing the plugin itself.

Skips the test if there are not enough resources configured.

pytest_multihost.transport

Objects for communicating with remote hosts

This class defines “SSHTransport” as ParamikoTransport (by default), or as OpenSSHTransport (if Paramiko is not importable, or the PYTESTMULTIHOST_SSH_TRANSPORT environment variable is set to “openssh”).

class pytest_multihost.transport.Command(argv, logger_name=None, log_stdout=True, get_logger=None)

A Popen-style object representing a remote command

Instances of this class should only be created via method of a concrete Transport, such as start_shell.

The standard error and output are handled by this class. They’re not available for file-like reading, and are logged by default. To make sure reading doesn’t stall after one buffer fills up, they are read in parallel using threads.

After calling wait(), stdout_text and stderr_text attributes will be strings containing the output, and returncode will contain the exit code.

Methods

wait([raiseonerr]) Wait for the remote process to exit
__init__(argv, logger_name=None, log_stdout=True, get_logger=None)
wait(raiseonerr=True)

Wait for the remote process to exit

Raises an excption if the exit code is not 0, unless raiseonerr is true.

class pytest_multihost.transport.OpenSSHTransport(host)

Transport that uses the ssh binary

Methods

file_exists(path)
get_file(remotepath, localpath) Copy a file from the remote host to a local file
get_file_contents(filename[, encoding])
get_next_command_logger_name()
mkdir(path)
mkdir_recursive(path) mkdir -p on the remote host
put_file(localpath, remotepath) Copy a local file to the remote host
put_file_contents(filename, contents)
remove_file(filepath)
rename_file(oldpath, newpath)
rmdir(path)
start_shell(argv[, log_stdout])
__init__(host)
class pytest_multihost.transport.ParamikoTransport(host)

Transport that uses the Paramiko SSH2 library

Attributes

sftp Paramiko SFTPClient connected to this host

Methods

file_exists(filename) Return true if the named remote file exists
get_file(remotepath, localpath)
get_file_contents(filename[, encoding]) Read the named remote file and return the contents as a string
get_next_command_logger_name()
mkdir(path)
mkdir_recursive(path) mkdir -p on the remote host
put_file(localpath, remotepath)
put_file_contents(filename, contents) Write the given string to the named remote file
remove_file(filepath)
rename_file(oldpath, newpath)
rmdir(path)
sftp_open(*args, **kwds) Context manager that provides a file-like object over a SFTP channel
start_shell(argv[, log_stdout, get_pty])
__init__(host)
file_exists(filename)

Return true if the named remote file exists

get_file_contents(filename, encoding=None)

Read the named remote file and return the contents as a string

put_file_contents(filename, contents)

Write the given string to the named remote file

sftp

Paramiko SFTPClient connected to this host

sftp_open(*args, **kwds)

Context manager that provides a file-like object over a SFTP channel

This provides compatibility with older Paramiko versions. (In Paramiko 1.10+, file objects from sftp.open are directly usable as context managers).

class pytest_multihost.transport.SSHCallWrapper(command)

Adapts a /usr/bin/ssh call to the paramiko.Channel interface

This only wraps what SSHCommand needs.

Methods

close()
invoke_shell()
makefile(mode)
makefile_stderr(mode)
recv_exit_status()
shutdown_write()
__init__(command)
class pytest_multihost.transport.SSHCommand(ssh, argv, logger_name, log_stdout=True, collect_output=True, encoding='utf-8', get_logger=None, get_pty=False)

Command implementation for ParamikoTransport and OpenSSHTranspport

Methods

wait([raiseonerr]) Wait for the remote process to exit
__init__(ssh, argv, logger_name, log_stdout=True, collect_output=True, encoding='utf-8', get_logger=None, get_pty=False)
pytest_multihost.transport.SSHTransport

alias of ParamikoTransport

class pytest_multihost.transport.Transport(host)

Mechanism for communicating with remote hosts

The Transport can manipulate files on a remote host, and open a Command.

The base class defines an interface that specific subclasses implement.

Methods

file_exists(filename) Return true if the named remote file exists
get_file(remotepath, localpath) Copy a file from the remote host to a local file
get_file_contents(filename[, encoding]) Read the named remote file and return the contents as a string
get_next_command_logger_name()
mkdir(path) Make the named directory
mkdir_recursive(path) mkdir -p on the remote host
put_file(localpath, remotepath) Copy a local file to the remote host
put_file_contents(filename, contents) Write the given string to the named remote file
remove_file(filepath) Removes files
rename_file(oldpath, newpath) Rename file
rmdir(path) Remove directory
start_shell(argv[, log_stdout]) Start a Shell
__init__(host)
file_exists(filename)

Return true if the named remote file exists

get_file(remotepath, localpath)

Copy a file from the remote host to a local file

get_file_contents(filename, encoding=None)

Read the named remote file and return the contents as a string

mkdir(path)

Make the named directory

mkdir_recursive(path)

mkdir -p on the remote host

put_file(localpath, remotepath)

Copy a local file to the remote host

put_file_contents(filename, contents)

Write the given string to the named remote file

remove_file(filepath)

Removes files

rename_file(oldpath, newpath)

Rename file

rmdir(path)

Remove directory

start_shell(argv, log_stdout=True)

Start a Shell

Parameters:
  • argv – The command this shell is intended to run (used for logging only)
  • log_stdout – If false, the stdout will not be logged (useful when binary output is expected)

Given a shell from this method, the caller can then use shell.stdin.write() to input any command(s), call shell.wait() to let the command run, and then inspect returncode, stdout_text or stderr_text.

pytest_multihost.util

class pytest_multihost.util.TempDir

Handle for a temporary directory that’s deleted on garbage collection

__init__()
pytest_multihost.util.check_config_dict_empty(dct, name)

Ensure that no keys are left in a configuration dict

pytest_multihost.util.shell_quote(string)

Quotes a string for the Bash shell

pytest_multihost.host

Host class for integration testing

class pytest_multihost.host.BaseHost(domain, hostname, role, ip=None, external_hostname=None, username=None, password=None, test_dir=None, host_type=None)

Representation of a remote host

See README for an overview of the core classes.

Attributes

config The Config that this Host is a part of
transport Provides means to manipulate files & run processs on the remote host

Methods

add_log_collector(collector) Register a log collector for this host
collect_log(filename) Call all registered log collectors on the given filename
from_dict(dct, domain) Load this Host from a dict
get_file_contents(filename[, encoding]) Shortcut for transport.get_file_contents
put_file_contents(filename, contents) Shortcut for transport.put_file_contents
remove_log_collector(collector) Unregister a log collector
reset_connection() Reset the connection
run_command(argv[, set_env, stdin_text, ...]) Run the given command on this host
to_dict() Export info about this Host to a dict
transport_class alias of ParamikoTransport
__init__(domain, hostname, role, ip=None, external_hostname=None, username=None, password=None, test_dir=None, host_type=None)
add_log_collector(collector)

Register a log collector for this host

collect_log(filename)

Call all registered log collectors on the given filename

config

The Config that this Host is a part of

classmethod from_dict(dct, domain)

Load this Host from a dict

get_file_contents(filename, encoding=None)

Shortcut for transport.get_file_contents

put_file_contents(filename, contents)

Shortcut for transport.put_file_contents

remove_log_collector(collector)

Unregister a log collector

reset_connection()

Reset the connection

The next time a connection is needed, a new Transport object will be made. This new transport will take into account any configuration changes, such as external_hostname, ssh_username, etc., that were made on the Host.

run_command(argv, set_env=True, stdin_text=None, log_stdout=True, raiseonerr=True, cwd=None, get_pty=False)

Run the given command on this host

Returns a Command instance. The command will have already run in the shell when this method returns, so its stdout_text, stderr_text, and returncode attributes will be available.

Parameters:
  • argv – Command to run, as either a Popen-style list, or a string containing a shell script
  • set_env – If true, env.sh exporting configuration variables will be sourced before running the command.
  • stdin_text – If given, will be written to the command’s stdin
  • log_stdout – If false, standard output will not be logged (but will still be available as cmd.stdout_text)
  • raiseonerr – If true, an exception will be raised if the command does not exit with return code 0
  • cwd – The working directory for the command
  • get_pty – If True, request a pseudo-terminal from the server.
to_dict()

Export info about this Host to a dict

transport

Provides means to manipulate files & run processs on the remote host

Accessing this property might connect to the remote Host (usually via SSH).

transport_class

alias of ParamikoTransport

class pytest_multihost.host.Host(domain, hostname, role, ip=None, external_hostname=None, username=None, password=None, test_dir=None, host_type=None)

A Unix host

Attributes

config The Config that this Host is a part of
transport Provides means to manipulate files & run processs on the remote host

Methods

add_log_collector(collector) Register a log collector for this host
collect_log(filename) Call all registered log collectors on the given filename
from_dict(dct, domain) Load this Host from a dict
get_file_contents(filename[, encoding]) Shortcut for transport.get_file_contents
put_file_contents(filename, contents) Shortcut for transport.put_file_contents
remove_log_collector(collector) Unregister a log collector
reset_connection() Reset the connection
run_command(argv[, set_env, stdin_text, ...]) Run the given command on this host
to_dict() Export info about this Host to a dict
transport_class alias of ParamikoTransport
class pytest_multihost.host.WinHost(domain, hostname, role, **kwargs)

Representation of a remote Windows host.

Attributes

config The Config that this Host is a part of
transport Provides means to manipulate files & run processs on the remote host

Methods

add_log_collector(collector) Register a log collector for this host
collect_log(filename) Call all registered log collectors on the given filename
from_dict(dct, domain) Load this Host from a dict
get_file_contents(filename[, encoding]) Shortcut for transport.get_file_contents
put_file_contents(filename, contents) Shortcut for transport.put_file_contents
remove_log_collector(collector) Unregister a log collector
reset_connection() Reset the connection
run_command(argv[, set_env, stdin_text, ...]) Run the given command on this host
to_dict() Export info about this Host to a dict
transport_class alias of ParamikoTransport
__init__(domain, hostname, role, **kwargs)