Basic executors

Executor with the core functionality.

mirakuru.base.ENV_UUID = 'mirakuru_uuid'

Name of the environment variable used by mirakuru to mark its subprocesses.

class mirakuru.base.Executor(command, cwd=None, shell=False, timeout=3600, sleep=0.1, sig_stop=<Signals.SIGTERM: 15>, sig_kill=<Signals.SIGKILL: 9>, envvars=None, stdin=-1, stdout=-1, stderr=None)

Bases: mirakuru.base.SimpleExecutor

Base class for executors with a pre- and after-start checks.

Initialize executor.

Parameters:

• list) command ((str,) – command to be run by the subprocess
• cwd (str) – current working directory to be set for executor
• shell (bool) – same as the subprocess.Popen shell definition. On Windows always set to True.
• timeout (int) – number of seconds to wait for the process to start or stop.
• sleep (float) – how often to check for start/stop condition
• sig_stop (int) – signal used to stop process run by the executor. default is signal.SIGTERM
• sig_kill (int) – signal used to kill process run by the executor. default is signal.SIGKILL (signal.SIGTERM on Windows)
• envvars (dict) – Additional environment variables
• stdin (int) – file descriptor for stdin
• stdout (int) – file descriptor for stdout
• stderr (int) – file descriptor for stderr

Note

timeout set for an executor is valid for all the level of waits on the way up. That means that if some more advanced executor establishes the timeout to 10 seconds and it will take 5 seconds for the first check, second check will only have 5 seconds left.

Your executor will raise an exception if something goes wrong during this time. The default value of timeout is None, so it is a good practice to set this.

after_start_check()

Check process after the start of executor.

Should be overridden in order to return boolean value if executor can be treated as started. :rtype: bool

check_subprocess()

Make sure the process didn’t exit with an error and run the checks.

Return type:bool Returns:the actual check status Raises:ProcessExitedWithError – when the main process exits with an error

pre_start_check()

Check process before the start of executor.

Should be overridden in order to return True when some other executor (or process) has already started with the same configuration. :rtype: bool

start()

Start executor with additional checks.

Checks if previous executor isn’t running then start process (executor) and wait until it’s started. :returns: itself :rtype: Executor

class mirakuru.base.SimpleExecutor(command, cwd=None, shell=False, timeout=3600, sleep=0.1, sig_stop=<Signals.SIGTERM: 15>, sig_kill=<Signals.SIGKILL: 9>, envvars=None, stdin=-1, stdout=-1, stderr=None)

Bases: object

Simple subprocess executor with start/stop/kill functionality.

Initialize executor.

Parameters:

• list) command ((str,) – command to be run by the subprocess
• cwd (str) – current working directory to be set for executor
• shell (bool) – same as the subprocess.Popen shell definition. On Windows always set to True.
• timeout (int) – number of seconds to wait for the process to start or stop.
• sleep (float) – how often to check for start/stop condition
• sig_stop (int) – signal used to stop process run by the executor. default is signal.SIGTERM
• sig_kill (int) – signal used to kill process run by the executor. default is signal.SIGKILL (signal.SIGTERM on Windows)
• envvars (dict) – Additional environment variables
• stdin (int) – file descriptor for stdin
• stdout (int) – file descriptor for stdout
• stderr (int) – file descriptor for stderr

Note

timeout set for an executor is valid for all the level of waits on the way up. That means that if some more advanced executor establishes the timeout to 10 seconds and it will take 5 seconds for the first check, second check will only have 5 seconds left.

Your executor will raise an exception if something goes wrong during this time. The default value of timeout is None, so it is a good practice to set this.

_clear_process()

Close stdin/stdout of subprocess.

It is required because of ResourceWarning in Python 3.

_kill_all_kids(sig)

Kill all subprocesses (and its subprocesses) that executor started.

This function tries to kill all leftovers in process tree that current executor may have left. It uses environment variable to recognise if process have origin in this Executor so it does not give 100 % and some daemons fired by subprocess may still be running.

Parameters:sig (int) – signal used to stop process run by executor. Returns:process ids (pids) of killed processes

:rtype list

_popen_kwargs

Get kwargs for the process instance.

Note

We want to open stdin, stdout and stderr as text streams in universal newlines mode, so we have to set universal_newlines to True.

Returns:

_set_timeout()

Set timeout for possible wait.

check_timeout()

Check if timeout has expired.

Returns True if there is no timeout set or the timeout has not expired. Kills the process and raises TimeoutExpired exception otherwise.

This method should be used in while loops waiting for some data.

Returns:True if timeout expired, False if not Return type:bool

command = None

Command that the executor runs.

err_output()

Return subprocess stderr.

kill(wait=True, sig=None)

Kill the process if running.

Parameters:

• wait (bool) – set to True to wait for the process to end, or False, to simply proceed after sending signal.
• sig (int) – signal used to kill process run by the executor. None by default.

Returns:

itself

Return type:

SimpleExecutor

output()

Return subprocess output.

process = None

A subprocess.Popen instance once process is started.

running()

Check if executor is running.

Returns:True if process is running, False otherwise Return type:bool

start()

Start defined process.

After process gets started, timeout countdown begins as well.

Returns:itself Return type:SimpleExecutor

stop(sig=None)

Stop process running.

Wait 10 seconds for the process to end, then just kill it.

Parameters:sig (int) – signal used to stop process run by executor. None for default. Returns:itself Return type:SimpleExecutor

Note

When gathering coverage for the subprocess in tests, you have to allow subprocesses to end gracefully.

stopped()

Stop process for given context and starts it afterwards.

Allows for easier writing resistance integration tests whenever one of the service fails. :yields: itself :rtype: SimpleExecutor

wait_for(wait_for)

Wait for callback to return True.

Simply returns if wait_for condition has been met, raises TimeoutExpired otherwise and kills the process.

Parameters:wait_for (callback) – callback to call Raises:mirakuru.exceptions.TimeoutExpired Returns:itself Return type:SimpleExecutor

mirakuru.base.cleanup_subprocesses()

On python exit: find possibly running subprocesses and kill them.

Executor that awaits for appearance of a predefined banner in output.

class mirakuru.output.OutputExecutor(command, banner, **kwargs)

Bases: mirakuru.base.SimpleExecutor

Executor that awaits for string output being present in output.

Initialize OutputExecutor executor.

Parameters:

• list) command ((str,) – command to be run by the subprocess
• banner (str) – string that has to appear in process output - should compile to regular expression.
• shell (bool) – same as the subprocess.Popen shell definition
• timeout (int) – number of seconds to wait for the process to start or stop. If None or False, wait indefinitely.
• sleep (float) – how often to check for start/stop condition
• sig_stop (int) – signal used to stop process run by the executor. default is signal.SIGTERM
• sig_kill (int) – signal used to kill process run by the executor. default is signal.SIGKILL (signal.SIGTERM on Windows)

_wait_for_output(*polls)

Check if output matches banner.

Warning

Waiting for I/O completion. It does not work on Windows. Sorry.

start()

Start process.

Returns:itself Return type:OutputExecutor

Note

Process will be considered started, when defined banner will appear in process output.

TCP executor definition.

class mirakuru.tcp.TCPExecutor(command, host, port, **kwargs)

Bases: mirakuru.base.Executor

TCP-listening process executor.

Used to start (and wait to actually be running) processes that can accept TCP connections.

Initialize TCPExecutor executor.

Parameters:

• list) command ((str,) – command to be run by the subprocess
• host (str) – host under which process is accessible
• port (int) – port under which process is accessible
• shell (bool) – same as the subprocess.Popen shell definition
• timeout (int) – number of seconds to wait for the process to start or stop. If None or False, wait indefinitely.
• sleep (float) – how often to check for start/stop condition
• sig_stop (int) – signal used to stop process run by the executor. default is signal.SIGTERM
• sig_kill (int) – signal used to kill process run by the executor. default is signal.SIGKILL (signal.SIGTERM on Windows)

after_start_check()

Check if process accepts connections.

Note

Process will be considered started, when it’ll be able to accept TCP connections as defined in initializer.

host = None

Host name, process is listening on.

port = None

Port number, process is listening on.

pre_start_check()

Check if process accepts connections.

Note

Process will be considered started, when it’ll be able to accept TCP connections as defined in initializer.

HTTP enabled process executor.

class mirakuru.http.HTTPExecutor(command, url, status='^2\d\d$', **kwargs)

Bases: mirakuru.tcp.TCPExecutor

Http enabled process executor.

Initialize HTTPExecutor executor.

Parameters:

• list) command ((str,) – command to be run by the subprocess
• url (str) – URL that executor checks to verify if process has already started.
• shell (bool) – same as the subprocess.Popen shell definition
• status (str|int) – HTTP status code(s) that an endpoint must return for the executor being considered as running. This argument is interpreted as a single status code - e.g. ‘200’ or ‘404’ but also it can be a regular expression - e.g. ‘4..’ or ‘(200|404)’. Default: any 2XX HTTP status code.
• timeout (int) – number of seconds to wait for the process to start or stop. If None or False, wait indefinitely.
• sleep (float) – how often to check for start/stop condition
• sig_stop (int) – signal used to stop process run by the executor. default is signal.SIGTERM
• sig_kill (int) – signal used to kill process run by the executor. default is signal.SIGKILL

DEFAULT_PORT = 80

Default TCP port for the HTTP protocol.

after_start_check()

Check if defined URL returns expected status to a HEAD request.

url = None

An urlparse.urlparse() representation of an url.

It’ll be used to check process status on.

Pid executor definition.

class mirakuru.pid.PidExecutor(command, filename, **kwargs)

Bases: mirakuru.base.Executor

File existence checking process executor.

Used to start processes that create pid files (or any other for that matter). Starts the given process and waits for the given file to be created.

Initialize the PidExecutor executor.

If the filename is empty, a ValueError is thrown.

Parameters:

• list) command ((str,) – command to be run by the subprocess
• filename (str) – the file which is to exist
• shell (bool) – same as the subprocess.Popen shell definition
• timeout (int) – number of seconds to wait for the process to start or stop. If None or False, wait indefinitely.
• sleep (float) – how often to check for start/stop condition
• sig_stop (int) – signal used to stop process run by the executor. default is signal.SIGTERM
• sig_kill (int) – signal used to kill process run by the executor. default is signal.SIGKILL (signal.SIGTERM on Windows)

Raises:

ValueError

after_start_check()

Check if the process has created the specified file.

Note

The process will be considered started when it will have created the specified file as defined in the initializer.

filename = None

the name of the file which the process is to create.

pre_start_check()

Check if the specified file has been created.

Note

The process will be considered started when it will have created the specified file as defined in the initializer.