pyina scripts documentation

ezpool script

helper script for pyina.mpi maps using the ‘worker pool’ strategy

Notes

this uses the same code as ezscatter, but with pyina.mpi_pool.

Warning

this is a helper script for pyina.mpi.Mapper – don’t use it directly.

_debug(boolean)

print debug statements

ezscatter script

helper script for pyina.mpi maps using the ‘scatter gather’ strategy

Notes

this uses the same code as ezpool, but with pyina.mpi_scatter.

Warning

this is a helper script for pyina.mpi.Mapper – don’t use it directly.

_debug(boolean)

print debug statements

mpi_world script

setup/query/kill the MPI environment

Notes

Commandline options are:

  • --help [prints this message]

  • --workers nodes [set mpi world (nodes is a list of worker nodes)]

  • --fetch N [get rank and names of ‘N’ worker nodes]

  • --kill [tear down mpi world]

'mpd &' must be run before setting the worker nodes.

Examples:

$ mpi_world --workers n00 n01 n02 n03
seting up mpi...

$ mpi_world --fetch 4
Rank: 0 -- n00.borel.local
Rank: 1 -- n01.borel.local
Rank: 3 -- n03.borel.local
Rank: 2 -- n02.borel.local
class Popen(args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=True, shell=False, cwd=None, env=None, universal_newlines=None, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, user=None, group=None, extra_groups=None, encoding=None, errors=None, text=None, umask=-1, pipesize=-1)

Bases: object

Execute a child program in a new process.

For a complete description of the arguments see the Python documentation.

Parameters:

  • args – A string, or a sequence of program arguments.

  • bufsize – supplied as the buffering argument to the open() function when creating the stdin/stdout/stderr pipe file objects

  • executable – A replacement program to execute.

  • stdin – These specify the executed programs’ standard input, standard output and standard error file handles, respectively.

  • stderr (stdout and) – These specify the executed programs’ standard input, standard output and standard error file handles, respectively.

  • preexec_fn – (POSIX only) An object to be called in the child process just before the child is executed.

  • close_fds – Controls closing or inheriting of file descriptors.

  • shell – If true, the command will be executed through the shell.

  • cwd – Sets the current directory before the child is executed.

  • env – Defines the environment variables for the new process.

  • text – If true, decode stdin, stdout and stderr using the given encoding (if set) or the system default otherwise.

  • universal_newlines – Alias of text, provided for backwards compatibility.

  • creationflags (startupinfo and) –

  • restore_signals (POSIX only) –

  • start_new_session (POSIX only) –

  • group (POSIX only) –

  • extra_groups (POSIX only) –

  • user (POSIX only) –

  • umask (POSIX only) –

  • pass_fds (POSIX only) –

  • errors (encoding and) – Text mode encoding and error handling to use for file objects stdin, stdout and stderr.

Variables:

returncode (stdin, stdout, stderr, pid,) –

Create new Popen instance.

__class_getitem__ = <bound method GenericAlias of <class 'subprocess.Popen'>>
__del__(_maxsize=9223372036854775807, _warn=<built-in function warn>)
__enter__()
__exit__(exc_type, value, traceback)
__repr__()

Return repr(self).

_check_timeout(endtime, orig_timeout, stdout_seq, stderr_seq, skip_check_and_raise=False)

Convenience for checking if a timeout has expired.

_child_created = False
_close_pipe_fds(p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite)
_communicate(input, endtime, orig_timeout)
_execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, gid, gids, uid, umask, start_new_session)

Execute program (POSIX version)

_get_devnull()
_get_handles(stdin, stdout, stderr)

Construct and return tuple with IO objects: p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite

_handle_exitstatus(sts, waitstatus_to_exitcode=<built-in function waitstatus_to_exitcode>, _WIFSTOPPED=<built-in function WIFSTOPPED>, _WSTOPSIG=<built-in function WSTOPSIG>)

All callers to this function MUST hold self._waitpid_lock.

_internal_poll(_deadstate=None, _waitpid=<built-in function waitpid>, _WNOHANG=1, _ECHILD=10)

Check if child process has terminated. Returns returncode attribute.

This method is called by __del__, so it cannot reference anything outside of the local scope (nor can any methods it calls).

_posix_spawn(args, executable, env, restore_signals, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite)

Execute program using os.posix_spawn().

_remaining_time(endtime)

Convenience for _communicate when computing timeouts.

_save_input(input)
_stdin_write(input)
_translate_newlines(data, encoding, errors)
_try_wait(wait_flags)

All callers to this function MUST hold self._waitpid_lock.

_wait(timeout)

Internal implementation of wait() on POSIX.

communicate(input=None, timeout=None)

Interact with process: Send data to stdin and close it. Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate.

The optional “input” argument should be data to be sent to the child process, or None, if no data should be sent to the child. communicate() returns a tuple (stdout, stderr).

By default, all communication is in bytes, and therefore any “input” should be bytes, and the (stdout, stderr) will be bytes. If in text mode (indicated by self.text_mode), any “input” should be a string, and (stdout, stderr) will be strings decoded according to locale encoding, or by “encoding” if set. Text mode is triggered by setting any of text, encoding, errors or universal_newlines.

kill()

Kill the process with SIGKILL

poll()

Check if child process has terminated. Set and return returncode attribute.

send_signal(sig)

Send a signal to the process.

terminate()

Terminate the process with SIGTERM

property universal_newlines
wait(timeout=None)

Wait for child process to terminate; returns self.returncode.

host(id)