Python Procedures as Subprocess Commands (xonsh.proc
)¶
Interface for running Python functions as subprocess-mode commands.
Code for several helper methods in the ProcProxy class have been reproduced without modification from subprocess.py in the Python 3.4.2 standard library. The contents of subprocess.py (and, thus, the reproduced methods) are Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se> and were licensed to the Python Software foundation under a Contributor Agreement.
-
class
xonsh.proc.
BufferedFDParallelReader
(fd, buffer=None, chunksize=1024)[source]¶ Buffered, parallel background thread reader.
- Parameters
- fdint
File descriptor from which to read.
- bufferbinary file-like or None, optional
A buffer to write bytes into. If None, a new BytesIO object is created.
- chunksizeint, optional
The max size of the parallel reads, default 1 kb.
-
class
xonsh.proc.
CommandPipeline
(specs)[source]¶ Represents a subprocess-mode command pipeline.
- Parameters
- specslist of SubprocSpec
Process specifications
- Attributes
- specSubprocSpec
The last specification in specs
- procPopen-like
The process in procs
- endedbool
Boolean for if the command has stopped executing.
- inputstr
A string of the standard input.
output
strNon-blocking, lazy access to output
- errorsstr
A string of the standard error.
- lineslist of str
The output lines
- starttimefloats or None
Pipeline start timestamp.
-
end
(tee_output=True)[source]¶ End the pipeline, return the controlling terminal if needed.
Main things done in self._end().
-
itercheck
()[source]¶ Iterates through the command lines and throws an error if the returncode is non-zero.
-
tee_stdout
()[source]¶ Writes the process stdout to the output variable, line-by-line, and yields each line. This may optionally accept lines (in bytes) to iterate over, in which case it does not call iterraw().
-
property
alias
¶ Alias the process used.
-
property
args
¶ Arguments to the process.
-
attrnames
= ('stdin', 'stdout', 'stderr', 'pid', 'returncode', 'args', 'alias', 'stdin_redirect', 'stdout_redirect', 'stderr_redirect', 'timestamps', 'executed_cmd', 'input', 'output', 'errors')¶
-
property
err
¶ Error messages as a string.
-
property
executed_cmd
¶ The resolve and executed command.
-
property
inp
¶ Creates normalized input string from args.
-
nonblocking
= (<class '_io.BytesIO'>, <class 'xonsh.proc.NonBlockingFDReader'>, <class 'xonsh.proc.ConsoleParallelReader'>)¶
-
property
out
¶ Output value as a str.
-
property
output
¶ Non-blocking, lazy access to output
-
property
pid
¶ Process identifier.
-
property
returncode
¶ Process return code, waits until command is completed.
-
property
rtn
¶ Alias to return code.
-
property
stderr
¶ Process stderr.
-
property
stderr_postfix
¶ Postfix to print after stderr, as bytes.
-
property
stderr_prefix
¶ Prefix to print in front of stderr, as bytes.
-
property
stderr_redirect
¶ Redirection used for stderr.
-
property
stdin
¶ Process stdin.
-
property
stdin_redirect
¶ Redirection used for stdin.
-
property
stdout
¶ Process stdout.
-
property
stdout_redirect
¶ Redirection used for stdout.
-
property
timestamps
¶ The start and end time stamps.
-
class
xonsh.proc.
ConsoleParallelReader
(fd, buffer=None, chunksize=1024, timeout=None)[source]¶ Parallel reader for consoles that runs in a background thread. This is only needed, available, and useful on Windows.
- Parameters
- fdint
Standard buffer file descriptor, 0 for stdin, 1 for stdout (default), and 2 for stderr.
- bufferctypes.c_wchar_p, optional
An existing buffer to (re-)use.
- chunksizeint, optional
The max size of the parallel reads, default 1 kb.
- timeoutfloat, optional
The queue reading timeout.
-
class
xonsh.proc.
FileThreadDispatcher
(default=None)[source]¶ Dispatches to different file handles depending on the current thread. Useful if you want file operation to go to different places for different threads.
- Parameters
- defaultfile-like or None, optional
The file handle to write to if a thread cannot be found in the registry. If None, a new in-memory instance.
- Attributes
- registrydict
Maps thread idents to file handles.
-
register
(handle)[source]¶ Registers a file handle for the current thread. Returns self so that this method can be used in a with-statement.
-
write
(s)[source]¶ Writes to this thread’s handle. This also flushes, just to be extra sure the string was written.
-
property
available
¶ True if the thread is available in the registry.
-
property
buffer
¶ Gets the buffer for this thread’s handle.
-
property
closed
¶ Is the thread’s handle closed.
-
property
encoding
¶ Gets the encoding for this thread’s handle.
-
property
errors
¶ Gets the errors for this thread’s handle.
-
property
handle
¶ Gets the current handle for the thread.
-
property
line_buffering
¶ Gets if line buffering for this thread’s handle enabled.
-
property
newlines
¶ Gets the newlines for this thread’s handle.
-
class
xonsh.proc.
HiddenCommandPipeline
(specs)[source]¶ - Parameters
- specslist of SubprocSpec
Process specifications
- Attributes
- specSubprocSpec
The last specification in specs
- procPopen-like
The process in procs
- endedbool
Boolean for if the command has stopped executing.
- inputstr
A string of the standard input.
output
strNon-blocking, lazy access to output
- errorsstr
A string of the standard error.
- lineslist of str
The output lines
- starttimefloats or None
Pipeline start timestamp.
-
class
xonsh.proc.
NonBlockingFDReader
(fd, timeout=None)[source]¶ A class for reading characters from a file descriptor on a background thread. This has the advantages that the calling thread can close the file and that the reading does not block the calling thread.
- Parameters
- fdint
A file descriptor
- timeoutfloat or None, optional
The queue reading timeout.
-
class
xonsh.proc.
PopenThread
(*args, stdin=None, stdout=None, stderr=None, **kwargs)[source]¶ A thread for running and managing subprocess. This allows reading from the stdin, stdout, and stderr streams in a non-blocking fashion.
This takes the same arguments and keyword arguments as regular Popen. This requires that the captured_stdout and captured_stderr attributes to be set following instantiation.
This constructor should always be called with keyword arguments. Arguments are:
group should be None; reserved for future extension when a ThreadGroup class is implemented.
target is the callable object to be invoked by the run() method. Defaults to None, meaning nothing is called.
name is the thread name. By default, a unique name is constructed of the form “Thread-N” where N is a small decimal number.
args is the argument tuple for the target invocation. Defaults to ().
kwargs is a dictionary of keyword arguments for the target invocation. Defaults to {}.
If a subclass overrides the constructor, it must make sure to invoke the base class constructor (Thread.__init__()) before doing anything else to the thread.
-
run
()[source]¶ Runs the subprocess by performing a parallel read on stdin if allowed, and copying bytes from captured_stdout to stdout and bytes from captured_stderr to stderr.
-
wait
(timeout=None)[source]¶ Dispatches to Popen.wait(), but also does process cleanup such as joining this thread and replacing the original window size signal handler.
-
property
returncode
¶ Process return code.
-
property
signal
¶ Process signal, or None.
-
-
class
xonsh.proc.
PrevProcCloser
(pipeline)[source]¶ Previous process closer thread for pipelines whose last command is itself unthreadable. This makes sure that the pipeline is driven forward and does not deadlock.
- Parameters
- pipelineCommandPipeline
The pipeline whose prev procs we should close.
-
class
xonsh.proc.
ProcProxy
(f, args, stdin=None, stdout=None, stderr=None, universal_newlines=False, close_fds=False, env=None)[source]¶ This is process proxy class that runs its alias functions on the same thread that it was called from, which is typically the main thread. This prevents the process from running on a background thread, but enables debugger and profiler tools (functions) be run on the same thread that they are attempting to debug.
-
class
xonsh.proc.
ProcProxyThread
(f, args, stdin=None, stdout=None, stderr=None, universal_newlines=False, close_fds=False, env=None)[source]¶ Class representing a function to be run as a subprocess-mode command.
- Parameters
- ffunction
The function to be executed.
- argslist
A (possibly empty) list containing the arguments that were given on the command line
- stdinfile-like, optional
A file-like object representing stdin (input can be read from here). If stdin is not provided or if it is explicitly set to None, then an instance of io.StringIO representing an empty file is used.
- stdoutfile-like, optional
A file-like object representing stdout (normal output can be written here). If stdout is not provided or if it is explicitly set to None, then sys.stdout is used.
- stderrfile-like, optional
A file-like object representing stderr (error output can be written here). If stderr is not provided or if it is explicitly set to None, then sys.stderr is used.
- universal_newlinesbool, optional
Whether or not to use universal newlines.
- close_fdsbool, optional
Whether or not to close file descriptors. This is here for Popen compatability and currently does nothing.
- envMapping, optional
Environment mapping.
-
poll
()[source]¶ Check if the function has completed.
- Returns
- None if the function is still executing, and the returncode otherwise
-
class
xonsh.proc.
QueueReader
(fd, timeout=None)[source]¶ Provides a file-like interface to reading from a queue.
- Parameters
- fdint
A file descriptor
- timeoutfloat or None, optional
The queue reading timeout.
-
xonsh.proc.
parse_proxy_return
(r, stdout, stderr)[source]¶ Proxies may return a variety of outputs. This handles them generally.
- Parameters
- rtuple, str, int, or None
Return from proxy function
- stdoutfile-like
Current stdout stream
- stdoutfile-like
Current stderr stream
- Returns
- cmd_resultint
The return code of the proxy
-
xonsh.proc.
partial_proxy
(f)[source]¶ Dispatches the appropriate proxy function based on the number of args.
-
xonsh.proc.
pause_call_resume
(p, f, *args, **kwargs)[source]¶ For a process p, this will call a function f with the remaining args and and kwargs. If the process cannot accept signals, the function will be called.
- Parameters
- pPopen object or similar
- fcallable
- argsremaining arguments
- kwargskeyword arguments
-
xonsh.proc.
populate_buffer
(reader, fd, buffer, chunksize)[source]¶ Reads bytes from the file descriptor and copies them into a buffer.
The reads happen in parallel using the pread() syscall; which is only available on POSIX systems. If the read fails for any reason, the reader is flagged as closed.
-
xonsh.proc.
populate_console
(reader, fd, buffer, chunksize, queue, expandsize=None)[source]¶ Reads bytes from the file descriptor and puts lines into the queue. The reads happened in parallel, using xonsh.winutils.read_console_output_character(), and is thus only available on windows. If the read fails for any reason, the reader is flagged as closed.
-
xonsh.proc.
populate_fd_queue
(reader, fd, queue)[source]¶ Reads 1 kb of data from a file descriptor into a queue. If this ends or fails, it flags the calling reader object as closed.
-
xonsh.proc.
proxy_five
(f, args, stdin, stdout, stderr, spec, stack)[source]¶ Calls a proxy function which takes four parameter: args, stdin, stdout, stderr, and spec.
-
xonsh.proc.
proxy_four
(f, args, stdin, stdout, stderr, spec, stack)[source]¶ Calls a proxy function which takes four parameter: args, stdin, stdout, and stderr.
-
xonsh.proc.
proxy_one
(f, args, stdin, stdout, stderr, spec, stack)[source]¶ Calls a proxy function which takes one parameter: args
-
xonsh.proc.
proxy_three
(f, args, stdin, stdout, stderr, spec, stack)[source]¶ Calls a proxy function which takes three parameter: args, stdin, stdout.
-
xonsh.proc.
proxy_two
(f, args, stdin, stdout, stderr, spec, stack)[source]¶ Calls a proxy function which takes two parameter: args and stdin.
-
xonsh.proc.
proxy_zero
(f, args, stdin, stdout, stderr, spec, stack)[source]¶ Calls a proxy function which takes no parameters.
-
xonsh.proc.
safe_fdclose
(handle, cache=None)[source]¶ Closes a file handle in the safest way possible, and potentially storing the result.
-
xonsh.proc.
safe_flush
(handle)[source]¶ Attempts to safely flush a file handle, returns success bool.
-
xonsh.proc.
safe_readable
(handle)[source]¶ Attempts to find if the handle is readable without throwing an error.
-
xonsh.proc.
safe_readlines
(handle, hint=- 1)[source]¶ Attempts to read lines without throwing an error.