xonsh.tools¶
Shared utility functions and helpers for the xonsh shell.
This module is the grab-bag of small, broadly-useful helpers used across
the xonsh codebase — anything that doesn’t naturally fit into one of the
more focused subsystems (parser, execer, environ, procs, completers,
prompt, history) ends up here. Downstream code — including xontribs —
can import from xonsh.tools as a stable public API.
The module groups its helpers roughly by topic:
Exceptions and error handling —
XonshError,XonshCalledProcessError,print_exception(),format_std_prepost().Type conversion and validation — the
is_*/to_*/*_or_defaultfamily used byEnvto validate and normalise environment variables (bools, ints, paths, lists, dicts, history units, log levels, color schemes, etc.).Path and file utilities —
expand_path(),expanduser_abs_path(),is_writable_file(),executables_in(),iglobpath(), cross-platform path normalisation helpers.String, token and AST helpers — quoting/escaping,
subexpr_from_unbalanced(),find_next_break(), regex constants for string prefixes and history tuples, theFlexibleFormatterfor prompt/format-string rendering.Subprocess and signal helpers —
on_main_thread(),suggest_commands(), signal-safe context managers, PTY helpers.Terminal and color handling — ANSI/ANSI256/RGB color parsing,
PTK_NEW_OLD_COLOR_MAP,WIN10_COLOR_MAPand related lookup tables,print_color()/format_color().Collections and functional helpers —
LazyObject,LazyDict,DefaultNotGivenType,always_true,always_false,all_permutations, small functional shims.Platform and environment probes — Windows/POSIX conditionals, locale detection, terminal-size queries, shell-level (
SHLVL) adjustment.
A handful of the oldest helpers (decode, encode,
cast_unicode, safe_hasattr, indent) were originally forked
from the IPython project and retain their upstream copyrights:
Copyright (c) 2008-2014, IPython Development Team
Copyright (C) 2001-2007 Fernando Perez <fperez@colorado.edu>
Copyright (c) 2001, Janko Hauser <jhauser@zscout.de>
Copyright (c) 2001, Nathaniel Gray <n8gray@caltech.edu>
- exception xonsh.tools.XonshCalledProcessError(returncode, command, output=None, stderr=None, completed_command=None)[source]¶
Raised when there’s an error with a called process
Inherits from XonshError and subprocess.CalledProcessError, catching either will also catch this error.
Raised after iterating over stdout of a captured command, if the returncode of the command is nonzero.
- add_note(object, /)¶
Exception.add_note(note) – add a note to the exception
- with_traceback(object, /)¶
Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
- args¶
- property stdout¶
Alias for output attribute, to match stderr
- exception xonsh.tools.XonshError[source]¶
- add_note(object, /)¶
Exception.add_note(note) – add a note to the exception
- with_traceback(object, /)¶
Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
- args¶
- class xonsh.tools.DefaultNotGivenType[source]¶
Singleton for representing when no default value is given.
- class xonsh.tools.FlexibleFormatter[source]¶
Support nested fields inside conditional formatters
e.g. template
{user:| {RED}{}{RESET}}will become| {RED}user{RESET}when user=user.- check_unused_args(used_args, args, kwargs)¶
- convert_field(value, conversion)¶
- format(format_string, /, *args, **kwargs)¶
- format_field(value, format_spec)¶
- get_field(field_name, args, kwargs)¶
- parse(format_string)¶
- vformat(format_string, args, kwargs)¶
- class xonsh.tools.redirect_stderr(new_target)[source]¶
Context manager for temporarily redirecting stderr to another file.
- class xonsh.tools.redirect_stdout(new_target)[source]¶
Context manager for temporarily redirecting stdout to another file:
# How to send help() to stderr with redirect_stdout(sys.stderr): help(dir) # How to write help() to a file with open('help.txt', 'w') as f: with redirect_stdout(f): help(pow)Mostly for backwards compatibility.
- xonsh.tools.adjust_shlvl(old_lvl: int, change: int)[source]¶
Adjusts an $SHLVL integer according to bash’s behaviour (variables.c::adjust_shell_level).
- xonsh.tools.all_permutations(iterable)[source]¶
Yeilds all permutations, not just those of a specified length
- xonsh.tools.ansicolors_to_ptk1_names(stylemap)[source]¶
Converts ansicolor names in a stylemap to old PTK1 color names
- xonsh.tools.argvquote(arg, force=False)[source]¶
Returns an argument quoted in such a way that that CommandLineToArgvW on Windows will return the argument string unchanged. This is the same thing Popen does when supplied with an list of arguments. Arguments in a command line should be separated by spaces; this function does not add these spaces. This implementation follows the suggestions outlined here: https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/
- xonsh.tools.backup_file(fname)[source]¶
Moves an existing file to a new name that has the current time right before the extension.
- xonsh.tools.balanced_parens(line, mincol=0, maxcol=None, lexer=None)[source]¶
Determines if parentheses are balanced in an expression.
- xonsh.tools.bool_to_str(x)[source]¶
Converts a bool to an empty string if False and the string ‘1’ if True.
- xonsh.tools.capturable(f)[source]¶
Decorator that specifies that a callable alias should be run with capturing. This is the default behavior.
- xonsh.tools.chdir(adir, mkdir=False, expanduser=True)[source]¶
Context manager for switching current directory to another. Note! You can use
with p'/path'.cd(): $[ls]to achieve the same.
- xonsh.tools.check_for_partial_string(x)[source]¶
- Returns the starting index (inclusive), ending index (exclusive), and
starting quote string of the most recent Python string found in the input.
check_for_partial_string(x) -> (startix, endix, quote)
- Parameters:
- xstr
The string to be checked (representing a line of terminal input)
- Returns:
- startixint (or None)
The index where the most recent Python string found started (inclusive), or None if no strings exist in the input
- endixint (or None)
The index where the most recent Python string found ended (exclusive), or None if no strings exist in the input OR if the input ended in the middle of a Python string
- quotestr (or None)
A string containing the quote used to start the string (e.g., b”, “, ‘’’), or None if no string was found.
- ` are ignored)
and # inside string literals (which are part of the string, not a comment). Known limitations / cases NOT covered:
PEP 701 f-strings with nested quotes of the same kind, e.g.
f"{ "hi" }"(Python 3.12+). The scanner sees this as two separate strings (f"{ "and" }") instead of one f-string. Quotes of a different kind nested inside f-string expressions work fine.Brace-balancing inside f-string expressions in general — anything that relies on knowing where
{ ... }starts and ends is out of scope.Comment characters that are not
#(xonsh has no others, but other shells’ line-comment markers are not recognized).Line continuation inside
#comments — irrelevant since Python comments always end at the next newline regardless of any trailing backslash.Arbitrary Python tokenization errors: malformed input (e.g. four double-quotes in a row) may be reported as a partial triple-quoted string rather than a syntax error; downstream code is responsible for validating syntax.
Improvements are welcome!
- xonsh.tools.check_quotes(s)[source]¶
Checks a string to make sure that if it starts with quotes, it also ends with quotes.
- xonsh.tools.columnize(elems, width=80, newline='\n')[source]¶
Takes an iterable of strings and returns a list of lines with the elements placed in columns. Each line will be at most width columns. The newline character will be appended to the end of each line.
- xonsh.tools.command_not_found(cmd, env)[source]¶
Uses various mechanism to suggest packages for a command that cannot currently be found.
- xonsh.tools.conda_suggest_command_not_found(cmd, env)[source]¶
Uses conda-suggest to suggest packages for a command that cannot currently be found.
- xonsh.tools.csv_to_bool_seq(x)[source]¶
Takes a comma-separated string and converts it into a list of bools.
- xonsh.tools.debian_command_not_found(cmd)[source]¶
Uses the debian/ubuntu command-not-found utility to suggest packages for a command that cannot currently be found.
- xonsh.tools.decode_bytes(b)[source]¶
Tries to decode the bytes using XONSH_ENCODING if available, otherwise using sys.getdefaultencoding().
- xonsh.tools.deprecated(deprecated_in=None, removed_in=None)[source]¶
Parametrized decorator that deprecates a function in a graceful manner.
Updates the decorated function’s docstring to mention the version that deprecation occurred in and the version it will be removed in if both of these values are passed.
When removed_in is not a release equal to or less than the current release, call
warnings.warnwith details, while raisingDeprecationWarning.When removed_in is a release equal to or less than the current release, raise an
AssertionError.- Parameters:
- deprecated_instr
The version number that deprecated this function.
- removed_instr
The version number that this function will be removed in.
- xonsh.tools.describe_waitpid_status(status)[source]¶
Describes
pid, status = os.waitpid(pid, opt)status.
- xonsh.tools.display_colored_error_message(exc_info, strip_xonsh_error_types=True, limit=None)[source]¶
- xonsh.tools.display_error_message(exc_info, strip_xonsh_error_types=True)[source]¶
Prints the error message of the given sys.exc_info() triple on stderr.
- xonsh.tools.ends_with_colon_token(line, lexer=None)[source]¶
Determines whether a line ends with a colon token, ignoring comments.
- xonsh.tools.ensure_string(x)[source]¶
Returns a string if x is not a string, and x if it already is. If x is None, the empty string is returned.
- xonsh.tools.env_path_to_str(x)[source]¶
Converts an environment path to a string by joining on the OS separator.
- xonsh.tools.escape_windows_cmd_string(s)[source]¶
Returns a string that is usable by the Windows cmd.exe. The escaping is based on details here and empirical testing: http://www.robvanderwoude.com/escapechars.php
- xonsh.tools.expand_case_matching(s)[source]¶
Expands a string to a case insensitive globable string.
- xonsh.tools.expand_path(s, expand_user=True)[source]¶
Takes a string path and expands ~ to home if expand_user is set and environment vars if EXPAND_ENV_VARS is set.
- xonsh.tools.expandvars(path)[source]¶
Expand shell variables of the forms $var, ${var} and %var%. Unknown variables are left unchanged.
- xonsh.tools.fallback(cond, backup)[source]¶
Decorator for returning the object if cond is true and a backup if cond is false.
- xonsh.tools.find_next_break(line, mincol=0, lexer=None)[source]¶
Returns the column number of the next logical break in subproc mode. This function may be useful in finding the maxcol argument of subproc_toks().
- xonsh.tools.findfirst(s, substrs)[source]¶
Finds whichever of the given substrings occurs first in the given string and returns that substring, or returns None if no such strings occur.
- xonsh.tools.format_color(string, **kwargs)[source]¶
Formats strings that may contain colors. This simply dispatches to the shell instances method of the same name. The results of this function should be directly usable by print_color().
- xonsh.tools.format_datetime(dt)[source]¶
Format datetime object to string base on $XONSH_DATETIME_FORMAT Env.
- xonsh.tools.format_std_prepost(template, env=None)[source]¶
Formats a template prefix/postfix string for a standard buffer. Returns a string suitable for prepending or appending.
- xonsh.tools.get_line_continuation()[source]¶
The line continuation characters used in subproc mode. In interactive mode on Windows the backslash must be preceded by a space. This is because paths on Windows may end in a backslash.
- xonsh.tools.get_logical_line(lines, idx)[source]¶
Returns a single logical line (i.e. one without line continuations) from a list of lines. This line should begin at index idx. This also returns the number of physical lines the logical line spans. The lines should not contain newlines
- xonsh.tools.get_portions(it, slices)[source]¶
Yield from portions of an iterable.
- Parameters:
- ititerable
- slicesa slice or a list of slice objects
- xonsh.tools.get_sep()[source]¶
Returns the appropriate filepath separator char depending on OS and xonsh options set
- xonsh.tools.globpath(s, ignore_case=False, return_empty=False, sort_result=None, include_dotfiles=None)[source]¶
Simple wrapper around glob that also expands home and env vars.
- xonsh.tools.hardcode_colors_for_win10(style_map)[source]¶
Replace all ansi colors with hardcoded colors to avoid unreadable defaults in conhost.exe
- xonsh.tools.iglobpath(s, ignore_case=False, sort_result=None, include_dotfiles=None)[source]¶
Simple wrapper around iglob that also expands home and env vars.
- xonsh.tools.indent(instr, nspaces=4, ntabs=0, flatten=False)[source]¶
Indent a string a given number of spaces or tabstops.
indent(str,nspaces=4,ntabs=0) -> indent str by ntabs+nspaces.
- Parameters:
- instrbasestring
The string to be indented.
- nspacesint (default: 4)
The number of spaces to be indented.
- ntabsint (default: 0)
The number of tabs to be indented.
- flattenbool (default: False)
Whether to scrub existing indentation. If True, all lines will be aligned to the same indentation. If False, existing indentation will be strictly increased.
- Returns:
- outstrstring indented by ntabs and nspaces.
- xonsh.tools.intensify_colors_for_cmd_exe(style_map)[source]¶
Returns a modified style to where colors that maps to dark colors are replaced with brighter versions.
- xonsh.tools.intensify_colors_on_win_setter(enable)[source]¶
Resets the style when setting the INTENSIFY_COLORS_ON_WIN environment variable.
- xonsh.tools.is_balanced(expr, ltok, rtok)[source]¶
Determines whether an expression has unbalanced opening and closing tokens.
- xonsh.tools.is_dynamic_cwd_width(x)[source]¶
Determine if the input is a valid input for the DYNAMIC_CWD_WIDTH environment variable.
- xonsh.tools.is_logfile_opt(x)[source]¶
Checks if x is a valid $XONSH_TRACEBACK_LOGFILE option. Returns False if x is not a writable/creatable file or an empty string or None.
- xonsh.tools.is_nonstring_seq_of_strings(x)[source]¶
Tests if something is a sequence of strings, where the top-level sequence is not a string itself.
- xonsh.tools.levenshtein(a, b, max_dist=inf)[source]¶
Calculates the Levenshtein distance between a and b.
- xonsh.tools.normabspath(p)[source]¶
Returns as normalized absolute path, namely, normcase(abspath(p))
- xonsh.tools.pathsep_to_seq(x)[source]¶
Converts a os.pathsep separated string to a sequence of strings.
- xonsh.tools.pathsep_to_upper_seq(x)[source]¶
Converts a os.pathsep separated string to a sequence of uppercase strings.
- xonsh.tools.print_above_prompt(msg)[source]¶
Print a message above the active prompt when using prompt_toolkit, otherwise fall back to stderr. Safe to call from background threads.
- xonsh.tools.print_color(string, color=None, **kwargs)[source]¶
Prints a string that may contain colors. This dispatched to the shell method of the same name. Colors will be formatted if they have not already been.
- xonsh.tools.print_exception(msg=None, exc_info=None, source_msg=None)[source]¶
Print given exception (or current if None) with/without traceback and set sys.last_exc accordingly.
- xonsh.tools.ptk2_color_depth_setter(x)[source]¶
Setter function for $PROMPT_TOOLKIT_COLOR_DEPTH. Also updates os.environ so prompt toolkit can pickup the value.
- xonsh.tools.qualified_name(obj) str[source]¶
Return fully qualified class name, e.g. ‘xonsh.environ.VarPattern’.
- xonsh.tools.register_custom_style(name, styles, highlight_color=None, background_color=None, base='default')[source]¶
Register custom style.
- Parameters:
- namestr
Style name.
- stylesdict
Token -> style mapping.
- highlight_colorstr
Hightlight color.
- background_colorstr
Background color.
- basestr, optional
Base style to use as default.
- Returns:
- styleThe style object created, None if not succeeded
- xonsh.tools.replace_logical_line(lines, logical, idx, n)[source]¶
Replaces lines at idx that may end in line continuation with a logical line that spans n lines.
- xonsh.tools.safe_hasattr(obj, attr)[source]¶
In recent versions of Python, hasattr() only catches AttributeError. This catches all errors.
- xonsh.tools.seq_to_upper_pathsep(x)[source]¶
Converts a sequence to an uppercase os.pathsep separated string.
- xonsh.tools.set_to_pathsep(x, sort=False)[source]¶
Converts a set to an os.pathsep separated string. The sort kwarg specifies whether to sort the set prior to str conversion.
- xonsh.tools.simple_random_choice(lst)[source]¶
Returns random element from the list with length less than 1 million elements.
- xonsh.tools.strip_simple_quotes(s)[source]¶
Gets rid of single quotes, double quotes, single triple quotes, and single double quotes from a string, if present front and back of a string. Otherwiswe, does nothing.
- xonsh.tools.subexpr_before_unbalanced(expr, ltok, rtok)[source]¶
Obtains the expression prior to last unbalanced left token.
- xonsh.tools.subexpr_from_unbalanced(expr, ltok, rtok)[source]¶
Attempts to pull out a valid subexpression for unbalanced grouping, based on opening tokens, eg. ‘(’, and closing tokens, eg. ‘)’. This does not do full tokenization, but should be good enough for tab completion.
- xonsh.tools.subproc_toks(line, mincol=-1, maxcol=None, lexer=None, returnline=False, greedy=False)[source]¶
Encapsulates tokens in a source code line in a uncaptured subprocess ![] starting at a minimum column. If there are no tokens (ie in a comment line) this returns None. If greedy is True, it will encapsulate normal parentheses. Greedy is False by default.
- xonsh.tools.suggest_commands(cmd, env)[source]¶
Suggests alternative commands given an environment and aliases.
- xonsh.tools.suggestion_sort_helper(x, y)[source]¶
Returns a score (lower is better) for x based on how similar it is to y. Used to rank suggestions.
- xonsh.tools.swap(namespace, name, value, default=<object object>)[source]¶
Swaps a current variable name in a namespace for another value, and then replaces it when the context is exited.
- xonsh.tools.swap_values(d, updates, default=<object object>)[source]¶
Updates a dictionary (or other mapping) with values from another mapping, and then restores the original mapping when the context is exited.
- xonsh.tools.threadable(f)[source]¶
Decorator that specifies that a callable alias should be run in a background thread. This is the default behavior.
- xonsh.tools.to_bool_or_none(x)[source]¶
Converts to a boolean or none in a semantically meaningful way.
- xonsh.tools.to_completions_display_value(x)[source]¶
Convert user input to value of
$COMPLETIONS_DISPLAY
- xonsh.tools.to_int_or_none(x) int | None[source]¶
Convert the given value to integer if possible. Otherwise return None
- xonsh.tools.to_logfile_opt(x)[source]¶
Converts a $XONSH_TRACEBACK_LOGFILE option to either a str containing the filepath if it is a writable file or None if the filepath is not valid, informing the user on stderr about the invalid choice.
- xonsh.tools.to_shlvl(x)[source]¶
Converts a value to an $SHLVL integer according to bash’s behaviour (variables.c::adjust_shell_level).
- xonsh.tools.uncapturable(f)[source]¶
Decorator that specifies that a callable alias should not be run with any capturing. This is often needed if the alias call interactive subprocess, like pagers and text editors.
- xonsh.tools.unthreadable(f)[source]¶
Decorator that specifies that a callable alias should be run only on the main thread process. This is often needed for debuggers and profilers.
- xonsh.tools.HISTORY_UNITS = {'': ('commands', <class 'int'>), 'b': ('b', <class 'int'>), 'byte': ('b', <class 'int'>), 'bytes': ('b', <class 'int'>), 'c': ('commands', <class 'int'>), 'cmd': ('commands', <class 'int'>), 'cmds': ('commands', <class 'int'>), 'command': ('commands', <class 'int'>), 'commands': ('commands', <class 'int'>), 'd': ('s', <function <lambda>>), 'day': ('s', <function <lambda>>), 'days': ('s', <function <lambda>>), 'f': ('files', <class 'int'>), 'files': ('files', <class 'int'>), 'gb': ('b', <function <lambda>>), 'gig': ('b', <function <lambda>>), 'gigabyte': ('b', <function <lambda>>), 'gigabytes': ('b', <function <lambda>>), 'gigs': ('b', <function <lambda>>), 'h': ('s', <function <lambda>>), 'hour': ('s', <function <lambda>>), 'hours': ('s', <function <lambda>>), 'hr': ('s', <function <lambda>>), 'kb': ('b', <function <lambda>>), 'kilobyte': ('b', <function <lambda>>), 'kilobytes': ('b', <function <lambda>>), 'm': ('s', <function <lambda>>), 'mb': ('b', <function <lambda>>), 'meg': ('b', <function <lambda>>), 'megabyte': ('b', <function <lambda>>), 'megabytes': ('b', <function <lambda>>), 'megs': ('b', <function <lambda>>), 'min': ('s', <function <lambda>>), 'mins': ('s', <function <lambda>>), 'mon': ('s', <function <lambda>>), 'month': ('s', <function <lambda>>), 'months': ('s', <function <lambda>>), 's': ('s', <class 'float'>), 'sec': ('s', <class 'float'>), 'second': ('s', <class 'float'>), 'seconds': ('s', <class 'float'>), 'tb': ('b', <function <lambda>>), 'terabyte': ('b', <function <lambda>>), 'terabytes': ('b', <function <lambda>>), 'y': ('s', <function <lambda>>), 'year': ('s', <function <lambda>>), 'years': ('s', <function <lambda>>), 'yr': ('s', <function <lambda>>), 'yrs': ('s', <function <lambda>>)}¶
Maps lowercase unit names to canonical name and conversion utilities.
- xonsh.tools.RE_BEGIN_STRING = re.compile('([bBprRuUf]*("""|\'\'\'|"|\'))')¶
Regular expression matching the start of a string, including quotes and leading characters (r, b, or u)
- xonsh.tools.RE_STRING_CONT = <xonsh.lib.lazyasd.LazyDict object>¶
Dictionary mapping starting quote sequences to regular expressions that match the contents of a string beginning with those quotes (not including the terminating quotes)
- xonsh.tools.RE_STRING_START = re.compile('[bBprRuUf]*')¶
Regular expression matching the characters before the quotes when starting a string (r, b, or u, case insensitive)