magmap.io.libmag module

Shared functions with the MagellanMapper package.

magmap.io.libmag.add_missing_keys(d_src, d_target, override=None)

Add dictionary items without overriding existing items.

Add key-value pairs from one dictionary to another if the key does not exist in the target dictionary or the corresponding value is an overridable value. Thisupdating allows these values to be overridden while protecting values that are explicitly set.

Parameters:

• d_src (dict) – Source dictionary, from which key-value pairs will be added to d_target.

• d_target (dict) – Target dictionary, to which which key-value pairs from d_src if each key is not in d_target or if d_target[key] is a value in override.

• override (Sequence[Any]) – Sequence of values to override even if the key is present; defaults to None to use (None,), where any existing value that is None will be overridden.

Returns:

d_target, modified in-place.

Return type:

dict

magmap.io.libmag.backup_file(path, modifier='', i=None)

Backup a file to the next given available path with an index number before the extension.

The backed up path will be in the format path-before-ext[modifier](i).ext, where [modifier] is an optional additional string, and i is the index number, which will be incremented to avoid overwriting an existing file. Will also backup any associated files as given by :const:_FILE_TYPE_GROUPS.

Parameters:

• path (Union[str, Path]) – Path of file to backup.

• modifier (str, default: '') – Modifier string to place before the index number.

• i (Optional[int], default: None) – Index to use; typically use default of None to iniate recursivie backup calls.

magmap.io.libmag.combine_arrs(arrs, filter_none=True, fn=None, **kwargs)

Combine arrays with array filtering.

Parameters:

• arrs (Sequence) – Sequence of arrays.

• filter_none (bool) – True to filter out None``s in ``arrs; defaults to True.

• fn (func) – Function to combine filtered arrays; defaults to None to use numpy.concatenate().

• **kwargs – Additional arguments to fn.

Returns:

Output of fn applied to the filtered arrs, or None if the filtered arrs is None or empty. If the filtered arrs has only one element, return this element as-is, without combining.

Return type:

Any

magmap.io.libmag.combine_paths(base_path, suffix, sep='_', ext=None, check_dir=False, keep_ext=False)

Merge two paths by appending suffix, replacing the extension in base_path.

Parameters:

• base_path (str) – Path whose dot-extension will be replaced by suffix. If None, suffix will be returned. If a directory as indicated by a trailing file separator, will simply be joined to suffix.

• suffix (str) – Replacement including new extension.

• sep (str, default: '_') – Separator between base_path and suffix.

• ext (Optional[str], default: None) – Extension to add or substitute; defaults to None to use the extension in suffix.

• check_dir (bool, default: False) – True to check if base_path is an existing directory, in which case it is simply joined to suffix; defaults to False.

• keep_ext (bool, default: False) – True to keep the base_path extension; defaults to False.

Returns:

Merged path.

See also

insert_before_ext() to splice in suffix instead.

magmap.io.libmag.compact_float(n, max_decimals=None)

Reduce a float to a more compact value.

Parameters:

• n – Floating point number.

• max_decimals (default: None) – Maximum decimals to keep; defaults to None.

Returns:

An integer if n is essentially an integer, or a string representation of n reduced to max_decimals numbers after the decimal point. Otherwise, simply returns n.

magmap.io.libmag.convert_bin_magnitude(val, orders)

Convert a number to another binary order of magnitude.

Parameters:

• val (int, float) – Value to convert.

• orders (int) – Orders of magnitude, such as 3 to convert a value in bytes to gibibytes (GiB), or -2 to convert a value in tebibytes (TiB) to mebibytes (MiB).

Returns:

magmap.io.libmag.convert_indices_to_int(dict_to_convert)

Convert indices of a dictionary to int if possible, including nested indices.

Parameters:

dict_to_convert – Dictionary whose indices will be converted.

Returns:

The converted dictionary.

magmap.io.libmag.coords_for_indexing(coords)

Convert array of coordinates to array for fancy indexing in a Numpy array.

Parameters:

coords – Array of shape (n, m), where n = number of coordinate sets, and m = number of coordinate dimensions.

Returns:

Array of coordinates split into axes, such as nd.array(rows_array, columns_array)). This array can then be used to index another array through arr[tuple(indices)].

magmap.io.libmag.copy_backup(src, target)

Wrapper to copy a file with backup for the target location.

Uses shutil.copy2() for the copying. Backs up target beforehand using backup_file() unless target is a directory.

Parameters:

• src (str) – Source path. If None or non-existant, no copy or backup will occur.

• target (str) – Target path.

magmap.io.libmag.create_symlink(src, target, **kwargs)

Wrapper to create symbolic link with error handling.

Creates a symlink through os.symlink().

Parameters:

• src (Union[str, Path]) – Source path.

• target (Union[str, Path]) – Target path.

• **kwargs – Extra arguments to os.symlink().

magmap.io.libmag.crop_mid_str(vals, max_chars=10, unique=True)

Crop out the middle portion of strings while keeping them unique.

The middle section is replaced with “…”.

Parameters:

• vals (Sequence[str]) – Sequence of strings to crop.

• max_chars (int, default: 10) – Maximum characters to retain. Half the characters will be from the start and the remaining characters from the end of the string. Defaults to 10. The final output size of each string is larger than this size by the number of intervening periods.

• unique (bool, default: True) – True to ensure that cropped strings are unique. A extra “.” is added iteratively so the string is unique in the output list. Uncropped strings are not checked for uniqueness.

Return type:

List[str]

Returns:

A list of cropped strings.

magmap.io.libmag.dtype_within_range(min_val, max_val, integer=None, signed=None)

Get a dtype that will contain the given range.

:const:_DTYPES will be used to specify the possible dtypes.

Parameters:

• min_val – Minimum required value, inclusive.

• max_val – Maximim required value, inclusive.

• integer (default: None) – True to get an int type, False for float. Defaults to None to determine automatically based on max_val.

• signed (default: None) – True for a signed int, False for unsigned; ignored for float. Defaults to None to determine automatically based on min_val.

Returns:

The dtype fitting the range specifications.

Raises:

TypeError if a dtype with the appropriate range cannot be found. –

magmap.io.libmag.enum_dict_aslist(d)

Summarize a dictionary with enums as keys as a shortened list with only the names of each enum member.

Parameters:

d – Dictionary with enums as keys.

Returns:

List of tuples for all dictionary key-value pairs, with each tuple containing the enum member name in place of the full enum as the key.

magmap.io.libmag.enum_names_aslist(c, lower=True)

Get an Enum class as a list of enum names.

Parameters:

• c (Enum) – Enum class.

• lower (bool) – True to get names as lower case; defaults to True for easier comparison with other strings.

Returns:

List of enum names.

Return type:

List

magmap.io.libmag.flatten(vals)

Flatten an arbitrarily nested sequence.

Parameters:

vals (Sequence[Any]) – Sequence of values with arbitrary levels of nesting.

Yields:

vals flattened to a single sequence.

Return type:

Generator[Any, None, None]

magmap.io.libmag.format_bytes(size)

Format bytes to human-readable unit.

Parameters:

size (Union[int, float, str]) – Size in bytes.

Return type:

str

Returns:

size formatted to human-readable unit.

magmap.io.libmag.format_num(val, dec_digits=1, allow_scinot=True)

Format a value to a given number of digits if the value is a number.

Parameters:

• val (float) – The value to format.

• dec_digits (int) – Maximum number of decimal place digits to keep; defaults to 1.

• allow_scinot (bool) – True to allow scientific notation in output; defaults to True.

Returns:

A formatted string with the number of decimal place digits reduced to digits if val is a number, or otherwise simply val.

Return type:

str

magmap.io.libmag.get_dict_keys_from_val(d, val)

Get keys whose value in a dictionary match a given value.

Parameters:

• d (dict) – Dictionary from which to get keys.

• val (Any) – Value whose matching keys will be extracted.

Returns:

List of keys whose values match val in d.

Return type:

List[Any]

magmap.io.libmag.get_dtype_info(arr)

Get the type information for the given array’s data type.

Parameters:

arr (np.ndarray) – Numpy array, assumed to be either an integer or floating point array.

Returns:

Numpy integer or floating point information object.

Return type:

np.iinfo, np.finfo

magmap.io.libmag.get_enum(s, enum_class)

Get an enum from a string where the enum class is assumed to have all upper-case keys, returning None if the key is not found.

Parameters:

• s (str) – Key of enum to find, case-insensitive.

• enum_class (Enum) – Enum class to search.

Returns:

The enum if found, otherwise None.

magmap.io.libmag.get_filename_without_ext(path)

Wrapper to splitext() for getting only the filename.

Parameters:

path (str) – Full path.

Return type:

str

Returns:

Filename alone without extension; simply returns the filename if no extension exists.

magmap.io.libmag.get_git_commit(repo_dir)

Get git commit hash.

Parameters:

repo_dir (str) – Path to repository root directory.

Return type:

Optional[str]

Returns:

Commit hash, or None if not found.

magmap.io.libmag.get_if_within(val, i, default=None)

Get a value from a sequence if available, otherwise returning the value if it is a scalar or a default value if the value at the index is not available.

Parameters:

• val (Any) – Scalar or sequence.

• i (int) – Index to extract an element from val if the index is available.

• default (Any) – Default value to return if val is a sequence but shorter than or equal in length to i.

Returns:

Element i from val if present, or default unless val is not a sequence, in which case val is simply returned.

Return type:

Any

magmap.io.libmag.get_int(val)

Cast a value as an integer or a float if not an integer, if possible.

Parameters:

val – Value to cast. If a tuple or list, each entry will be casted recursively.

Returns:

Value casted to int, falling back to a float, None if none (case-insensitive), or the original value if any error occurs during casting.

magmap.io.libmag.get_version(git=False)

Get package version from installed metadata.

The version string is based on the version at time of installation, which is only updated when reinstalling the package.

Parameters:

git (bool, default: False) – True to add a short hash of the current Git commit.

Return type:

str

Returns:

The version string. If git is True, the current Git commit is appended as <ver>-<short-hash>. If package metadata is not available, the version is given as “n/a”.

magmap.io.libmag.insert_before_ext(name, insert, sep='')

Merge two paths by splicing in insert just before the extention in name.

Parameters:

• name (Union[str, Path]) – Path; if no dot is present in the basename, simply merge the string components.

• insert (str) – String to insert before the extension in name.

• sep (str, default: '') – Separator between name and insert; defaults to an empty string.

Returns:

name with insert inserted just before the extension.

Return type:

str

See also

:func:combine_paths to use the extension from insert.

magmap.io.libmag.is_binary(img)

Check if image is binary.

Parameters:

img – Image array.

Returns:

True if the image is composed of only 0 and 1 values.

magmap.io.libmag.is_int(val)

Check if a value is an integer, with support for alternate integer types such as Numpy integers.

Parameters:

val (Any) – Value to check.

Returns:

True if val is equal to itself after casting to an int.

Return type:

bool

magmap.io.libmag.is_nan(val)

Check if a value can be cast to NaN.

Parameters:

val (Any) – Value or sequence of values to check.

Returns:

True if the value can be cast to np.nan; False if not

or any error, including strings that do cannot be cast to float; or sequence of booleans if val is a sequence.

Return type:

Any

magmap.io.libmag.is_number(val)

Check if a value is a number by attempting to cast to float.

Parameters:

val – Value to check.

Returns:

True if the value was successfully cast to float; False otherwise.

magmap.io.libmag.is_seq(val)

Check if a value is a non-string sequence.

Arg:

val (Any): Value to check.

Returns:

True if the value is a list, tuple, or Numpy array.

Return type:

bool

magmap.io.libmag.last_lines(path, n)

Get the last lines of a file by simply loading the entire file and returning only the last specified lines, without depending on any underlying system commands.

Parameters:

• path – Path to file.

• n – Number of lines at the end of the file to extract; if the file is shorter than this number, all lines are returned.

Returns:

The last n lines as a list, or all lines if n is greater than the number of lines in the file.

magmap.io.libmag.log_once(fn_log, msg)

Log a message only once.

Parameters:

• fn_log (Callable[[str], None]) – Log function.

• msg (str) – Message

magmap.io.libmag.make_acronym(val, delim=' ', ignore=None, caps=False, num_single=3)

Make an acronymn from a string.

Parameters:

• val (Optional[str]) – String to abbreviate.

• delim (str, default: ' ') – Delimiter to split val.

• ignore (Optional[Sequence[str]], default: None) – Sequence of split strings to ignore. Defaults to None, in which case “of” and “the” will be ignored, case-insensitive.

• caps (bool, default: False) – True to capitalize the abbreviation; defaults to False.

• num_single (int, default: 3) – Number of characters to keep if val splits into only a single word.

Return type:

str

Returns:

Abbreviation of val. Return val unchanged if it is empty.

magmap.io.libmag.make_out_path(base_path=None, prefix=None, suffix=None, combine_prefix=False)

Make output path based on prefix and suffix settings.

The base path is typically the default path, while the prefix and suffix are given by the user. The prefix will override all other settings unless flagged to combine it with them.

Parameters:

• base_path (Optional[str], default: None) – Base path from which to construct the output path. Defaults to None to use magmap.settings.config.filename if no prefix is given.

• prefix (Optional[str], default: None) – Path that normally overrides base_path and suffix. Defaults to None, which causes magmap.settings.config.prefix_out to be used if available, falling back to magmap.settings.config.prefix. Set to “” to ignore. If given with a trailing file separator, the prefix will be combined with base_path and suffix even if combine_prefix is False.

• suffix (Optional[str], default: None) – String to append to end of path just before the extension; defaults to None to use magmap.settings.config.suffix.

• combine_prefix (bool, default: False) – True to combine the prefix with the basename of the base path and with the suffix; defaults to False. Often used when multiple files will be output.

Return type:

str

Returns:

Output path.

magmap.io.libmag.match_ext(path, path_to_match)

Match extensions for two paths.

Parameters:

• path – Path with extension that will be kept; will be ignored if only an extension with dot.

• path_to_match – Path whose extension will be replaced with that of path.

Returns:

path_to_match with extension replaced by that of path if it has an extension; otherwise, path_to_match is simply returned.

magmap.io.libmag.normalize(array, minimum, maximum, in_range='image')

Normalizes an array to fall within the given min and max.

Parameters:

• array (Sequence[Union[int, float]]) – Array to normalize.

• minimum (Union[int, float]) – Minimum value for the array.

• maximum (Union[int, float]) – Maximum value for the array. Assumed to be greater than min.

• in_range (Union[str, Tuple[Union[int, float], Union[int, float]]], default: 'image') – Range within array to rescale. Defaults to “image”, which uses the range from array itself.

Return type:

Union[Sequence[Union[int, float]], ndarray]

Returns:

The normalized array.

magmap.io.libmag.npstr_to_array(s, shape=None)

Convert a string representation of a Numpy array back to an array.

Parameters:

• s – String representation of a Numpy array such as from a print command.

• shape (default: None) – Tuple of ints by which to reshape the Numpy array. Defaults to None, in which case the output will be a 1-D array even if the input is multi-dimensional.

Returns:

A numpy array, or None if the string could not be converted.

magmap.io.libmag.pad_seq(seq, n, pad=None)

Pad a sequence with a given value or truncate to fit a given length.

Parameters:

• seq (Sequence[Any]) – Sequence to fill in-place.

• n (int) – Target length.

• pad (Optional[Any], default: None) – Value with which to fill; defaults to None. If a sequence, missing values in seq will be filled with corresponding values in pad.

Return type:

Sequence[Any]

Returns:

A truncated view of seq if the sequence is longer than n or seq with pad appended to reach a length of n. seq is modified in-place if it is extended and not a NumPy array.

magmap.io.libmag.print_compact(arr, msg=None, mid=False)

Print a Numpy array in a compact form to visual comparison with other arrays.

The array will be rounded, converted to integer, and optionally reduced to a single plane to maximize chance of printing a non-truncated array (or plane) to compare more easily with modified versions of the array or other arrays.

Parameters:

• arr – Numpy array.

• msg (default: None) – Message to print on separate line before the array; defaults to None, in which case the message will not be printed.

• mid (default: False) – True to only print the middle element of the array.

Returns:

The compact array as a new array.

magmap.io.libmag.printcb(s, fn_callback)

Print a string to the terminal and call a callback function.

Parameters:

• s (str) – String to print and call in fn_callback.

• fn_callback (func) – Callback function taking a string argument.

magmap.io.libmag.printv(*s)

Print to console only if verbose.

Parameters:

s – Variable number of strings to be printed if :attr:config.verbose is true.

magmap.io.libmag.remove_file(path)

Remove a file with error catching.

Exceptions from files that are not found or are directories will be caught and displayed.

Parameters:

path (str) – Path of file to remove.

Returns:

True if the file was successfully deleted.

Return type:

bool

magmap.io.libmag.replace_seq(seq, replacement)

Replace elements in one sequence with those from another.

Sequences will be converted to NumPy arrays to support multiple dimensions.

Parameters:

• seq (Sequence[Any]) – Sequence to be replaced.

• replacement (Sequence[Any]) – Sequence to replace in seq. Must be of the same shape and type as those of seq.

Return type:

ndarray

Returns:

seq as a new NumPy array with as many values ass possible replaced by replacement.

magmap.io.libmag.roll_elements(arr, shift, axis=None)

Roll elements in a tuple safe manner.

Essentially calls Numpy.roll, but checks for tuple beforehand and converts it to a Numpy array beforehand and back to a new tuple afterward.

Parameters:

arr – Array, which can be a tuple, list, or Numpy array.

Returns:

The array with elements rolled. If arr is a tuple, the returned value

will be a new tuple. If arr is a Numpy array, a view of the array will be turned.

magmap.io.libmag.scale_slice(sl, scale, size)

Scale slice values by a given factor.

Parameters:

• sl (slice) – Slice object to scale.

• scale (int, float) – Scaling factor.

• size (int) – Size of the full range, used if sl.stop is None and generating a sequence of indices.

Returns:

Either a new slice object after scaling if scale is >= 1, or a np.ndarray of scaled indices with the same number of elements as would be in the original sl range.

magmap.io.libmag.series_as_str(series)

Get the series as a string for MagellanMapper filenames, ensuring 5 characters to allow for a large number of series.

Parameters:

series – Series number, to be padded to 5 characters.

Returns:

Padded series.

magmap.io.libmag.show_full_arrays(on=True)

Show full Numpy arrays, except for particularly large arrays.

Parameters:

on (default: True) – True if full printing should be turned on; False to reset all settings.

magmap.io.libmag.splice_before(base, search, splice, post_splice='')

Splice in a string before a given substring.

Parameters:

• base (str) – String in which to splice.

• search (str) – Splice before this substring. If not found, splice is simply appended to base.

• splice (str) – Splice in this string.

• post_splice (str, default: '') – String to add after the spliced string; defaults to “”.

Return type:

str

Returns:

Spliced string.

magmap.io.libmag.splitext(path)

Split a path at its extension in a way that supports extensions with multiple periods as identified in :const:_EXTENSIONS_MULTIPLE.

Parameters:

path – Path to split.

Returns:

Tuple of path prior to extension and the extension, including leading period. If an extension start is not found in :const:_EXTENSIONS_MULTIPLE, the path will simply be split by :meth:os.path.splitext.

magmap.io.libmag.str_to_disp(s)

Convert a string to a user-friendly, displayable string by replacing underscores with spaces and trimming outer whitespace.

Parameters:

s – String to make displayable.

Returns:

New, converted string.

magmap.io.libmag.swap_elements(arr, axis0, axis1, offset=0)

Swap elements within an list or tuple.

Parameters:

• arr – List or tuple in which to swap elements.

• axis0 – Index of first element to swap.

• axis1 – Index of second element to swap.

• offset (default: 0) – Offsets for indices; defaults to 0.

Returns:

The array with elements swapped. If the original array is actually

a tuple, a new tuple with the elements swapped will be returned, so the return object may be different from the passed one.

magmap.io.libmag.to_seq(val, non_none=True)

Wrap a value in a sequence if not already a sequence.

Parameters:

• val (Any) – Value to wrap in a sequence.

• non_none (bool) – True to only wrap in a sequence if the value is not None; False to allow creating [None].

Returns:

A sequence of the value if it is not already a sequence.

Return type:

list

magmap.io.libmag.transpose_1d(arr, plane)

magmap.io.libmag.transpose_1d_rev(arr, plane)

magmap.io.libmag.truncate_decimal_digit(val, repeats=3, trim_near=False)

Truncate floats that may have gained unintended decimal point digits because of floating point representation.

Floating point representation may cause a number input as 3.0 to be displayed as 3.0000000000000004, for example. To remove the unintended numbers, assume that consecutive repeats of a given digit indicates the place at which the decimal points can be truncated.

Repeated 0’s immediately after the decimal point in values between 0-1 (eg. 0.00000000012 will be retained on the assumption that the repeated 0’s are intended for the small value.

Parameters:

• val (float) – Number to truncate.

• repeats (int) – Number of consecutive decimal digits repeats at which to truncate val; defaults to 3. Only the first digit of the repeated value stretch is retained.

• trim_near (bool) – True to remove even this first digit if the repeated value is “near”, ie 0 or 9, which are assumed to be floating point errors (eg 3.0000001 or 2.999999998); defaults to False.

Returns:

String representation of val with truncated decimals.

Return type:

str

magmap.io.libmag.warn(msg, category=<class 'UserWarning'>, stacklevel=2)

Print a warning message.

Parameters:

• msg (str) – Message to print.

• category (Exception) – Warning category class.

• stacklevel (default: 2) – Warning message level.