magmap.cv.chunking module#

Divides a region into smaller chunks and reassembles it.

class magmap.cv.chunking.SharedArr(arr)[source]#

Bases: object

Shared array storage class.

Attrbibutes:: arr: Shared ctypes raw array. shape: Shape of the original ndarray array. dtype: Data type of the ndarray.

class magmap.cv.chunking.SharedArrsContainer[source]#

Bases: object

Container class for storing shared arrays for multiprocessing.

Arrays are taken as ndarrays, stored as raw arrays, and accessed back as ndarrays.

Variables:: shared_arrs – Dictionary of registered names to shared image instances.

classmethod build_pool_init(arrs)[source]#

Build the initializer and its arguments for multiprocessing Pool.

Parameters:: arrs (Dict[Any, ndarray]) – Dictionary of ndarrays to convert to shared arrays.
Return type:: Tuple[Callable, Tuple[Callable, Dict[Any, SharedArr]]]
Returns:: Tuple of init_shared_labels() as the Pool initializer function and a tuple of the arguments to this initializer, consisting of SharedArrsContainer.setup_shared_arrs`() and a dictionary of the shared arrays.

classmethod convert_shared_arr(arr_key)[source]#

Convert labels image as a shared array to a NumPy array.

Return type:: Optional[ndarray]
Returns:: The shared array as an ndarray, or None if not found.

classmethod setup_shared_arrs(shared_arrs)[source]#: Set up shared arrays for reconstructing as ndarray.

shared_arrs: Dict[Any, SharedArr] = None#

magmap.cv.chunking.get_mp_pool(initializer=None, initargs=None)[source]#

Get a multiprocessing Pool object, configured based on config settings.

Parameters:

initializer (Optional[Callable], default: None) – Function to be called on initialization for each process; defaults to None.
initargs (Optional[Tuple], default: None) – Arguments to initializer; defaults to None.

Return type:

Pool

Returns:

Pool object with number of processes and max tasks per process determined by command-line and the main (first) ROI profile settings.

magmap.cv.chunking.get_split_stack_total_shape(sub_rois, overlap=None)[source]#

Get the shape of a chunked stack.

Useful for determining the final shape of a stack that has been chunked and potentially scaled before merging the stack into an output array of this shape.

Variables:

sub_rois – Array of sub regions, in (z, y, x, …) dimensions.
overlap – Overlap size between sub-ROIs. Defaults to None for no overlap.

Returns:

The shape of the chunked stack after it would be merged.

magmap.cv.chunking.init_shared_container(fn, *args)[source]#

Provide generic initialization for spawned multiprocessing.

Parameters:

fn (Callable) – Function that takes args for initializing each process.
args – Arguments to fn.

magmap.cv.chunking.is_fork()[source]#

Check if the multiprocessing start method is set to “fork”.

Returns:: True if the start method is “fork”, False if otherwise.
Return type:: bool

magmap.cv.chunking.merge_blobs(blob_rois)[source]#

Combine all blobs into a master list so that each overlapping region will contain all blobs from all sub-ROIs that had blobs in those regions, obviating the need to pair sub-ROIs with one another.

Parameters:: blob_rois (np.ndarray) – Blob from each sub-region defined by stack_splitter(). Blobs are assumed to be a 2D array in the format [[z, y, x, ...], ...].
Returns:: Merged blobs in 2D format of the format, [[z, y, x, ..., sub_roi_z, sub_roi_y, sub_roi_x], ...], where sub-ROI coordinates have been added as the final columns.
Return type:: np.ndarray

magmap.cv.chunking.merge_split_stack(sub_rois, max_pixels, overlap)[source]#

Merges sub regions back into a single stack.

See :func:merge_split_stack2 for a much faster implementation if the final output array size is known beforehand.

Parameters:

sub_rois (ndarray) – Array of sub regions, in z, y, x, ... dimensions.
max_pixels (Sequence[int]) – Max pixels for each side in z, y, x order. Assumes that the full stack was at least this large.
overlap (ndarray) – Overlap size between sub-ROIs.

Return type:

ndarray

Returns:

The merged stack.

magmap.cv.chunking.merge_split_stack2(sub_rois, overlap, offset, output)[source]#

Merges sub regions back into a single stack, saving directly to an output variable such as a memmapped array.

Parameters:

sub_rois – Array of sub regions, in (z, y, x, …) dimensions.
overlap – Overlap size between sub-ROIs given as an int seq in z,y,x. Can be None for no overlap.
offset – Axis offset for output array.
output – Output array, such as a memmapped array to bypass storing the merged array in RAM.

Returns:

The merged stack.

magmap.cv.chunking.set_mp_start_method(val=None)[source]#

Set the multiprocessing start method.

If the start method has already been applied, will skip.

Parameters:: val (str) – Start method to set; defaults to None to use the default for the platform. If the given method is not available for the platform, the default method will be used instead.
Returns:: The applied start method.
Return type:: str

magmap.cv.chunking.stack_splitter(shape, max_pixels, overlap=None)[source]#

Split a stack into multiple sub regions.

Parameters:

shape (Sequence[int]) – Shape of the stack to split.
max_pixels (Sequence[int]) – Max pixels for each side in z, y, x order.
overlap (Optional[Sequence[int]], default: None) – Overlap size between sub-ROIs in z, y, x order. Defaults to None for no overlap.

Return type:

Tuple[ndarray, ndarray]

Returns:

Tuple of sub_roi_slices, sub_rois_offsets, where sub_roi_slices is a Numpy object array where each element contains a tuple of slice objects defining the corresponding sub-region at that position, and sub_rois_offsets is a Numpy array of corresponding offsets for each sub-ROI in (z, y, x) order coordinates.