magmap.io.sqlite module

Connects with a SQLite database for experiment and image analysis storage.

class magmap.io.sqlite.ClrDB[source]

Bases: object

MagellanMapper experiment database handler.

Stores detection related data for ground truth sets, automated detections, and their comparisons.

Variables:

  • ( (cur) – obj:`sqlite3.Connection): Connection object.

  • ( – obj:`sqlite3.Cursor): Cursor object.

  • blobs_truth (List[int]) – Truth blobs list as returned by select_blobs_confirmed().

  • path (str) – Path from which the database was loaded.

blobs_truth = None
conn = None
cur = None
get_rois(exp_name, ignore_ext=True)[source]

Get all ROIs for the given experiment.

Parameters:

  • exp_name (str) – Name of experiment.

  • ignore_ext (bool) – True to ignore any extension in exp_name and experiment names stored in the database; defaults to True.

Returns:

Sequence of all ROIs for the experiment.

Return type:

List[sqlite3.Row]

insert_blob_matches(roi_id, matches)[source]

Insert blob matches.

Parameters:
Returns:

List of blob match IDs.

Return type:

list[int]

load_db(path, new_db=False)[source]

Load a database from an existing path, raising an exception if the path does not exist.

Parameters:

  • path (str) – Path from which to load a database.

  • new_db (bool) – If True or if path does not exist, a new database will be created; defaults to False.

Returns:

The :class:ClrDB database at the given location.

Raises:

FileNoutFoundError` if path is not found –

load_truth_blobs()[source]
select_blob(roi_id, blob)[source]

Select a blob based on position and status.

Parameters:

  • roi_id (int) – ROI ID.

  • blob (np.ndarray) – Blob as a 1D array of z, y, x, r, confirmed, truth, channel.

Returns:

The blob as a 1D array. The blob ID if available. If more than one blow is found, only the first blob is returned.

Return type:

np.ndarray, int

select_blob_by_id(blob_id)[source]

Select a blob by ID.

Parameters:

blob_id (int) – Blob ID.

Returns:

The blob as a 1D array.

Return type:

np.ndarray, int

select_blob_matches(roi_id, offset=None, shape=None)[source]

Select blob matches for the given ROI.

Parameters:

  • roi_id (int) – ROI ID.

  • offset (Optional[Sequence[int]], default: None) – ROI offset in z,y,x; defaults to None.

  • shape (Optional[Sequence[int]], default: None) – ROI shape in z,y,x; defaults to None.

Return type:

BlobMatch

Returns:

Blob matches.

select_blob_matches_by_blob_id(row_id, blobn, blob_ids, max_params=100000)[source]

Select blob matches corresponding to the given blob IDs in the given blob column.

Parameters:

  • row_id (int) – Row ID.

  • blobn (int) – 1 or 2 to indicate the first or second blob column, respectively.

  • blob_ids (Sequence[int]) – Blob IDs.

  • max_params (int, default: 100000) – Maximum number of parameters for the SELECT statements; defaults to 100000. The max is determined by SQLITE_MAX_VARIABLE_NUMBER set at the sqlite3 compile time. If this number is exceeded, this function is called recursively with half the given max_params.

Return type:

BlobMatch

Returns:

Blob match object, which is empty if not matches are found.

Raises:

  • sqlit3.OperationalError – if the maximum number of

  • parameters is < 1. –

Deprecated: 1.6.0

Use select_blob_matches() instead.

select_blobs_by_position(roi_id, offset, size)[source]

Select blobs from the given region defined by offset and size.

An ROI ID is still required in cases blobs have been inserted from overlapping ROIs.

Parameters:

  • roi_id (int) – ROI ID.

  • offset (List[int]) – Offset in x,y,z as absolute coordinates.

  • size (List[int]) – Size in x,y,z.

Returns:

Blobs in the given ROI defined by coordinates. List of blob IDs if available.

Return type:

np.ndarray, List[int]

select_blobs_by_roi(roi_id)[source]

Select blobs from the given ROI.

Parameters:

roi_id (int) – ROI ID.

Returns:

Blobs in the given ROI. List of blob IDs if available.

Return type:

numpy.ndarray, list[int]

select_experiment(name=None)[source]

Selects an experiment from the given name.

Parameters:

name (Optional[str], default: None) – Name of the experiment. Defaults to None to get all experiments.

Return type:

List[Row]

Returns:

Sequence of all the experiment rows with the given name.

select_or_insert_experiment(exp_name=None, date=None)[source]

Selects an experiment from the given name, or inserts the experiment if not found.

Parameters:

  • exp_name (Optional[str], default: None) – Name of the experiment, typically the filename. Defaults to None to use the first experiment found.

  • date (Optional[datetime], default: None) – The date if an experiment is inserted; defaults to None.

Return type:

int

Returns:

The ID of the selected or inserted experiment.

start_db(path=None, new_db=False)[source]

Start the database.

Parameters:

  • path (Optional[str], default: None) – Path where the new database resides; if None, defaults to :attr:DB_NAME.

  • new_db (bool, default: False) – If True or if path does not exist, a new database will be created; defaults to False.

magmap.io.sqlite.clean_up_blobs(db)[source]

Clean up blobs from pre-v.0.5.0, where user-added radii have neg values rather than 0.0, and remove all unconfirmed blobs since dragging/dropping decreases the number of delete/adds required for proper blob placement.

Parameters:

db – Database to clean up, typically a truth database.

magmap.io.sqlite.delete_blobs(conn, cur, roi_id, blobs)[source]

Deletes blobs matching the given blobs’ ROI ID and coordinates.

Parameters:

  • conn – The connection.

  • cur – Connection’s cursor.

  • roi_id (int) – ROI ID.

  • blobs – Array of blobs arrays, assumed to be formatted accorind to :func:detector.format_blob.

Returns:

The number of rows deleted.

magmap.io.sqlite.get_exp_name(path)[source]

Get experiment name formatted for the database.

Parameters:

path (str) – Path to format, typically the loaded image5d path.

Returns:

path deconstructed to remove image suffixes and any extension while preserving the sub-image string.

Return type:

str

magmap.io.sqlite.get_roi_offset(roi)[source]
magmap.io.sqlite.get_roi_size(roi)[source]
magmap.io.sqlite.insert_about(conn, cur, version, date)[source]

Inserts an experiment into the database.

Parameters:

  • conn – The connection.

  • cur – Connection’s cursor.

  • name – Name of the experiment.

  • date – The date as a SQLite date object.

magmap.io.sqlite.insert_blobs(conn, cur, roi_id, blobs)[source]

Inserts blobs into the database, replacing any duplicate blobs.

Parameters:

  • conn – The connection.

  • cur – Connection’s cursor.

  • roi_id (int) – ROI ID.

  • blobs – Array of blobs arrays, assumed to be formatted according to :func:detector.format_blob. “Confirmed” is given as -1 = unconfirmed, 0 = incorrect, 1 = correct.

magmap.io.sqlite.insert_experiment(conn, cur, name, date=None)[source]

Inserts an experiment into the database.

Parameters:

  • ( (cur) – class:`sqlite3.Connection): SQLite connection object.

  • ( – class:`sqlite3.Cursor): SQLite cursor object.

  • name (str) – Name of the experiment.

  • date (datetime) – The experiment date; defaults to None to use the date now.

magmap.io.sqlite.insert_roi(conn, cur, exp_id, series, offset, size)[source]

Inserts an ROI into the database.

Parameters:

  • ( (cur) – obj:`sqlite3.Connection): Connection object.

  • ( – obj:`sqlite3.Cursor): Cursor object.

  • exp_id (int) – ID of the experiment.

  • series (int) – Series within the experiment. Can be None, in which case the series will be set to 0.

  • offset (List[int) – ROI offset as (x, y, z).

  • size (List[int) – ROI size as (x, y, z)

Returns:

ID of the selected or inserted row and the feedback string.

Return type:

int, str

magmap.io.sqlite.load_truth_db(filename_base)[source]

Convenience function to load a truth database associated with an image.

Parameters:

filename_base – MagellanMapper-oriented base name associated with an image path. If the file is not found, :const:DB_SUFFIX_TRUTH will be appended and directories removed.

Returns:

The :class:ClrDB truth database associated with the image.

magmap.io.sqlite.main()[source]

Run main SQLite access commands after loading CLI.

magmap.io.sqlite.match_elements(src, delim, repeat)[source]

Repeat a value to match the same number of delimited string elements.

Parameters:

  • src (str) – Delimited string.

  • delim (str) – Delimiter.

  • repeat (str) – String to repeat for each delimited element.

Return type:

str

Returns:

String with repeat repeated a number of times equal to the number of delim-delimited elements in src.

magmap.io.sqlite.merge_truth_dbs(img_paths)[source]
magmap.io.sqlite.select_about(conn, cur)[source]
magmap.io.sqlite.select_blobs_confirmed(cur, confirmed)[source]

Selects ROIs from the given experiment

Parameters:

  • cur – Connection’s cursor.

  • confirmed (int) – Blob confirmation status.

Returns:

Blobs in the given ROI.

magmap.io.sqlite.select_or_insert_roi(conn, cur, exp_id, series, offset, size)[source]

Selects an ROI from the given parameters, or inserts the ROI if not found.

Parameters:

  • ( (cur) – obj:`sqlite3.Connection): Connection object.

  • ( – obj:`sqlite3.Cursor): Cursor object.

  • exp_id (int) – ID of the experiment.

  • series (int) – Series within the experiment. Can be None, in which case all series will be allowed.

  • offset (List[int) – ROI offset as (x, y, z).

  • size (List[int) – ROI size as (x, y, z)

Returns:

ID of the selected or inserted row and feedback on the result.

Return type:

int, str

magmap.io.sqlite.select_roi(cur, roi_id)[source]

Selects an ROI from the ID.

Parameters:

  • cur – Connection’s cursor.

  • roi_id – The ID of the ROI.

Returns:

The ROI.

magmap.io.sqlite.select_rois(cur, exp_id)[source]

Selects ROIs from the given experiment.

Parameters:

  • ( (cur) – obj:`sqlite3.Cursor): Connection’s cursor.

  • exp_id (int) – ID of the experiment.

Returns:

Sequence of all ROIs for the experiment.

Return type:

List[sqlite3.Row]

magmap.io.sqlite.update_rois(conn, cur, offset, size)[source]

Updates ROI positions and size.

Parameters:

  • cur – Connection’s cursor.

  • offset – Amount to subtract from the offset.

  • size – Amount to add to the size.

magmap.io.sqlite.upgrade_db(conn, cur)[source]
magmap.io.sqlite.verification_stats(db, exp_name, treat_maybes=0)[source]

Calculate accuracy metrics based on blob verification status in the database.

Parameters:

  • db (ClrDB) – Database.

  • exp_name (str) – Experiment name in the database.

  • treat_maybes (int, default: 0) – Pass to detector.meas_detection_accurarcy() for how to treat maybe flags.

Return type:

Tuple[float, float, str]

Returns:

Output from detector.meas_detection_accurarcy() for all blobs in an experiment matching exp_name.