sourcefinder.image#

Some generic utility routines for number handling and calculating (specific) variances

Attributes#

logger

Classes#

ImageData

Encapsulates an image in terms of a numpy array + meta/headerdata.

Module Contents#

class sourcefinder.image.ImageData(data, beam, wcs, conf: sourcefinder.config.Conf = Conf(image=ImgConf(), export=ExportSettings()))[source]#

Bases: object

Encapsulates an image in terms of a numpy array + meta/headerdata.

This is your primary contact point for interaction with images: it includes facilities for source extraction and measurement, etc.

Parameters:

data2D np.ndarray: Observational image data. Must be a regular np.ndarray, since image data read from e.g. a FITS file is not a MaskedArray.
beamtuple: Clean beam specification as (semi-major axis, semi-minor axis, position angle) with the axes in pixel coordinates and the position angle in radians
wcsutility.coordinates.wcs: World coordinate system specification, in our case it is always about sky coordinates.
confConf, default: Conf(image=ImgConf(), export=ExportSettings()): Configuration options for source finding. This includes settings related to image processing (e.g., background and rms noise estimation, thresholds) as well as export options (e.g., source parameters and output maps).

Parameters:

conf (sourcefinder.config.Conf)

__grids()[source]#

Calculate background and RMS grids of this image.

These grids can be interpolated up to make maps of the original image dimensions: see _interpolate().

This is called automatically when ImageData.backmap, ImageData.rmsmap or ImageData.fdrmap is first accessed.

_interpolate(grid, inds, roundup=False)[source]#

Interpolate a grid to produce a map of the dimensions of the image.

Parameters:

gridnp.ma.MaskedArray: The grid to be interpolated.
roundupbool, default: False: If True, values of the resultant map which are lower than the input grid are trimmed. Default is False.

Returns:

np.ma.MaskedArray: The interpolated map.

Notes

This function is used to transform the RMS, background or FDR grids produced by _grids() to a map we can compare with the image data.

_pyse(detectionthresholdmap, analysisthresholdmap, labelled_data=None, labels=np.array([], dtype=np.int32))[source]#

Run Python-based source extraction on this image.

Parameters:

detectionthresholdmapnp.ma.MaskedArray: 2D array of floats with the same shape as the observational image (self.rawdata). The detection threshold map imposes an extra threshold for source detection and is therefore higher than the analysis threshold map.
analysisthresholdmapnp.ma.MaskedArray: 2D array of floats with the same shape as the observational image (self.rawdata). analysisthresholdmap imposes the primary threshold for source detection. All the pixels within the island that exceed this will be used when measuring the source. It is lower (or equal) than detectionthresholdmap, or else we would be left with too few pixels for proper source shape measurements, in some cases. This map is computed as analysis_threshold * self.rmsmap.
labelled_datanp.ndarray, optional, default=None: Labelled island map (output of np.ndimage.label()). Will be calculated automatically if not provided.
labelsnp.ndarray, optional, default=np.array([], dtype=np.int32): Array of integers representing the labels in the island map to use for fitting.

Returns:

A utility.containers.ExtractionResults instance or a Pandas
DataFrame containing the results of the source extraction.

Notes

This is described in detail in the “LOFAR Transients Pipeline” article by John D. Swinbank et al., see https://doi.org/10.1016/j.ascom.2015.03.002

static box_slice_about_pixel(x, y, box_radius)[source]#

Returns a slice centred about (x,y), of width = 2 * int(box_radius) + 1.

Parameters:

xint: Desired row index.
yint: Desired column index.
box_radiusfloat: Radius of the box in pixel coordinates.

Returns:

tuple of slice: Slice centred about (x,y) with width = 2*box_radius + 1.

clearcache()[source]#

Zap any calculated data stored in this object.

Clear the background and rms maps, labels, clip, and any locally held data. All of these can be reconstructed from the data accessor.

Note that this must be run to pick up any new settings.

extract(noisemap=None, bgmap=None, labelled_data=None, labels=None)[source]#

Kick off conventional (ie, rms island finding) source extraction.

Parameters:

noisemapnp.ndarray, default: None: Noise map, i.e. the standard deviation (rms) of the background noise across the observational image
bgmapnp.ndarray, default: None: Background map, i.e. the mean of the background noise across the observational image.
labelled_datanp.ndarray, default: None: The output of a connected component analysis of the image, with a unique label for each source. Should have the same shape as the observational image.
labelsnp.ndarray, default: None: Labels array, i.e. a 1D integer array of labels for each source.

Returns:

A utility.containers.ExtractionResults instance or a
Pandas DataFrame containing the results of the source
extraction.

static extract_parms_image_slice(some_image, inds, labelled_data, label, dummy, maxpos, maxi, npix)[source]#

Find the highest pixel value and its position.

For an island, indicated by a group of pixels with the same label, find the highest pixel value and its position, first relative to the upper left corner of the rectangular slice encompassing the island, but finally relative to the upper left corner of the image, i.e. the [0, 0] position of the Numpy array with all the image pixel values. Also, derive the number of pixels of the island.

Parameters:

some_imagenp.ndarray: 2D array with all the pixel values, typically self.data_bgsubbed.data.
indsnp.ndarray: Array of four indices indicating the slice encompassing an island. Such a slice would typically be a pick from a list of slices from a call to scipy.ndimage.find_objects. Since we are attempting vectorized processing here, the slice should have been replaced by its four coordinates through a call to slices_to_indices.
labelled_datanp.ndarray: Array with the same shape as some_image, with labelled islands with integer values and zeroes for all background pixels.
labelint: The label (integer value) corresponding to the slice encompassing the island. Or actually it should be the other way round, since there can be multiple islands within one rectangular slice.
dummynp.ndarray: Artefact of the implementation of guvectorize: Empty array with the same shape as maxpos. It is needed because of a missing feature in guvectorize: There is no other way to tell guvectorize what the shape of the output array will be. Therefore, we define an otherwise redundant input array with the same shape as the desired output array. Defined as int32, but could be any type.
maxposnp.ndarray: Array of two integers indicating the indices of the highest pixel value of the island with label = label relative to the position of pixel [0, 0] of the image.
maxinp.float32: Float32 equal to the highest pixel value of the island with label=label.
npixnp.int32: Integer indicating the number of pixels of the island.

Returns:

None: No return values, because of the use of the guvectorize decorator: ‘guvectorize() functions don’t return their result value: they take it as an array argument, which must be filled in by the function’. In this case maxpos, maxi and npix will be filled with values.

fd_extract(alpha, noisemap=None, bgmap=None)[source]#

False Detection Rate based source extraction.

The FDR procedure guarantees that the False Detection Rate (FDR) is less than alpha.

Parameters:

alphafloat: Maximum allowed fraction of false positives. Must be between 0 and 1, exclusive.
noisemapnp.ndarray, default: None: Noise map, i.e. the standard deviation (rms) of the background noise across the observational image
bgmapnp.ndarray, default: None: Background map, i.e. the mean of the background noise across the observational image.

Returns:

A`utility.containers.ExtractionResults` instance or a
Pandas Dataframe containing the results of the source
extraction.

Notes

See Hopkins et al., AJ, 123, 1086 (2002) for more details. http://adsabs.harvard.edu/abs/2002AJ….123.1086H

fit_fixed_positions(positions, boxsize, threshold=None, fixed='position+shape', ids=None)[source]#

Convenience function to fit a list of sources at the given positions.

This function wraps around fit_to_point().

Parameters:

positionslist of tuples: List of (RA, Dec) tuples. Positions to be fit, in decimal degrees.
boxsizeint: Length of the square section of the image to use for the fit.
thresholdfloat, default: None: Threshold below which data is not used for fitting.
fixedstr, default: ‘position+shape’: If set to position, the pixel coordinates are fixed in the fit.
idstuple, default: None: List of identifiers. If not None, must match the length and order of the requested fits.

Returns:

tuple: A list of successful fits. If ids is None, returns a single list of sourcefinder.extract.Detection s. Otherwise, returns a tuple of two matched lists: ([detections], [matching_ids]).

Notes

boxsize is in pixel coordinates, not in sky coordinates.

static fit_islands(fudge_max_pix_factor, beamsize, correlation_lengths, fixed, island)[source]#: This function was created to enable the use of ‘partial’ such that we can parallellize source measurements

fit_to_point(x: int, y: int, boxsize: int, threshold: float, fixed: str)[source]#

Fit an elliptical Gaussian to a specified point on the image.

Parameters:

xint: Pixel x-coordinate of the point to fit.
yint: Pixel y-coordinate of the point to fit.
boxsizeint: Length of the square section of the image to use for the fit.
thresholdfloat: Threshold below which data is not used for fitting (in units of rmsmap).
fixedstr: If set to position, the pixel coordinates are fixed in the fit.

Returns:

Detection: An instance of sourcefinder.extract.Detection containing the fit results.

Parameters:

x (int)
y (int)
boxsize (int)
threshold (float)
fixed (str)

label_islands(detectionthresholdmap, analysisthresholdmap)[source]#

Return a labelled array of pixels for fitting.

Parameters:

detectionthresholdmapnp.ma.MaskedArray: Detection threshold map with shape (nrow, ncol), matching the shape of the observational image (self.rawdata). The values are of dtype np.float32.
analysisthresholdmapnp.ma.MaskedArray: Analysis threshold map with shape (nrow, ncol), matching the shape of the observational image (self.rawdata). The values are of dtype np.float32.

Returns:

tuple

labels_above_det_thr (np.ndarray): 1D array of labels above detection threshold, with shape (num_islands_above_detection_threshold,) and dtype np.int64. Note that the length of this array may be smaller than the total number of islands above the analysis threshold, as some labels may have been filtered out due to a peak spectral brightness lower than the local detection threshold.
labelled_data (np.ndarray): Array of labelled pixels, where each pixel with a nonzero label corresponds to an island above the analysis threshold. The array has the same shape as the observational image (self.rawdata) and contains integer values corresponding to the labels of the islands. Pixels that do not belong to any island are assigned a label of 0. The number of islands above the analysis threshold is equal to the number of unique labels in this array, which is equal to or larger than num_islands_above_detection_threshold, i.e. the number of islands above the detection threshold. This array has dtype np.int32.
num_islands_above_detection_threshold (int): Number of islands above detection threshold.
maxposs_above_det_thr (np.ndarray): Array of indices of the maximum pixel values above detection threshold, with shape (num_islands_above_detection_threshold, 2) and dtype np.int32.
maxis_above_det_thr (np.ndarray): Array of maximum pixel values above detection threshold, with shape (num_islands_above_detection_threshold,) and dtype np.float32.
npixs_above_det (np.ndarray): 1D array of pixel counts for each island with peak spectral brightness above the detection threshold, with shape (num_islands_above_detection_threshold,) and dtype np.int32.
all_indices_above_det_thr (np.ndarray): Array of indices of the islands above detection threshold, with shape (num_islands_above_detection_threshold, 4) and dtype np.int32.

reverse_se()[source]#

Run source extraction on the negative of this image.

This process can be used to estimate the false positive rate, as there should be no sources in the negative image.

Returns:

sourcefinder.utility.containers.ExtractionResults
To prevent interference with the normal extraction process, cached
data (background map, clips, etc.) is cleared before and after
running this method. If this method is used frequently, a separate
cache may be implemented in the future.

static slices_to_indices(slices)[source]#: Convert the list of tuples of slices generated by scipy.ndimage.find_objects into a 2D int32 array with number of rows equal to the number of islands and 4 columns, i.e 4 integers per island, containing the same information as the slices, but more suitable for compilation by Numba