sourcefinder.accessors#

Data accessors.

These can be used to populate ImageData objects based on some data source (FITS file, array in memory… etc.).

Submodules#

Classes#

`Conf`
`ExportSettings`	Selection of output, related to detected sources and/or intermediate
`FitsImageBlob`	A FITS image blob. Same as `sourcefinder.accessors.fitsimage.FitsImage`
`ImageData`	Encapsulates an image in terms of a numpy array + meta/headerdata.
`ImgConf`	Configuration that should cover all the specifications for processing the image.
`LofarFitsImage`	Use PyFITS to pull image data out of a FITS file.

Functions#

`open`(path, args, *kwargs)	Returns an accessor object (if available) for the file or
`sourcefinder_image_from_accessor`(image[, conf])	Create a sourcefinder.image.ImageData object from an image
`writefits`(data, filename[, header])	Dump a NumPy array to a FITS file.

Package Contents#

class sourcefinder.accessors.Conf[source]#

export: ExportSettings#

image: ImgConf#

class sourcefinder.accessors.ExportSettings[source]#

Bases: _Validate

Selection of output, related to detected sources and/or intermediate image processing products

csv: bool = False#

file_type: str = 'csv'#

islands: bool = False#

output_dir: str = '.'#

pandas_df: bool = True#

regions: bool = False#

residuals: bool = False#

rmsmap: bool = False#

sigmap: bool = False#

skymodel: bool = False#

source_params: list[str]#

source_params_file: list[str]#

class sourcefinder.accessors.FitsImageBlob(hdulist, plane=None, beam=None, hdu_index=0)[source]#

Bases: sourcefinder.accessors.fitsimage.FitsImage

A FITS image blob. Same as sourcefinder.accessors.fitsimage.FitsImage but constructed from an in-memory FITS file, not a FITS file on disk.

Parameters:

hdulistastropy.io.fits.HDUList: The HDU list representing the in-memory FITS file.
planeint, default: None: If the data is a datacube, specifies which plane to use.
beamtuple, default: None: Beam parameters in degrees, in the form (bmaj, bmin, bpa). If not supplied, the method will attempt to read these from the header.
hdu_indexint, default: 0: The index of the HDU to use from the HDU list.

_get_header(*args)[source]#

Retrieve the header from the specified HDU.

Parameters:

*argstuple

Positional arguments where:

args[0] is the HDU list (astropy.io.fits.HDUList).
args[1] is the index of the HDU to use.

Returns:

astropy.io.fits.Header: The header of the specified HDU.

read_data(*args)[source]#

Read and process the data from the specified HDU.

Parameters:

*argstuple

Positional arguments where:

args[0] is the HDU list (astropy.io.fits.HDUList).
args[1] is the index of the HDU to use.
args[2] is the plane index (int) if the data is a datacube.

Returns:

numpy.ndarray: The processed 2D data array. Processing here means remove axes of length 1, select the plane index from the datacube if needed, and transpose.

data#

header#

pixelsize#

url = 'AARTFAAC streaming image'#

wcs#

class sourcefinder.accessors.ImageData(data, beam, wcs, conf: sourcefinder.config.Conf = Conf(image=ImgConf(), export=ExportSettings()))[source]#

Bases: object

Encapsulates an image in terms of a numpy array + meta/headerdata.

This is your primary contact point for interaction with images: it includes facilities for source extraction and measurement, etc.

Parameters:

data2D np.ndarray: Observational image data. Must be a regular np.ndarray, since image data read from e.g. a FITS file is not a MaskedArray.
beamtuple: Clean beam specification as (semi-major axis, semi-minor axis, position angle) with the axes in pixel coordinates and the position angle in radians
wcsutility.coordinates.wcs: World coordinate system specification, in our case it is always about sky coordinates.
confConf, default: Conf(image=ImgConf(), export=ExportSettings()): Configuration options for source finding. This includes settings related to image processing (e.g., background and rms noise estimation, thresholds) as well as export options (e.g., source parameters and output maps).

Parameters:

conf (sourcefinder.config.Conf)

__grids()[source]#

Calculate background and RMS grids of this image.

These grids can be interpolated up to make maps of the original image dimensions: see _interpolate().

This is called automatically when ImageData.backmap, ImageData.rmsmap or ImageData.fdrmap is first accessed.

_interpolate(grid, inds, roundup=False)[source]#

Interpolate a grid to produce a map of the dimensions of the image.

Parameters:

gridnp.ma.MaskedArray: The grid to be interpolated.
roundupbool, default: False: If True, values of the resultant map which are lower than the input grid are trimmed. Default is False.

Returns:

np.ma.MaskedArray: The interpolated map.

Notes

This function is used to transform the RMS, background or FDR grids produced by _grids() to a map we can compare with the image data.

_pyse(detectionthresholdmap, analysisthresholdmap, labelled_data=None, labels=np.array([], dtype=np.int32))[source]#

Run Python-based source extraction on this image.

Parameters:

detectionthresholdmapnp.ma.MaskedArray: 2D array of floats with the same shape as the observational image (self.rawdata). The detection threshold map imposes an extra threshold for source detection and is therefore higher than the analysis threshold map.
analysisthresholdmapnp.ma.MaskedArray: 2D array of floats with the same shape as the observational image (self.rawdata). analysisthresholdmap imposes the primary threshold for source detection. All the pixels within the island that exceed this will be used when measuring the source. It is lower (or equal) than detectionthresholdmap, or else we would be left with too few pixels for proper source shape measurements, in some cases. This map is computed as analysis_threshold * self.rmsmap.
labelled_datanp.ndarray, optional, default=None: Labelled island map (output of np.ndimage.label()). Will be calculated automatically if not provided.
labelsnp.ndarray, optional, default=np.array([], dtype=np.int32): Array of integers representing the labels in the island map to use for fitting.

Returns:

A utility.containers.ExtractionResults instance or a Pandas
DataFrame containing the results of the source extraction.

Notes

This is described in detail in the “LOFAR Transients Pipeline” article by John D. Swinbank et al., see https://doi.org/10.1016/j.ascom.2015.03.002

static box_slice_about_pixel(x, y, box_radius)[source]#

Returns a slice centred about (x,y), of width = 2 * int(box_radius) + 1.

Parameters:

xint: Desired row index.
yint: Desired column index.
box_radiusfloat: Radius of the box in pixel coordinates.

Returns:

tuple of slice: Slice centred about (x,y) with width = 2*box_radius + 1.

clearcache()[source]#

Zap any calculated data stored in this object.

Clear the background and rms maps, labels, clip, and any locally held data. All of these can be reconstructed from the data accessor.

Note that this must be run to pick up any new settings.

extract(noisemap=None, bgmap=None, labelled_data=None, labels=None)[source]#

Kick off conventional (ie, rms island finding) source extraction.

Parameters:

noisemapnp.ndarray, default: None: Noise map, i.e. the standard deviation (rms) of the background noise across the observational image
bgmapnp.ndarray, default: None: Background map, i.e. the mean of the background noise across the observational image.
labelled_datanp.ndarray, default: None: The output of a connected component analysis of the image, with a unique label for each source. Should have the same shape as the observational image.
labelsnp.ndarray, default: None: Labels array, i.e. a 1D integer array of labels for each source.

Returns:

A utility.containers.ExtractionResults instance or a
Pandas DataFrame containing the results of the source
extraction.

static extract_parms_image_slice(some_image, inds, labelled_data, label, dummy, maxpos, maxi, npix)[source]#

Find the highest pixel value and its position.

For an island, indicated by a group of pixels with the same label, find the highest pixel value and its position, first relative to the upper left corner of the rectangular slice encompassing the island, but finally relative to the upper left corner of the image, i.e. the [0, 0] position of the Numpy array with all the image pixel values. Also, derive the number of pixels of the island.

Parameters:

some_imagenp.ndarray: 2D array with all the pixel values, typically self.data_bgsubbed.data.
indsnp.ndarray: Array of four indices indicating the slice encompassing an island. Such a slice would typically be a pick from a list of slices from a call to scipy.ndimage.find_objects. Since we are attempting vectorized processing here, the slice should have been replaced by its four coordinates through a call to slices_to_indices.
labelled_datanp.ndarray: Array with the same shape as some_image, with labelled islands with integer values and zeroes for all background pixels.
labelint: The label (integer value) corresponding to the slice encompassing the island. Or actually it should be the other way round, since there can be multiple islands within one rectangular slice.
dummynp.ndarray: Artefact of the implementation of guvectorize: Empty array with the same shape as maxpos. It is needed because of a missing feature in guvectorize: There is no other way to tell guvectorize what the shape of the output array will be. Therefore, we define an otherwise redundant input array with the same shape as the desired output array. Defined as int32, but could be any type.
maxposnp.ndarray: Array of two integers indicating the indices of the highest pixel value of the island with label = label relative to the position of pixel [0, 0] of the image.
maxinp.float32: Float32 equal to the highest pixel value of the island with label=label.
npixnp.int32: Integer indicating the number of pixels of the island.

Returns:

None: No return values, because of the use of the guvectorize decorator: ‘guvectorize() functions don’t return their result value: they take it as an array argument, which must be filled in by the function’. In this case maxpos, maxi and npix will be filled with values.

fd_extract(alpha, noisemap=None, bgmap=None)[source]#

False Detection Rate based source extraction.

The FDR procedure guarantees that the False Detection Rate (FDR) is less than alpha.

Parameters:

alphafloat: Maximum allowed fraction of false positives. Must be between 0 and 1, exclusive.
noisemapnp.ndarray, default: None: Noise map, i.e. the standard deviation (rms) of the background noise across the observational image
bgmapnp.ndarray, default: None: Background map, i.e. the mean of the background noise across the observational image.

Returns:

A`utility.containers.ExtractionResults` instance or a
Pandas Dataframe containing the results of the source
extraction.

Notes

See Hopkins et al., AJ, 123, 1086 (2002) for more details. http://adsabs.harvard.edu/abs/2002AJ….123.1086H

fit_fixed_positions(positions, boxsize, threshold=None, fixed='position+shape', ids=None)[source]#

Convenience function to fit a list of sources at the given positions.

This function wraps around fit_to_point().

Parameters:

positionslist of tuples: List of (RA, Dec) tuples. Positions to be fit, in decimal degrees.
boxsizeint: Length of the square section of the image to use for the fit.
thresholdfloat, default: None: Threshold below which data is not used for fitting.
fixedstr, default: ‘position+shape’: If set to position, the pixel coordinates are fixed in the fit.
idstuple, default: None: List of identifiers. If not None, must match the length and order of the requested fits.

Returns:

tuple: A list of successful fits. If ids is None, returns a single list of sourcefinder.extract.Detection s. Otherwise, returns a tuple of two matched lists: ([detections], [matching_ids]).

Notes

boxsize is in pixel coordinates, not in sky coordinates.

static fit_islands(fudge_max_pix_factor, beamsize, correlation_lengths, fixed, island)[source]#: This function was created to enable the use of ‘partial’ such that we can parallellize source measurements

fit_to_point(x: int, y: int, boxsize: int, threshold: float, fixed: str)[source]#

Fit an elliptical Gaussian to a specified point on the image.

Parameters:

xint: Pixel x-coordinate of the point to fit.
yint: Pixel y-coordinate of the point to fit.
boxsizeint: Length of the square section of the image to use for the fit.
thresholdfloat: Threshold below which data is not used for fitting (in units of rmsmap).
fixedstr: If set to position, the pixel coordinates are fixed in the fit.

Returns:

Detection: An instance of sourcefinder.extract.Detection containing the fit results.

Parameters:

x (int)
y (int)
boxsize (int)
threshold (float)
fixed (str)

label_islands(detectionthresholdmap, analysisthresholdmap)[source]#

Return a labelled array of pixels for fitting.

Parameters:

detectionthresholdmapnp.ma.MaskedArray: Detection threshold map with shape (nrow, ncol), matching the shape of the observational image (self.rawdata). The values are of dtype np.float32.
analysisthresholdmapnp.ma.MaskedArray: Analysis threshold map with shape (nrow, ncol), matching the shape of the observational image (self.rawdata). The values are of dtype np.float32.

Returns:

tuple

labels_above_det_thr (np.ndarray): 1D array of labels above detection threshold, with shape (num_islands_above_detection_threshold,) and dtype np.int64. Note that the length of this array may be smaller than the total number of islands above the analysis threshold, as some labels may have been filtered out due to a peak spectral brightness lower than the local detection threshold.
labelled_data (np.ndarray): Array of labelled pixels, where each pixel with a nonzero label corresponds to an island above the analysis threshold. The array has the same shape as the observational image (self.rawdata) and contains integer values corresponding to the labels of the islands. Pixels that do not belong to any island are assigned a label of 0. The number of islands above the analysis threshold is equal to the number of unique labels in this array, which is equal to or larger than num_islands_above_detection_threshold, i.e. the number of islands above the detection threshold. This array has dtype np.int32.
num_islands_above_detection_threshold (int): Number of islands above detection threshold.
maxposs_above_det_thr (np.ndarray): Array of indices of the maximum pixel values above detection threshold, with shape (num_islands_above_detection_threshold, 2) and dtype np.int32.
maxis_above_det_thr (np.ndarray): Array of maximum pixel values above detection threshold, with shape (num_islands_above_detection_threshold,) and dtype np.float32.
npixs_above_det (np.ndarray): 1D array of pixel counts for each island with peak spectral brightness above the detection threshold, with shape (num_islands_above_detection_threshold,) and dtype np.int32.
all_indices_above_det_thr (np.ndarray): Array of indices of the islands above detection threshold, with shape (num_islands_above_detection_threshold, 4) and dtype np.int32.

reverse_se()[source]#

Run source extraction on the negative of this image.

This process can be used to estimate the false positive rate, as there should be no sources in the negative image.

Returns:

sourcefinder.utility.containers.ExtractionResults
To prevent interference with the normal extraction process, cached
data (background map, clips, etc.) is cleared before and after
running this method. If this method is used frequently, a separate
cache may be implemented in the future.

static slices_to_indices(slices)[source]#: Convert the list of tuples of slices generated by scipy.ndimage.find_objects into a 2D int32 array with number of rows equal to the number of islands and 4 columns, i.e 4 integers per island, containing the same information as the slices, but more suitable for compilation by Numba

_conf#

property backmap#: Mean background map

beamsize#

clip: dict[float, numpy.ndarray]#

property conf: sourcefinder.config.Conf#

Return type:: sourcefinder.config.Conf

correlation_lengths#

property data#: Masked image data

property data_bgsubbed#: Background subtracted masked image data

fudge_max_pix_factor#

property grids#: Gridded RMS and background data for interpolating

labels: dict[float, tuple[numpy.ndarray, int]]#

property pixmax#: Maximum pixel value (pre-background subtraction)

property pixmin#: Minimum pixel value (pre-background subtraction)

rawdata#

property rmsmap#: root-mean-squares map, i.e. the standard deviation of the local background noise, interpolated across the image.

wcs#

property xdim#: X pixel dimension of (unmasked) data

property ydim#: Y pixel dimension of (unmasked) data

class sourcefinder.accessors.ImgConf[source]#

Bases: _Validate

Configuration that should cover all the specifications for processing the image.

alpha: float = 0.01#

alpha_brightness1: float = 1.5#

alpha_brightness2: float = 1.5#

alpha_maj1: float = 2.5#

alpha_maj2: float = 0.5#

alpha_min1: float = 0.5#

alpha_min2: float = 2.5#

analysis_thr: float = 3.0#

back_size_x: int | None = None#

back_size_y: int | None = None#

bmaj: float | None = None#

bmin: float | None = None#

bpa: float | None = None#

clean_bias: float = 0.0#

clean_bias_error: float = 0.0#

deblend_mincont: float = 0.005#

deblend_nthresh: int = 0#

detection_image: str | None = None#

detection_thr: float = 10.0#

eps_dec: float = 0.0#

eps_ra: float = 0.0#

ew_sys_err: float = 0.0#

fdr: bool = False#

ffbox: float = 3.0#

fixed_posns: str | None = None#

fixed_posns_file: str | None = None#

force_beam: bool = False#

frac_flux_cal_error: float = 0.0#

grid: int | None = 64#

interpolate_order: int = 1#

margin: int = 0#

median_filter: int = 0#

mf_threshold: int = 0#

nr_threads: int | None = None#

ns_sys_err: float = 0.0#

radius: float = 0.0#

remove_edge_sources: bool = True#

rms_filter: float = 0.001#

structuring_element: list[list[int]] = [[1, 1, 1], [1, 1, 1], [1, 1, 1]]#

vectorized: bool = True#

class sourcefinder.accessors.LofarFitsImage(url, plane=False, beam=False, hdu=0)[source]#

Bases: sourcefinder.accessors.fitsimage.FitsImage, sourcefinder.accessors.lofaraccessor.LofarAccessor

Use PyFITS to pull image data out of a FITS file.

Provide standard attributes, as per DataAccessor. In addition, we provide a telescope attribute if the FITS file has a TELESCOP header.

Parameters:

urlPath or str: The path or URL to the FITS file.
planeint, default: None: If the data is a datacube, specifies which plane to use.
beamtuple, default: None: Beam parameters in degrees, in the form (bmaj, bmin, bpa). If not supplied, the method will attempt to read these from the header.
hdu_indexint, default: 0: The index of the HDU to use from the HDU list.

antenna_set#

ncore#

nintl#

nremote#

subbands#

subbandwidth#

sourcefinder.accessors.open(path, *args, **kwargs)[source]#

Returns an accessor object (if available) for the file or directory ‘path’.

This function attempts to find an appropriate accessor for the given file or directory path. Accessors are tried in order from most specific to least specific. For example, an accessor providing LofarAccessor is preferred over one providing DataAccessor, but the latter will be used if no better match is found.

Parameters:

pathstr or HDUList: The file path or HDUList object to be processed.
*argstuple: Additional positional arguments to pass to the accessor constructor.
**kwargsdict: Additional keyword arguments to pass to the accessor constructor.

Returns:

DataAccessor or subclass: An accessor object for the given file or directory.

Raises:

OSError: If the file does not exist, cannot be read, or no matching accessor class is found.
Exception: If the path parameter is neither a string nor an HDUList.

sourcefinder.accessors.sourcefinder_image_from_accessor(image, conf: sourcefinder.config.Conf = Conf(image=ImgConf(), export=ExportSettings()))[source]#

Create a sourcefinder.image.ImageData object from an image ‘accessor’.

This function initializes a sourcefinder.image.ImageData object using the data, beam, and WCS information provided by the given image accessor.

Parameters:

imageDataAccessor: FITS/AIPS/HDF5 image available through an accessor.
confConf, default: Conf(image=ImgConf(), export=ExportSettings()): Configuration options for source finding. This includes settings related to image processing (e.g., background and rms noise estimation, thresholds) as well as export options (e.g., source parameters and output maps).

Returns:

ImageData: A sourcefinder.image.ImageData object.

Parameters:

conf (sourcefinder.config.Conf)

sourcefinder.accessors.writefits(data, filename, header={})[source]#

Dump a NumPy array to a FITS file.

This function writes a given NumPy array to a FITS file, optionally including header information. The header can be provided as a dictionary containing key-value pairs to be added to the FITS file’s metadata.

Parameters:

datanumpy.ndarray: The NumPy array to be written to the FITS file.
filenamePath or str: The path to the output FITS file.
headerdict, default: {}: A dictionary containing key-value pairs for the FITS header.

Raises:

OSError: If the file cannot be written due to permission issues or other errors.

Notes

The data is transposed before writing to match the transpose from fitsimage.FitsImage.read_data(). This is necessary to ensure that the data is stored in the correct orientation in the FITS file.

sourcefinder.accessors#

Submodules#

Classes#

Functions#

Package Contents#

This Page