Interpret DQ flags

The reftools.interpretdq module contains functions for interpreting DQ flags.

Examples

>>> from reftools.interpretdq import ImageDQ, DQParser

Create a class instance for HST/ACS image using pre-defined DQ definitions. Then, display the associated metadata and translation table:

>>> acsdq = ImageDQ.from_fits('j12345678q_flt.fits', ext=('DQ', 2))  
>>> acsdq.parser.metadata  
<Table length=1>
   key     val
  str10    str3
---------- ----
INSTRUMENT  ACS
>>> acsdq.parser.tab
<Table length=17>
DQFLAG ...                         LONG_DESCRIPTION
uint16 ...                              str74
------ ... ---------------------------------------------------------------
     0 ...                                                      Good pixel
     1 ...                                         Lost during compression
     2 ...                                          Replaced by fill value
     4 ... Bad detector pixel or beyond aperture or HRC upper-right defect
     8 ...              Masked by aperture feature or HRC occulting finger
    16 ...                                                       Hot pixel
   ... ...                                                             ...
   512 ...     Bad pixel in reference file (FLAT, polarizer, or dust mote)
  1024 ...                                                     Charge trap
  2048 ...                                          A-to-D saturated pixel
  4096 ...       Cosmic ray and detector artifact (AstroDrizzle, CR-SPLIT)
  8192 ...                                             Cosmic ray (ACSREJ)
 16384 ...                                        Manually flagged by user
 32768 ...                                                        Not used

Interpret DQ value for a single pixel at IRAF-style coordinate X=1457 and Y=170. Also display the original pixel value for comparison:

>>> acsdq.interpret_pixel(1457, 170)  
<Table length=3>
DQFLAG SHORT_DESCRIPTION LONG_DESCRIPTION
uint16        str9            str74
------ ----------------- ----------------
    16               HOT        Hot pixel
    32               CTE         CTE tail
  1024              TRAP      Charge trap
>>> acsdq.data[169, 1456]  
1072

This is the same as above, except pixel is given as 0-indexed values:

>>> acsdq.interpret_pixel(1456, 169, origin=0)  

Interpret DQ values for all pixels. Then, extract mask for interpreted DQ value of 16 (hot pixel) and display pixel values for that mask:

>>> acsdq.interpret_all()  
Parsing DQ flag(s)...
Done!
Run time: 2.822 s
N_FLAGGED: 550656/8388608 (6.564%)
FLAG=1    : 0 (0.000%)
FLAG=2    : 0 (0.000%)
FLAG=4    : 0 (0.000%)
FLAG=8    : 0 (0.000%)
FLAG=16   : 78901 (0.941%)
FLAG=32   : 78915 (0.941%)
FLAG=64   : 343917 (4.100%)
FLAG=128  : 20373 (0.243%)
FLAG=256  : 15202 (0.181%)
FLAG=512  : 26623 (0.317%)
FLAG=1024 : 5128 (0.061%)
FLAG=2048 : 15117 (0.180%)
FLAG=4096 : 0 (0.000%)
FLAG=8192 : 0 (0.000%)
FLAG=16384: 0 (0.000%)
FLAG=32768: 0 (0.000%)
>>> hotmask = acsdq.dq_mask(16)  
>>> acsdq.data[hotmask]  
array([528, 528, 528, ..., 560, 560, 560], dtype=int16)

If you are only interested in a subset of the data, e.g., the 100x100 corner, you can also pass the subarray of interest into ImageDQ like this:

>>> from astropy.io import fits
>>> dqdata = fits.getdata('jbt7a3k7q_flc.fits', 3)  # DQ, 1  
>>> dqparser = DQParser.from_instrument('ACS')
>>> acsdq = ImageDQ(dqdata[:100, :100], dqparser=dqparser)  
>>> acsdq.interpret_all()  
Parsing DQ flag(s)...
Done!
Run time: 0.006 s
N_FLAGGED: 366/10000 (3.660%)
FLAG=1    : 0 (0.000%)
FLAG=2    : 0 (0.000%)
FLAG=4    : 0 (0.000%)
FLAG=8    : 0 (0.000%)
FLAG=16   : 111 (1.110%)
FLAG=32   : 21 (0.210%)
FLAG=64   : 115 (1.150%)
FLAG=128  : 0 (0.000%)
FLAG=256  : 0 (0.000%)
FLAG=512  : 100 (1.000%)
FLAG=1024 : 0 (0.000%)
FLAG=2048 : 0 (0.000%)
FLAG=4096 : 49 (0.490%)
FLAG=8192 : 0 (0.000%)
FLAG=16384: 0 (0.000%)
FLAG=32768: 0 (0.000%)
>>> acsdq.data.shape  
(100, 100)

Create a class instance for HST/WFC3 DQ parser (without image). Then, interpret a given DQ value:

>>> wfc3dq = DQParser.from_instrument('WFC3')
>>> wfc3dq.interpret_dqval(16658)
<Table length=4>
DQFLAG SHORT_DESCRIPTION                 LONG_DESCRIPTION
uint16        str9                            str69
------ ----------------- ------------------------------------------------
     2            FILLED                           Replaced by fill value
    16               HOT                                        Hot pixel
   256         SATURATED              Full-well or A-to-D saturated pixel
 16384             CRMAX Pixel has more than max CRs, ghost, or crosstalk
class reftools.interpretdq.DQParser(definition_file)

Class to handle parsing of DQ flags.

Definition Table

A “definition table” is an ASCII table that defines each DQ flag and its short and long descriptions. It can have optional comment line(s) for metadata, e.g.:

# INSTRUMENT = HSTGENERIC

It must have three columns:

  1. DQFLAG contains the flag value (uint16).

  2. SHORT_DESCRIPTION (string).

  3. LONG_DESCRIPTION (string).

Example file contents:

# INSTRUMENT = HSTGENERIC
DQFLAG SHORT_DESCRIPTION LONG_DESCRIPTION
0      "OK"              "Good pixel"
1      "LOST"            "Lost during compression"
...    ...               ...

The table format must be readable by astropy.io.ascii.

Parameters:
definition_filestr

ASCII table that defines the DQ flags (see above).

Attributes:
tabastropy.table.Table

Table object from given definition file.

metadataastropy.table.Table

Table object from file metadata.

Methods

from_instrument(instrument)

Use pre-defined DQ flags for the given instrument (HST only for now).

interpret_array(data[, verbose])

Interpret DQ values for an array.

interpret_dqval(dqval)

Interpret DQ values for a single pixel.

classmethod from_instrument(instrument)

Use pre-defined DQ flags for the given instrument (HST only for now). The data files are distributed with this module.

Parameters:
instrument{‘ACS’, ‘WFC3’}

Currently not all instruments are supported. If the instrument is not found, a generic definition is used.

Returns:
clsDQParser

An instance of this class.

interpret_array(data, verbose=True)

Interpret DQ values for an array.

Warning

If the array is large and has a lot of flagged elements, this can be resource intensive.

Parameters:
datandarray

DQ values.

verbosebool

Print info to screen.

Returns:
dqs_by_flagdict

Dictionary mapping each interpreted DQ value to indices of affected array elements.

interpret_dqval(dqval)

Interpret DQ values for a single pixel.

Parameters:
dqvalint

DQ value.

Returns:
dqsastropy.table.Table

Table object containing a list of interpreted DQ values and their meanings.

class reftools.interpretdq.ImageDQ(data, dqparser=None)

Class to handle DQ flags in an image.

Parameters:
datandarray

DQ data array to interpret.

dqparserDQParser or None

DQ parser for interpretation. If not given, default is used.

Attributes:
datandarray

Same as input.

parserDQParser

DQ parser for interpretation.

Methods

dq_mask(dqval)

Generate mask for the given interpreted DQ value.

from_fits(filename[, ext, inskey])

Use data from given FITS image and create DQ parser based on header value.

interpret_all([verbose])

Interpret DQ values for all flagged pixels.

interpret_pixel(x, y[, origin])

Construct a user-friendly table object with interpreted DQ values for a given pixel location.

pixlist([origin])

Convert cached results to a list of (X, Y) pixel coordinates.

Raises:
ValueError

Invalid image dimension.

dq_mask(dqval)

Generate mask for the given interpreted DQ value.

Parameters:
dqvalint

Interpreted DQ value.

Returns:
maskndarray

Boolean mask where the affected pixels are marked True.

Raises:
ValueError

Missing interpreter result or invalid DQ value.

classmethod from_fits(filename, ext=('DQ', 1), inskey='INSTRUME')

Use data from given FITS image and create DQ parser based on header value.

Parameters:
filenamestr

Image filename.

extstr or int or tuple

Extension identifier, as accepted by astropy.io.fits.

inskeystr

Keyword value in PRIMARY header that defines the instrument.

Returns:
clsImageDQ

An instance of this class.

interpret_all(verbose=True)

Interpret DQ values for all flagged pixels. Results are stored in self._dqs_by_flag internal cache.

Warning

If the image is large and has a lot of flagged pixels, this can be resource intensive.

Parameters:
verbosebool

Print info to screen.

interpret_pixel(x, y, origin=1)

Construct a user-friendly table object with interpreted DQ values for a given pixel location.

Parameters:
x, yint

X and Y pixel coordinates. Also see origin.

origin{0, 1}

Indexing system for the pixel values given. Use 0 for Python 0-indexed system and 1 for IRAF 1-indexed system.

Returns:
dqsastropy.table.Table

Table object containing a list of interpreted DQ values and their meanings.

pixlist(origin=1)

Convert cached results to a list of (X, Y) pixel coordinates.

Parameters:
origin{0, 1}

Indexing system for the pixel values returned. Use 0 for Python 0-indexed system and 1 for IRAF 1-indexed system.

Returns:
pixlist_by_flagdict

Dictionary mapping each interpreted DQ value to a list of coordinates.