The file suffixes given to ACS data products are described in Table
2.1 and closely mimic the suffixes used by STIS. The initial input
files to the calibration pipeline are the raw (RAW) files from Generic
Conversion and the association (ASN) table, if applicable, for the
complete observation set.
For CCD images, a temporary file, with the suffix BLV_TMP, is created once
bias levels are subtracted and the overscan regions are trimmed. This file
is renamed with the FLT extension after the standard calibrations (flat
fielding, dark subtraction, etc.) are complete. The FLT files will serve
as input for cosmic ray rejection (if required). For CR-SPLIT exposures,
a temporary 'cr-combined image' (CRJ_TMP) is created and then renamed with
the CRJ extension once basic calibrations are complete. Single MAMA images
are given the FLT suffix once calibrations are complete. By definition,
these images do not have an overscan region and are not affected by
cosmic rays. The calibrated products of a REPEAT-OBS association will be
several individually calibrated FLT exposures and a summed flat-fielded
(but not cosmic-ray cleaned) SFL image.
ACS File filenames (J)
| CADC file names (proxy) | File Suffix | Description | Units | Example |
|---|
| jxxxxxxxx.raw | _RAW | Raw data | DN | J8C032XUQ.RAW |
| jxxxxxxxx_asn | _ASN | Association file for observation set | | J8C032XUQ_ASN |
| jxxxxxxxx.spt | _SPT | Telemetry and engineering data | | J8C032XUQ.SPT |
| jxxxxxxxx.trl | _TRL | Trailer File with processing comments | | J8C032XUQ.TRL |
| jxxxxxxxx.flt | _FLT | Calibrated, Flat fielded individual exposure | electrons | J8C032XUQ.FLT |
| jxxxxxxxx.crj | _CRJ | Calibrated, CR-rejected, Combined image | electrons | J8C032XUQ.CRJ |
| jxxxxxxxx.sfl | _SFL | Calibrated, Repeat-Obs, Combined image | electrons | J8C032XUQ.SFL |
| jxxxxxxxx_drz | _DRZ | Calibrated, Geometrically Corrected, Dither-Combined image | electrons/sec | J8C032XUQ_DRZ |
| CADC value added products |
|---|
J8C032XUQ.
File id: J6M901010.ASN_040502211658_ASN
File id: J6M901010.JIF_060502014044_OMS
File id: J6M901010.JIT_060502014044_OMS
File id: J6M901010_DRZ
File id: J6M901010_PREV
File id: J6M901010_PREV.FITS.GZ
File id: J6M901010_PREV.PNG
File id: J6M901010_DRZ.CAT
File id: J6M901010_DRZ.STATS
File id: J6M901010_DRZ.VOT
File id: J6M901010_DRZ_APERTURE
File id: J6M901010_DRZ_BACKGROUND
File id: J6M901010_DRZ_HIS_G.PNG
File id: J6M901010_DRZ_HIS_S.PNG
File id: J6M901010_DRZ_HIS_T.PNG
File id: J6M901010_DRZ_MAG_ERR.PNG
File id: J6M901010_DRZ_SEGMENTATION
File id: J6M901010_DRZ_SGS.PNG
File id: J8C032XUQ.DRZ_190402202926_CAL
File id: J8C032XUQ.RAW_190402202926_CAL
File id: J8C032XUQ.SPT_190402202926_CAL
File id: J8C032XUQ.TRL_190402202926_CAL
File id: J8C032XUQ_DRZ
File id: J8C032XUQ_FLT
Association Tables
Association tables are useful for keeping track of the complex set of
relationships that can exist between exposures taken with ACS, especially
with REPEAT-OBS, CR-SPLIT, and dithered exposures. Images taken at a given
dither position may be additionally CR-SPLIT into multiple exposures. In
these cases, associations are built to describe how each exposure relates
to the desired final product. As a result, ACS associations will create
one or more science products from the input exposures, unlike NICMOS or
STIS associations. The relationships defined in the association table
determine how far through the calibration pipeline the exposures are
processed and when the calibrated exposures are combined into sub-products
for further calibration.
ACS data files are given the following definitions:
- An exposure is a single image, the "atomic unit" of HST data.
- A dataset is a collection of files having a common root name (first 9 characters).
- A sub-product is a dataset created by combining a subset of the exposures in an association.
- A product is a dataset created by combining sub-products of an association.
ACS association tables were designed to closely resemble the NICMOS
association format, with three primary columns: MEMNAME, MEMTYPE, and
MEMPRSNT. The column MEMNAME gives the name of each exposure making up the
association and output product name(s). The column MEMTYPE specifies the
role the file has in the association. A unique set of MEMTYPES specific
to ACS were adopted to provide the support for multiple products.
Exposure types in ACS associations. The suffix "n" is appended to the MEMTYPE when multiple sets are present within a single association.
| MEMTYPE | Description |
|---|
| EXP-CRJ | Input CR-SPLIT exposure (single set) |
| EXP-CRn | Input CR-SPLIT exposure for CR-combined image n (multiple sets) |
| PROD-CRJ | CR-combined output product (single set) |
| PROD-CRn | CR-combined output product n (multiple sets) |
| EXP-RPT | Input REPEAT-OBS exposure (single set) |
| EXP-RPn | Input REPEAT-OBS exposure for repeated image n (multiple sets) |
| PROD-RPT | REPEAT-OBS combined output product (single set) |
| PROD-RPn | REPEAT-OBS combined output product n (multiple sets) |
| EXP-DTH | Input dither exposure |
| PROD-DTH | Dither-combined output product |
A sample association table for a two-position dithered observation
with CR-SPLIT=2 is presented in Table 2.3. This example shows how both
MEMNAME and MEMTYPE are used to associate input and output products. The
MEMTYPE for each component of the first CR-SPLIT exposure, JxxxxxECM
and JxxxxxEGM, are given the type EXP-CR1. The sub-product Jxxxxx011 is
designated in the table with a MEMTYPE of PROD-CR1. The last digit of
the product filename corresponds to the output product number in the
MEMTYPE. A designation of zero for the last digit in the filename is
reserved for the dither-combined product.
The column MEMPRSNT indicates whether a given file already exists. For
example, if cosmic ray rejection has not yet been performed by CALACS,
the PROD-CRn files will have a MEMPRSNT value of "no". The sample
association table in Table 2.3 shows the values of MEMPRSNT prior to
CALACS processing.
Sample Association Table Jxxxxx010_ASN
| MEMNAME | MEMTYPE | MEMPRSNT |
|---|
| JxxxxxECM | EXP-CR1 | yes |
| JxxxxxEGM | EXP-CR1 | yes |
| Jxxxxx011 | PROD-CR1 | no |
| JxxxxxEMM | EXP-CR2 | yes |
| JxxxxxEOM | EXP-CR2 | yes |
| Jxxxxx012 | PROD-CR2 | no |
| Jxxxxx010 | PROD-DTH | no |
Trailer Files
Each task in the CALACS package creates messages during processing
which describe the progress of the calibration and which are sent to
STDOUT. In the pipeline processing for other HST instruments, trailer
files were created by simply redirecting the STDOUT to a file. Because
multiple output files can be processed in a single run of CALACS, creating
trailer files presents a unique challenge. Each task within the CALACS
package must decide which trailer file should be appended with comments
and automatically open, populate, and close each trailer file.
CALACS will always overwrite information in trailer files from previous
runs of CALACS while preserving any comments generated by Generic
Conversion. This ensures that the trailer files accurately reflect
the most recent processing performed. The string CALACSBEG will mark
the first comment added to the trailer file. If a trailer file already
exists, CALACS will search for this string to determine where to append
processing comments. If it is not found, the string will be written to
the end of the file and all comments will follow. Thus any comments from
previous processing are overwritten and only the most current calibrations
are recorded.
As each image is processed, an accompanying trailer file with the '.trl'
ending will be created. Further processing will concatenate all trailer
files associated with the output product into a single file. Additional
messages will then be appended to this concatenated file. Thus, some
information is duplicated across multiple trailer files, but for any
product processed within the pipeline, the trailer file is ensured to
contain processing comments from each input file.
Linking trailer files together can result in multiple occurrences of the
CALACSBEG string. Only the first, however, determines where CALACS will
begin overwriting comments if an observation is reprocessed.
ACS FILESTRUCTURE
The ACS calibration pipeline assembles data received from HST into
datasets, applies the standard calibrations, and stores the uncalibrated
datasets in the HST Data Archive. The structure of these data products is
based on the STIS and NICMOS file format and consists of multi-extension
FITS files which store science (SCI), data quality (DQ) and error (ERR)
arrays, as shown in Figure 2.1.
ACS WFC data come from two CCD chips and are treated as separate
observations, with a SCI, DQ and ERR array for each chip, and with both
chips being stored in the same FITS file. The result is a FITS file with
6 data extensions plus a global header for a single WFC exposure. The WFC
apertures are plotted with respect to the V2/V3 reference frame in Figure
2.2 and are oriented such that the x-axis is approximately to the right
and the y-axis is approximately straight up. For pipeline data products,
chip 2 is below chip 1 in y-pixel coordinates and was therefore defined
as extension 1. Thus, the chip/extension notation is counterintuitive. To
display the science image for chip 1, the user must specify the extension
'file.fits[sci,2]'. Similarly, the data quality and error array for chip
1 are specified as [dq,2] and [err,2].
A single HRC or SBC exposure comes from a single chip and has only 3 data
extensions plus a global header. While the raw and calibrated WFC images
contain 6 data extensions, the drizzled product will always contain 3
data extensions, no matter which detector was used. When WFC data is
drizzled, both chips are included in a single FITS extension.
Figure 2.1 illustrates the basic format for storing ACS images. An
uncalibrated (RAW) exposure contains a primary header plus, for each chip,
a SCI extension (in 16-bit integer format), an empty ERR extension, and
an empty DQ array. The calibrated product from CALACS contains a primary
header plus, for each chip, a SCI extension (in 32-bit float format),
an ERR extension (32-bit float), and a DQ extension (16-bit integer).
MultiDrizzle removes the effects of the geometric distortion and produces
a drizzled (DRZ) product which has the multi-extension FITS file shown
in Figure 2.1. The SCI extension contains the distortion-corrected data
(32-bit float), the WHT extension contains the weight mask (32-bit float),
and the CTX extension contains the context image (32-bit integer).
|
(Subscribe to new public observations)
