Miscellaneous DOT utilities

There are many small command line utilities available on most DOT computers to handle DOT data. A quick manual for the most important utilities are given below. Some of them work on the raw FITS files, others on the “burstlogs”, a file containing information about the image bursts recorded during observations. A description of the burstlog format will be given first.

Burstlogs

The camera computers create a directory for each day of observations. In each directory there is a file called “burstlog_name”, where “name” is the name of the camera computer. Each line of the burstlog corresponds with a burst.

An example of the contents of a burstlog file:

1282723608.989774 /storage/disk1/2010-08-25/W37S40_ARf_001_1282723608_virgil.fits 6 20 20 0.064419 0.063959 435.365084 nan nan
1282723612.666336 /storage/disk1/2010-08-25/W37S40_ARf_002_1282723612_virgil.fits 19 20 20 0.064234 0.063116 436.321210 nan nan
1282723616.667386 /storage/disk1/2010-08-25/W37S40_ARf_003_1282723616_virgil.fits 17 20 20 0.065223 0.064327 436.056806 nan nan
1282723620.343960 /storage/disk1/2010-08-25/W37S40_ARf_004_1282723620_virgil.fits 0 20 20 0.063681 0.063137 436.761275 nan nan
1282723624.885687 /storage/disk1/2010-08-25/W37S40_ARf_005_1282723624_virgil.fits 11 20 20 0.063341 0.062657 437.212978 nan nan
...

There are 10 columns in the burstlog file:

  1. Timestamp in UNIX time with microsecond resolution.
  2. Filename of the file containing the raw images of that burst.
  3. Number of the best image in the burst (counting starts at 0).
  4. Actual number of images saved in the burst.
  5. Number of images that should have been saved.
  6. RMS contrast of the best image in the burst (dimensionless, higher is better).
  7. Average RMS contrast of the whole burst (dimensionless, higher is better).
  8. Mean intensity of the whole burst (in counts).
  9. Displacement in the x-axis of the best image (in pixels).
  10. Displacement in the y-axis of the best image (in pixels).

Columns 4 and 5 should contain the same number, only when an error occured during recording these numbers can differ. Columns 9 and 10 are usually not used (containing “nan”), only when correlation tracking is enabled will this contain valid information (this feature has almost never been used so far).

Column 1 can be converted to human readable time with the date tool, for example:

[dot@jeff]~>date -u -d@1282723608
Wed Aug 25 08:06:48 UTC 2010

Column 3 is used when creating frameselected FITS files. Columns 6 and 7 can be used to select the best bursts. However, the burstlog can contain information about multiple targets, when selecting the best bursts take care that you first select only those entries containing a single target.

burstselect

cat burstlog | burstselect [-p] [-v] [<timerange>...]

The burstselect utility selects only those entries from a burstlog that fall within specified time range(s), and prints the list of filenames from those entries. The utility is written as a filter and expects the burstlog on standard input. It can have any number of time ranges as arguments. A time range can have these two forms:

HH:MM-HH:MM
HH:MM+MM

Where HH stands for hour, MM for minutes. The first form selects those entries from the first up to (but excluding) the second time. The second form selects those entries starting from the first time up to (but excluding) the first time plus the second number of minutes. So, these two time ranges are equivalent:

12:00-13:30
12:00+90

You can omit the :MM part if you want to start and/or end at whole hours. For example, the two ranges below are equivalent to the two ranges above:

12-13:30
12+90

This utility will only select complete burst sets if one has recorded line scans.

There are two options that can be given to change the behaviour. The -p option lets the tool write complete burstlog entries instead of just the filenames of the selected bursts, the -v option inverts the selection criteria (i.e., it will print those entries that do not fall within the specified time ranges).

burstselectbest

cat burstlog | burstselectbest [-p] [-v] <period>

This utility works much like the burstselect utility, but instead of selecting burst based on time ranges, it selects only the best burst set from each period, given in minutes. It uses the average RMS (column 7 from the burstlog) to determine which bursts have the best contrast.

Examples

To copy the best H-alpha bursts from 15 minute periods between 12:00 and 13:30 to the directory despeckle, one would give this command:

cp `cat burstlog_tintin | burstselect -p 12-13:30 | burstselectbest 15` despeckle/