## AstroPy work 9/23/07

New code: AstroPy.092307.tar.gz (all scripts).

### Automatic alignment of the night side of Venus

So far this month, I've been working on trying to better align the night limb using 2 different methods. The first is normalized parametric cross correlations (the same formula used by FITSFlow) from multiple regions on either the dark limb, or distributed over the night side. This technique does not align the planet directly to a circle, but rather to another previously aligned image. For this test, I have continued to use the previously processed and medium-aligned images #195-249 from 7/24/07. For the reference image, I have chosen #215, as it is rated as the sharpest, or almost sharpest, of that set in the automated rankings. Using FITSRegister, I have manually aligned the dark limb to the nearest pixel by shifting the processed image by (13, -4), and placed it inside a circle of radius 200 centered at (255, 255): I have written a script called 'finealign.py' which creates a set of flow elements similar to those used by FITSFlow. These elements can be distributed just over the presumed position of the dark limb, or across the night side. The template size used is 33x33 pixels, while the range is 99x99. Each flow element computes the normalized parametric cross correlation of the template (in image to be aligned) with all same-size subregions in the range (in reference image), and returns the offset of the maximum value of the correlation surface. Currently, offsets are only computed to whole pixel values, but the use of several flow elements allows the average of these offsets to be fractional values. A shift vector is shown at the center of each flow element, magnified by a factor of 5x:

```> finealign.py 072407/processed/im0215.a.fits 2
n =  48
mean =  13.00   -4.00
min =  13.00   -4.00
max =  13.00   -4.00
std =   0.00    0.00
``` ```> finealign.py 072407/processed/im0215.a.fits 3
n =  48
mean =  13.00   -4.00
min =  13.00   -4.00
max =  13.00   -4.00
std =   0.00    0.00
``` Here are some examples of using this script to align single images:

```> finealign.py 072407/processed/im0195.a.fits 0
n =  48
mean =  11.42   -4.73
min =   3.00   -7.00
max =  13.00   -1.00
std =   1.38    0.97
``` ```> finealign.py 072407/processed/im0195.a.fits 1
n =  48
mean =  13.38   -5.73
min =   5.00  -32.00
max =  31.00   23.00
std =   5.56    6.89
``` ```> finealign.py 072407/processed/im0235.a.fits 0
n =  48
mean =  14.06   -3.42
min =  10.00   -6.00
max =  15.00   -1.00
std =   0.92    0.84
``` ```> finealign.py 072407/processed/im0235.a.fits 1
n =  48
mean =  15.56   -4.83
min =   5.00  -32.00
max =  31.00   29.00
std =   4.92    8.36
``` Note that 2 things happen when using flow elements distributed over the clouds rather than just at the limb:

1. The estimated shifts are larger,
2. The variation in vectors is greater.

These are because the vectors toward the middle of the planet tend to get longer and more random. I'm not sure about the reason for this, but my guess is that the interior of the clouds looks more like a random featureless region, and that the correlation is finding another random featureless region to match. I haven't looked at the correlation surfaces in detail. Each vector is chosen as the first of possibly several identical maxima. However, even in the featureless interior of the clouds, each correlation maximum appears to be a unique point not at the edge of the array. Nevertheless, because of this, all additional analyses on this page will make use only of flow elements which are centered over the limb.

The finealign.py script can also be run on a set of files. When it is run on processed images #195-249, the following output is obtained:

```072407/processed/im0195.a.fits      11.416667       -4.729167       1.381927        0.973387
072407/processed/im0198.a.fits      11.229167       -4.020833       0.871770        0.777002
072407/processed/im0199.a.fits      10.562500       -4.083333       1.078314        0.640095
072407/processed/im0200.a.fits      11.895833       -3.937500       1.025500        0.851622
072407/processed/im0201.a.fits      10.083333       -3.479167       1.483708        1.224568
072407/processed/im0202.a.fits      12.479167       -3.854167       0.978723        0.889513
072407/processed/im0203.a.fits      11.458333       -3.104167       1.189509        0.871770
072407/processed/im0204.a.fits      11.270833       -3.770833       1.410372        1.211741
072407/processed/im0205.a.fits      11.395833       -3.708333       1.334472        1.019770
072407/processed/im0206.a.fits      13.875000       -4.937500       0.780625        0.774765
072407/processed/im0207.a.fits      14.104167       -5.208333       2.247587        1.189509
072407/processed/im0208.a.fits      11.666667       -4.208333       1.462494        1.171863
072407/processed/im0209.a.fits      11.604167       -4.125000       1.523969        1.183656
072407/processed/im0210.a.fits      12.020833       -3.458333       1.163500        0.934486
072407/processed/im0211.a.fits      11.208333       -2.708333       1.059841        0.888780
072407/processed/im0212.a.fits      12.250000       -3.354167       1.376893        0.721387
072407/processed/im0213.a.fits      14.437500       -4.312500       1.153008        0.845484
072407/processed/im0214.a.fits      13.458333       -3.312500       0.956520        0.582961
072407/processed/im0215.a.fits      13.000000       -4.000000       0.000000        0.000000
072407/processed/im0216.a.fits      11.937500       -3.000000       1.688271        0.204124
072407/processed/im0217.a.fits      10.625000       -2.958333       2.429206        2.798499
072407/processed/im0218.a.fits      15.708333       -4.312500       1.719476        0.982265
072407/processed/im0219.a.fits      14.187500       -3.687500       1.438406        0.982265
072407/processed/im0220.a.fits      13.666667       -4.750000       2.365845        0.901388
072407/processed/im0221.a.fits      14.270833       -3.416667       1.035607        0.862007
072407/processed/im0222.a.fits      15.104167       -4.229167       1.828474        1.065356
072407/processed/im0223.a.fits      14.666667       -3.375000       1.637240        0.665363
072407/processed/im0224.a.fits      15.187500       -4.041667       1.317451        1.117257
072407/processed/im0225.a.fits      13.187500       -3.604167       1.889182        1.185671
072407/processed/im0226.a.fits      12.395833       -3.833333       0.973387        0.772802
072407/processed/im0227.a.fits      12.020833       -3.604167       1.520548        1.113171
072407/processed/im0228.a.fits      12.791667       -4.000000       1.274074        1.290994
072407/processed/im0229.a.fits      14.020833       -4.333333       1.391785        1.086534
072407/processed/im0230.a.fits      13.895833       -3.729167       1.758783        1.149992
072407/processed/im0231.a.fits      13.020833       -2.479167       2.056388        1.457589
072407/processed/im0232.a.fits      13.395833       -3.020833       1.270492        0.923977
072407/processed/im0233.a.fits      12.375000       -2.291667       1.536026        0.978058
072407/processed/im0234.a.fits      13.166667       -3.250000       1.067187        0.877971
072407/processed/im0235.a.fits      14.062500       -3.416667       0.922096        0.837490
072407/processed/im0236.a.fits      12.125000       -3.062500       0.949232        0.626041
072407/processed/im0237.a.fits      13.645833       -3.229167       1.127120        0.684336
072407/processed/im0238.a.fits      12.000000       -2.312500       2.362908        1.596301
072407/processed/im0239.a.fits      13.333333       -3.729167       0.920447        0.929596
072407/processed/im0240.a.fits      12.395833       -2.958333       1.220307        1.306368
072407/processed/im0241.a.fits      13.750000       -3.916667       1.070436        0.975392
072407/processed/im0242.a.fits      14.062500       -4.208333       0.689391        0.644151
072407/processed/im0243.a.fits      14.708333       -4.062500       1.098452        0.966227
072407/processed/im0244.a.fits      13.979167       -4.166667       1.626468        0.874007
072407/processed/im0245.a.fits      13.645833       -3.291667       1.127120        0.888780
072407/processed/im0246.a.fits      13.916667       -4.604167       8.810111        2.619952
072407/processed/im0247.a.fits      14.729167       -5.020833       1.655038        6.796107
072407/processed/im0248.a.fits      13.687500       -5.895833       1.634985        7.743921
072407/processed/im0249.a.fits      14.312500       -6.354167       2.887879        8.754736
```

Columns 2 and 3 are the calculated x and y shifts. Columns 4 and 5 are the standard deviations of the vectors. Note that most of the images produce shift stds of about 1-2 pixels (the exception being #215, the reference image, for which the stds = 0). However, images #246-249 produce larger stds because part of the night limb is out of the frame, resulting in overestimation of the y shift amount and an aligned image which is too low:

```> finealign.py 072407/processed/im0249.a.fits 2
n =  48
mean =  14.31   -6.35
min =   6.00  -33.00
max =  28.00    8.00
std =   2.89    8.75
``` This effect can be partially alleviated if those flow elements whose templates intersect the unshifted image edge (i.e. have a minimum value = 0) are ignored:

```> finealign.py 072407/processed/im0249.a.fits 2
x =      294.733866, y =      451.013316, template min =      288.927172, range min =      247.350609
x =      283.870726, y =      452.905233, template min =      282.143725, range min =      243.091851
x =      272.919829, y =      454.195582, template min =      279.097813, range min =      229.723192
x =      261.914460, y =      454.880440, template min =      263.966812, range min =      211.626914
x =      250.888074, y =      454.957726, template min =      251.912884, range min =      208.793846
x =      239.874187, y =      454.427204, template min =      218.574186, range min =      205.079685
x =      228.906278, y =      453.290488, template min =      218.574186, range min =      195.642732
x =      218.017685, y =      451.551033, template min =      218.574186, range min =      199.417165
x =      207.241507, y =      449.214125, template min =      230.660430, range min =      199.417165
x =      196.610499, y =      446.286869, template min =      220.222879, range min =      199.417165
x =      186.156976, y =      442.778162, template min =      217.628400, range min =      163.750461
x =      175.912714, y =      438.698670, template min =      223.657145, range min =      163.750461
x =      165.908852, y =      434.060792, template min =      213.156715, range min =      163.750461
x =      156.175798, y =      428.878628, template min =      210.688585, range min =      163.750461
x =      146.743139, y =      423.167928, template min =      208.532136, range min =      163.750461
x =      137.639545, y =      416.946052, template min =      208.858794, range min =      163.750461
x =      128.892690, y =      410.231911, template min =      209.359174, range min =      163.750461
x =      120.529160, y =      403.045916, template min =      208.367965, range min =      163.750461
x =      112.574379, y =      395.409908, template min =      208.041397, range min =      163.750461
x =      105.052525, y =      387.347100, template min =      201.782897, range min =      163.750461
x =       97.986464, y =      378.881998, template min =      212.040115, range min =      163.750461
x =       91.397674, y =      370.040336, template min =      205.823190, range min =      191.947068
x =       85.306182, y =      360.848987, template min =      204.783229, range min =      189.386942
x =       79.730505, y =      351.335892, template min =      208.139780, range min =      190.764814
x =       74.687591, y =      341.529966, template min =      204.868623, range min =      189.941604
x =       70.192768, y =      331.461017, template min =      202.143618, range min =      189.983148
x =       66.259701, y =      321.159651, template min =        0.000000, range min =      191.667467 *** ignored
x =       62.900343, y =      310.657181, template min =        0.000000, range min =      196.248161 *** ignored
x =       60.124907, y =      299.985531, template min =        0.000000, range min =        0.000000 *** ignored
x =       57.941828, y =      289.177140, template min =        0.000000, range min =        0.000000 *** ignored
x =       56.357743, y =      278.264860, template min =        0.000000, range min =        0.000000 *** ignored
x =       55.377467, y =      267.281864, template min =        0.000000, range min =        0.000000 *** ignored
x =       55.003979, y =      256.261534, template min =        0.000000, range min =        0.000000 *** ignored
x =       55.238414, y =      245.237370, template min =        0.000000, range min =        0.000000 *** ignored
x =       56.080061, y =      234.242880, template min =        0.000000, range min =        0.000000 *** ignored
x =       57.526361, y =      223.311486, template min =        0.000000, range min =        0.000000 *** ignored
x =       59.572917, y =      212.476415, template min =        0.000000, range min =        0.000000 *** ignored
x =       62.213509, y =      201.770602, template min =        0.000000, range min =      222.873985 *** ignored
x =       65.440110, y =      191.226589, template min =        0.000000, range min =      226.363215 *** ignored
x =       69.242912, y =      180.876427, template min =      234.542611, range min =      226.363215
x =       73.610355, y =      170.751577, template min =      245.992704, range min =      228.126198
x =       78.529166, y =      160.882814, template min =      245.342177, range min =      235.153391
x =       83.984391, y =      151.300138, template min =      241.807387, range min =      233.149358
x =       89.959448, y =      142.032676, template min =      261.197423, range min =      240.312100
x =       96.436176, y =      133.108599, template min =      280.689299, range min =      244.053498
x =      103.394887, y =      124.555032, template min =      293.030226, range min =      253.810122
x =      110.814428, y =      116.397976, template min =      312.110335, range min =      255.316986
x =      118.672248, y =      108.662226, template min =      336.101513, range min =      269.964243
n =  35
mean =  15.00   -2.94
min =   6.00   -9.00
max =  28.00    0.00
std =   2.76    1.24
``` If images #246-249 are re-aligned using this technique, their output is as follows:

```072407/processed/im0246.a.fits      15.622222       -4.222222       0.824322        0.726908
072407/processed/im0247.a.fits      14.297297       -2.783784       1.352972        0.775822
072407/processed/im0248.a.fits      13.405405       -3.081081       1.324600        0.784250
072407/processed/im0249.a.fits      15.000000       -2.942857       2.756810        1.240803
```

These std values are similar to the rest of the aligned images, and places the resulting images in better alignment with the reference image and circle: I have written a script called 'shift2.py' which can read a file of processed images and shift values in the format above, and write the aligned images to a new directory:

```> shift2.py 072407/intermediates/shifts.processed.195-249.txt
072407/aligned/im0195.a.fits        11.416667       -4.729167
072407/aligned/im0198.a.fits        11.229167       -4.020833
072407/aligned/im0199.a.fits        10.562500       -4.083333
072407/aligned/im0200.a.fits        11.895833       -3.937500
072407/aligned/im0201.a.fits        10.083333       -3.479167
...
072407/aligned/im0245.a.fits        13.645833       -3.291667
072407/aligned/im0246.a.fits        15.622222       -4.222222
072407/aligned/im0247.a.fits        14.297297       -2.783784
072407/aligned/im0248.a.fits        13.405405       -3.081081
072407/aligned/im0249.a.fits        15.000000       -2.942857
```

I have also written a script called 'loopfits.py' that reads a list of images and presents them all in an image window in an animated loop. Keys can be used to run the animation forward, backward, or reset, somewhat like QuickTime (but all in Python and PyQt). I may add additional key commands to (x, y) shift individual images within the animation (and record the cumulative shifts in both an external file and the header of the image), so that this program could become a mini-FITSRegister: Although the resulting aligned images are pretty good (enough for stacking, in my opinion), there are some slight variations which are worth noting. In particular, images #214 and #216 (next to the reference #215) appear to be slightly out of alignment based on visual observation of the cloud features near the top of the limb. Running finealign.py on the aligned images suggests why this is so:

```> finealign.py 072407/aligned/im0214.a.fits
n =  48
mean =  -0.10   -0.23
min =  -2.00   -4.00
max =   4.00    0.00
std =   0.82    0.74
> finealign.py 072407/aligned/im0216.a.fits
n =  48
mean =  -0.04    0.00
min =  -2.00   -1.00
max =   2.00    1.00
std =   1.84    0.20
``` Even though the average limb is in good alignment, local parts of the limb are not, especially in image #216. This further indicates the anisotropy of the distortions in these images. Even though the average shift of the aligned image is insignificant, the recommended shift for specific parts of the limb might be several pixels. Therefore, rather than a single geometric object, each image should be considered as a discrete set of points which abberate semi-independently of one another. At the fundamental level this is true, as each image is actually composed of the time integrated superposition of many speckled interferograms. Although readout noise may preclude this, it might be interesting to try to reconstruct deconvolved images from many very short exposure frames, wherein the individual PSFs might be regarded as constant over the frame and in time. Eliot and I did some preliminary work on this topic several years ago which appeared promising.

The second technique I have begun to try is based on radial analysis of the image attenuation. Specifically, I have written a script called 'finealign2.py' that uses the results of finealign.py. This script measures the image values at radial cuts perpendicular to the limb at several different angles. At each major angle, several (3-5) cuts spaced 0.01 radian apart are used to construct an average of image value v. radius at that angle for several pixels inside and outside of the presumed limb. Here is an example using 3 cuts/angle within +-25 pixels (measured as a vector parallel to radius) of the limb: The idea here is to use the individual plots to locate points on the limb, and then to use the circle fitting routine from roughalign.py and roughalign2.py to fit a circle to the night limb. Although the intensity falloff is clear and sharp at some angles, it is dubious for others. Presumably some measure of both intensity and variation would be used to distinguish foreground from background. However, at present it is unclear how to quantitatively distinguish the location of the limb in each of the radial plots.

### Still to do

1. Continue with automatic night alignment.
2. Create additional Python/PyQt registration tools.
3. Complete bracket-gamma subtractions.
4. Perform all steps of image processing on other days in 7/07.