AstroPy work 9/23/07


New code: AstroPy.092307.tar.gz (all scripts).

Automatic alignment of the night side of Venus

So far this month, I've been working on trying to better align the night limb using 2 different methods. The first is normalized parametric cross correlations (the same formula used by FITSFlow) from multiple regions on either the dark limb, or distributed over the night side. This technique does not align the planet directly to a circle, but rather to another previously aligned image. For this test, I have continued to use the previously processed and medium-aligned images #195-249 from 7/24/07. For the reference image, I have chosen #215, as it is rated as the sharpest, or almost sharpest, of that set in the automated rankings. Using FITSRegister, I have manually aligned the dark limb to the nearest pixel by shifting the processed image by (13, -4), and placed it inside a circle of radius 200 centered at (255, 255):

I have written a script called 'finealign.py' which creates a set of flow elements similar to those used by FITSFlow. These elements can be distributed just over the presumed position of the dark limb, or across the night side. The template size used is 33x33 pixels, while the range is 99x99. Each flow element computes the normalized parametric cross correlation of the template (in image to be aligned) with all same-size subregions in the range (in reference image), and returns the offset of the maximum value of the correlation surface. Currently, offsets are only computed to whole pixel values, but the use of several flow elements allows the average of these offsets to be fractional values. A shift vector is shown at the center of each flow element, magnified by a factor of 5x:

> finealign.py 072407/processed/im0215.a.fits 2
   n =  48
mean =  13.00   -4.00
 min =  13.00   -4.00
 max =  13.00   -4.00
 std =   0.00    0.00

> finealign.py 072407/processed/im0215.a.fits 3
   n =  48
mean =  13.00   -4.00
 min =  13.00   -4.00
 max =  13.00   -4.00
 std =   0.00    0.00

Here are some examples of using this script to align single images:

> finealign.py 072407/processed/im0195.a.fits 0
   n =  48
mean =  11.42   -4.73
 min =   3.00   -7.00
 max =  13.00   -1.00
 std =   1.38    0.97

> finealign.py 072407/processed/im0195.a.fits 1
   n =  48
mean =  13.38   -5.73
 min =   5.00  -32.00
 max =  31.00   23.00
 std =   5.56    6.89

> finealign.py 072407/processed/im0235.a.fits 0
   n =  48
mean =  14.06   -3.42
 min =  10.00   -6.00
 max =  15.00   -1.00
 std =   0.92    0.84

> finealign.py 072407/processed/im0235.a.fits 1
   n =  48
mean =  15.56   -4.83
 min =   5.00  -32.00
 max =  31.00   29.00
 std =   4.92    8.36

Note that 2 things happen when using flow elements distributed over the clouds rather than just at the limb:

  1. The estimated shifts are larger,
  2. The variation in vectors is greater.

These are because the vectors toward the middle of the planet tend to get longer and more random. I'm not sure about the reason for this, but my guess is that the interior of the clouds looks more like a random featureless region, and that the correlation is finding another random featureless region to match. I haven't looked at the correlation surfaces in detail. Each vector is chosen as the first of possibly several identical maxima. However, even in the featureless interior of the clouds, each correlation maximum appears to be a unique point not at the edge of the array. Nevertheless, because of this, all additional analyses on this page will make use only of flow elements which are centered over the limb.

The finealign.py script can also be run on a set of files. When it is run on processed images #195-249, the following output is obtained:

072407/processed/im0195.a.fits      11.416667       -4.729167       1.381927        0.973387       
072407/processed/im0198.a.fits      11.229167       -4.020833       0.871770        0.777002       
072407/processed/im0199.a.fits      10.562500       -4.083333       1.078314        0.640095       
072407/processed/im0200.a.fits      11.895833       -3.937500       1.025500        0.851622       
072407/processed/im0201.a.fits      10.083333       -3.479167       1.483708        1.224568       
072407/processed/im0202.a.fits      12.479167       -3.854167       0.978723        0.889513       
072407/processed/im0203.a.fits      11.458333       -3.104167       1.189509        0.871770       
072407/processed/im0204.a.fits      11.270833       -3.770833       1.410372        1.211741       
072407/processed/im0205.a.fits      11.395833       -3.708333       1.334472        1.019770       
072407/processed/im0206.a.fits      13.875000       -4.937500       0.780625        0.774765       
072407/processed/im0207.a.fits      14.104167       -5.208333       2.247587        1.189509       
072407/processed/im0208.a.fits      11.666667       -4.208333       1.462494        1.171863       
072407/processed/im0209.a.fits      11.604167       -4.125000       1.523969        1.183656       
072407/processed/im0210.a.fits      12.020833       -3.458333       1.163500        0.934486       
072407/processed/im0211.a.fits      11.208333       -2.708333       1.059841        0.888780       
072407/processed/im0212.a.fits      12.250000       -3.354167       1.376893        0.721387       
072407/processed/im0213.a.fits      14.437500       -4.312500       1.153008        0.845484       
072407/processed/im0214.a.fits      13.458333       -3.312500       0.956520        0.582961       
072407/processed/im0215.a.fits      13.000000       -4.000000       0.000000        0.000000       
072407/processed/im0216.a.fits      11.937500       -3.000000       1.688271        0.204124       
072407/processed/im0217.a.fits      10.625000       -2.958333       2.429206        2.798499       
072407/processed/im0218.a.fits      15.708333       -4.312500       1.719476        0.982265       
072407/processed/im0219.a.fits      14.187500       -3.687500       1.438406        0.982265       
072407/processed/im0220.a.fits      13.666667       -4.750000       2.365845        0.901388       
072407/processed/im0221.a.fits      14.270833       -3.416667       1.035607        0.862007       
072407/processed/im0222.a.fits      15.104167       -4.229167       1.828474        1.065356       
072407/processed/im0223.a.fits      14.666667       -3.375000       1.637240        0.665363       
072407/processed/im0224.a.fits      15.187500       -4.041667       1.317451        1.117257       
072407/processed/im0225.a.fits      13.187500       -3.604167       1.889182        1.185671       
072407/processed/im0226.a.fits      12.395833       -3.833333       0.973387        0.772802       
072407/processed/im0227.a.fits      12.020833       -3.604167       1.520548        1.113171       
072407/processed/im0228.a.fits      12.791667       -4.000000       1.274074        1.290994       
072407/processed/im0229.a.fits      14.020833       -4.333333       1.391785        1.086534       
072407/processed/im0230.a.fits      13.895833       -3.729167       1.758783        1.149992       
072407/processed/im0231.a.fits      13.020833       -2.479167       2.056388        1.457589       
072407/processed/im0232.a.fits      13.395833       -3.020833       1.270492        0.923977       
072407/processed/im0233.a.fits      12.375000       -2.291667       1.536026        0.978058       
072407/processed/im0234.a.fits      13.166667       -3.250000       1.067187        0.877971       
072407/processed/im0235.a.fits      14.062500       -3.416667       0.922096        0.837490       
072407/processed/im0236.a.fits      12.125000       -3.062500       0.949232        0.626041       
072407/processed/im0237.a.fits      13.645833       -3.229167       1.127120        0.684336       
072407/processed/im0238.a.fits      12.000000       -2.312500       2.362908        1.596301       
072407/processed/im0239.a.fits      13.333333       -3.729167       0.920447        0.929596       
072407/processed/im0240.a.fits      12.395833       -2.958333       1.220307        1.306368       
072407/processed/im0241.a.fits      13.750000       -3.916667       1.070436        0.975392       
072407/processed/im0242.a.fits      14.062500       -4.208333       0.689391        0.644151       
072407/processed/im0243.a.fits      14.708333       -4.062500       1.098452        0.966227       
072407/processed/im0244.a.fits      13.979167       -4.166667       1.626468        0.874007       
072407/processed/im0245.a.fits      13.645833       -3.291667       1.127120        0.888780       
072407/processed/im0246.a.fits      13.916667       -4.604167       8.810111        2.619952       
072407/processed/im0247.a.fits      14.729167       -5.020833       1.655038        6.796107       
072407/processed/im0248.a.fits      13.687500       -5.895833       1.634985        7.743921       
072407/processed/im0249.a.fits      14.312500       -6.354167       2.887879        8.754736       

Columns 2 and 3 are the calculated x and y shifts. Columns 4 and 5 are the standard deviations of the vectors. Note that most of the images produce shift stds of about 1-2 pixels (the exception being #215, the reference image, for which the stds = 0). However, images #246-249 produce larger stds because part of the night limb is out of the frame, resulting in overestimation of the y shift amount and an aligned image which is too low:

> finealign.py 072407/processed/im0249.a.fits 2
   n =  48
mean =  14.31   -6.35
 min =   6.00  -33.00
 max =  28.00    8.00
 std =   2.89    8.75

This effect can be partially alleviated if those flow elements whose templates intersect the unshifted image edge (i.e. have a minimum value = 0) are ignored:

> finealign.py 072407/processed/im0249.a.fits 2
x =      294.733866, y =      451.013316, template min =      288.927172, range min =      247.350609
x =      283.870726, y =      452.905233, template min =      282.143725, range min =      243.091851
x =      272.919829, y =      454.195582, template min =      279.097813, range min =      229.723192
x =      261.914460, y =      454.880440, template min =      263.966812, range min =      211.626914
x =      250.888074, y =      454.957726, template min =      251.912884, range min =      208.793846
x =      239.874187, y =      454.427204, template min =      218.574186, range min =      205.079685
x =      228.906278, y =      453.290488, template min =      218.574186, range min =      195.642732
x =      218.017685, y =      451.551033, template min =      218.574186, range min =      199.417165
x =      207.241507, y =      449.214125, template min =      230.660430, range min =      199.417165
x =      196.610499, y =      446.286869, template min =      220.222879, range min =      199.417165
x =      186.156976, y =      442.778162, template min =      217.628400, range min =      163.750461
x =      175.912714, y =      438.698670, template min =      223.657145, range min =      163.750461
x =      165.908852, y =      434.060792, template min =      213.156715, range min =      163.750461
x =      156.175798, y =      428.878628, template min =      210.688585, range min =      163.750461
x =      146.743139, y =      423.167928, template min =      208.532136, range min =      163.750461
x =      137.639545, y =      416.946052, template min =      208.858794, range min =      163.750461
x =      128.892690, y =      410.231911, template min =      209.359174, range min =      163.750461
x =      120.529160, y =      403.045916, template min =      208.367965, range min =      163.750461
x =      112.574379, y =      395.409908, template min =      208.041397, range min =      163.750461
x =      105.052525, y =      387.347100, template min =      201.782897, range min =      163.750461
x =       97.986464, y =      378.881998, template min =      212.040115, range min =      163.750461
x =       91.397674, y =      370.040336, template min =      205.823190, range min =      191.947068
x =       85.306182, y =      360.848987, template min =      204.783229, range min =      189.386942
x =       79.730505, y =      351.335892, template min =      208.139780, range min =      190.764814
x =       74.687591, y =      341.529966, template min =      204.868623, range min =      189.941604
x =       70.192768, y =      331.461017, template min =      202.143618, range min =      189.983148
x =       66.259701, y =      321.159651, template min =        0.000000, range min =      191.667467 *** ignored
x =       62.900343, y =      310.657181, template min =        0.000000, range min =      196.248161 *** ignored
x =       60.124907, y =      299.985531, template min =        0.000000, range min =        0.000000 *** ignored
x =       57.941828, y =      289.177140, template min =        0.000000, range min =        0.000000 *** ignored
x =       56.357743, y =      278.264860, template min =        0.000000, range min =        0.000000 *** ignored
x =       55.377467, y =      267.281864, template min =        0.000000, range min =        0.000000 *** ignored
x =       55.003979, y =      256.261534, template min =        0.000000, range min =        0.000000 *** ignored
x =       55.238414, y =      245.237370, template min =        0.000000, range min =        0.000000 *** ignored
x =       56.080061, y =      234.242880, template min =        0.000000, range min =        0.000000 *** ignored
x =       57.526361, y =      223.311486, template min =        0.000000, range min =        0.000000 *** ignored
x =       59.572917, y =      212.476415, template min =        0.000000, range min =        0.000000 *** ignored
x =       62.213509, y =      201.770602, template min =        0.000000, range min =      222.873985 *** ignored
x =       65.440110, y =      191.226589, template min =        0.000000, range min =      226.363215 *** ignored
x =       69.242912, y =      180.876427, template min =      234.542611, range min =      226.363215
x =       73.610355, y =      170.751577, template min =      245.992704, range min =      228.126198
x =       78.529166, y =      160.882814, template min =      245.342177, range min =      235.153391
x =       83.984391, y =      151.300138, template min =      241.807387, range min =      233.149358
x =       89.959448, y =      142.032676, template min =      261.197423, range min =      240.312100
x =       96.436176, y =      133.108599, template min =      280.689299, range min =      244.053498
x =      103.394887, y =      124.555032, template min =      293.030226, range min =      253.810122
x =      110.814428, y =      116.397976, template min =      312.110335, range min =      255.316986
x =      118.672248, y =      108.662226, template min =      336.101513, range min =      269.964243
   n =  35
mean =  15.00   -2.94
 min =   6.00   -9.00
 max =  28.00    0.00
 std =   2.76    1.24

If images #246-249 are re-aligned using this technique, their output is as follows:

072407/processed/im0246.a.fits      15.622222       -4.222222       0.824322        0.726908       
072407/processed/im0247.a.fits      14.297297       -2.783784       1.352972        0.775822       
072407/processed/im0248.a.fits      13.405405       -3.081081       1.324600        0.784250       
072407/processed/im0249.a.fits      15.000000       -2.942857       2.756810        1.240803       

These std values are similar to the rest of the aligned images, and places the resulting images in better alignment with the reference image and circle:

I have written a script called 'shift2.py' which can read a file of processed images and shift values in the format above, and write the aligned images to a new directory:

> shift2.py 072407/intermediates/shifts.processed.195-249.txt 
072407/aligned/im0195.a.fits        11.416667       -4.729167      
072407/aligned/im0198.a.fits        11.229167       -4.020833      
072407/aligned/im0199.a.fits        10.562500       -4.083333      
072407/aligned/im0200.a.fits        11.895833       -3.937500      
072407/aligned/im0201.a.fits        10.083333       -3.479167      
...
072407/aligned/im0245.a.fits        13.645833       -3.291667      
072407/aligned/im0246.a.fits        15.622222       -4.222222      
072407/aligned/im0247.a.fits        14.297297       -2.783784      
072407/aligned/im0248.a.fits        13.405405       -3.081081      
072407/aligned/im0249.a.fits        15.000000       -2.942857      

I have also written a script called 'loopfits.py' that reads a list of images and presents them all in an image window in an animated loop. Keys can be used to run the animation forward, backward, or reset, somewhat like QuickTime (but all in Python and PyQt). I may add additional key commands to (x, y) shift individual images within the animation (and record the cumulative shifts in both an external file and the header of the image), so that this program could become a mini-FITSRegister:

Although the resulting aligned images are pretty good (enough for stacking, in my opinion), there are some slight variations which are worth noting. In particular, images #214 and #216 (next to the reference #215) appear to be slightly out of alignment based on visual observation of the cloud features near the top of the limb. Running finealign.py on the aligned images suggests why this is so:

> finealign.py 072407/aligned/im0214.a.fits 
   n =  48
mean =  -0.10   -0.23
 min =  -2.00   -4.00
 max =   4.00    0.00
 std =   0.82    0.74
> finealign.py 072407/aligned/im0216.a.fits
   n =  48
mean =  -0.04    0.00
 min =  -2.00   -1.00
 max =   2.00    1.00
 std =   1.84    0.20

Even though the average limb is in good alignment, local parts of the limb are not, especially in image #216. This further indicates the anisotropy of the distortions in these images. Even though the average shift of the aligned image is insignificant, the recommended shift for specific parts of the limb might be several pixels. Therefore, rather than a single geometric object, each image should be considered as a discrete set of points which abberate semi-independently of one another. At the fundamental level this is true, as each image is actually composed of the time integrated superposition of many speckled interferograms. Although readout noise may preclude this, it might be interesting to try to reconstruct deconvolved images from many very short exposure frames, wherein the individual PSFs might be regarded as constant over the frame and in time. Eliot and I did some preliminary work on this topic several years ago which appeared promising.

The second technique I have begun to try is based on radial analysis of the image attenuation. Specifically, I have written a script called 'finealign2.py' that uses the results of finealign.py. This script measures the image values at radial cuts perpendicular to the limb at several different angles. At each major angle, several (3-5) cuts spaced 0.01 radian apart are used to construct an average of image value v. radius at that angle for several pixels inside and outside of the presumed limb. Here is an example using 3 cuts/angle within +-25 pixels (measured as a vector parallel to radius) of the limb:

The idea here is to use the individual plots to locate points on the limb, and then to use the circle fitting routine from roughalign.py and roughalign2.py to fit a circle to the night limb. Although the intensity falloff is clear and sharp at some angles, it is dubious for others. Presumably some measure of both intensity and variation would be used to distinguish foreground from background. However, at present it is unclear how to quantitatively distinguish the location of the limb in each of the radial plots.

Still to do

  1. Continue with automatic night alignment.
  2. Create additional Python/PyQt registration tools.
  3. Complete bracket-gamma subtractions.
  4. Perform all steps of image processing on other days in 7/07.


İSky Coyote 2007