Category: Software

Posted on

New python utility

Hi All,

Recently Casandra prepared the Monthly Observing Hours 2021 for Google Sheets to keep track of total observing time. I have written a small python script that automatically totals up the observing time for a month from the Observing Log and writes it in hours to the Monthly Observing Hours. It’s usage is as follows:

From the Documents folder on MonitorYTLA:

python obshours.py

This will default to the current month, so ideal usage would be to run at the end of the month after all observations for that month are completed. However, there is an option to specify the month you are totaling:

python obshours.py YYYY-MM

Let me know if you have any questions.

-Adam

Posted on

secondary gain calibration

Introduction

We are currently lacking a strong planet during observation that can be used as a primary calibrator (bandpass/flux/phase). To fulfill the calibration requirements, we have the following:

  • bandpass cal / flux conversion
    • –> ‘auto_planet_cal.py mean‘
    • Mars mean bandpass (for 1.92GHz bw)
    • Each night, the digital delay may vary and is not accounted for in the mean bandpass cal.
    • The flux conversion is fixed. As long as the loading factor is stable, so will the derived SEFD. A separate gain cal is needed.
  • delay cal
    • –> ‘solve_delay.py‘ on a weaker planet or a strong quasar
    • phase center is set to this calibrator
  • gain cal (new)
    • –> ‘calibrate_avgh5.py‘ with the ‘-sec‘ option
    • If a planet is used as the secondary calibrator, or if the source flux is known, the absolute gain can be obtained.
    • Otherwise, the relative gain that flattens the flux across baselines is returned.
    • phase center is reset to this calibrator
  • to apply the calibration to other targets:
    • –> ‘copy_table.py‘ (new)
    • This script replaces the ‘correct_delay.py’ and supports copying other calibrations.

Example:

The following is the SEFD from Mars tracking data (on 210106):

  • Original process (mean bandpass + delay cal):
## antenna SEFD: (MJy)
         Ant0   Ant1   Ant2   Ant3   Ant4   Ant5   Ant6
LSB::    1.07   1.00   1.43   0.64   3.03   0.90   1.05
USB::    1.11   0.95   1.33   0.59   2.33   0.86   0.94
--
## antenna SEFD: (MJy)
         Ant0   Ant1   Ant2   Ant3   Ant4   Ant5   Ant6
LSB::    0.98   1.09   1.39   0.59   2.30   0.83   1.15
USB::    0.95   1.10   1.29   0.57   1.55   0.84   1.08

Spectra:

  • (new) Additionally with the secondary gain cal:
## antenna SEFD: (MJy)
         Ant0   Ant1   Ant2   Ant3   Ant4   Ant5   Ant6
LSB::    1.07   0.94   1.52   0.67   3.04   0.97   0.98
USB::    1.11   0.87   1.41   0.63   2.42   0.93   0.87
--
## antenna SEFD: (MJy)
         Ant0   Ant1   Ant2   Ant3   Ant4   Ant5   Ant6
LSB::    0.99   1.01   1.51   0.63   2.28   0.90   1.49
USB::    0.95   1.01   1.38   0.62   1.58   0.92   1.50

Spectra:

— Kyle

Posted on

compare_sefd.py

I have updated the SEFD plotting tool (formerly ‘plot_sefd4.py’) into ‘compare_sefd.py’ and moved it into the pipeline directory.

It retrieves SEFD analysis results from the .avgh5 files in the current directory and saves the summary plots into a sub-folder ‘figures/’.

It also requires a mandatory argument of ‘original’ or ‘renorm’, that specifies whether the original SEFD values or the renormalized SEFD values should be used. The idea of the renormalization is to separate the weather effect out of the stability analysis. The steps are:

  • From multiple datasets, normalize the SEFD of each antenna by its median value.
  • Take the mean (over antennas) of the normalized SEFD from the same dataset as its level.
    • Some antennas are excluded:
      • Ant3 due to its drastic change before and after using the Al-coating
      • Ant4 due to its poor bandpass
      • Ant6 due to power fluctuation (in X)
  • The gain is defined as the level divided by its median over datasets.
  • Renormalize the SEFD by the gain
    • The gain of each dataset is shown in the SEFD vs. date plot
    • Percentage variation (in RMS) of the SEFD is also shown for each antenna (see examples below).

The plots shown below uses the same datasets Casandra used in her post.

Original SEFD:

(command:  compare_sefd.py o –exclude 6)

Renormalized SEFD:

(command:  compare_sefd.py r –exclude 6)

After normalizing out the overall gain (weather effect), we are only seeing 2% to 4% variation in RMS except:

  • Ant1 had a slightly larger variation since its secondary was taped with foil (Nov_20; last 3 datasets)
  • Ant3 swapped from the original dish to an Al-coated one (since Sep/25; 3rd dataset).
  • Ant4 needs a new bias setting, and Ant6 were a bit unstable?

— Kyle

Posted on

mean bandpass with Mars for the 1.92GHz BW correlator

Since Jupiter is no longer available during the night time observations after we switch to the 1.92GHz bw configuration, we have obtained about 10 sets of 60-min tracks on Mars from late October to early December when Mars was brighter than 200Jy. And since Mars flux is diminishing quickly, we will rely on the mean bandpass derived from these Mars observations for future calibrations. Here I will summarize the steps of this analysis.

Procedures and results

  • 10 sets of data from Oct/21 (497Jy) — Dec/04 (194Jy).
  • Raw bandpass: individual bandpass (measured vis. / source flux; complex) are derived with the updated Mars flux estimates. The raw bandpass has been spectrally smoothed with a Gaussian window (sigma = 6ch). 

X-pol: 

Y-pol: 

  • Delay correction: each setup of the correlator may introduce a different delay to the complex bandpass. this is fitted and removed from each data set on a per baseline basis.

X-pol: 
Y-pol:

  • Scaled bandpass: an individual bandpass is also affected by weather and/or instrumental gain effects (e.g. misalignment, bias error, etc.). Note that later stage gain variation is mostly removed when the cross-correlation was normalized by the auto-correlation. The dominant effects should mainly come from the RF section. Here I simply scaled the individual bandpasses to the mean level. Then I performed a weighted average of the scaled bandpasses (weighted by the gain). The thick gray band in each plot indicates the derived mean bandpass.

X-pol:
Y-pol:

  • Deviation: deviation of the individual bandpass from the mean is also examined (in both amplitude and phase). To guide the eye, I have indicated [-2*std, 2*std] by the filled gray area, where ‘std’ is the standard deviation (of the 10 data sets) in each channel. The mean value of the ‘std’ between channel [20, 820] (as indicated by the vertical dashed lines) is written in each subplot. For amplitude, the ‘rms’ (or ‘std’) is a fractional change. For phase, the ‘rms’ is in radian.

Note that for 1-hr integration and 10-ch smoothing, the sensitivity we expect from each channel is about 4Jy. Therefore a 1% to 2% thermal noise is expected in the individual bandpasses. Most of the deviations are consistent with thermal noise, while Ant4 (both X and Y) showed larger variation in amplitude.

Similarly, the thermal noise translates to about 0.01 to 0.02 rad deviation in phase. Only a few baselines are consistent with thermal noise in phase. Most show a little spectral structure. Ant2Y showed the most significant change in phase structure (0.03rad to 0.04rad in rms, or about 2deg). Do we have a corresponding hardware change during this period?

X-pol: 

Y-pol: 

Summary and discussion.

  • The mean bandpass has been derived for the 1.92GHz bw configuration. It is available for use in the ‘auto_planet_cal.py’ and ‘calibrate_avgh5.py’ scripts. The scripts would recognize the data being processed and look for the appropriate mean bandpass (e.g. Mars-derived for 1.92GHz new data, and Jupiter-derived for 2.24GHz old data).
  • With only a limited sample of 10 data sets, we can only roughly estimate the bandpass variation. It is roughly a 2% effect in amplitude and a 2deg effect in phase.
  • We also noticed a small ripple in the bandpass phase response. The ripples are separated by roughly 200MHz. Ranjani estimated the corresponding length of the multi-reflection to be about 50cm in co-ax, or maybe 70cm in air.  Perhaps this is from between the feedhorn and the antenna? It is quite stable though.

 

— Kyle