S4 Data Quality Review

Data Quality Activities

I know of the following people working on evaluating the S4 data quality Note: As of now all results should be considered preliminary.

The following table summarizes the live-time loss of each of the data problems. All times are in minutes and percentages are the fraction of single-ifo science modes times.
 
Symptom H1 H2 L1
ADC Overflows 22  (0.06%) 146  (0.42%) 59  (0.19%)
Calibration line 1s dropout 7  (0.02%) 10  (0.03%) 7  (0.02%)
Calibration single sample dropout 1  (<.01%) 1  (<.01%) 1  (<.01%)
Optical coupling 2037  (5.95%) 1177  (3.41%) -  (0.00%)
High Winds 4290  (12.54%) 3359  (9.72%) -  (0.00%)
Acoustic 1377  (4.02%) 1765  (5.11%) 2927  (9.25%)
Seismic 2359  (6.90%) 2131  (6.17%) 1536  (4.85%)
Total science mode (min) 34,213  (100.%) 34,563  (100.%) 31,654  (100.%)

Still to be done:

  1. WFS not running as usual.

Data Quality Monitor (DataQual)

A data quality monitor based on the PSLmon program was running during the entire S4 run. This monitor made trends of band limited RMS and glitch rates on several channels. The DataQual configuration files can be viewed by following these links for the LLO configuration and for the LHO configuration. The current status of the monitors at LHO and LLO  are also available online.
 

S4 Summary trends

Trends of AS_Q band RMS and Glitch rates are available as pdf files. Warning: The narrow band rms files are enormous (750 pages, ~150MB).
 
As_Q band limited RMS (narrow bands - LARGE Files) H1 H2 L1
As_Q band limited RMS (wide bands) H1 H2 L1
Glitch Rates H1 H2 L1

These plots currently cover the entire S4 run)  (February 22, 2005 14:00 UTC through March 24, 2005 ~6:00 UTC. The plotted RMS points are average variance (sigma^2) within the specified band for each 1-minute period while the interferometer was locked and in common mode. The glitch plots show the glitch rates in Hz, averaged over 1 minute. Each plot corresonds to 1 day and the x axis is the GPS hour (note that this differs from UTC by the net number of leap-seconds, 13 seconds during S3?). The bands are indicated in the title of each plot. The error bars are calculated from the distribution of the fifteen 4-second measurements made during each GPS minute. A histogram of all points is plotted at the beginning of the trends for each band. Thick red bars near the bottom of each trend plot indicate the science mode segments. On some plots I have also drawn a green line at an arbitrary value to "guide the eye".
 

Scan of DAQ overflow channels

I have scanned the minute trends for non-zero overflow counts as measured by the DAQ overflow channels for the LSC subsystem. At present, errors are scanned and tabulated through ~18:00 UTC 2005-03-24. All channels named Xn:LSC-*_OVERFLOW were scanned except Xn:LSC-MASTER_OVERFLOW and Xn:LSC-RESET_OVERFLOW. The RESET_OVERFLOW channels are supposed to have a single count each time the MASTER_OVERFLOW counter is reset (nominally once per second).

 Times with overflows are summarized in the following table and the raw results are in H1, H2 and L1. Note that minutes that are not completely contained in a lock segment are ignored. The 1-minute cut around each of these overflows is arbitrary and may not cover all the bad data in the region of the overflow, as shown in the following figure:
 

Calibration lines

Calibration lines should have been injected throughout the S4 run, but it appears that there may be stability problems in the injection. To verify the stability of the calibration lines, I ran a 6 sigma glitch finder on the excitation channels in the RDS (Xn:LSC-DARM_CTRL_EXC_DAQ) after notching out the three calibration lines. In theory, once the notch filters have settled, the resulting signal should be just the difference between the measured signal and a perfect sine wave. If the only error is numeric round-off from the single precision float used to store the data, the signal should be A/2^24 ~ 7e-9.

Dropout rates and classification

To get an overview of the calibration line dropouts, I summarized the trigger rates in the glitch finder minute trends by counting the number of minutes with non-zero trigger rate for each interferometer. The dead-time resulting from calibration line glitches is tabulated here for H1, H2 and L1 from the start of S4 through ~March 18th. The fraction of data in which glitches are seen in the excitation channels is presented in the table below.
 
Ifo Minutes with Glitches Fraction with Glitches
H1 146 0.56%
H2 12574 49.12%
L1 54 0.24%

It is clear that the rate of glitches found is drastically different for H2 than it is for H1 and L1. In fact the H2 data seem to exhibit two modes. From 2/26 10:35 UTC - 3/8 16:03UTC nearly 100% of the data were lost. The rest of the time, a relatively small fraction shows glitches. As described below, the high-rate H2 glitch are of very small amplitude and can probably be ignored. I then summarized the results by classifying triggers into the following groups:

1 Second Dropouts

The calibration lines dropped to zero on occasion for 1 second. I selected these drop-outs by looking for triggers with amplitude aproximately the same as the calibration line amplitude (H1 minval=0.15, H2 minval=0.11, L1 minval=0.48) and a fractional part of the trigger time ~ 0.875. In general, eac dropout produced two such triggers: one at the start of the dropout and one at the end of the dropout. To avoid double counting the dropouts, all triggers that followed another dropout by exactly one second were eliminated. The resulting list of trigger times are listed here for H1, H2 and L1. These dropouts have been used to form data quality segments with the CALIB_LINE_DROPOUT flag.

Single Sample Dropouts

Single-sample dropouts have been discovered in all the interferometers. These were first noticed at the end of injection periods, but some have been found independent of hardware injections. An example of one such dropout is shown in the figure below:
 
The single sample droput triggers are selected by looking for trigger with very high significance (H1 minsig=110, H2 minsig=110 and L1 minsig=59) and a fractional time of 0.936767578 or 0.937011719. Complete lists of the glitches are here for H1, H2 and L1.

Periodic H2 micro-glitches

As described above, the H2 interferometer calibration line excitation channel seems to have a periodic glitch. The figure below shows the H2 data at GPS 793450069 (2/26 10:47 UTC). There is clear pseudo-random noise with signal ~5e-9. On top of this, every ~3.8 seconds there is a ~1200Hz ring-down with an initial amplitude of ~6e-8. It would appear that there is some change in the calibration line parameters (Amplitude or Phase) every 3.8s during the "noisy" H2 running mode.
 
More extensive investigation of these glitches shows that the interval between glitches is not uniform in the short term (e.g. for part of the time, the glitch intervals followed a pattern of 3 glitches each separated from the previous by 4.2s followed by a fourth glitch after 2.2s) or over the long term (the inter-glitch period expanded to 6-8s). A complete list of triggers is available here.
 

Front-end synchronization errors

I scanned for front-end synchronization errors by looking at minute trends of the following channels:
Xn:SUS-*_SYNC, Xn:[AL]SC-ETP_00
 

Optical Coupling

When one of the Hanford interferometers is out of lock, the poorly controlled mirrors can reflect stray light into the other interferometer. For this reason, H1 data should only be analysed when H2 is in lock (although not necessarily in science mode) and vice versa. The science mode segments where this problem could occur were found by scanning state vector value in the LockLoss monitor minute trends and demanding that the "other IFO" was in a state ≥ 2. The segments where H1 was in science mode and the H2 LockLoss state vector < 2 and those where H2 was in science mode and the H1 LockLoss state vector < 2 are tabulated here. Note that these are conservative lists in that if an interferometer was unlocked for any part of a specified minute a veto is generated for the whole minute.

Wind (at LHO)

The local wind speed is reportes from five stations at LHO located at EX, MX, LVEA, MY and EY. The wind speed at LLO was never seen to exceed 20 MPH. Windy conditions were much more of a problem at LHO where the wind gusted strongly enough to knock the interferometers out of lock on several occasions. I have tabulated locked segments where the peak wind speed excedes 20 MPH as inferred from the H0:PEM-<station>_WINDMPH minute trends. In doing this I notice that several minutes have mismeasured wind speeds (>100 MPH are listed here). The times with excess wind speed that were isolated from other windy segments were removed from the final list of windy segments which can be seen here for H1 raw, summary and here for H2 raw, summary.
 

Tighter Cuts (>35 MPH)

Robert suggested rerunning the wind scans with higher thresholds, specifically >30~MPH at the LVEA and >35~MPH at the end stations. The resulting segment files are here for H1 raw, summary and here for H2 raw, summary. Again, there were apparent hardware errors which appear as excessive wind speeds. These were removed from the summary file. The Mid-Y station didn't measure any non-pathological wind speeds >35~MPH. This reduces the dead-time significantly (337 minutes instead of 4290 minutes for >20~MPH).

Acoustic Noise

The RMS in the band from 62-118Hz (61-119Hz at LLO) was trended for all microphone channels. The maximum 10 second RMS in each minute of each channel is plotted in pdf files for LHO and LLO. It should be noted that several of the LLO microphones switch between 2-3 RMS levels. This seems to be a periodic broad-band noise increase, perhaps from fans turning on and off. The following spectrum shows the L0:PEM-BSC5_MIC acoustic noise spectra in the low (black - GPS793179008) and high (red - GPS793177216) noise states.
 

There are several segments in which wind, etc causes noise for a prolonged period. At LHO these times are:
 
Start (GPS) Date (UTC) Duration(s) Comments
793321200 Feb 24 23:00 32400 PEM Injections?
793758000 Mar  2 00:20 6000 Work on ISCT4
793834200 Mar  3 21:30 5400 LSB chiller vibration damping installation
794336400 Mar  8 09:00 6600 Misc. repairs in LVEA (PSL water, zero RM oplev, move BSC1_ACCY)
794682000 Mar 12 17:00 27000 High winds
794776500 Mar 13 19:15 8100 High winds
794953800 Mar 15 20:30 3600 Entered LVEA for maintenance
795029460 Mar 16 17:30 32400 High winds
795283200 Mar 19 16:00 18000 Repairs in LVEA?
795363300 Mar 20 14:15 3600 Noise in Ex (log says localized wind!)
795403800 Mar 21 01:30 41400 Wind
795448800 Mar 21 14:00 3600 Noise in Ey
795625200 Mar 23 15:00 32400 High winds

At LLO these times are:
 
Start (GPS) Date (UTC) Duration(s) Comments
793227600 Feb 23 23:00 5400 Stormy weather
793314000 Feb 24 23:00 7200 Stormy weather
793683000 Mar 1 03:30 3600 Fire alarm (false)
794971800 Mar 16 01:30 3600 ???
795355200 Mar 20 12:00 7200 Stormy weather

In general the L0:PEM-BSC4_MIC (X end) channel is much noisier than other locations. This would argue for increasing the threshold on BSC4 or removing the BSC4 microphone from any audio veto. The segments with elevated noise are tabulated for H1 (raw, all channels), H2 (raw, all channels) and L1 (raw, all channels).

Seismic Noise

I have created trends of the seismic noise in the 0.8-2Hz bands. These are plotted as PDFS for LHO and for LLO. The segments with elevated seimic noise in this band are tabulated for H1 (raw, all channels), H2 (raw, all channels) and L1 (raw, all channels).

Wave-Front Sensors

 

Last modified:April 18, 2005 11:00 PDT by  John Zweizig