S2 Data Quality Review

Data Quality Activities

I know of the following people working on evaluating the S2 data quality

Betsy has scanned the E-Log and conlog looking for H1 & H2 science mode segments that were out of configuration or had timing issues. Her results can be found here. I believe that this study is still in progress as of 4/21/03.
Brian and the operators at LLO are looking for similar problems in L1. In progress as of 4/21/03.
The Data Quality monitor ran during S2. Some of the statistical measures it recorded are summarized below.
Gaby has plotted Livingston IFO channel trends, histogrammed the esults and recommends eliminating a few segments containing outliers.
Mike Ashley has plotted the average glitchMon glitch rates vs. time (this requires the usual PSU username/password).
Peter Shawhan and Keith Riles have generated a list of locked segments, and are keeping a data base of information about each segment. Keith has a (preliminary) graphical view of these segments.
I have reviewed the data near BitTest triggers for indications of DAQ dropouts or LSC system saturation in the AS_Q channels.
I have scanned reconstructed L1:LSC-AS1_Q data for repeated data. This reveal numerous data stretches where the photodiode or ADC were saturated.

Note: As of now all results should be considered preliminary.

Data Quality Monitor (DataQual)

A data quality monitor based on the PSLmon program was running during the entire S2 run. This monitor made trends of band limited RMS and glitch rates on several channels. The DataQual configuration files can be viewed by following these links for the LLO configuration and for the LHO configuration. The current status of the monitors at LHO and LLO -may also be viewed.

S2 Summary trends

The trends of the wide AS_Q bands' RMS and the 6-sigma Glitch rates are available as pdf files:

Band Limited RMS	H1	H2	L1
Glitch Rates	H1	H2	L1

The plotted RMS points are average variance (sigma^2) within the specified band for each 1-minute period when the interferometer was in science mode. Each plot corresonds to 1 day and the x axis is the GPS hour (note that this differs from UTC by the net number of leap-seconds, 13 seconds during S2). The bands are indicated in the title of each plot and are 10-100Hz, 100-200Hz, 200-400Hz, 400-1kHz and 1kHz-7kHz. The error bars are calculated from the distribution of the 15 4-second RMS measurements made during each GPS minute. A histogram of all points is plotted at the beginning of the trends for each band. I have plotted a line at an arbitrary value on some of the bands to "guide the eye".

Extreme Rates

I scanned the trends for the lowest and highest rates over a period of 10 minutes found on the AS_Q channel by the 4-sigma, 100Hz. The 10-minute averages The results are as follows:

Interferometer	H1	H2	L1
Maximum average rate (Hz)	5.45	5.99	25.68
Start time of Max. period (GPS)	729730080	731564100	734359740
Minimum average rate (Hz)	0.0116667	0.0716667	0
Start time of Min. period (GPS)	729695640	733074300	731489100

DataQual Notes:

H1

Rms: H1 is as expected the most stable by far. I think that a simple cut on average rms might actually be useful for the 200-400Hz and 400-1kHz bands. Unfortunately the 100-200Hz band seems to be dominated by the 110 Hz peak that came and went much of the run. The green lines give a zeroth order cut with very little loss of live-time.

Glitches: The LHO glitch rate measurements before March 5, 2003 are bogus because the filters were wrongly specified before this time (Note: that this has now been rerun for most times JGZ 6/9/03).

H2

Rms: All the bands in H2 vary over a range of at least 1 order of magnitude. This must be tracked by the analysis codes so it may not be a major concern. There are places however where there is an unusually large minute to minute variation in the rms which may be problematic for the analysis and should probably be flagged (see the 400Hz and 1kHz bands on April 13th 16-24h for example - FWIW, the E-Log says that the wind speed was high during this time).

Glitches: The LHO glitch rate measurements before March 5, 2003 are bogus because the filters were wrongly specified before this time (Note: that this has now been rerun for most times JGZ 6/9/03).

L1

Rms: L1 has quite a variety of effects going on. The ringing down of optic and suspension resonances is very obvious in several bands. There is also a rapid increase in noise rms at the end of some lock segments, presumably due to deteriorating alignment. In some locked segments the IFO never reaches a stable state because of one or both of these effects. I think that we should at least flags these periods, and should probably recommend that they not be used for analysis. There is also an interesting periodic variation in the 100-200Hz band on April 13th 15-24UTC. The 400-1kHz Band is dominated by the 927Hz calibration line after it was increased in magnitude. There are a few points where the 400Hz band RMS decreases below the level expected from the calibration lines. The calibration line is probably missing at these points.

BitTest Triggers

BitTest monitors a list of channels for apparent stuck bits or repeated values. During S2, the BitTest processes were configured to generate a trigger any time the value of one of the AS_Q channels was repeated at least 32 times. Since the AS_Q channels vary continuously, these repetitions tend to indicate DAQ system dropouts or saturation of the AS_Q signal. To get more detailed information than the BitTest triggers which are produced at most once every 20 minutes, I rescanned the data in the interval indicated by the trigger for repeated values in the specified AS_Q channel. Each such repeat group was summarized for LHO and LLO. These files contain the following information for each repeat group:

The channel name.
The GPS time of the first sample truncated to 1 second.
The ID of the segment (from Peter's list ) that contains the repeat group or "-".
The repeated value
The number of times the value was repeated (note that a sequence containing N samples of the same value would have a repeat count of N-1).

The following effects can be seen in the data:

H1:LSC-AS_Q is set to zero for ~½ second (usually 7790 samples) ~10-13 seconds after the end of each lock. This is probably a side effect of the autolock script.
H2:LSC-AS_Q is seen to saturate at various values (the most frequent are +616 and -609). These are usually when the IFO is out of lock, although in two cases the saturation occurs at the end of the locked segment and may have been connected with the loss of lock.
In two cases H2:LSC-AS_Q went on strike (values of zero) for exactly 1 second during a lock stretch. It is not clear whether this was a DAQ dropout or whether the data were lost somewhere else along the line.
L1:LSC-AS_Q saturated regularly at a wide range of values. All detected saturations were while the IFO was out of lock.
L1:LSC-AS_Q also had multiple DAQ dropouts. Several 1/16th second dropout occurred in clusters while the IFO was in lock. Several 1s dropouts were also seen, but didn't affect the locked segments.

The repeat groups occuring during locked segments are summarized in the following table:

Segment	Time (GPS)	Duration (s)	Comments
H2-56	730022077	1	Saturated at -609.526
H2-58	730058792	1	Saturated at 616.316
H2-232	732171047	1	1 second DAQ dropout
H2-407	734255477	1	1 second DAQ dropout
L1-17	729415602	33	Multiple 1/16th s DAQ dropouts
L1-303	731655665	6	Multiple 1/16th s DAQ dropouts

Scan of reconstructed L1:LSC-AS1_Q data

I have scanned the L1:LSC-AS1_Q data from S2 science mode segments for saturation by looking for data that were repeated more than 4 times. Because the photodiode data were not recorded, the data were reconstructed by running the correct linear combination of L1:LSC-AS_Q and L1:LSC-AS_I through the inverse of the dewhitening filter. The raw results are listed here. Each line of the results file indicates one or more groups of repeated data in the same GPS second. If more than one repeat group was found during the same second, the repeated value for the first group and the sum of the repeat counts are listed.

For the most part the repeated values were the minimum or maximum values representable by a 16-bit data word (-32768 or 32767). Of the 496 seconds with repeated data, 329 were in the last 1-2 seconds of the lock segment and are most likely associated with the loss of lock. I would suggest removing these times from the ends of the segments in the next version of the segment list. An additional 13 seconds correspond to previously identified DAQ dropouts (in L1-17 and L1-303). The remaining times should at least be flagged. in the segment lists.

Narrow resonances (line features) seen in L1:LSC-AS_Q

There are three obvious line features seen in L1:LSC-AS_Q. These are summarized in the table below

Frequency (Hz)	Decay constant (sec^-1)	Comments
340-345	0.016	Suspension Violin mode
680-690	0.030	Violin mode second harmonic
3730	0.0025	Beam splitter butterfly(?) internal mode.
5623-6813	0.018	Test mass butterfly(?) internal mode.

The first 3 to 60 minutes of each lock segment were fit to sqrt(a^2*exp(-2bt)+c^2). The results of these fits are displayed in the pdf files linked by the comment lines in the table.

Search for Rms-Based Epoch Vetos

Several questionable time-segments have been identified based on As_Q Rms and/or glitch rates. Since the As_Q channel is (hopefully) higly sensitive to gravitational waves, it is dangerous to veto these time periods based only on As_Q. I have therefore searched the Band-limited Rms of other channels in hopes of finding an indication of the effect seen in As_Q in an insensitive auxillary channel.

The following table gives times and links for glitchy segments that were used to search for epoch vetos. The sixth column of the table contains a link to a list of the 20 channels with the largest differences in one or more bands. The band given is the band with the greates fractional difference in power (i.e. x = (sqrt(Pn) -sqrt(Pq)/sqrt(Pn+Pq) where Pn is average power in band for the noisy segment and Pq is the same for the quiet segment). Note that L1:LSC-AS_DC figures prominantly in the L1 lists. This is not surprising in that this channel has been identified as possible veto channel, for the high frequency noise.

The 'Spectra' column contains links to spectra from several of the listed channels. For each channel and segment pair, the red curve is the spectrum of the noisy section and the blue curve is the spectrum of the nearby quiet spectrum.

The last column contains a trend of the power in several bands of a selected channel for the entire S2 period. These can be compared with the glitch rate trends linked above.

IFO	NoisyDate (UTC)	Noisy (GPS)	Comparison (GPS)	Epoch Length	Channel List	Spectra	Trend (Channel)
H1	Feb 28 21:46	730504000	730524000	600s	h1-list-1	pdf	H2:LSC-AS_Q
H2	Mar 4 14:40	730824000	730814000	1000s	h2-list-1	pdf	H2:LSC-POB_Q
H2	Mar 6 20:00	731016000	731009000	1000s	h2-list-2	pdf
L1	Feb 16 20:03	729461000	729443000	1000s	l1-list-1	pdf	L1:LSC-AS_DC
L1	Mar 15 20:40	731796000	731825000	1000s	l1-list-2	pdf	L1:LSC-AS_DC

Potential epoch vetos are listed in the following table

IFO	Channel Name	Band (Hz)	Threshold¹ (ct²/Hz)	Duration (min)	Vetoed² (min)	Dead (%)
H1	H1:LSC-AS_Q	200-400	2.0e-4	5	<344	<0.6
H2	H2:LSC-POB_Q	46-56	0.5	-	295	0.6
L1	L1:LSC-AS_DC	300-1k	1.0	-	2485	7.7
L1	L1:LSC-AS_DC	1-10	1.0e+7	-	3792	12.3

Notes:

The links from the "Threshold" column contain 3 plots:

a scatter plot of the 6-sigma glitch rate versus the log of the Veto Rms
a glitch rate histogram for all minutes in segments and for all minutes with the veto Rms > threshold (in Blue). The green vertical line indicates the cut used in the blue portion of the next plot
a Veto RMS histogram for all minutes in segments and (in blue) for all minutes with the glitch rate > a specified cut (usually 0.1 Hz). The vertical line gives the threshold used for the previous plot and the red curve gives a quality factor defined by Q=100*<live_fraction>^2/<noise_fraction>.

The link from the "Vetoed" colum contains a list of times with the specified vetoes. To assess the effect of the minumum duration cut in the H1 veto, all segments with >1 vetoed minute in a 10 minute interval are listed. The "<344" indicates the number of minutes that would be vetoed if all listed minutes were included.

Suspension Sensor Glitches

During the S3 running, Daniel Sigg noticed that there are numerous large glitches in the large optic suspension sensor (OSEM) signals. These can potentially be fed into the interferometer length in those optics where the sensor signals are used for local damping during running (notably the ITMs). I am in the process of generating triggers for these glitches on all SENSOR channels during the S2 Run. Each trigger jobs is run for 100,000 seconds of data. The following table contains the appropriate links. Each GPS time range contains a link to an xml file with the Sensor triggers for that time range. Note that if you click on the links instead of downloading them directly, your web browser may try to view the XML and choke.

730000000-730099999	730100000-730199999	730200000-730299999	730300000-730399999	730400000-730499999
730500000-730599999	730600000-730699999	730700000-730799999	730800000-730899999	730900000-730999999
731000000-731099999	731100000-731199999	731200000-731299999	731300000-731399999	731400000-731499999
731500000-731599999	731600000-731699999	731700000-731799999	731800000-731899999	731900000-731999999
732000000-732099999	732100000-732199999	732200000-732299999	732300000-732399999	732400000-732499999
732500000-732599999	732600000-732699999	732700000-732799999	732800000-732899999	732900000-732999999
733000000-733099999	733100000-733199999	733200000-733299999	733300000-733399999	733400000-733499999
733500000-733599999	733600000-733699999	733700000-733799999	733800000-733899999	733900000-733999999

PEM Investigations

Note that this section is still under construction.

LHO Airplane Events

I believe that S2 airplane overflights can be spotted most easily by looking at the band-limited RMS of microphone channels in the 60-110Hz band. Fortuately the RmsBands monitor has trended the 62-100Hz band rms of 5 microphones at LHO during the S2 and S3 runs. They are listed in the following table.

Channel Name	Location
`H0:PEM-PSL2_MIC`	H2 PSL table (LVEA)
`H0:PEM-BSC5_MIC`	X mid-station
`H0:PEM-BSC6_MIC`	Y mid-station
`H0:PEM-BSC9_MIC`	X end-station (poor data quality)
`H0:PEM-BSC10_MIC`	Y end-station

The X-end station (BSC9) microphone seems to have been glitching frequently during S3 so I will ignore it in this discussion. I have plotted the rms of the other four microphones in the 62-100Hz band. Each page of plots shows the RMS in the maximum 4 seconds of each 1 minute bin for all four microphones during a single day of S2. The x-axis is the UTC hour. The red line indicates times when H1 was running in science mode, and the green line is an arbitrary threshold (to 'guide the eye').

A quick visual scan of the microphone channels yields the following periods of high noise levels. I checked the eLog during the specified times and came up with the possible reasons for the elevated noise levels described in the comments field.

Start (UTC)	Start (GPS)	Duration (s)	Comments
Feb 14 00:45	729218700	1800	PSL2 noise burst (Check FMY Hardware filters)
Feb 14 02:15	729224100	2700	BSC6 noise burst
Feb 14 09:15	729249300	3600	PSL2 noise burst
Feb 15 23:00	729385200	2400	Missing data
Feb 17 07:45	729503100	4500	Wind (>40 MPH)
Feb 19 04:00	729662400	900	Missing data
Feb 20 19:00	729802800	16200	Wind (20 MPH)
Feb 21 03:00	729831600	4500	Wind (30-40 MPH)
Feb 21 20:00	729892800	14400	Wind (20 MPH)
Feb 22 00:00	729907200	7200	Wind
Mar 04 21:00	730846800	21600	Missing data
Mar 05 20:30	730931400	12600	Wind
Mar 06 03:30	730956600	27000	Wind
Mar 06 19:45	731015100	31500	Wind
Mar 11 18:15	731441700	27900	Wind
Mar 12 05:30	731482200	600	Wind.
Mar 13 00:45	731551500	1800	PSL2 noisy. PEM RF Injections? Robert in LVEA
Mar 13 20:00	731620800	1800	PSL2 noisy. PEM Injections? (Acoustic)
Mar 14 20:00	731707200	3600	PSL2 noisy. Robert in LVEA?
Mar 16 03:20	731820000	1200	PSL2 noisy??
Mar 16 04:35	731824500	6000	PSL2 noisy??
Mar 17 01:30	731899800	1800	missing data (PSL2, BSC6)
Mar 17 15:10	731949000	23700	missing/corrupt data (PSL2, BSC6)
Mar 20 04:15	732168900	5400	Wind?
Mar 21 03:20	732252000	600	PSL2 noisy; Testpoints?
Mar 21 07:40	732267600	600	PSL2 noisy; Testpoints?
Mar 21 08:25	732270300	600	PSL2 noisy; Testpoints?
Mar 21 12:50	732286200	1200	PSL2 noisy; Testpoints?
Mar 21 14:15	732291300	1800	PSL2 noisy; Testpoints?
Mar 21 15:05	732294300	900	PSL2 noisy; Testpoints?
Mar 21 15:40	732296400	600	PSL2 noisy; Testpoints?
Mar 24 08:00	732528000	2700	PSL2 noisy
Mar 24 09:00	732531600	900	PSL2 noisy
Apr 01 16:30	733249800	9900	BSC5 data missing/corrupt
Apr 04 17:00	733510800	21600	data missing/corrupt

Once these bad periods have been eliminated, candidate airplane events can be found by looking at the RMS in the band 62-100Hz of


H0:PEM-PSL2_MIC

(this is the only LVEA microphone for which this band was trended). The following figure shows the RMS of the loudest 4 second sub-interval in each minute of S3 running. Except for the noisy periods listed above, no selection was made for e.g. IFO being in lock. Note that the red line indicates the threshold used below (1500). This seems to be within ±200 units of the break between a gaussian background peak and a power law amplitude distribution. The blue line is the power (11127) of the airplane at GPS 732000700 that was found in the S2 science data by WaveBurst. Note that some DAQ related noise may still be contained in this distribution.

Maximum 4 second RMS in 62-100Hz band for all minutes of S3

Maximum 4 second RMS in 62-100Hz band for all minutes of S3

The maximum RMS in the 62-100 Hz band averaged over all minutes of S3 of H0:PEM-PSL2_MIC varies with the time of day as shown below. The x-axis is the UTC hour. Note that there are no noise spikes at night (midnight - 6 AM local time corresponds to 8-14 UTC), but large spikes in the average noise are visible at certain times (scheduled flight times?). Note that some DAQ related noise may still be contained in this distribution.

Power in H2 PSL mic vs Time of day

A complete list of airplane candidates seen while the interferometers were in science mode can be found here for H1, H2 and H1H2L1 triple coincidence. Each line of this file has the start time in GPS seconds and as a UTC date string, the number of minutes that the rms was above the threshold, the RMS value and the H1 segment number. The 20 loudest "Airplane" candidates seen in triple coincidence running (H1, H2, L1 were simultaneously in science mode) are:

Start Time (GPS)	Start Time (UTC)	Duration (min)	RMS	Segment
732778020	Mar 27 05:26	1	5022	HL-310
733555620	Apr 5 05:26	1	5145	HL-388
729845460	Feb 21 06:50	1	5444	HL-054
731231100	Mar 9 07:44	1	5617	HL-177
730877640	Mar 5 05:33	1	6439	HL-140
733383120	Apr 3 05:31	2	6935	HL-364
734218200	Apr 12 21:29	1	7200	HL-432
730588200	Mar 1 21:09	1	7350	HL-105
729555840	Feb 17 22:23	2	7831	HL-030
732864480	Mar 28 05:27	1	7895	HL-317
733699080	Apr 6 21:17	2	8520	HL-404
729552600	Feb 17 21:29	1	9037	HL-029
732678060	Mar 26 01:40	1	10034	HL-298
730417260	Feb 27 21:40	1	10446	HL-080
732000660	Mar 18 05:30	2	11127	HL-209
731741700	Mar 15 05:34	1	13551	HL-204
733685220	Apr 6 17:26	1	17308	HL-402
734149380	Apr 12 02:22	2	42866	HL-427
731679480	Mar 14 12:17	1	53838	HL-199
731140800	Mar 8 06:39	2	260674	HL-160

Note that the row highlighted in red corresponds to the time at which the WaveBurst candidate was found. The rows highlighted in yellow have signals only in the LVEA.