Weekly Report for Week Ending Jan 04, 2007

Past Weekly Reports

LIGO Hanford Observatory (LHO) and Interferometer Operations (Raab)

LIGO Livingston Observatory (LLO) and Interferometer Operations (Giaime)

S5 Run (Valera Frolov):

The science data taking uptime was 70% during the two week holiday break. The downtime was mostly due to the seasonally high micro-seismic activity. The problem with Y end station building temperature control caused the temperature change but the disruption of the data taking was minimal since the repair was done during the power outage at LHO. The BNS range was 12-13 Mpc most of the time. There were two interferometer related problems reported during this period: one was a new 45 Hz line and harmonics in the DARM spectrum, which was tracked to the RM optical lever laser, and the other was the low frequency 3-4 Hz intermittent noise in DARM, which is still under investigation. The replacement for the RM optical laser is planned on Friday.

[J.G.] Note on Y end temperature problem: There were strange multiple failures in the Y end HVAC equipment, discovered on Dec 22, that caused poor temperature control in the CDS highbay. The temperature in the Y VEA also changed, though not as much because the heat load there is so small. Repairs were complete Dec 28.

LIGO computing and network security (Roddy)
--------------------------------------------------------------------

Reported under General Computing, see below.

General computing and LDAS admin (Giardina)
---------------------------------------------------------------------
Reported under General Computing, see below

Reported under LDAS System Administration, see below

Data analysis & computing (Yakushin)
---------------------------------------------------------------------
Storage/Condor/LDAS admin:
Reported under LDAS System Administration, see below

Data analysis:

Reported under Data Analysis activities, see below

CDS Computing (Lisa Bogue):
- The holidays were reletively quiet. There was a small dmt problem
that was readily resolved by John Z.
- Working on re-writing the perl scripts that auto-generate some of the
S5 pages.

LIGO Science Education Center (Thacker)

Prepared programs for and conducted outreach to 3 groups this week: Livingston Parish Science Teachers, Pointe Coupee Parish AMPS Mentoring Group, and Tangi Home School Group. Prepared for Saturday Teacher Professional Development workshop (Tangi Schools MSP project)

Ordered materials for LA GEAR UP workshop next month.

Drafted Activity protocols for: Colored Shadows, Polish Portraits and Gravity Blanket

Mechanical and Optical Systems (Coyne)

See Advanced LIGO

Controls and Data Systems (Bork)

See Advanced LIGO

40-Meter Interferometer (Weinstein)

• IFO commissioning, Electronics, controls, computers:
  • Rana and Kirk worked on DRMI lock acquisition, and then Sam measured the locking time using his "run ten times, show the average acquire time" script. With some tweaking, he got it down to 2.5 seconds on average to lock.
  • Sam got the DRMI to lock on each of the lower and upper 166 MHz RF sidebands (forming optical spring or antispring), deterministically, by changing the REFL_166_Q demod phase.
  • Rana worked on Bram's autoalignment script, and it works pretty well for the xarm. One problem is that the input TT-PZT mirrors are too close together to make a tilt-free translation of the input beam. It would be a lot better if they were spaced further apart (~ 1 m).
  • We vented on Wednesday, with no problems, and opened up the output optic chamber. Rana locked the MC in air at the "wrong" polarization (to reduce the finesse and thus the stored power), to get light to the AS port. Go and Osamu will be installing the motorized mirror for the squeezing experiment, and Rana and company will be locating and eliminating the clipping that is observed on the AS beam, which is somewhere between TT-PZT1 and TT-PZT2.
• IFO modeling:
  • Kirk and Rana are using Optickle to understand the signals for the smooth locking of the PRFPMI. They have good error signals for CARM and DARM, and are looking for a good MICH DC locking signal (when locked to 0.8 of bright fringe).
• DC detection:
  • Before the holidays, Rob measured the resonances in the OMC TT-PZT sterring mirrors and found "a broad, fertile valley from 2-7 kHz where dither lines will be able to flourish." He locked 2 DOFs (both loops on the second tip/tilt: the angles) at relatively high bandwidth (50Hz), and this bandwidth can surely be pushed higher. Working on diagonalizing the sensing/actuation scheme.
  • Ben has finished the new DCPD schematic and pcb layout. It is being reviewed by Rob and Rana.
• Vacuum squeezing:
  • Before the holidays, Go and Osamu measured broadband squeezing while locking the OPO cavity with the subcarrier which was phase-locked to the MOPA laser and noise-locking the squeeze angle.
  • Osamu and Go locked a signal recycled Michelson (SRMI) for the squeezed vacuum injection, with greatly reduced sidebands exiting the AS port. They are looking for a good set of signals for controlling the MICH offset.
  • Shally implemented a new servo for phase-locking the second laser to the MOPA laser, and he and Go took the transfer functions of the fast and slow open loop gains.
  • Go is preparing a paper draft on the generation of a stable low-frequency squeezed vacuum field with periodically-poled KTP.
• Lab Infrastructure, Bake lab:
  • We ran out of N2 for the vacuum valves over Christmas, and the pressure rose to almost 4 mTorr. The PSL shutter interlock tripped, and the OMC PZT HV was turned off. Steve came in and restored vacuum mormal, shutters, and HV.

Thermal Noise Interferometer (Black)

Some of the picomotor mirror mounts came in over the holidays, and Greg has started unpacking them and preparing for installation.

LASTI (Ottaway)

ISI pre-isolator hardware (R.Mittleman, F.Matichard)

• The Compliance results for the apparatus were showing a difference from the model of about 28%. We brought one of our electronics crates and a set of sensors to the manufacturer (ADE) who is look to be checked. We found three different problems, 2 of which were ours and one of which was theirs. We fixed our problems and now the sensors should be good to 1% (the remaining ADE problem) and the compliance matrix diagonal terms now agree with the model. The off diagonal terms have been reduced, but the system is till slightly asymmetric.

• We found an effect that we were previously identifying as friction between stages is dues to a DC drift, most likely due to the temperature of the laboratory.

• We are now using a combination of higher harmonics and transfer function measurements to characterize the interference of the system. We think that we currently have a small interference.


HAM SAS (D. Ottaway, A. Stochino, M. MacInnes and B. LaLiberte)

Most of this weeks effort centered on getting various parts of the HAM SAS from the loading dock to the lab. The HAM SAS was de-crated and moved to the optics lab adjacent the highbay. We will vent start taking doors off tommorrow (Friday) and will begin installing the elevator fixturing
In addition to this measurements on clustered geophones show an order of magnitude separation between the geophone noise and ground motion in the highbay of around an order of magnitude in amplitude in the middle of the night at 0.1 Hz. This means that we should be able to measure the isolation at 0.1Hz using only geophones and will not have to resort to putting an STS-2 on the HAM SAS.

Cavity Locking Tests
Ken M has built up a solid works model of this experiment and we will begin to order parts for this soon.

CIT Science Group (Weinstein)

  Kent Blackburn

TCLGLOBUS

Still investigating the issues arising in the LDAS software associated
with GSI socket buffering.

GRID APPLICATION DEVELOPMENT

Currently running full HIPE with local cache at UCSD for meeting
the first OSG milestone which is using 25 slots at UCSD continuously
in one week.

After running 4 days (Dec 20, 2006 - Dec 24, 2006), LIGO jobs were
limited to only 4 slots because CDF and CMS jobs had maxed out the
slots from Dec 25, 2006 to Jan 1, 2007. This was reported to the USCD
and a meeting is planned for this Friday.

Restarted the run on Jan 1, 2007 and it has been running continuously
since then.

Started testing nano HIPE with SRM/dcache at UCSD. Pushed input LIGO
GWFs to LIGO partition under the SRM server but experienced failures
which are being investigated.

Met with Kent Blackburn and David Meyers to discuss initial steps
for benchmarking full HIPE at CIT LDG cluster without having to moving
any data. This requires some Pegasus property changes. We are working
with Pegasus developers to enable recognition that the data is already
available locally.

Held weekly teleconference with Pegasus developers to discuss progress
on meeting first OSG milestone, dynamic data clean up, and a conference
paper on dynamic data clean up in workflows. Discussed LSC publication
policies with group.


OPEN SCIENCE GRID INTEGRATION TESTBED

Reported problem on the ITB site validation table caused by change
in certificate needed by GridScan and VORS submit host. This has now
been resolved by John Rosheck at Grid Operations Center (GOC).

Attended weekly Integration Testbed teleconference.

OPEN SCIENCE GRID VALIDATION TESTBED

Configured NFS and updated kernel to 2.6.17-1.2142-FC4 on worker
node1 and CE. Updated to 6.8.2 Condor release. Successfully ran
site_verify on VTB 1.6.0 cache.

Attended weekly Validation Testbed teleconference

OPEN SCIENCE GRID MANAGEMENT

Developed and discussed an OSG Change Control Process to be presented
at this weeks OSG Executive Team meeting.

Signed off on the FermiLab Statement of Work as OSG Resource Manager.

Reviewed OSG Security Plan with subset of OSG Executive Team.

Filed first monthly OSG excel spreadsheet capturing progress made on
WBS tasks specific to Caltech and the Pegasus developers at ISI.

______
Gregory Mendell

Since the last report I have worked with other members of the pulsar
group on the S4 PowerFlux, StackSlide, Hough paper, with the latest
draft sent to the reviewers yesterday. The paper is expected to go
through several more iterations before a final draft ready.

______
Igor Yakushin

LDAS admin:

1) ldas-grid froze 3 times during recent couple weeks at LLO and couple
times at LHO. At the moment the cause is not known. I am running 'top'
as a cron job each two minutes to catch an abusive process in action.
Since I started doing this 4 days ago, there were no more problems with
ldas-grid so far.
2) Ejected/imported tapes on Tuesday.
3) Installed Solaris 10 update 3 on the new dataserver.
4) Wrote a segment replication monitor:
http://ldas.ligo-la.caltech.edu/ldas_outgoing/archiver/Status_Pages/SEGMENT_
REPLICATION/index.html

Data analysis:

Working on Coherent WaveBust online infrastructure.

______
Xavier Siemens

- Worked on neutron star populations
- Written the final data selection c code for Einstein@Home (this one's
finished)
- Shifts at LLO (Jan 2 - Jan 8th)

Laboratory Computing (Anderson)

LDAS Software (Maros)

Work is proceeding on preventing the diskcacheAPI from hanging when
scanning non responsive file systems due to NFS issues (PR#3015). During
this process, it has been discovered that the frameAPI will not come up
if an entry in the mount point table suffers from NFS issues. As a
result, the code originally developed for the diskcacheAPI has been
moved to the genericAPI so the frameAPI can benefit from the solution.

To reduce the number of new processes created at the TCL layer, a touch
procedure was written (PR#3103). This replaces the need for calling the
system utility touch. It has been incorporated into cmonClient,
cntmonPI, ligolwAPI, and the genericAPI.

The manager now rejects jobs for which an API required by a command
macro is not available (PR#3087).

System tests for LDAS were done using version 1.8.408 of the software.
Some of the regression tests failed in FrDump and FrVerify. These are
being investigated.

A solid week of testing on four systems: ldas-dev, ldas-test,
ldas-suntest1, and ldasbox1. Most of the errors were timing issues which
resulted in the datacondAPI terminating the job. "Bad Symbol" (meaning
some data did not arrive at the datacondAPI in a timely manner and hence
was not available to the algorithm section) was the reason for the
datacondAPI's action. On lasbox1, the frameAPI was rebooted twice by the
manager due to memory limit issues. This was expected due to the small
amount of resources available.

LDAS System Administration (Anderson)

Caltech
-------
(Dan Kozak)
* Fixed stuck staging (reported by David Meyers).
* Ejected tapes for shelf storage at both sites & CIT.
* Helped Mike with globus-rls-cli, pools and some info about the difference between /data and /archive.
* Fixed tapes marked as 'tp' (generic tape) at LLO.
* Helped Larry identify and work around the talitha newfs problem (newfs ignores the -i for filesystems over 1TB).
* Replaced a 3510 disk (first in a long time).
* Set up 6140 RAIDs on IP, other config ongoing as I'm dealing with Sun on a couple of issues.
* Fixed stuck tape robot daemons at CIT.
* FrChecking all frame files on node 304 for Erik (ongoing).

(Phil Ehrens)
* Coordinated with Larry, Veronica, Cindy, and Albert to install the Visitor Appointment Application online form on an appropriate machine.
* Investigated RAID failure on node304. Replaced a single bad drive in port 1, and ran rebuild with the ignoreECC option to rebuild the array.
* Investigated a problem identified by Mary Lei with LDAS on the dev system. Some log file links were stale. I created a patch for the genericAPI log.tcl library that is being tested by Mary. It seems that Tcl reports a dead link as a file that doesn't exist, which is inconsistent with existing file existence checks.
* Ongoing cleanup of expired certificates throughout the LDAS computing systems.
* General certificate related support.

(Erik Espinoza)
* Built various "LIGO" FC4 distributions to support new Sun hardware.
* Shipped 420 sticks of ram to ASA Computers.
* Further training of spam filters on GC Spam Gateway.
* Testing kickstart w/ Latest LIGO FC4 x86_64 Distro.
* Requested DVD Burner for D610 Laptop.
* Rebuilt RAID, ran fsck on /usr1 on node304.

(Stuart Anderson)
* Stress tested the 5 new X4600M2 machines at Caltech under Solaris 10 Update 3.
* Built a newer Linux kernel for the X4600M2 machines that appears to have
solved a kernel panic problem with the latest FC4 kernel.
* Ordered UPS and rack mount equipment to install the new Sun RAID systems
and servers for the LDAS-CIT cluster.

MIT
---
(Fred Donovan)
* All quiet, replacement power strips (2) ordered.

Livingston
----------
(Igor Yakushin)
* ldas-grid froze 3 times during recent couple weeks at LLO and couple times at LHO. At the moment the cause is not known. I am running 'top' as a cron job each two minutes to catch an abusive process in action. Since I started doing this 4 days ago, there were no more problems with ldas-grid so far.
* Ejected/imported tapes on Tuesday.
* Installed Solaris 10 update 3 on the new dataserver.
* Wrote a segment replication monitor: http://ldas.ligo-la.caltech.edu/ldas_outgoing/archiver/Status_Pages/SEGMENT_REPLICATION/index.html

(Dwayne Giardina)
* Noticed connection problems to ldas-gridmon from LLO GC DHCP addresses. eth1 on gridmon had no network connection. It does now, and connections are fine.
* Rack-mounted new 3511s and new x4600 with Bernie and Igor.
* Packaged tapes for shipment to CIT.

Hanford
-------
(Greg Mendell)
* The main sys admin activity since the last report, besides routine tasks, was to make sure the LDAS systems and services recovered completely from the power failure at LHO on Dec. 25, 2006. I am also making sure that archiving and RDS generation at the sites and the transfer of this data to CIT is up-to-date.

(Ben Johnson)
* Replaced bad T3 disk drive.
* Put new x4600M2 into a rack with a dedicated 2200 KVA UPS; installed Sol10 U3; currently burntesting the unit. The current location for the server is temporary pending more thought.

General Computing (Wallace)

MIT:
(Fred)
-Ram tests tintagel, crashing, 1 bad ram stick RMA Genstor
-Replacement for thinkmate cpu's arrived
-Building horde, for webmail IMP interface
-More hosts added to nagios, more testing done
-Resolved some misc printer xp issues
-Built pellinore (sunblade 1500) powered on, virtual machine
(zone) "ligo" created for backup mail server; plan to create other
virtual machine (emvogil-2, oyster, + more) as my global backup machine.

Livingston:
(Dwayne)
- spam filter cleanup
- other usual user requests and support
LLO mail summary for 21 December 2006 - 4 January 2007:
Rejected: 42,091 (107 containing virus)
Accepted: 2,926
Total: 45,017

(Shannon)
-Laying out new VLANs for the switch gear. Previously, almost
everything was in one VLAN, now we need to segregate ports as the new
allocation comes online in places other than the computer users room.
-Upgraded the new fileserver to Solaris 10 U3. Added in ZFS hotspare
functionality.
-Contacted Arkeia on a support issue concerning ZFS. Issue resolved.
-Contacted Arkeia on a library support issue. Issue unresolved at the
present.

Hanford:
(Christine)
- I was sick last week, so I postponed rack mounting all the new
equipment until this weekend, 1/6-7.
- Postponing the rack mounting turned out to be a good thing because
Richard had his electrician in this week to install connections to the
new UPS for the server room. Richard bought some rack mountable power
strips that have network connections and on board computers to report
power statistics for the strips. I'm borrowing two of those for my
server room racks.
- I also received the Veritas backup software to be used with the new
raided disk arrays and tape drive.
- Purchased more wireless keyboards, mice, and USB drives. Everyone
with a laptop wants a wireless keyboard and mouse and many are using the
120 GB USB drives as portable backup for their laptops.
- Helped a couple of users get acquainted with their new laptops.
- Ordered a new laptop for another Operator.

CIT:
(Mike)
-Finish up working on Primavera/Prism servers. All software is now
updated and running correctly.
-Worked on our VPN server establishing a new connection for a new user
in LHO. This user now has a VPN connection as well. I'm also looking
into the IP database to free up some more IP numbers to accommodate
additional advance LIGO users that will need an offsite connection to
our Primavera/Prism server located here on campus.
-Moved backup server Repository and backup software to another server.
This new server will be our permanent storage for all NT servers with
plenty of room for growth. I now have all client servers backing up to
the new server.
-Bridge Annex server rooms: Larry and I installed and fired up SunFire
X4600. We had to moved another server out of RM:19 to RM:18 to free up
some power consumption in RM:19 to run the X4600.
-Ran end of month ghost backups on all NTSRV's.
-Continued work over the Holiday's on Spam Filters and working the mail
servers.
-Additional user support plus other sysadmin tasks.

(Veronica)
- LIGO: Homepage redo. Various web updates.
- LSC: Updates of the databases of publications and technical
papers. Addressed an issue with the discrepancy of LSC-related
mailing lists (lsc-all is an alias, and the actual list is generated by
merging 2 separate lists). I got a complaint of someone not getting
messages even though the person was in the lsc-all.
- CaJAGWR: Web updates/ user support.

(Christian)-Osamu Miyakawa- Configured new system with the engineering
Ligo image for Osamu.
-Lloyd, Dorothy- replaced Dot's old CRT monitor with a new LCD monitor.
Irene Baldon- replaced Irene's old CRT monitor with a new LCD monitor.
-Shourov Chatterji-- Hard drive failure on laptop. Rebuilt new hard-drive
with our standard ligo image.
-Rod Luna- - Upgraded Rod?s workstation from Windows 2000 to Windows XP.
-Finished upgrading Flexml license server from windows 2000 to 2003.
-Worked on the Spam Filters with Mike and Larry.
-Other misc.: Continued onsite software/phone support.

(Larry)
-Worked a number of purchases. A number of new notebooks systems have
arrived and Christian is getting them ready for distribution. We should
also receive more related items over the next few weeks.
The wireless modem card for the PC so far has tested out positive. We've
purchased another one and will probably purchase one more in the near
future.
Checking on some licenses supplied by campus that are due to expire.
-Did the monthly backups. Presently, in the process of redoing some of
them. After checking some of the tapes it was discovered some items did
not get backed up completely so I am doing a little more checking but
things appear to be OK now.
-Working on moving home accounts to the new server. So far so good. I
should start moving over a number of people in the next few days, after
I get rid of some of the dead accounts.
-Installed some new h/w and reworked some of the cabling in the computer
rooms.
-Setup a few new accounts and modified a number of mail alias lists.
Presently, going over the lsc mailing lists with Veronica. Also, going
through the whitelist for the new mail server.
-Working with PMA and others on some of the room modifications that are
taking place.
-Misc. e-mail work. Spam filter cleaning and related items. We are still
blocking over 10K/messages a day on the main mail server.
Also, took care of a couple of mailman issues. Still need to process a
number of requests for new mailman lists.
-Did some testing on a couple of lowend Gige switches. They are OK for
offices or small labs but not for the core network system.

Advanced LIGO and Supporting R&D (Shoemaker)

For additional information about this report, contact Albert Lazzarini or Phil Lindquist