Weekly Report for
Week Ending June 1, 2006
Due to the LSC and Burst F2F meetings at MIT June 2 - 6 , there will be no LIGO Executive Committee meeting June
5, 2006.
Special Announcements: Advanced
LIGO Baseline Review completed. See Advanced LIGO Development
below.
·
Peter Saulson attended
the Elba meeting, where there were discussions
on many aspects of global collaboration. The LIGO-Virgo MOU was the subject of
some private discussions with Benoit Mours and
others. Most detector projects presented their plans for commissioning,
running, and upgrades. We had a round-table discussion on network scheduling
issues, at which we grappled with the problems of ensuring enough coverage to
do good observing, while also not unwisely delaying the advent of more
sensitive interferometers.
LIGO Laboratory Administration
(Lindquist)
STATUS OF LSC MOUs (Lloyd)
SITE TELECONFERENCE (Lindquist)
- There
are currently no open action items.
The list of assigned actions updated through December 01, 2005 (the
last update) will be found Here.
PROPERTY MANAGEMENT (Luna)
>
DOCUMENT CONTROL CENTER
(Turner, Mak)
>From: Linda Turner
<turner@ligo.caltech.edu>
- Finished
the "final" draft of the conversion requirements and statement
of work with FileHold -- waiting for all the
signatures of both sides of the fence.
- Worked
non-stop to prepare our current database for conversion over to
Synergy. This involved writing numerous queries to isolate
"pockets" of documents that could be moved over to the new
system. Beginning with 45,000 records, queries so far have defined
and allocated about 13,000 records. Work will continue in creating
queries that further assign document from our side over to their new
location in Synergy.
- Attending
the LSC to facilitate receipt and posting of all presentations.
>From: Cleveland Mak
<mak_c@ligo.caltech.edu>
- Worked
on assembling multiple complete copies of the Cooperative Agreement with
all its amendments.
- Worked
on assembling queries in preparation for the migration of current data
into the new database.
- Worked
on "cleaning out" the keyword and organization lists in
preparation for migration to the new database.
- *Update
on Scanning* - Progress continues on scanning of contract closeout files.
- Additional
Activity:
|
Week Ending
05/18/2006
|
In
|
Out
|
|
Packages
|
17
|
2
|
|
Faxes
|
18
|
21
|
FINANCIAL SYSTEMS (Cronin, Brambila, Kaufman)
>
>From: "Brambila, Ruth" <Ruth.Brambila@caltech.edu>
- Completed the package for change order #5 to The
Exploratorium and routed it for approvals. It is pending approval.
- Attended the weekly team meeting and several campus
meetings.
- Placed purchase orders as requested.
- Completed the p.o.
closeouts for May 2006 and submitted the monthly report. Researched
open orders with balances that are over 3 months old. Closed several po's.
- Followed up on the status of orders not yet delivered
to check on due dates.
- Responded to various inquiries related to invoices.
- Renewed the maintenance agreement for the fax machine
in Bridge. There are problems with Oracle in printing out this p.o.
- Renewed the maintenance
agreement for Linda's document scanner
>From: Florence Kaufman <>
SUBCONTRACTS MANAGEMENT (Jasnow, Salone)
>From: Gina Salone <gsalone@ligo.caltech.edu>
- Nothing
significant to report.
>From: Ed Jasnow <jasnow@ligo.caltech.edu>
PROPOSALS and REPORTS (Lindquist)
- The
NSF has provided the LIGO Operating funds for the last half of FY 2006.
- Requests
for material for the LIGO Operations Annual Report have been
distributed. Contributions are due Friday, June 23, 2006.
CHANGE CONTROL/CONTINGENCY MANAGEMENT (Lindquist)
HUMAN RESOURCES (Akutagawa)
>From: Cindy Akutagawa <cindy@ligo.caltech.edu>
- No special activities to report other than preparing
for the summer students.
Quality/Safety (Tyler)
>From: Bill Tyler tyler@ligo.caltech.edu
No report.
LIGO Hanford
Observatory (LHO) and Interferometer Operations (Raab)
Summary of
S5 Activities at LIGO Hanford
Observatory (compiled by M. Landry)
The latter part of this week saw some of the finest
running the project has ever had; see here for an example FOM
that shows (2X14 +7)Mpc for
extended lock stretches. Duty cycles for the IFOs
for the week were 84% on H1 and 90% on H2. H1 did struggle for periods
this week at a lowly 12Mpc; part of the problem was a bad setting on the QPDs.
Elog highlights
are bulleted below:
- Our
standard pie chart showing downtime contributors was updated.
Also shown there is a comparison with previous pies: winds are down, and
we now have a better accounting for why we lose lock, i.e. less
"unaccounted for" losses
- A scimon study of truck traffic on the 240 highway past
EY was performed
- Mt. St. Helens resumed activity last
week and may have been responsible for some lock losses
- Another
photon cal line at lower frequency (719.1Hz) was added briefly to H2, in
order to study sytematics between the photon cal
and official cal procedures
- The
weekly range and duty cycle report can be found here
- Recall
the problematic injection machine h1awg0 has required near-daily reboots
for the last several months. The most recent spate of reboots has
occurred since the April commissioning period. An important advance was
made on this problem when loading of the cpu was reduced by removing the unused photon
calibrator line (we alternately arms monthly but typically left both test
points defined), and now monitor slow channels instead of test points whenever
possible. The result has been a far more stable h1awg0, with no
reboots during this week.
- The
TCS laser on the X-arm showed power
instabilities, and a fix was prescribed.
- h2susepics
resumed flaky
behaviour
Outreach (Ingram)
- LHO
welcomed real-time lunar images from the Rattlesnake Mountain Observatory
into our auditorium as part of this year's multi-partner National
Astronomy Day program. Thanks to Richard McCarthy and Josh Myers
for creating the capability for LHO to receive RMO's
image transmissions, to Greg Mendell for his
fine presentation, to Justin Garofoli (wearing
his AASTA hat) for his service on the mountain and to the staff for
additional backup.
LIGO Livingston
Observatory (LLO) and Interferometer Operations (Zucker)
L1 in S5 (Yakushin)
- We
had a very good week with 75% duty cycle and about 14Mpc inspiral range. On Sunday our duty cycle was even 94%!
- The
lock was interrupted only by earthquakes, trains and construction.
L1 CDS (Bogue)
- DMT
filled up the disk space on london
over the weekend. Keith Riles and I got enough disk space to keep
going for awhile. This is a larger problem that will need to be
addressed soon.
- Modified
the london web server to
run perl as a cgi
rather than a module. The next big project in cds
should be replacing the web server with something more up-to-date and
easier to maintain.
- There
is a new design for the cds web pages ready to
go live.
- Wrote
some perl scripts to play audio alerts in the
control room when we drop out of science mode or when a gamma ray burst
event occurs.
- Worked
on the corner weather station code. The code is now stable, but I
would like to spend some time verifying the values coming out of it.
Safety/Security (Riesen)
- Nearing
completion of the last 3 Action Items from our Safety Audit. They
should be completed by the end of next week.
- Working with Allen on the fire alarm system re-work.
The site security system is now receiving correct data from the new fire
alarm system.
- Rupal Amin and I are getting
the Optics Lab ready for his Optical Power Amp experiments. All
safety hardware has been verified as operational,
the area has been re-cleaned and organized for Rupal's
move in.
- Ordered
laser safety glasses with an OD>5.00 at 808nm.(an
OD of >3 meets the requirements, but these would be special order and
hard to get).
- An
LLO Laser Safety SOP addendum has been written for the CEO Optical Power
Amplifier and will be sent to Mike Zucker for
review and sign off prior to the starting of the experiment and adding it
to the current LLO Laser SOP plus all associated signage.
- Conducting
Safety orientation for the arriving students and new staff members.
LLO Outreach (Thacker)
- Prepared
for upcoming exhibit training.
AdvLIGO Suspensions and Isolation, Outreach and
Very Very Large Art (Romie)
- (MZ
for Romie) Janeen is
at MIT participating in the AdvLIGO Baseline
Review.
LIGO Computing and Network Security (Roddy)
- Mail
server lost an internal disk this morning.
I am temporarily running on an external disk to get services back
up. There will need to be some scheduled downtime in the near future to
install more reliable disks.
- Ran a
network scan of the LLO network from offsite. The scan originated
from the LSU network. Several things to follow up on.
- Coordinated
with CIT to run a network scan of the CIT networks.
- Currently
have partial results. There will most likely be a large number of
items to follow up on.
- Still
working with Charter Communications on contractual issues for the backup
network connection.
- Started
setting up replacement dial-up server for modem access. Had to stop
on this to restore mail services.
- Working
on some of the technical BGP issues for the backup network
connection. We may or may not need some additional hardware for the
connection. It is also possible that we may need the same additional
hardware just for reliability issues.
- Ordered
various bits of hardware.
- Ordered
new backup tapes. About to run out of what we have...
- Investigating
a couple of security issues - one relating to some grid software
- Investigating
a couple of IDS alerts at LHO-CDS. One relating to the FMCS web
server.
Data Analysis (Yakushin)
Storage/Condor/LDAS
admin:
- Studying
Q-replication and WebSphere documentation (Q-replication
and WebShpere are used to keep segment databases
at sites in sync).
- After
last week reboot of dataserver and gateway I
have not noticed any more network interruptions inside LDAS network and
decided not to reboot the network switch for now.
- LDRed SG1C_S5 burst MDC frames from PSU to LLO and CIT.
Data analysis:
- Continued
working on LIGO-GEO simulations.
- Generated
waveburst triggers on SG1C_S5 burst MDC frames
that cover a period between Feb 15 and May 2.
Initial LIGO Detector Science & Engineering (Coyne)
No report (participating in AdvLIGO
baseline review).
40-Meter Interferometer (Weinstein)
No report.
Thermal Noise Interferometer (Black)
·
Nothing exciting to report this week. We are
debugging electronics and getting ready for SURF students.
LASTI (Ottaway)
No report.
Data Analysis and Computing (Lazzarini)
Simulation and Modeling (Yamamoto)
No report this week – attending Elba Conference.
Data Analysis Activities (Lazzarini)
Shawhan:
- Produced
segment lists for offline burst analysis for S5 through May 2, using a
number of data quality flags.
- Released
new versions of a few LIGOtools packages.
- Drafted
the remaining sections of the paper describing the S4 LIGO untriggered burst search.
Sutton:
- I
spent the past week at the GWADW 2006 in Elba, Italy.
I delivered a talk on "Recent advances in coherent network searches
for gravitational-wave bursts".
- During
the previous week I was in Benevento at the Universita degli studi del Sannio,
talking about time-frequency burst detection with Innocenzo
Pinto and Maria Principe, and giving some seminars on burst detection
and coherent search techniques.
Yakushin:
- The
results of the waveburst simulations on SG1_S5,
covering the period from the beginning of S5 to May 2, are published
at
http://ldas-jobs.ligo-la.caltech.edu/~waveburst/S5_LIGO_ONLINE_2/SIMULATIONS/index.html
- The
events found by high-threshold analysis up to now are summarized at
http://ldas-jobs.ligo-la.caltech.edu/~waveburst/S5_LIGO_ONLINE_2/bigtable.html
- The techical document describing the waveburst
version currently used for online analysis is submitted to DCC: LIGO-T060112-00-Z
LIGO Data Analysis System
Software Systems (Blackburn)
LDAS
- Using
TCL 8.4.12 has shown problems within LDAS. Using this version of TCL,
there appears to be a higher rate of missed tracing of variables. Work has
focused on modifying the trace syntax to use that of TCL 8.4.12 instead of
the legacy syntax based on TCL 8.3.5 which is causing the problems. This
has shown some improvement. This effort should be completed by next week.
- Work
has started on supporting LDAS on the Opteron
running Solaris 10. The current stumbling block is GCC and building the
C++ compiler. It is a known issue and is slated to be fixed in a future
release. Efforts are shifting to use Sun Studio.
- System
testing was done using ldas version 1.8.193 and
using TCL 8.4.12. Weekend testing were not
carried out do to a bug in the start up of the test scripts. Higher than
normal job failure rates continue in association with the TCL 8.4.12 upgrade.
TCLGLOBUS
- Investigation
of the connections timing out when connecting to LDAS manager while using
GLOBUS 4.0.2 is still under investigation. Three versions (XIO-based TCL
Channels, raw TclGlobus, C program using Globus API) of the same program have been created in
an attempt to determine which level of interaction is causing the delay.
These programs need to be enhanced to record at the millisecond
resolution.
- Attended
the VDT weekly "office hours" telecon.
No follow up on the inclusion of TclGlobus into
VDT at this time. The VDT has formally requested the OSG executive board
to review the inclusion of TclGlobus into VDT.
GRID COMPUTING
- Received
a response from the Condor team requesting additional debug information
regarding registered bug related to FC4/glibc/DAGMan. Planning has begun
to set up one system for debugging with Condor 6.7.18 - 6.7.20 to
demonstrate which combinations of glibc and
Condor DAGMan abort while attempting to execute
a work flow consisting of an Inspiral analysis
on the OSG.
- Using
a simple Burst analysis provided by Shourov in MatLab R13, constructed a work flow consisting of set
of DAG nodes designed to execute the Burst analysis on the LIGO Caltech
OSG cluster. Condor DAGMan has successfully
executed the Burst analysis including the MatLab
interpreter on the LIGO-CIT-ITB OSG cluster.
- Worked
with the Michael, and Kent on the LIGO work flow
planner specification. The work flow planner code base has been checked
into CVS.
- The
OSG Council meet to discuss agenda and plans for
the upcoming Consortium meeting August 21-23 in Seattle.
Hardware Systems (Anderson)
Caltech
(Dan Kozak)
- Helped
a little with racking equipment to ship to the observatories.
- Worked
with Eric to try to figure out the dataserver-cit
automount of node170 problem. We've tried
swapping the Linux hardware, checking the Linux software
installation.
- Provided
some small support for recovery from hung T3 in /archive at LLO.
- Closed
case with Sun regarding problem 3510 (-10/12). After replacing 6 of
the 12 drives, the unit has been functioning fine for almost a week.
- Figured
out which LLO tapes to eject for shelf storage.
- In
the course of getting caught up on all L0s, found a corrupt (truncated) L0
frame file at LLO. Ben believes this was caused by the dd
from /frames to /archive failing. The ingesting procedure flagged it, but
we'll need to look into getting the info early, while there's still a
change to re-copy it from /frames.
- Still
trying to figure out why rsync claims that
500,000+ files would need to be rsync'd from LLO's /archive/frames/S5/L0 to CIT when it is not the
case.
(Phil Ehrens)
- Wrote
and installed certcheck.tcl automatic
certificate expiration detection and notification tool. This script is
called as a daily cron task and logs imminent
cert expiration beginning one week before expiration. The log entries are
recognized by log_mon.tcl and email is sent to
the appropriate admin.
- Wrote
mcelog module for log_mon.tcl
which dumps mcelog data into a sensible file
hierarchy that is visible via a web browser.
(Erik Espinoza)
- Worked
with Phil to log Machine Check Exceptions.
- Reconfigured
all IPMI cards due to a few machines being swapped.
- Configured
lm_sensors, working on getting rrdtool to create charts.
- Coordinated
telecon w/ AMD, SuperMicro,
ASA, & Kingston
for RAM stability issues.
- Looked
into Symantec Antivirus Issue.
- Tested
a few different options on node170:
-
Raised number of nfsd, didn't
work
-
Swapped hardware, also didn't work
- Replaced
disk on node155.
- Rebooted
node40 (rwsem.h crash).
- Upgraded
ldasbox5, dataserver (dev), metaserver
(dev), & beowulf
(dev) to latest available updates + glibc for
testing.
- Looked
into possibility of Jumbo packets on Cisco 6509 + nodes.
- Added
xfig/transfig packages to kickstart.
- Installed
fftw3 development package on couch.
- Looking
into BIOS issue on "slow" returned nodes.
MIT
(Keith Bayer)
- Supporting
AdvLIGO meeting.
Livingston
(Igor Yakushin)
- Continue
studying WebSphere documentation.
- t3-24, which is part of /archive, froze on Monday
morning and had to be power cycled. A few corrupted L0, L1, L3 frames were found and fixed.
- Installed
xfig and transfig on ldas-grid and the nodes per Shourov's
request.
- Per
Laura's request copying S5 L3 frames to the nodes.
- Found
several 'damaged' L3 LHO S5 frames. Ran 'undamage'
on them. At first glance it helped.
Hanford
(Greg Mendell)
- I am
still working on realtime SFT publishing.
(Ben Johnson)
- Researching
IPFirewall rules for LDAS sites. Successfully firewalled both client and server for NFSv4 using only
port 2049.
- Publishing
and/or datafind server problem revealed this
weekend. Online user, inspiralbns, was not
getting the full list of files it needed. The segments were there, but not
all URLs. The problem has since gone away. Will follow up.
- A
photon calibrator was turned off this Saturday, May 27. H1AWG0 has been running
stably since (as of yesterday at least). Only ONE CRC mismatch frame at
LHO has been found in the 5 days since (it occurred
on May 30).
General Computing (Wallace)
MIT
(Keith)
- Spent
this week doing various prep related stuff for
the Advanced LIGO Review and the LSC conference.
Livingston
(Shannon)
- Mail
server lost an internal disk this morning. I am temporarily running
on an external disk to get services back up. There will need to be
some scheduled downtime in the near future to install more reliable disks.
- Ran
a network scan of the LLO network from offsite. The scan originated
from the LSU network. Several things to follow up on.
- Coordinated
with CIT to run a network scan of the CIT networks. Currently have partial
results. There will most likely be a large number of items to follow
up on.
- Still
working with Charter Communications on contractual issues for the backup
network connection.
- Started
setting up replacement dial-up server for modem access. Had to stop
on this to restore mail services.
- Working
on some of the technical BGP issues for the backup network connection. We
may or may not need some additional hardware for the connection. It
is also possible that we may need the same additional hardware just for
reliability issues.
- Ordered
various bits of hardware.
- Ordered
new backup tapes. About to run out of what we have.
- Investigating
a couple of security issues - one relating to some grid software.
- Investigating
a couple of IDS alerts at LHO-CDS. One relating to the FMCS web server.
Hanford
(Christine)
- Installed
two additional paper drawers on the OSB color printer. One of the paper
drawers was damaged in shipment, but thanks to some help the drawer was
fixed without having to be sent back.
- Set
up more new computers, still have 6 more
computers to set up.
CIT
(Mike)
- Ghost
backups on NTSRV's. I ran end of month ghost
backups on all critical NT servers.
- Went
through ten SUN Ultra 10 workstations that we had in stock. I ended up
keeping 2 good one's, and surplus 8 workstations. Most of these workstation's are inoperable.
- Spent
time on maintaining the mail servers, through out the week including the
weekend.
- Continued
work on the DCC servers. More issues this week that required much time
keeping these servers up and running. One issue was user error this had
the public side of the house down for a couple of hours.
- Held
down the fort while Larry was out on his day of vacation.
- Worked
on the wireless access points; removing old mac
addresses and adding a few new ones.
- Spam
Filters: There was a lot of work over the holiday weekend keeping up with
the filters, searching for false positives.
(Veronica)
- Ongoing
support of the AdvLIGO NSF Review. Misc. user
support. Addressed a DCC server issue. Updates of the LSC mailing lists.
(Christian)
- Mike
Smith - configured new system with the engineering Ligo
image for Mike and transferred all his data.
- Lee Cardena - upgraded Lee from Windows 2000 to Windows XP
and transferred all his data.
- Worked
on the Spam Filters with Mike and Larry.
- Other
misc. - continued onsite software/phone support.
(Larry)
- Still
working the procurements. Purchased a number of UPS units to replace the old
worn out units. Still trying to get Monarch to respond on getting the
server repaired. PMA has a new contact on campus that has worked with
Monarch in the past, I will try to get them to
help out. Purchased more supply items. Received the SUN equipment and
presently going through the order list to make sure we got everything that
was ordered.
- Setup
one of the new SUN workstations running Solaris10. Attached a dual
24inch monitor system on it. It looks pretty nice. Something I may need to
consider for future upgrades for those that need a lot of screen space.
- Worked
with Mike, Christian and Rod in getting a bunch of old equipment removed.
We are now just waiting for transportation to take it away.
- Worked
with Keith on a couple of logistical issues concerning the NSF review.
- Still
working on account cleanup and removal. There are quite a few accounts that
I still need to check out.
- Worked
a number of DCC issues. Both for the DCC and some of the endusers.
- Assisted
a number of users with different issues.
- Started
monthly backups.
- The
e-mail saga continues. The filter database is getting to be in pretty bad shape
and will probably need a rebuild in the not too far future.
Mail Statistics for May 25-31, 2006
|
Mail Statistics
|
May 18, 2006
|
|
Rejected Messages
|
34,599
|
|
Virus Messages
|
1,557
|
|
Accepted Messages
|
25,281
|
|
Total Messages
|
59,880
|
Advanced LIGO and Supporting R&D (Shoemaker)
Advanced LIGO Systems
- Advanced LIGO Baseline Review completed. It was a
resounding success, thanks to the work of the
entire LIGO/LSC Advanced LIGO Team.
For additional information about this report, contact Stan Whitcomb or Phil Lindquist