The Web Newsletter Front Page Hanford Livingston MIT

LIGO Caltech News

Microway Alpha Workstation Speed Testing
LIGO Employees Honored For Long-Term Service

Microway Alpha Workstation Speed Testing

- Contributed by James Kent Blackburn

The LIGO interferometers will begin continuous collection of data soon after the start of the new millennium. Analysis of this data will allow for both near real time on-line filtering of the instruments' response, and off-line searches requiring more resources than are planned for the sites to detect gravitational waves from a range of astrophysical sources. Many sources are intelligible through our present understanding of the physics producing the gravitational waves LIGO intends to detect. These include such catastrophic events as the supernova end point in the life of a massive star--exploding in a largely undetermined burst of gravitational radiation--to the highly deterministic inspiral as two massive neutron stars or black holes coalesce, and even the ringing of a large spinning black hole that has been excited by interactions with infalling matter, much like a bell that has been struck. LIGO will open a new window on the universe, and the possibility of seeing serendipitous sources of gravitational radiation is highly likely and will require less specialized, more robust means of detection.

Figure 1. Microway Alpha based 'Screamer' Workstation Carrying out searches for known sources of gravitational waves will rely heavily on the use of optimal filtering methods. These methods require that the data captured as a time ordered collection of signals, be recast into the frequency domain, preserving the amplitude and phase information. Then it must be correlated with expected frequency domain waveforms in order to achieve the maximum signal to noise ratio for those astrophysical sources where we do have insight into the physics and evolution of the systems. Chief among these are the binary inspiral of two neutron stars or black holes and the ringdown of excited massive black holes. In these cases it is possible to estimate the number of independent waveforms each segment of data must be compared with to have a reasonable chance of seeing a gravitational wave signal of sufficient strength above the instrumental noise. For example, on a per interferometer basis it will require nearly 35 thousand filters to look for the binary inspiral of two neutron stars or black holes where each companion has a mass greater than or equal to 1.2 solar mass units. It will require nearly 1000 ringdown filters to look for massive spinning black holes with quality factors less than 30. It is estimated that 80 percent of the computation associated with these filters is in carrying out the Fast Fourier Transforms (FFTs). If a computer could deliver 200 million floating point operations per second (MFLOPS) while carrying out the FFT algorithm, then LIGO would need roughly 35 such computer nodes working in parallel to perform these two classes of filtering per interferometer data stream (the binary inspiral search makes up about 90 percent of this).

Demonstration of such performance is often difficult on the basis of published benchmarks. Performance of any piece of computer hardware depends on many aspects of the overall system design, such as processor performance, memory cache, operating system and compilers, just to name a few. To understand if the target of 35 200-MFLOPS computers is in scope for LIGO's on-line data analysis needs, an Alpha workstation (see Figure 1. at left) from Microway, Inc. has been tested by LIGO.

Figure 2. 600 MHz Alpha LX Motherboard The workstation was purchased in Spring 1998, consisting of an Alpha LX motherboard (see Figure 2. at right) with a 600 MHz Alpha CPU, a 4MB 9ns SRAM cache (the largest available on the market at that time, and capable of going up to 8MB), 128 MB of dynamic RAM and a 4 GB ultra wide SCSI hard drive. The unit was shipped with a modified version of the Redhat Linux 5.0 operating system. The motherboard also includes 32 and 64 bit PCI bus interface. The workstation includes a standard DEC keyboard and a IIyama 17 inch monitor and a three button Logitech mouse.

A series of FFT benchmarks was carried out on this workstation using the ANSI C FFTW 1.3 library available from MIT using the GNU C compiler shipped by Microway, Inc. This was compared with a Sun Ultra 30 with a 300 MHz Ultrasparc processor, using both GNU's C compiler on the Sun and Sun's Workshop 4.2 C compiler. The FFTW package includes its own benchmark algorithms which were used as the "standard" for comparing both machines. Since the Sun compiler was expected to demonstrate superior optimization over the GNU C compiler, it was decided that an additional benchmark, using the DEC C compiler to build the FFTW benchmark routines, would be a good idea if it was deemed possible to run DEC compiled binaries on the LINUX Alpha operating system. It was found that the LINUX kernel could be modified to allow statically linked binaries from DEC Unix to run on the LINUX system. A second vendor, Alta Technology, offered to provide LIGO with static versions of this needed FFTW benchmark program, using both floating point precision and double precision complex FFTs. The results are shown in Figure 3., below right.

Figure 3. Performance comparison for complex FFTW benchmark code Gauging the results we see that for both the Sun and the Alpha, peak performance for the FFTW occurs for very small data size (order 32 complex data points). The most likely reason for this is the number of data registers in the CPUs. A sharp fall off begins at above 1024 data points, tailing off at about 16384 sample points, where the Alpha is delivering about 250 MFLOPS of FFT performance. This is short of the earlier design estimates. Those estimates suggested that working with million point FFTs would be optimal if the performance was flat in the region between 128K and 2048K samples. The FFTs could be shortened to roughly 256K samples without modifying the underlying convolution algorithm and its performance for the optimal filtering techniques. This is driven by the time the binary inspiral waveforms are in the interferometers' most sensitive frequency band--a 1.2, 1.2 solar mass binary will be in initial LIGO's band for 70 seconds--and by the significance of higher frequency structure in the waveforms to the detection process. In this specific area, the Alpha is delivering the best performance, but only at the level of about 150 MFLOPS for FFTs (down by a factor of 5 from peak performance). The added cache size on the Alpha workstation helps maintain the large sample size performance to twice that of the Sun with its 2MB cache. Clearly the larger cache is needed for mega-point FFTs.

The Alpha workstation was also performance-tested by the end-to-end group. It gave the best showing of any computer tested by a factor of between two to three. This demonstrates the tremendous numerical performance afforded by the Alpha CPU to a broad scope of problems. (The end-to-end model is not dominated by FFTs.) However, there was a clear downside to using the Alpha workstation, at least with the Redhat LINUX 5.0 operating system. The Alpha port of LINUX was both incomplete (many system header files were missing) and unstable in its behavior and ability to compile system level code. Redhat LINUX 5.1 is available and will soon be installed on the Alpha workstation. It is hoped that this new version will fix many of the software problems encountered when using the Alpha. Greater optimization of the GNU C compiler for this platform would also be extremely beneficial.

Microway has already dropped this unit's price by 25 percent. A 633 MHz unit is now available, and a 700 MHz system--delivering over twice the integer and floating point performance--has been announced for delivery the third quarter of 1998. The other Alpha vendor, Alta Technology, is commercially producing mini-Beowulf boxes containing eight 533 MHz Alpha motherboards loaded with 128 MB of memory, 4 GB disks, intelligent power supply and an internal switch for under $2000 per node. If this trend holds and the Alpha Linux stabilizes, then distributed computer engines, like the Beowulf on Alphas, represent a respectable performance per cost advantage in today's market.


LIGO Employees Honored For Long-Term Service

- Contributed by Linda Turner

Cindy Akutagawa, Gerry Stapfer, and Irena Petrac Sumptuous Spread In Honor Of Long-Time LIGOnians Four LIGO employees were honored by Caltech this past Spring for their long terms of service--35 years for Bill Althouse and Gerry Stapfer, and 20 years for Cindy Akutagawa and Irena Petrac. After the official campus ceremony, a lovely reception was given to the quartet by their LIGO colleagues.

In the photograph at left, Cindy, Gerry, and Irena are seen slicing the cake brought in their honor. (Not pictured is Bill Althouse, who was working at the LIGO Hanford Observatory in Washington at the time.) At right is a view of the banquet table before being descended upon by the hungry LIGO celebrants.

As described below, each of the honorees had a unique journey to their present jobs within the LIGO family. Perhaps you'll discover something about them you didn't know...

Celebrating 35 Years:

While still a full-time engineering student at Cal Poly, and with a new family to support, Bill Althouse began employment at Caltech in 1963. Hired by Robbie Vogt, he was given an unparalleled opportunity for a student, to set up from scratch the Space Radiation Laboratory (SRL). Bill worked 24 years in the SRL, rising from an electronics technician to Chief Scientist. In 1983, he enjoyed the distinction of being appointed a member of the professional staff at Caltech. Bill was invited to join the ranks of the LIGO Project in 1987, sharing responsibility for the overall management of LIGO. During the past 11 years on LIGO, Bill has served in many different capacities for the project, using his varied experiences and expertise in the on-going success of LIGO. He looks forward to continued association with the project and the eventual discovery of the first gravitational wave.

Born in Switzerland, Gerry Stapfer emigrated to the United States in 1956. Employment at JPL began in 1962 as a Cognizant Engineer in Section 342 developing solar panels for the Ranger spacecraft. Gerry remained in Section 342 for twenty years, steadily promoted until he became Group Supervisor for the Nuclear Power Sources Group. In 1983, he transferred to Project SP-100 which was researching and developing nuclear space reactor power. He began his decade-long tenure on the project as Technical Development Manager and finished as the System Project Manager. Moving to the Caltech campus in 1993, he joined the LIGO Project in the capacity of Chief Deputy Engineer and, by 1994, was Deputy Group Leader for Facilities. Progress in the construction of LIGO's observatories afforded him a move to Louisiana as Site Operations Manager at the Livingston site. Looking back on his JPL experience, Gerry thrived on the unlimited opportunities afforded to employees; looking forward he anticipates staying with LIGO until his retirement.

Celebrating 20 Years:

What began as a part-time job in 1976, became an on-going career at Caltech for Cindy Akutagawa. Cindy's first position was in Biology as an animal caretaker, working for Dr. Mark Koneshi and her husband, Gene. Her duties revolved primarily around the care of the many birds housed in the Campbell Greenhouse and bird aviary thus allowing the flexibility of bringing her two small children to work with her. After eight years in Biology, Cindy worked for Jerry Burk in the Physics Department as a clerk. In this position, she grew professionally, developing self-confidence in her abilities. After about a year, she became secretary to Ron Drever and the Gravity Physics group and what was the beginnings of the LIGO Project. As the LIGO Project grew and management changed, Cindy took on an administrative role with duties pertaining to the accounting and finances of a growing Project. During the past 20 years, Cindy has enjoyed the academic environment of Caltech and looks forward to the second half of her career supporting the LIGO Project and the Caltech campus.

Irena Petrac came to JPL in the summer of 1978 as a member of Procurement. She served as Contract Negotiator for Alternate Energy Sources. After approximately three years, Irena went on to support various flight projects: Solar Polar Mission, Magellan, Topix, and Cirtaf. Her favorite assignment was with Magellan where she enjoyed working with its managers and engineers. In early 1992, Irena moved to the Caltech campus to join the LIGO Project as their Sub-Contracts Manager, supporting the beginning of construction for two observatories. She is now dedicated to procuring and administering the sub-contracts necessary for the successful construction and completion of the detector for LIGO's observatories.