Daily Battles At The Tevatron

A quiet moment at the Tevatron Main Control Room. Murphy's Law usually runs the Tevatron. Anything that can go wrong, will go wrong. Highlights from April 7 to 9, 2008: Booster kicker magnet (MP02) fails. 10E10 of antiprotons lost. Lightening strikes cause store to abort and TeV to quench. Here's the full 5 page report for April 9 to 11, 2008, from the Fermilab Accelerator Update. "The day shift began with the Tevatron (TeV) in quench recovery. . ."

Well, its a big place with most of the aging equipment dating from 1982. "The Tevatron tunnel is slowly sagging." V. Lebedev writes in his paper on Tevetron Operation Status And Possible Lessons For The LHC. As a result, "During 2006 shutdown about 50 quadrupoles we unrolled." And ". . . because of compression of thermo-insulating spacers . . . It took 3 years and 3 major shutdowns to finish shimming for all 772 Tevatron SC dipoles this year. (2006)"

A lot of experiments and projects make for a hectic schedule, over 50, with some 20 centered at the Tevatron. A thousand superconducting magnets, 15 miles of helium pipe, 24 cryogenic refrigerator houses, but utterly dwarfed by the power and size of the monster LHC. A lot of small annoyances, like from the magnets, though supercooled why should they smell? Other strange smells cropping up, odd vapors, but no fires or radiation hot spots detected by biohazard teams. Mysterious and spooky in the old bowels of the accelerators, grimy cement floors and poor light, an aging industrial twilight city, zones contaminated with radiation and off limits to personnel.

The main problem is a plumber's nightmare. Stuck valves, helium leaks that mushroom 700 times in volume from the cryogenic state, nitrogen leaks as the secondary coolant, vacuum leaks from the beam lines, heat leaks usually electrical, and radiation leaks which contaminate the magnets and force nearly all the superconducting magnet quenches. And they quench a lot, an average of 8 quenches per month in 2006, down from 16 per month in 2001.

Even the roof leaks at the Meson Lab. And when there's a prairie thunderstorm it's pandemonium. Power and communications disrupted. Fire alarms triggered all over dozens of buildings. Even without a direct hit the lightening surging through the ground can terminate the Tevatron beam, an emergency situation, an instant beam dump or its 2 beams drilling holes through anything. Wacky? No, it already happened at the Tevatron twice in December, 2003. A catastrophic beam loss. Ron Moore, Tevatron Department Head, Beams Division, referring to one of the beam loss accidents, recently said ". . . the high energy causes damage even within short time periods. Within 16 nanoseconds, one beam burned through about 1.5 meters (about 5 feet) of solid steel."

The Tevatron's Main Injector ring has had an intermittent beam loss problem with a neutrino beam that's fired underground to a target detector in an abandoned mine in northern Minnesota, the Soudan mine, 735 km away. Not science fiction either, but you need Fermilab authorization before you can access the failure data. There's just a note published in Fermilab Today of December 7, 2006. "NuMI (Neutrino Main Injector) suffers intermittent beam loss when MI stashes" occurring December 4-6. The NuMI-MINOS project on neutrino physics is a big one that will probably go on without the old Tevatron, only the new Main Injector required to produce the beam.

The big worry is radiation contamination, like tritium, usually found in nuclear power plant cooling water and then your local river. Fermilab also relies on water cooling for certain elements of the Tevatron, even collecting rainwater for some of their needs. It's an eco friendly gesture. In November 2005, low levels of tritium were found in the waters around Fermilab, in Indian Creek and ponds on the site. First time in 30 years. But don't forget that 2 years before, the Tevatron had at least two beam accidents in 2003, the December 5 hit that quenched two-thirds of its superconducting magnets, and later that month, the December 20 failure about as serious. But how could radioactive tritium have been produced from protons (even though extracted from hydrogen) and antiprotons (from nickel), unless one or the other acted on something else, producing neutrons then acting on boron or lithium to make tritium. It's up to Fermilab to find out how this happened. All they have been doing is taking water samples. How about some basic research? Well almost. They did locate the tritium leak, a pipe from Fermilab connecting two cooling ponds. Fermilab says the problem is fixed with no further tritium detected. But where is the tritium that was discharging into the ponds? Still in the Tevatron system? And how was tritium produced in the first place? According to Lebedev in the same paper cited above on page 5, there is a lithium lens, apparently in the Debuncher. Lebedev adds that the lithium lens was to be upgraded by 2007. So was it damaged then? Transformed into tritium? No analysis provided by Lebedev.

There are many lessons here for the LHC just in Lebedev's 5 page paper. No doubt CERN is fully aware of them and have revised their engineering and operations with Fermilab's help. The problem is CERN is not building a new and better Tevatron, but a much bigger machine with energies surpassing the Tevatron's beam by an enormous factor of 150.

Two reassuring things though. On the main desk at the Main Control Room, a lowtech box of Puffs and a flashlight. Click to enlarge.