Volunteer Computing

When people first hear about Hadoop and MapReduce they often ask, “How is it differen from SETI@home?” SETI, the Search for Extra-Terrestrial Intelligence, runs a projec called SETI@home in which volunteers donate CPU time from their otherwise idl computers to analyze radio telescope data for signs of intelligent life outside Earth SETI@home is the most well known of many volunteer computing projects; other include the Great Internet Mersenne Prime Search (to search for large prime numbers) an Folding@home (to understand protein folding and how it relates to disease)

Volunteer computing projects work by breaking the problems they are trying to solve int chunks called work units, which are sent to computers around the world to be analyzed For example, a SETI@home work unit is about 0.35 MB of radio telescope data, and take hours or days to analyze on a typical home computer. When the analysis is completed, th results are sent back to the server, and the client gets another work unit. As a precaution t combat cheating, each work unit is sent to three different machines and needs at least tw results to agree to be accepted

Although SETI@home may be superficially similar to MapReduce (breaking a proble into independent pieces to be worked on in parallel), there are some significan differences. The SETI@home problem is very CPU-intensive, which makes it suitable fo running on hundreds of thousands of computers across the world[9] because the time t transfer the work unit is dwarfed by the time to run the computation on it. Volunteers ar donating CPU cycles, not bandwidth

MapReduce is designed to run jobs that last minutes or hours on trusted, dedicate hardware running in a single data center with very high aggregate bandwidt interconnects. By contrast, SETI@home runs a perpetual computation on untruste machines on the Internet with highly variable connection speeds and no data locality.