Foldit is a multiplayer online game in which players collaborate and compete to create accurate protein structure models. For specific hard problems, Foldit player solutions can in some cases outperform state-of-the-art computational methods. However, very little is known about how collaborative gameplay produces these results and whether Foldit player strategies can be formalized and structured so that they can be used by computers. To determine whether high performing player strategies could be collectively codified, we augmented the Foldit gameplay mechanics with tools for players to encode their folding strategies as “recipes” and to share their recipes with other players, who are able to further modify and redistribute them. Here we describe the rapid social evolution of player-developed folding algorithms that took place in the year following the introduction of these tools. Players developed over 5,400 different recipes, both by creating new algorithms and by modifying and recombining successful recipes developed by other players. The most successful recipes rapidly spread through the Foldit player population, and two of the recipes became particularly dominant. Examination of the algorithms encoded in these two recipes revealed a striking similarity to an unpublished algorithm developed by scientists over the same period. Benchmark calculations show that the new algorithm independently discovered by scientists and by Foldit players outperforms previously published methods. Thus, online scientific game frameworks have the potential not only to solve hard scientific problems, but also to discover and formalize effective new strategies and algorithms.
The circumstances surrounding the recent announcement of results from an HIV vaccine trial in Thailand are troubling. The sponsors of the US$119-million phase III clinical trial, a consortium led by the US Army, the National Institutes of Health and the Thai government, announced on 24 September that the trial had been a success: an analysis of the data showed that the vaccine had a statistically significant effect on preventing infection.
Other scientists could not immediately assess that claim, however: the full data from the trial were not made available until 20 October, when they were presented at an AIDS vaccine conference in Paris and in an article published online the same day (S. Rerks-Ngarm et al. N. Engl. J. Med. doi:10.1056/nejmoa0908492; 2009). The article contained two other data analyses, not mentioned in the initial announcement, showing smaller effects that were not statistically significant (see page 1187).
The trial’s sponsors defend the premature announcement on the grounds that they had promised to inform the Thai people of the results first; 24 September is also Mahidol Day, the anniversary of the death of the king’s father and a day of national observance in Thailand. The sponsors also argue that announcing the less-upbeat analyses along with the positive result would have been too complicated for the public to understand; they wanted to quickly deliver a clear-cut message on the trial’s findings. Making the full data immediately available to scientists on 24 September would also have been impossible, they add, because of the conference and journal embargoes.
The trial sponsors argue that announcing the less-upbeat analyses along with the positive result would have been too complicated for the public to understand.
To their credit, the scientists involved did emphasize in their public statements that any vaccine effect was “modest”, and that the vaccine itself was of no immediate public-health utility. At the same time, however, they hammered home the message that this was “the first time an HIV vaccine has successfully prevented HIV infection in humans”, and implied that the event was somehow historic. Such statements, together with the selective initial presentation of the data, are well outside the scientific norms for presenting the results of clinical trials. They inevitably create suspicion that the trial sponsors may have put an excessively positive spin on results that are far from clear-cut, in a trial that has long been controversial (T. V. Padma Nature Med. 10, 1267; 2004). The trial has also been six years in the works, and so there seems no particular public-health urgency to justify publication by press conference.
Fortunately, such stories are still rare in science. Witness the way scientists have behaved since the beginning of the current H1N1 flu pandemic, in which the urgent threat to health creates legitimate tensions between getting results out fast and respecting peer review. Most researchers have negotiated this tension well, through a combination of fast-track publication by journals and online pre-publication sharing of preliminary data — but not through hyping their results.
Yet the temptation for scientists and their institutions to spin their research to the media, or to go publicity-mongering, is always there. And — as illustrated by the excessive public-relations campaign surrounding Ida, a fossil presented as a missing link in human evolution (see Nature 459, 484; 2009 and Nature 461, 1040; 2009) — too many in the media will buy into the initial hype.
Such behaviour is corrosive to the process of scholarly scientific communication. Research institutions must not allow it to become the norm.
Recent Comments