Self-Selection and the Diminishing Returns of Research

New paper from Science Advances highlights the replication problem in science, “….We use publicly available data to show that published papers in top psychology, economics, and general interest journals that fail to replicate are cited more than those that replicate. This difference in citation does not change after the publication of the failure to replicate. Only 12% of post replication citations of nonreplicable findings acknowledge the replication failure. Existing evidence also shows that experts predict well which papers will be replicated. Given this prediction, why are nonreplicable papers accepted for publication in the first place? A possible answer is that the review team faces a trade-off. When the results are more “interesting,” they apply lower standards regarding their reproducibility….”

Core result, “…Our main finding is that papers that fail to replicate in (5-7) are cited more than those that are replicable. We find no significant change in citation trends, even after the publication of the failed replication. Notably, only a minority of publications after the failed replications were published acknowledge the failure. The “quality” of citations of papers that failed to replicate is similar to that of papers that were replicated: We do not find a difference in how often the citations of nonreplicable publications are cited by others or in the impact factor of the journals in which citations are published…”

The evidence, "...Figure 1 shows the distribution of total citation counts by the end of 2019 of the papers included in the replication projects, separately depending on replicability for Nature/Science, economics journals, and psychology journals. Replicability is measured according to the criterion that the replication project featured a P value of 0.05 or lower in a two-sided test, with an effect in the same direction as the original test….”

new advances in understanding Maxwell’s demon"...The team, led by Manzano, wondered if there was a way toimplement something like Maxwell’s demon but without the information requirements. They imagined a two-compartment system with a door, as before. But in this case, the door would open and close on its own. Sometimes particles would randomly separate themselves into hotter and colder compartments. The demon could only watch this process and decide when to turn the system off. In theory this process could create a small temperature imbalance, and therefore a useful heat engine, if the demon was smart about when to end the experiment and lock any temperature imbalance in place, much as a smart gambler on a hot streak knows when to leave the table. “You can either play all night on the roulette table, or you can stop if you win $100,” said Édgar Roldán, a physicist at the International Center for Theoretical Physics in Italy who was a co-author on the study. “We’re saying we don’t need such a complicated device as Maxwell’s demon to extract work in the second law. We can be more relaxed.”The researchers then implemented such a gambling demon in a nanoelectronic device, to show it was possible..."

Jonathan O'Callaghan, "How Maxwell’s Demon Continues to Startle Scientists,"Quanta Magazine, April 22, 2021, https://www.quantamagazine.org/how-maxwells-demon-continues-to-startle-scientists-20210422/

How Maxwell’s Demon Continues to Startle Scientists

The universe bets on disorder. Imagine, for example, dropping a thimbleful of red dye into a swimming pool. All of those dye molecules are going to slowly spread throughout the water. Physicists quantify this tendency to spread by counting the number of possible ways the dye molecules can be arranged. There’s one possible state where the molecules are crowded into the thimble. There’s another where, say, the molecules settle in a tidy clump at the pool’s bottom. But there are uncountable billions of permutations where the molecules spread out in different ways throughout the water. If the universe chooses from all the possible states at random, you can bet that it’s going to end up with one of the vast set of disordered possibilities.

Seen in this way, the inexorable rise in entropy, or disorder, as quantified by the second law of thermodynamics, takes on an almost mathematical certainty. So of course physicists are constantly trying to break it.

One almost did. A thought experiment devised by the Scottish physicist James Clerk Maxwell in 1867 stumped scientists for 115 years. And even after a solution was found, physicists have continued to use “Maxwell’s demon” to push the laws of the universe to their limits.

In the thought experiment, Maxwell imagined splitting a room full of gas into two compartments by erecting a wall with a small door. Like all gases, this one is made of individual particles. The average speed of the particles corresponds to the temperature of the gas — faster is hotter. But at any given time, some particles will be moving more slowly than others.

What if, suggested Maxwell, a tiny imaginary creature — a demon, as it was later called — sat at the door. Every time it saw a fast-moving particle approaching from the left-hand side, it opened the door and let it into the right-hand compartment. And every time a slow-moving particle approached from the right, the demon let it into the left-hand compartment.

After a while, the left-hand compartment would be full of slow, cold particles, and the right-hand compartment would grow hot. This isolated system would seem to grow more orderly, not less, because two distinguishable compartments have more order than two identical compartments. Maxwell had created a system that appeared to defy the rise of entropy, and thus the laws of the universe.

“He tried to prove a system where the entropy would decrease,” said Laia Delgado Callico, a physicist at King’s College London. “It’s a paradox.”

Two advances would be crucial to solving Maxwell’s demon. The first was by the American mathematician Claude Shannon, regarded as the founder of information theory. In 1948, Shannon showed that the information content of a message could be quantified with what he called the information entropy. “In the 19th century, no one knew about information,” said Takahiro Sagawa, a physicist at the University of Tokyo. “The modern understanding of Maxwell’s demon was established by Shannon’s work.”

The second vital piece of the puzzle was the principle of erasure. In 1961, the German American physicist Rolf Landauer showed that any logically irreversible computation, such as the erasing of information from a memory, would result in a minimal nonzero amount of work converted into heat dumped into the environment, and a corresponding rise in entropy. Landauer’s erasure principle provided a tantalizing link between information and thermodynamics. “Information is physical,” he later proclaimed.

In 1982, the American physicist Charles Bennett put the pieces of the puzzle together. He realized that Maxwell’s demon was at core an information-processing machine: It needed to record and store information about individual particles in order to decide when to open and close the door. Periodically it would need to erase this information. According to Landauer’s erasure principle, the rise in entropy from the erasure would more than compensate for the decrease in entropy caused by the sorting of the particles. “You need to pay,” said Gonzalo Manzano, a physicist at the Institute for Quantum Optics and Quantum Information in Vienna. The demon’s need to make room for more information inexorably led to a net increase in disorder.

Then in the 21st century, with the thought experiment solved, the real experiments began. “The most important development is we can now realize Maxwell’s demon in laboratories,” said Sagawa.

In 2007 scientists used a light-powered gate to demonstrate the idea of Maxwell’s demon in action; in 2010, another team devised a way to use the energy produced by the demon’s information to coax a bead uphill; and in 2016 scientists applied the idea of Maxwell’s demon to two compartments containing not gas, but light.

“We switched the roles of matter and light,” said Vlatko Vedral, a physicist at the University of Oxford and one of the study’s co-authors. The researchers were ultimately able to charge a very small battery.

Others wondered if there might be less demanding ways to use information to extract useful work from a similar system. And research published in February in Physical Review Letters seems to have found a way to do so. The work makes the demon into a gambler.

The team, led by Manzano, wondered if there was a way to implement something like Maxwell’s demon but without the information requirements. They imagined a two-compartment system with a door, as before. But in this case, the door would open and close on its own. Sometimes particles would randomly separate themselves into hotter and colder compartments. The demon could only watch this process and decide when to turn the system off. In theory this process could create a small temperature imbalance, and therefore a useful heat engine, if the demon was smart about when to end the experiment and lock any temperature imbalance in place, much as a smart gambler on a hot streak knows when to leave the table. “You can either play all night on the roulette table, or you can stop if you win $100,” said Édgar Roldán, a physicist at the International Center for Theoretical Physics in Italy who was a co-author on the study. “We’re saying we don’t need such a complicated device as Maxwell’s demon to extract work in the second law. We can be more relaxed.” The researchers then implemented such a gambling demon in a nanoelectronic device, to show it was possible.

Ideas like this could prove useful in designing more efficient thermal systems, like refrigerators, or even in developing more advanced computer chips, which may be approaching a fundamental limit dictated by Landauer’s principle.

For the time being, though, our laws of the universe are safe, even when placed under the greatest scrutiny. What has changed is our understanding of information in the universe, and with it our appreciation of Maxwell’s demon, first a troublesome paradox, and now an invaluable concept — one that has helped to illuminate the remarkable link between the physical world and information.