A new study by researchers from Rice University and Stanford University suggests that the quality of generative AI may decrease if the AI engine is trained based on machine-generated input rather than human input. Essentially, if AI models learn from each other in a self-consuming manner, it may affect the long-term quality of the system.

How can generative AI models go "mad"?

Researchers refer to this effect as "Model Autophagy Disorder" (MAD) and compare it to mad cow disease. Mad cow disease is a neurodegenerative disease that occurs in cows that consume infected remains of other cows. Just as those cows are affected by consuming beef, AI systems can be influenced by obtaining information from other systems.

AI models require fresh data from the real world to work effectively, both in terms of quality and diversity. In the future, AI models may be at risk due to the influence of individuals attempting to expedite the learning process.

Richard Baraniuk, a computer engineer at Rice University, told Scientific Alert magazine, "Some consequences are obvious: without enough fresh, real data, future generative models are destined to fall into a MAD state."

These findings come from testing visual generative AI models trained on three types of data: fully synthetic data, a mixture of synthetic and real fixed data, and a mixture of synthetic and real fresh data. In the first two cases, the model's output becomes increasingly flawed.

For example, computer-generated faces have noticeable artifacts, and digits become increasingly difficult to read. Faces also start to become more and more similar, highlighting the lack of diversity in models trained on synthetic data over time.

"Our team has extensively studied this feedback loop, and the bad news is that even after several generations of such training, new models may become irreparably damaged," said Baraniuk.

While this experiment focused on image generation, it can be inferred to have far-reaching consequences. As AI is applied in more and more systems, researchers warn of a potential "doomsday scenario."

"If left unchecked, MAD could undermine the data quality and diversity of the entire internet," said Baraniuk. "Moreover, it seems inevitable that even in the near future, AI autophagy will produce unforeseen consequences."