Science Notes: Generative AI and Its Impacts

Source Domain: www.rockinst.org

Generative AI has roots reaching back to the 1960s, when Joseph Weizenbaum created ELIZA, an early natural language program designed to mimic empathetic conversation. Generative AI refers to certain machine learning models that can create new content, such as text, images, audio, or code, by learning patterns from large amounts of data and using those patterns to produce original outputs in response to user prompts.

To understand the recent growth and potential impacts of generative AI, and how it works, it is helpful to examine how models referred to as artificial neural networks (ANNs) evolved into the models used today. These models were inspired by how the human brain works, where billions of connected neurons pass signals that allow us to think, move, and make decisions. ANNs use a simplified version of this idea, using small units (“neurons”) stacked in layers that pass information and learn patterns from data.1 A major turning point came with the rise of deep neural networks, which use many layers and can learn far more complex patterns than earlier models. As ANNs evolved, they were increasingly used to implement two long-standing data modeling approaches: discriminative and generative models. Discriminative models focus primarily on prediction tasks, learning relationships between inputs and outputs to classify or separate outcomes. Generative models, by contrast, aim to learn how data itself is produced by capturing the underlying data distribution, making it possible to generate new examples that resemble the original data.2

After a series of developments in crucial models—including variational autoencoders (VAEs), generative adversarial networks (GANs), diffusion models, and transformers—the ability of AI systems to generate realistic data at scale expanded significantly. A breakthrough for generative models came with the emergence of foundation models and, ultimately, today’s large-scale generative AI…

Source