Foundation Models

A foundation model is a large, broadly trained AI model that can serve as a reusable base for many different applications. It learns general patterns from large and diverse datasets during pretraining, after which developers can adapt it to particular tasks. Its defining feature is broad adaptability, not merely its size or ability to generate content.

How it works

A foundation model is developed through resource-intensive pretraining and then reused rather than trained separately for every task.

• Pretraining commonly uses self-supervised learning, in which the training data itself supplies learning signals, such as predicting missing or subsequent elements.
• The base model can be adapted through prompting, fine-tuning, retrieval of external information, or additional task-specific components.
• Foundation models may process text, images, audio, code or several modalities together; models handling multiple data types are called multimodal models.

Relationship with other AI terms

The term describes a model's broad and reusable role within an AI ecosystem. It should not be treated as synonymous with every large or generative model.

• Many modern large language models are foundation models because they are broadly pretrained and can be adapted to numerous language tasks.
• Generative AI describes systems that create content, whereas a foundation model may also support classification, prediction, search or representation tasks.
• An AI application usually combines the foundation model with prompts, data sources, interfaces, safeguards and task-specific software.

Why foundation models matter for regulation

Because one upstream model may support numerous downstream systems, its capabilities and failures can spread across sectors and applications. Regulation therefore examines risks at both the model level and the level of each deployed use.

• Important concerns include biased outputs, inaccurate generated content, privacy leakage, security vulnerabilities, misuse and limited transparency.
• Risk management may require testing and evaluation, documentation, access controls, incident monitoring and communication of limitations to downstream developers.
• The severity of risk depends not only on the model but also on the deployment context, affected persons and degree of human oversight.

How UPSC asks this