By the Engine Policy Team
This blog continues our series called “AI Essentials,” which aims to bridge the knowledge gap surrounding AI-related topics. It discusses what a foundation model is, how startups are leveraging them to drive greater innovation in the AI ecosystem, and what policymakers need to keep in mind.
Developing cutting-edge AI systems requires vast amounts of compute and data, making the process prohibitively expensive for many companies. Yet, recent innovations in AI have enabled startups and smaller organizations to harness powerful AI without needing to build everything from scratch.
Foundation models are large-scale AI models trained on vast amounts of diverse data, making them adaptable to a variety of tasks. These models serve as the backbone for AI applications, providing a base level of knowledge that other developers, like startups, can refine for specific uses. You can think of foundation models as a handyman, individuals that are generally useful for a wide range of home improvement tasks, but with a bit more specialized training can become very good at specific tasks, like plumbing or electrical.
Rather than developing an AI system from the ground up, companies can leverage foundation models to accelerate innovation, reducing both costs and complexity. These models can be closed or open-source — for example, OpenAI’s GPT-4 (closed) and Meta’s LLaMA (open) are both foundation models. Through techniques like fine-tuning, foundation models can be adapted for specific applications, like reviewing pitch decks, improving public services, or enhancing the capacity of small businesses.
Foundation models streamline AI development by providing a starting point, saving startups from the immense resource burden of training a model from scratch. Training a foundation model requires vast computational resources, often amounting to millions of dollars in cloud costs and specialized hardware. Most startups do not have access to those sorts of resources. Even if they did aim to build their own foundation model to then fine tune, it would likely be out of date by the time they finished training, based upon the rapid increases in performance in models from leading AI labs. By using a foundation model (or multiple), developers can focus on optimizing performance for their unique needs, significantly lowering barriers for startups to bring AI-based products to market.
Policymakers’ approach to AI regulation — and to foundation models in particular — will impact far beyond just the foundation model developers themselves, given the ecosystem of startups and others building upon them. For example, AI rules that incorrectly assign liability to developers of foundation models rather than malign actors will restrict the availability of those models, because developers will not want to be liable for actions of others that they do not have control over. This disincentive will be particularly acute for open-source models, because open source developers lack formal relationships with and awareness of those who use and build with the technology they make widely available. Therefore, it’s essential for policymakers to understand how foundation models are leveraged by startups to avoid stifling innovators.
Engine is a non-profit technology policy, research, and advocacy organization that bridges the gap between policymakers and startups. Engine works with government and a community of thousands of high-technology, growth-oriented startups across the nation to support the development of technology entrepreneurship through economic research, policy analysis, and advocacy on local and national issues.
