Constitutional AI (CAI), pioneered by Anthropic, is an approach to training AI systems that leverages a set of principles, akin to a constitution, to guide the AI’s behavior. This method prioritizes implementation of human value through these established principles, supplemented by minimal examples for fine-tuning prompts. It aims to reduce reliance on extensive human labeling for tasks like ensuring harmlessness.
The term “constitutional” suggests that building and deploying a general AI system requires establishing core principles to guide its development and use, even if those principles remain implicit or unspoken. CAI involves a two-stage process:
- A supervised learning phase, where an initial model generates responses, critiques its own responses according to the principles, revises the responses, and is fine-tuned on the revised responses.
- 2. A reinforcement learning phase, where the fine-tuned model generates response pairs, evaluates which is better according to the principles, and this AI feedback is used to train a preference model. Reinforcement learning is then performed using the preference model as the reward signal.
Balancing Helpfulness and Harmlessness
By encoding AI training objectives in a set of natural language instructions or principles, CAI aims to make AI decision-making more transparent, understandable, and controllable. This approach reduces the reliance on extensive human feedback, making the supervision of AI models more efficient and cost-effective. CAI also addresses the crucial challenge of balancing helpfulness and harmlessness, encouraging AI to engage and explain its objections to harmful requests. Furthermore, by using principles to guide AI behavior, CAI can help ensure that AI systems are fair, unbiased, and do not perpetuate harmful stereotypes. With its potential to scale supervision, improve transparency, and enable faster iteration, Constitutional AI holds great promise for the development of safer and more aligned AI systems.
Judicial AI
Luminos.AI is pioneering what it calls Judicial AI, a novel approach that both trains and evaluates AI systems using a custom-built “constitution” of principles. This constitution governs the AI’s behavior, offering a more targeted approach compared to the broad principles outlined in the original Constitutional AI paper. Judicial AI provides a specific framework for operationalizing and implementing laws and rules governing AI, similar to how a constitution’s high-level values are translated into granular legal provisions aligned with existing laws and regulations.
Luminos AI Judges can be used at all phases of the model lifecycle – from training, where each judge can help optimize for the legality of the AI’s outputs, to deployments, where models can use the Judge’s legality scoring to ensure each output aligns with the right laws and principles.
In Judicial AI, an AI “Judge” evaluates each AI output against specific provisions of the constitution, which can be drafted collaboratively by your team and Luminos.AI. This granular score ensures the AI adheres to each of the established principles. Luminos.AI provides a starter set of provisions that can be tailored to your specific needs, guaranteeing alignment between the AI system and your organization’s goals and values.
Judicial AI’s provisions encompass both prohibitions and affirmations. Prohibitions prevent the AI from engaging in harmful or unethical actions, such as biased decision-making, specific types of privacy violations, or deceptive practices. Affirmations, on the other hand, encourage desirable qualities like empathy, explainability, respect, and promoting fairness and justice. This two-pronged approach aligns to legal frameworks and fosters the development of AI systems that are not only safe and ethical but also aligned with human values and expectations.
For more information about Luminos.AI, which just came out of stealth this month, reach out to contact@luminos.ai.
If you enjoyed this post please support our work by encouraging your friends and colleagues to subscribe to our newsletter: