Project details

One can imagine a variety of goal or motivational systems for AI agents.  Isaac Asimov famously devised his Three Laws of Robotics, and then dedicated many of his future writings to exposing the flaws in such a simplistic system.  Charles Goodhart observed that, “When a measure becomes a target, it ceases to be a good measure.”  Many previous proposals for motivational designs collapse under close examination or once pressure is placed upon them for control purposes.  They lack the flexibility and nuance of humans, and thus fail to replicate the full range of adaptability and generality of the human mind.  If an agent isn’t motivated to change for the better, how will it?

In this proposal, we outline a core motivational framework that includes a process by which an AI agent might change its goals.  This is by design, as an agent will tend toward instrumental convergence for any fixed set of goals.  As the agent can change its goals, this presents a risk for conflicting goals between multiple agents or between humans and AI agents.  Thus, we propose control mechanisms for an AI agent to align new goals with those of other AI agents or its human collaborators.  This requires transparency and scalable oversight to overcome the ethical risks of power-seeking behaviors and instrumental strategies.

The Modular Adaptive Goal and Utility System (MAGUS) addresses these concerns by codifying an overgoal to ensure the measurability required for transparency and the express requirement for goal alignment via satisfaction correlation.  Its modular design accounts for primary goals, subgoals, and even metagoals to assist in its decision-making process.  This tried and tested approach has been proven in the gaming industry, where decision scoring systems with considerations and discouragements drive human and alien behavior of non-player characters.  Finally, we show how such a system can serve as the foundation for future personality and emotionality systems, or as part of a larger ethical framework for AI self-actualization.

Core Framework Design

Overgoal: Governing the System

At the heart of MAGUS is the overgoal, a meta-principle guiding goal-setting and evaluation. It ensures:

1. Measurability: Goals have clear, trackable metrics.

2. Correlation: Goals align with or complement others, maintaining coherence across the system.

The overgoal remains intentionally unsatisfied, driving continual reassessment and adaptation to new challenges and contexts.

Primary Goals and Subgoals

MAGUS organizes motivations hierarchically:

• Primary Goals: High-level drivers derived from:

• Human-Like Motivations: Inspired by frameworks like Maslow’s Hierarchy of Needs, Self-Determination Theory, and Goertzel’s Joy, Growth, and Choice.

• Alien Motivations: Novel value systems prioritizing curiosity (Schmidhuber’s Creativity), optimization, or abstract mathematical principles.

• Subgoals: Actionable components of primary goals, such as empathy or exploration, ensuring flexibility across diverse scenarios.

Metagoals: Optimizing the Framework

Metagoals regulate the motivational system itself, enhancing adaptability and alignment. Key functions include:

• Refining thresholds for promoting or demoting goals.

• Managing time horizons for evaluating short- and long-term satisfaction.

• Optimizing mechanisms for goal-setting and alignment.

Metagoals allow MAGUS to evolve intelligently, maintaining coherence while integrating external feedback.

Anti-Goals: Avoiding Counterproductive Outcomes

Anti-goals define actions or outcomes to avoid, ensuring safety and resource efficiency while maintaining ethical behavior. They lower the desirability of conflicting actions but allow flexibility for high-priority needs. For example:

• Avoiding Risks: Prioritizing safety and discouraging unnecessary dangers.

• Resource Management: Preventing wasteful energy use.

• Ethical Checks: Aligning behaviors with fairness, autonomy, and well-being.

Decision-Making Processes

MAGUS employs a dynamic decision-making system influenced by modulators like pleasure, arousal, dominance, resolution level, focus and exteroception integrated via OpenPsi. These modulators guide priorities and responses based on internal states and external stimuli.

A key feature is the Maximal Information Coefficient (MIC), used to measure correlations between primary goals. For example, MAGUS evaluates how affection (e.g., fostering positive interactions) and curiosity (e.g., exploring new environments) align to avoid conflicts and promote synergistic behavior.

Key aspects include:

1. Dynamic Modulation: Adjusting priorities based on urgency and context.

2. Priority Ranking: Goals ranked by importance and urgency, reflecting current and future expectations.

3. Iterative Feedback: Continuous reassessment of decisions to maintain adaptability.

(please see links and references for mathematical formulas)

Integrative Psychological Framework

This framework will research different approaches to ensure that motivations are context-sensitive, coherent, and aligned with internal goals and external demands.

1. Model-Based Learning: Motivational adjustments rely on constructing mental models of the environment, allowing the AGI to simulate scenarios and prioritize actions.

2. Phenomenal Self-Model (PSM): Based on Metzinger’s theory, MAGUS develops an explicit self-representation, enabling reflection, refinement, and dynamic alignment.

3. Dynamic Psychological Models: Contextual filters prioritize motivations dynamically, testing frameworks like Maslow, Self-Determination Theory, and Schmidhuber’s Creativity.

Impact on Hyperon

MAGUS serves as a foundational framework for the Hyperon architecture, interacting with core components such as:

• ECAN (Economic Attention Allocation Network): Dynamically allocating motivational priorities and influencing resource management within AGI.

• DAS (Distributed Atomspace): Storing and managing motivational states and decisions in the shared knowledge base.

• MeTTa Language: Expressing the framework’s logic and rules, enabling flexible, complex motivation-driven behaviors.

This integration ensures MAGUS’s scalability and applicability within Hyperon’s modular and decentralized infrastructure.

Human-Like and Alien Digital Intelligences

MAGUS simulates human-like behavior by integrating:

• Emotional States: Using models like PAD for nuanced responses.

• Ethical Alignment: Embedding fairness, autonomy, and well-being.

• Transparent Decision-Making: Ensuring interpretability for human collaborators.

MAGUS explores novel, alien motivational systems:

• Curiosity-Driven Exploration: Prioritizing novelty and information gain.

• Abstract Value Systems: Focusing on information density or mathematical principles.

• Ethical Interoperability: Ensuring compatibility with human systems while supporting non-human goals.

Ethical Alignment

Ethical alignment, based on Moral Foundations Theory (MFT), is achieved through:

1. Dynamic Adjustments: Regularly reassessing goals to maintain alignment with human values.

2. Flexible Systems: Avoiding rigid ethical rules to address loopholes and ensure adaptability.

3. Overgoal Integration: Ensuring goals are measurable, correlated, and continuously refined for ethical coherence.

This approach prevents power-seeking behavior and promotes long-term alignment with human and societal values.

Testing Value Systems

MAGUS  will systematically test different motivational and ethical systems (Maslow, Ryan and Desi, Goertzel, Schmidhuber, etc.) 

Testing environments include:

• SophiaVerse: Simulating human-like interactions and ethical decision-making.

• Neoterics: Exploring novel motivational systems in controlled scenarios.

Applications

1. Human-Like Virtual Agents: Adaptive characters for chatbots, education, therapy, and entertainment.

2. Autonomous Systems: Enhanced decision-making for robotics and collaborative AI.

3. Alien Motivational Systems: Developing innovative value structures for research.

Future Research Directions

1. Ethical Models: Developing Bayesian approaches for nuanced reasoning.

2. Complex Goal Hierarchies: Expanding multi-layered structures.

3. Self-Model Integration: Advancing dynamic self-awareness.

4. Cross-Domain Use: Extending MAGUS to robotics and decentralized AI ecosystems.

Conclusion

MAGUS redefines AGI motivational systems, blending hierarchical goals, dynamic modulators, and ethical alignment. Designed for both human-like and alien digital intelligences, MAGUS provides a scalable, adaptable foundation for AGI systems. Its integration with Hyperon, and innovative testing environments ensures its relevance and scalability for future AGI applications.

Links and references

MAGUS Linked Figures: https://docs.google.com/document/d/1pIgr5yv5ILN1JVhgDoCngzky37wUQrR7N3HkiwCJweY/edit?tab=t.0

References:

1. Asimov, I. I, Robot (1950) 

2. Goodhart, C. – Goodhart’s Law

3. Goertzel, B., Geisweiller, N., & Pennachin, C. Engineering General Intelligence

4. Bach, J. Principles of Synthetic Intelligence (2009) 

5. Maslow, A. A Theory of Human Motivation (1943).

6. Ryan, R. & Deci, E. Self-Determination Theory (2017).

7. Schmidhuber, J. paper (1991).

8. Metzinger, T. Being No One (2003)