Evaluating Reasoning – A Study

chevron-icon
RFP Proposals
Top
chevron-icon
project-presentation-img
Soumil Rathi
Project Owner

Evaluating Reasoning – A Study

Expert Rating

n/a

Overview

Symbolic approaches face the problem of generality, whereas connectivist approaches face a problem with representation and data. Neuro-symbolic models are needed. This proposal evaluates various neuro-symbolic architectures (Eg. PyNeuraLogic and KANs) on their ability to embed logical rules, generalize, and efficiently model relational data. It will also contain a POC designed in MeTTa demonstrating the capability of such systems to improve efficiency and capability in AIRIS-like agents. The POC will be an AI system having to model a complex social setting with relational/structured interests; the system will have to efficiently embed and generalize over the rules for such a world.

RFP Guidelines

Neuro-symbolic DNN architectures

Internal Proposal Review
  • Type SingularityNET RFP
  • Total RFP Funding $160,000 USD
  • Proposals 9
  • Awarded Projects n/a
author-img
SingularityNET
Oct. 4, 2024

This RFP invites proposals to explore and demonstrate the use of neuro-symbolic deep neural networks (DNNs), such as PyNeuraLogic and Kolmogorov Arnold Networks (KANs), for experiential learning and/or higher-order reasoning. The goal is to investigate how these architectures can embed logic rules derived from experiential systems like AIRIS or user-supplied higher-order logic, and apply them to improve reasoning in graph neural networks (GNNs), LLMs, or other DNNs.

Proposal Description

Proposal Details Locked…

In order to protect this proposal from being copied, all details are hidden until the end of the submission period. Please come back later to see all details.

Proposal Video

Not Avaliable Yet

Check back later during the Feedback & Selection period for the RFP that is proposal is applied to.

  • Total Milestones

    7

  • Total Budget

    $80,000 USD

  • Last Updated

    7 Dec 2024

Milestone 1 - Comprehensive Study

Description

A key component of this RFP is a comprehensive study on different neuro-symbolic architectures evaluating them on their ability to embed and generalize over provided logic rules. This milestone includes researching architectures such as LRNNs KANs AlphaProof and evaluating them on their potential for use in Hyperon. This milestone take place in parallel with the POC.

Deliverables

The deliverable for this milestone The deliverable for this milestone will be a report containing the evaluations of various neuro-symbolic architectures for their logic embedding and relational learning capabilities. The evaluations will be well documented and well explained such that they promote future work and exploration.

Budget

$25,000 USD

Success Criterion

The submitted report should cover various neuro-symbolic architectures with their evaluations on logical embedding and reasoning. It should be well explained and understandable to members of the community.

Milestone 2 - Environmental Design

Description

This milestone covers designing the MeTTa environment for the proof of concept. This includes designing the other NPCs with their data presented through conversations (this also includes the rewards) with the agent. Thus the agent will be able to learn information about the others at the party through conversations and observations. This milestone also covers programming the possible actions an agent will be able to take within the environment.

Deliverables

The deliverable for this milestone will be the demo MeTTa script containing the party environment with each character being able to conduct various conversations depending on the agent’s actions.

Budget

$10,000 USD

Success Criterion

The MeTTa script successfully runs and simulates a party environment as described.

Milestone 3 - Agent Architecture

Description

The AI system is an AIRIS-based system which perceives data from its environment - makes rules based on it - and uses simulations and reward maximisation as its decision-making module. This milestone builds out the data perception and decision making (input and output) modules for the agent.

Deliverables

The deliverable will be the codebase which will now contain an agent that can receive data from its environment in a compatible format and can output actions that directly impact the environment

Budget

$5,000 USD

Success Criterion

The agent should be able to receive and process data incoming from the environment. This includes any data, information, rewards, goals, etc. Any actions taken by the agent should be implemented within the environment.

Milestone 4 - Rule Generation

Description

Symbolic rule generation is one of the most important components of the AI system. This milestone will implement the AIRIS-based rule generation system as the agent executes through the environment.

Deliverables

The deliverable for this milestone will be a codebase where the agent can now generate rules symbolically as it executes through the environment.

Budget

$5,000 USD

Success Criterion

The module is implemented successfully if the agent is generating symbolic rules based on its interactions. These rules should be directly based on its perceptions and should be logical.

Milestone 5 - Rule Embedding

Description

This is the primary milestone for this proof of concept and implements the LRNN (PyNeuraLogic) architecture after the rule generation to embed the rules. Embedding the rules then allows the representations to continually update and learn more about the world symbolically. Thus the milestone covers the learning process for the neuro-symbolic model.

Deliverables

The deliverable for this milestone will be the codebase containing an agent fitted with a neuro-symbolic model to embed logical rules. This model will be able to generalize over these rules as well as assign them weights based on all observations of the agent.

Budget

$17,500 USD

Success Criterion

The agent will now be learning symbolic rules by embedding them into the neuro-symbolic LRNN model. It would also be continuously updating its weights based on real-time observations.

Milestone 6 - Planning

Description

The AIRIS system uses a planning and modelling component to identify the best reward path. This milestone implements the LRNN model into that component, thus allowing the system to use the neuro-symbolic model within decision making. This completes the AI system and allows the agent to act based on its learnt and embedded rules.

Deliverables

The deliverable will be the codebase, which will now contain the neuro-symbolic model integrated within the planning and decision-making phase of the system.

Budget

$15,000 USD

Success Criterion

The agent should be able to operate autonomously based on its reward system. It should now be able to embed its generated rules and use those within decision making.

Milestone 7 - Final

Description

This milestone will document the entire Proof-Of-Concept including the codebase, the architecture, the environment, and any other information. The original codebase alongside all related documentation will be released publicly.

Deliverables

The deliverables for this milestone will include the final codebase for the POC and all associated documentation.

Budget

$2,500 USD

Success Criterion

The codebase should run without errors with the entire environment being replicable and verifiable by others in the community. The documentation should be transparent and ensure full reproducibility.

Join the Discussion (0)

Expert Ratings

Reviews & Ratings

    No Reviews Avaliable

    Check back later by refreshing the page.

feedback_icon