chevron-icon
Active RFP

PLN guidance to LLMs

Top
chevron-icon
SingularityNET
RFP Owner

PLN guidance to LLMs

Utilizing PLN to provide guidance to LLMs, such as providing an alternative to graphRAG for augmenting LLM memory

  • Type SingularityNET RFP
  • Total RFP Funding $80,000 USD
  • Proposals 3
  • Awarded Projects n/a

Overview

  • Est. Complexity

    💪 75/ 100

  • Est. Execution Time

    ⏱️ 6 Months

  • Proposal Winners

    🏆 Single

  • Max Funding / Proposal

    $80,000USD

RFP Details

Short summary

This RFP seeks proposals to explore how Probabilistic Logic Networks (PLN) can be used to provide guidance to LLMs. We are particularly interested in applying PLN to develop an alternative to graphRAG for augmenting LLM memory using Atomspace knowledge graphs.

Main purpose

The purpose of this RFP is to investigate how PLN can interface with LLMs, to enhance their functionality. Proposals that offer innovative approaches to improving the capabilities and usefulness of LLMs through PLN integration are particularly encouraged.

 

Long description

Context and background:

SingularityNET Foundation, in collaboration with other partners such as the OpenCog Foundation and TrueAGI, is working toward a scalable implementation of the Hyperon AGI framework running on decentralized infrastructure, and toward implementation of the PRIMUS cognitive architecture within this framework.

Hyperon and PRIMUS are complex systems involving multiple components, which need to demonstrate appropriate functionalities both individually and in combination. This RFP aims to address a portion of this overall need, via funding the initial iteration of one significant component of PRIMUS within Hyperon.

The goal of this RFP is to explore how Probabilistic Logic Networks (PLN) can be applied to enhance LLMs. By leveraging PLN’s probabilistic reasoning capabilities, the aim is to improve memory systems, reasoning capabilities, and information retrieval processes. While a comparison to existing methods, such as retrieval-augmented generation (RAG), is of interest, this research encourages proposals that consider alternative approaches to integrating PLN with LLMs.

There are two suggested avenues for exploration, but researchers are welcome to propose other innovative methods where PLN could enhance LLM performance.

  1. Batch Mode PLN on Atomspace Knowledge Graphs:
    This approach applies PLN in batch mode on an Atomspace knowledge graph to enrich the graph with more informative nodes and probabilistic links. The enhanced knowledge graph would then support LLM memory by improving the structure of the graph to facilitate more accurate and relevant information retrieval. The output of the PLN-enriched knowledge graph can be used to develop a probabilistically weighted graphRAG, improving retrieval without the need for real-time inference. This method operates in the background, refining the knowledge graph over time and offering a probabilistic edge over static graphs commonly used in graphRAG systems.
  2. Real-Time PLN Queries on Atomspace:
    Another potential method involves using PLN to perform real-time queries on the knowledge graph. Instead of relying solely on pre-built graphs, this approach would allow LLMs to engage dynamically with the knowledge graph. By running live PLN queries, LLMs could reason probabilistically in real time, enabling context-sensitive decisions based on the updated graph. While this approach offers more nuanced reasoning capabilities, it may be limited by the current speed of MeTTa interpreters, making real-time inference less practical at this time. However, future improvements in interpreter performance could make this a highly adaptive solution for LLM memory and reasoning.

Expected Outcomes:

  • PLN-enhanced Knowledge Graphs: Demonstrate how PLN can be used to enrich knowledge graphs, improving their structure and utility for LLM memory and reasoning tasks. The graph should include useful nodes and probabilistically weighted links that enhance retrieval processes.
  • Improved LLM Memory and Reasoning: Develop and demonstrate systems where PLN improves LLM memory retrieval and reasoning, either through batch-mode knowledge graph enrichment or real-time queries, compared to existing methods like graphRAG.
  • Performance Evaluation: Conduct a thorough evaluation of the proposed approaches (batch mode, real-time PLN queries, or alternative methods), including performance metrics, improvements in reasoning, and memory accuracy.
  • Comparative Study: Compare the effectiveness of PLN-enhanced systems against traditional RAG or graphRAG approaches, demonstrating the unique benefits PLN offers for improving LLM memory, retrieval, and reasoning.

Functional Requirements

Must Have:

  • PLN Integration: Implement a solution where PLN interfaces with LLMs, either by:
    • Enhancing knowledge graphs in batch mode to support LLM retrieval, or
    • Using real-time PLN queries for dynamic reasoning and memory enhancement.
  • Test and Demonstrate: The system must demonstrate clear improvements in LLM memory retrieval or reasoning based on PLN-enhanced processes. 
  • Empirical Results: Provide testing and validation results for the selected method, showing quantifiable improvements in reasoning or retrieval accuracy compared to standard approaches (e.g., graphRAG).

Should Have:

  • Comparative Evaluation: Include a comparison of the chosen PLN method (batch mode, real-time, or other) with existing techniques like graphRAG. Highlight the performance trade-offs in terms of reasoning quality, speed, and retrieval effectiveness.
  • Broader Applicability: Demonstrate how the PLN-guided approach can be generalized or adapted to other systems beyond LLMs. 
  • Demonstration Tools: Provide visualizations, reports, or other outputs to showcase how PLN-enhanced memory or reasoning processes work in practice, ensuring results are clear and easy to interpret.

Could Have:

  • Hybrid Approaches: Explore and implement a combination of batch-mode and real-time PLN queries, or other novel methods that balance performance with reasoning flexibility.
  • Alternative Use Cases: Suggest innovative uses of PLN to guide external systems that go beyond memory enhancement for LLMs, identifying new domains where PLN can add value.

Non-functional Requirements

  • Architecture: The solution should leverage Atomspace as the underlying knowledge graph system, with PLN integrated within the OpenCog Hyperon framework. The architecture must support PLN’s probabilistic reasoning to guide LLMs, whether through batch or real-time processes.
  • Programming language: The system should be implemented in languages compatible with Hyperon (preferably in MeTTa, but Python, Rust or C++ are also acceptable). Flexibility in programming is essential to allow PLN integration, batch processing, and real-time querying.
  • Performance: The solution must be optimized for moderate datasets and allow efficient processing, particularly in batch-mode operations. While real-time PLN queries may be relatively slower, correctness and reasoning accuracy should take priority in this context.
  • Scalability: The solution does not need to be fully scalable for production, but it should be able to handle moderate knowledge graphs and perform meaningful experimentation, demonstrating feasibility for broader implementation.
  • Documentation: Provide clear, well-organized documentation detailing the methods, architecture, and processes used. Ensure that the solution can be replicated, extended, or adapted for future research, including instructions for reproducing results and modifying the system for other applications.

Main evaluation criteria

Alignment with requirements and objective

  • Does the proposal meet the requirements and advances the objectives of the RFP

Pre-existing R&D

  • Has the team previously done similar or related research or development work in other platforms / languages / contexts?

Team competence

  • Does the team have relevant skills?

Cost

  • Does the proposal offer good value for money?

Timeline

  • Does the proposal include a set of clearly defined milestones?

Other resources

References

Hyperon and related AI-platforms are quickly evolving! This is a bit of a moving target, but the internal SingularityNET team will be available for help and expert advice, where needed. Also included:

  • SingularityNET technology links 
  • Educational materials and resources for learning MeTTa
  • SingularityNET holds MeTTa study group calls every other week. Proposers are welcome to attend for support from our researchers and community.
  • Recurring Hyperon study group calls for community are currently being planned. These will cover MOSES, ECAN, PLN, and other key components of the OpenCog and PRIMUS Hyperon cognitive architectures.
  • Access to the SingularityNET World Mattermost server, with a dedicated channel for discussion and support among the RFP-winning teams and SingularityNET resources.

RFP Status

Completed & Awarded

The community and public are invited to view the full proposals and give feedback. During this time the RFP committee will doing their formal selection process to award winning proposals.

View Awarded Projects
3 proposals
rfp=proposal-img
EXPERT REVIEW 4.0

Plantagenet: A Guidance System for LLMs

  • Type SingularityNET RFP
  • Funding Request n/a
  • RFP Guidelines PLN guidance to LLMs
author-img
Luke Mahoney (MLabs)
Dec. 5, 2024
rfp=proposal-img
EXPERT REVIEW 3.6

PSL-Augmented Atomspace Knowledge Graphs

  • Type SingularityNET RFP
  • Funding Request n/a
  • RFP Guidelines PLN guidance to LLMs
author-img
Mayank Kejriwal
Oct. 24, 2024
rfp=proposal-img
EXPERT REVIEW 2.3

PLN Guidance for Large Language Models (LLMs)

  • Type SingularityNET RFP
  • Funding Request n/a
  • RFP Guidelines PLN guidance to LLMs
author-img
aasavravi1234
Dec. 7, 2024
0 projects

No Projects Avaliable

Check back later by refreshing the page.

Join the Discussion (0)

feedback_icon