Direct the MOSES Evolutionary Exploration via LLMs

chevron-icon
RFP Proposals
Top
chevron-icon
project-presentation-img
Patrick Nercessian
Project Owner

Direct the MOSES Evolutionary Exploration via LLMs

Expert Rating

n/a

Overview

This proposal seeks to promote the application of reinforcement learning principles within the MOSES (Meta-Optimizing Semantic Evolutionary Search) algorithm through the strategic integration of Large Language Models (LLMs). The design is conceptually inspired by Q-learning, and our approach would be to experiment with using LLMs in place of Q-learning functions to estimate the utility of specific evolutionary directions in MOSES. By integrating LLMs into the generational modeling and fitness estimation stages of the evolutionary search, we believe there is potential to guide evolutionary operations to efficiently explore and exploit the search space for optimal solutions.

RFP Guidelines

Utilize LLMs for modeling within MOSES

Internal Proposal Review
  • Type SingularityNET RFP
  • Total RFP Funding $150,000 USD
  • Proposals 10
  • Awarded Projects n/a
author-img
SingularityNET
Oct. 9, 2024

This RFP invites proposals to explore the integration of LLMs into the MOSES evolutionary algorithm. Researchers can pursue one of several approaches, including generation modeling, fitness function learning, fitness estimation, investigation into domain-independent “cognitively motivated” fitness functions, or propose new innovative ways to leverage LLMs to enhance MOSES's capabilities within the OpenCog Hyperon framework.

Proposal Description

Proposal Details Locked…

In order to protect this proposal from being copied, all details are hidden until the end of the submission period. Please come back later to see all details.

Proposal Video

Not Avaliable Yet

Check back later during the Feedback & Selection period for the RFP that is proposal is applied to.

  • Total Milestones

    4

  • Total Budget

    $60,000 USD

  • Last Updated

    8 Dec 2024

Milestone 1 - Utility Estimation Framework Development

Description

Develop the building blocks for the LLM integration with MOSES.

Deliverables

A preliminary framework for LLMs to perform utility evaluation of variation operators. Design and integration of the data structure for input to the LLM (current evolutionary states and actions). Simulated initial utility estimate outputs for these actions.

Budget

$20,000 USD

Success Criterion

We will have succeeded here if we have a framework which can take in a data structure representing the state of an individual and possible action(s) and output an esimated utility for the action(s).

Milestone 2 - Expand Framework and Integrate with MOSES

Description

A more cohesive framework which will allow relatively easy setup (for a new problem domain) to run an evolutionary algorithm using LLMs to direct favorable variation operators.

Deliverables

An implemented modular architecture for integrating utility estimation. An implemented state-action representation for generational modeling & fitness estimation. An LLM layer for fitness estimation (directed mutation/crossover) Integration of above deliverables into MOSES, and simulation of initial search process.

Budget

$15,000 USD

Success Criterion

We will have succeeded here if a typical software engineer could use our framework to run MOSES evolutionary algorithms with LLM-directed variation for a new problem domain.

Milestone 3 - Experiments and Iterative Decision-Making

Description

Run initial experiments, and utilize the results to make decisions on chosen LLMs, prompts, and other hyper-parameters or design choices. Iterate on the framework repeatedly and improve its performance based on these findings.

Deliverables

Evaluate prospective LLMs via empirical testing on utility estimation tasks (or related tasks) and select a production set of LLMs based on their performance. Integrate the selected LLMs with MOSES and continuously tune the prompts for each model. Run initial evolutionary experiments to assess the impact of LLM-informed decisions.

Budget

$12,500 USD

Success Criterion

We will have succeeded here by getting initial results from our new framework, having made decisions on tweaks to the framework based on preliminary results, and have repeated these steps multiple times.

Milestone 4 - Finalize Findings and Documentation

Description

Consolidate results of our experiments by comparing results based on: A) differences between the runs B) similar runs executed via standalone MOSES usage

Deliverables

Analyze the data from experiments to identify successful strategies and common patterns. Compare the performance of different LLM-enhanced MOSES configurations (e.g., Moses-GPT-4, Moses-Claude-Sonnet-3.5, Moses-Llama3-70B) against traditional MOSES implementations. Compile documentation regarding implementation and experimental methods. Prepare findings for publication and further research. Document and release the enhanced MOSES framework with comprehensive instructions.

Budget

$12,500 USD

Success Criterion

We will have succeeded here by having a final report documenting our journey and findings as it relates to framework-internal decisions and to baseline MOSES attempts.

Join the Discussion (0)

Expert Ratings

Reviews & Ratings

    No Reviews Avaliable

    Check back later by refreshing the page.

feedback_icon