Direct the MOSES Evolutionary Exploration via LLMs

chevron-icon
Back
Top
chevron-icon
project-presentation-img
Patrick Nercessian
Project Owner

Direct the MOSES Evolutionary Exploration via LLMs

Status

  • Overall Status

    ⏳ Contract Pending

  • Funding Transfered

    $0 USD

  • Max Funding Amount

    $60,000 USD

Funding Schedule

View Milestones
Milestone Release 1
$20,000 USD Pending TBD
Milestone Release 2
$15,000 USD Pending TBD
Milestone Release 3
$12,500 USD Pending TBD
Milestone Release 4
$12,500 USD Pending TBD

Project AI Services

No Service Available

Overview

This proposal seeks to promote the application of reinforcement learning principles within the MOSES (Meta-Optimizing Semantic Evolutionary Search) algorithm through the strategic integration of Large Language Models (LLMs). The design is conceptually inspired by Q-learning, and our approach would be to experiment with using LLMs in place of Q-learning functions to estimate the utility of specific evolutionary directions in MOSES. By integrating LLMs into the generational modeling and fitness estimation stages of the evolutionary search, we believe there is potential to guide evolutionary operations to efficiently explore and exploit the search space for optimal solutions.

RFP Guidelines

Utilize LLMs for modeling within MOSES

Complete & Awarded
  • Type SingularityNET RFP
  • Total RFP Funding $150,000 USD
  • Proposals 10
  • Awarded Projects 1
author-img
SingularityNET
Oct. 9, 2024

This RFP invites proposals to explore the integration of LLMs into the MOSES evolutionary algorithm. Researchers can pursue one of several approaches, including generation modeling, fitness function learning, fitness estimation, investigation into domain-independent “cognitively motivated” fitness functions, or propose new innovative ways to leverage LLMs to enhance MOSES's capabilities within the OpenCog Hyperon framework.

Proposal Description

Company Name (if applicable)

Vectorial

Project details

Vectorial consists of 5 software, AI, and ML engineers. For 2 years, we have implemented advanced LLM and ML solutions for our customers. Our expertise originates from several advanced degrees in Data Science, Computer Science, and Machine Learning. Our engineers also have working experience at companies such as AWS, Oracle, Home Depot, and Fortune 50 financial corporations. Our President, Patrick Nercessian, has significant experience with evolutionary computation and machine learning. An example of he and his team's work is linked in the Links and References section.

The objective of this proposal is to apply principles inspired by Q-learning to evolutionary algorithms, specifically targeting integration within the MOSES framework. We intend to utilize the proven zero-shot learning capabilities of LLMs (as well as relying on the ability of transformers to abstract global context) to guide the evolutionary process. In this framework, LLMs will serve as dynamic utility estimators, evaluating the potential of various evolutionary actions based on the current state of the system. This integration aims to:

  • Enhance the strategic decision-making process in MOSES by leveraging the predictive power of LLMs to estimate the potential success of possible evolutionary operations.

  • Implement a more directed search strategy that prioritizes actions with higher expected utility, leading to faster convergence and discovery of superior solutions.

  • Take advantage of capabilities inherent to transformer models and especially LLMs to enable directed parameter optimization 

We believe that LLMs can serve MOSES specifically in the areas of mutation and selection - both in identifying probabilistically beneficial populations, high impact mutations, and evaluating the fitness during the selection and culling process. Use of already existing techniques in LLM application such as top-k sampling and temperature can be used to exercise stochasticism within the evolutionary process to prevent typical gradient-based optimization topology issues. 

Our intention is to investigate the viability of LLMs applied in this capacity by comparing various configurations of MOSES (control, LLM-based population selection, using different LLMs, etc). After developing the underlying framework for the integration, we will generate our set of candidates using open-source LLMs chosen via literature review. We will compare them to non-integrated MOSES and each other across a series of evolutionary benchmark tasks. Resource requirements, resulting function complexity (of the MOSES output), and of course, fitness. Benchmarks would surround classification tasks, regression analysis, and other forms of functions that are commonly approximated in evolutionary computing and machine learning. 

This project will require a series of bespoke components in order to support the experimental desires. This includes a framework by which LLMs can be provided state spaces and state action pairs in order to estimate the utility of the functions, as well as an integration into the current fitness evaluation implemented within the MOSES architecture. The architectures will be deployed and hosted using cloud resources and managed by our team. If budgetary constraints allow, and initial results are favorable, we would also like to explore closed-source performance (OpenAI, Anthropic, etc) on our benchmark set by building API integrations for the LLM components. 

The project will follow these steps:

  1. Develop a mechanism to represent the current state of evolutionary processes and possible actions to LLMs.

  2. Implement a system for the LLM to provide utility estimates for each possible action.

  3. Simulate decision-making in evolutionary algorithms based on utility estimations provided by LLMs.

  4. Evaluate the effectiveness of this integration in guiding MOSES to achieve better optimization results.

Open Source Licensing

MIT - Massachusetts Institute of Technology License

Proposal Video

Not Avaliable Yet

Check back later during the Feedback & Selection period for the RFP that is proposal is applied to.

Group Expert Rating (Final)

Overall

5.0

  • Feasibility 4.7
  • Desirabilty 4.7
  • Usefulness 4.7

New reviews and ratings are disabled for Awarded Projects

Overall Community

4.7

from 3 reviews
  • 5
    2
  • 4
    1
  • 3
    0
  • 2
    0
  • 1
    0

Feasibility

4.7

from 3 reviews

Viability

4.7

from 3 reviews

Desirabilty

4.7

from 3 reviews

Usefulness

0

from 3 reviews

Sort by

3 ratings
  • Expert Review 1

    Overall

    5.0

    • Compliance with RFP requirements 5.0
    • Solution details and team expertise 5.0
    • Value for money 5.0
    A solid proposal that fully addresses the RFP's scope in a believable way

    I like the focus on directed crossover and mutation here, I think this is a very solid use of LLMs, less trivial than just using the LLM as a fitness function. The team appears to have practical experience making complex ML algos work. A very credible proposal.

  • Expert Review 2

    Overall

    5.0

    • Compliance with RFP requirements 5.0
    • Solution details and team expertise 5.0
    • Value for money 5.0
    Excellent proposal, simple yet clear and to the point.

    In spite of being rather concise the methodology is quite clear and the proposal is perfectly on topic. The team looks good too and the description gives a sense of clear understanding.

  • Expert Review 3

    Overall

    4.0

    • Compliance with RFP requirements 4.0
    • Solution details and team expertise 4.0
    • Value for money 4.0

    Solid, comprehensive, and targeted proposal for using LLMs for modeling and generation within MOSES.

  • Total Milestones

    4

  • Total Budget

    $60,000 USD

  • Last Updated

    3 Feb 2025

Milestone 1 - Utility Estimation Framework Development

Status
😐 Not Started
Description

Develop the building blocks for the LLM integration with MOSES.

Deliverables

A preliminary framework for LLMs to perform utility evaluation of variation operators. Design and integration of the data structure for input to the LLM (current evolutionary states and actions). Simulated initial utility estimate outputs for these actions.

Budget

$20,000 USD

Success Criterion

We will have succeeded here if we have a framework which can take in a data structure representing the state of an individual and possible action(s) and output an esimated utility for the action(s).

Link URL

Milestone 2 - Expand Framework and Integrate with MOSES

Status
😐 Not Started
Description

A more cohesive framework which will allow relatively easy setup (for a new problem domain) to run an evolutionary algorithm using LLMs to direct favorable variation operators.

Deliverables

An implemented modular architecture for integrating utility estimation. An implemented state-action representation for generational modeling & fitness estimation. An LLM layer for fitness estimation (directed mutation/crossover) Integration of above deliverables into MOSES, and simulation of initial search process.

Budget

$15,000 USD

Success Criterion

We will have succeeded here if a typical software engineer could use our framework to run MOSES evolutionary algorithms with LLM-directed variation for a new problem domain.

Link URL

Milestone 3 - Experiments and Iterative Decision-Making

Status
😐 Not Started
Description

Run initial experiments, and utilize the results to make decisions on chosen LLMs, prompts, and other hyper-parameters or design choices. Iterate on the framework repeatedly and improve its performance based on these findings.

Deliverables

Evaluate prospective LLMs via empirical testing on utility estimation tasks (or related tasks) and select a production set of LLMs based on their performance. Integrate the selected LLMs with MOSES and continuously tune the prompts for each model. Run initial evolutionary experiments to assess the impact of LLM-informed decisions.

Budget

$12,500 USD

Success Criterion

We will have succeeded here by getting initial results from our new framework, having made decisions on tweaks to the framework based on preliminary results, and have repeated these steps multiple times.

Link URL

Milestone 4 - Finalize Findings and Documentation

Status
😐 Not Started
Description

Consolidate results of our experiments by comparing results based on: A) differences between the runs B) similar runs executed via standalone MOSES usage

Deliverables

Analyze the data from experiments to identify successful strategies and common patterns. Compare the performance of different LLM-enhanced MOSES configurations (e.g., Moses-GPT-4, Moses-Claude-Sonnet-3.5, Moses-Llama3-70B) against traditional MOSES implementations. Compile documentation regarding implementation and experimental methods. Prepare findings for publication and further research. Document and release the enhanced MOSES framework with comprehensive instructions.

Budget

$12,500 USD

Success Criterion

We will have succeeded here by having a final report documenting our journey and findings as it relates to framework-internal decisions and to baseline MOSES attempts.

Link URL

Join the Discussion (0)

Expert Ratings

Reviews & Ratings

Group Expert Rating (Final)

Overall

5.0

  • Feasibility 4.7
  • Desirabilty 4.7
  • Usefulness 4.7

New reviews and ratings are disabled for Awarded Projects

  • Expert Review 1

    Overall

    5.0

    • Compliance with RFP requirements 5.0
    • Solution details and team expertise 5.0
    • Value for money 5.0
    A solid proposal that fully addresses the RFP's scope in a believable way

    I like the focus on directed crossover and mutation here, I think this is a very solid use of LLMs, less trivial than just using the LLM as a fitness function. The team appears to have practical experience making complex ML algos work. A very credible proposal.

  • Expert Review 2

    Overall

    5.0

    • Compliance with RFP requirements 5.0
    • Solution details and team expertise 5.0
    • Value for money 5.0
    Excellent proposal, simple yet clear and to the point.

    In spite of being rather concise the methodology is quite clear and the proposal is perfectly on topic. The team looks good too and the description gives a sense of clear understanding.

  • Expert Review 3

    Overall

    4.0

    • Compliance with RFP requirements 4.0
    • Solution details and team expertise 4.0
    • Value for money 4.0

    Solid, comprehensive, and targeted proposal for using LLMs for modeling and generation within MOSES.

feedback_icon