Automated Peer Review of Recent Preprints in MeTTa

chevron-icon
RFP Proposals
Top
chevron-icon
project-presentation-img
Expert Rating 2.3
Nassim Dehouche
Project Owner

Automated Peer Review of Recent Preprints in MeTTa

Expert Rating

2.3

Overview

We propose developing an AI-powered system for automated peer review of bioRxiv preprints using MeTTa. The exponential growth of preprints in computational biology and computer science has created an urgent need for rapid evaluation mechanisms, as these fields evolve too quickly for traditional peer review cycles. Our system will leverage MeTTa's unique capabilities to provide rapid, transparent, and justified assessments of preprint quality and impact.

RFP Guidelines

Develop interesting demos in MeTTa

Complete & Awarded
  • Type SingularityNET RFP
  • Total RFP Funding $100,000 USD
  • Proposals 21
  • Awarded Projects 4
author-img
SingularityNET
Aug. 12, 2024

Create educational and/or useful demos using SingularityNET's own MeTTa programming language. This RFP aims at bringing more community adoption of MeTTa and engagement within our ecosystem, and to demonstrate and expand the utility of MeTTa. Researchers must maintain demos for a minimum of one year.

Proposal Description

Company Name (if applicable)

HaAI Labs - Preprints.io

Project details

Core Components

  1. Data Integration
  • Direct integration with bioRxiv API
  • Structured representation of papers in MeTTa's atomspace
  • Extraction of key claims, methods, and results
  1. Analysis Engine
  • MeTTa-based agents for different review aspects:
    • Methods validation
    • Results verification
    • Statistical analysis
    • Impact prediction
  • Dynamic code generation with runtime justifications
  • Integration with BioAtomspace for domain knowledge
  1. Review Generation
  • Automated synthesis of review points
  • Line-by-line logical deduction for transparency
  • Real-time interaction for clarification queries

MeTTa-Specific Features

  • Runtime justification generation for each review point
  • Dynamic updating of evaluation criteria based on field trends
  • Integration with existing AI systems through MeTTa's flexible architecture

Open Source Licensing

MIT - Massachusetts Institute of Technology License

Links and references

We are https://haai.info. Our team has successfully developed https://discover.preprints.io, a platform that uses network science to predict preprints impact and quality, in addition to our experience in publishing with https://app.preprints.io and LLM-agents (https://chat.preprints.io

This experience positions us uniquely to build an AI-agent based system using MeTTa. The proposed system will extend beyond network metrics to include content-based analysis and reasoning.

Proposal Video

Not Avaliable Yet

Check back later during the Feedback & Selection period for the RFP that is proposal is applied to.

  • Total Milestones

    3

  • Total Budget

    $20,000 USD

  • Last Updated

    2 Dec 2024

Milestone 1 - Initial Infrastructure and BioRxiv Integration

Description

Develop core MeTTa infrastructure for preprint processing and integrate with BioRxiv API. Set up basic atomspace structure for paper representation.

Deliverables

Working BioRxiv API integration Basic MeTTa infrastructure code Paper representation schema

Budget

$6,999 USD

Success Criterion

Success Criteria: Successfully import and represent 100 recent bioRxiv papers in MeTTa

Milestone 2 - AI Agent Development and Knowledge Integration

Description

Develop core review agents for methods validation statistical analysis and results verification.

Deliverables

Working review agents BioAtomspace integration Initial automated reviews

Budget

$6,999 USD

Success Criterion

Agents successfully analyze and provide justified reviews for 50 papers

Milestone 3 - Documentation and Community Release

Description

Validate system performance optimize processing speed and implement feedback mechanisms. Compare results with human expert reviews. Create comprehensive documentation tutorials and prepare system for community use. Deploy on Hyperon instance.

Deliverables

Optimized codebase Complete documentation Tutorial materials Deployed system

Budget

$6,002 USD

Success Criterion

Successful deployment and demonstration of system to SingularityNET team

Join the Discussion (0)

Expert Ratings

Reviews & Ratings

Group Expert Rating (Final)

Overall

2.3

  • Feasibility 3.7
  • Desirabilty 2.7
  • Usefulness 2.3
  • Expert Review 1

    Overall

    3.0

    • Compliance with RFP requirements 5.0
    • Solution details and team expertise 3.0
    • Value for money 3.0
    Too vague

    Ambitious proposal but while it aligns conceptually with the RFP, the approach is vague. Needs significantly more specificity to evaluate its viability.

  • Expert Review 2

    Overall

    3.0

    • Compliance with RFP requirements 3.0
    • Solution details and team expertise 3.0
    • Value for money 3.0

  • Expert Review 3

    Overall

    1.0

    • Compliance with RFP requirements 3.0
    • Solution details and team expertise 2.0
    • Value for money 1.0
    This is just way too big and hard for the time and price involved... it would need quite advanced reasoning. We can do this in Hyperon for sure but it's not a quick simple demo at all.

feedback_icon