Scalable MeTTa Knowledge Graphs

chevron-icon
Back
project-presentation-img
Adam Vandervorst
Project Owner

Scalable MeTTa Knowledge Graphs

Funding Awarded

$120,000 USD

Expert Review
Star Filled Image Star Filled Image Star Filled Image Star Filled Image Star Filled Image 0
Community
Star Filled Image Star Filled Image Star Filled Image Star Filled Image Star Filled Image 4 (9)

Status

  • Overall Status

    ⏳ Contract Pending

  • Funding Transfered

    $0 USD

  • Max Funding Amount

    $120,000 USD

Funding Schedule

View Milestones
Milestone Release 1
$24,000 USD Pending TBD
Milestone Release 2
$55,000 USD Pending TBD
Milestone Release 3
$20,000 USD Pending TBD
Milestone Release 4
$12,000 USD Pending TBD
Milestone Release 5
$9,000 USD Pending TBD

Project AI Services

No Service Available

Overview

Use the new performant Hyperon backend, MeTTa Optimal Reduction Kernel (MORK), to host knowledge graphs core to the SingularityNET ecosystem. By using Hyperon for this task, we have the opportunity to practice dogfooding on core capabilities like importing relevant data from existing platforms and file formats, storing large knowledge bases in a compressed yet accessible form, supporting both SPARQL and MeTTa queries, and communicating to other services. Furthermore, by the nature of Hyperon's Metagraphs, we can store the code, documentation, version control, and knowledge graph data in the same representation. All functionality would be available via a rich online MeTTa editor.

Proposal Description

Our Team

We have successfully collaborated on relevant projects, contributing our expertise in AI, mathematics, UI, and security. Our experience with MeTTa is invaluable for the scalable knowledge-graph implementation. Adam, a former SingularityNET employee, developed a Hyperon backend, and Anneline used Metagraph querying in automated planning and is contributing to MeTTa examples. Helder is a leading voice in the AI and HCI space. Tim is an experienced full-stack developer and ML hacker.

View Team

Company Name (if applicable)

Qoba.ai

Our specific solution to this problem

We provide a complete and performant solution based on Hyperon's core strengths: Embedding the (Deep Funding) knowledge graphs in an Atom Space and using MeTTa for data transformations. Our contribution is twofold:
1) develop a knowledge graph platform that wraps up the MORK Hyperon implementation and,
2) extend the SingularityNET software ecosystem to accommodate the import and export, networking, and UI needs put forward by the Content Knowledge Graph RFP. 

 By reusing Hyperon's design and its implementation of many core capabilities, we can focus on scalability and an excellent user experience and avoid locking in third-party interface constraints. The value provided transfers well to individual developers and companies that want to take advantage of
1) the conversion of various data formats to graphs,
2) the interface for editing the knowledge graph, and 
3) the ability to program their own APIs to access the data.

Project details

(Please see Milestones & Budget for a low-level expansion of features and provided utility.)

The knowledge graph platform consists of a server database and a web editor.

Database Server

The server thinly wraps the Hyperon (meta)graph and exposes a flat API for submitting queries. The MeTTa queries serve many purposes: adding and removing data, retrieving subgraphs, performing complex transformations, and consulting APIs like those provided by OpenAI and the AI Marketplace.

On top of the core functionality and security, token-based authentication allows for fine-grained (tracked and potentially metered) changes to the knowledge graph. Synergizing with the retrieval of subgraphs, authorized users can quickly provide private and public views of the data themselves. MeTTa is the lingua franca for consulting other Hyperon databases, advanced AI solutions, and interlinked services. The server is hosted on DigitalOcean and includes request-level load balancing. It is also possible to run the server locally, with a stretch goal to be able to run it in the browser via WASM.

Web Portal and Editor

The web editor serves as a friendly entry point for exploring the (DeepFunding) graph and also gives power users power tools. Designed on top of event sourcing, we provide reproducible behavior with proper undo's in version control and—leveraging our extensive HCI research in input optimization and end-user programming—lay the groundwork for the direct manipulation of complex data and intuitive task composition and automation.

By relying on powerful software, we aim to minimize the complexity and maintenance costs owned by this project. We are determined to make a maintainable and future-proof cornerstone of the SNet ecosystem, with a no-install environment filled with examples and guidance to produce value.

Existing resources

On the theory side, Ben Goertzel, Linas Vepstas, Alexey Potapov, and others provided us with the tools to reason about heterogeneous (graph) data in a unified and computational fashion (see respectively  https://arxiv.org/pdf/2112.08272, https://github.com/opencog/atomspace, and https://metta-lang.dev). The graph generalizations (hypergraphs and metagraphs) are purpose-designed to represent heterogeneous graph data, meta-data, programs, and more in a unified format. Being first-class citizens, we can work with annotations as if they were regular data without the typical associated technical depth. Furthermore, by building on a simple generic data format (sequence of S-expressions), we avoid the trouble with version conflicts and opaque or hard-to-parse custom formats. The MeTTa language is built for data transformations: its core statement is (transform <pattern> <template>). While not required for prototyping, MORK Hyperon provides an industry-grade approach to handle metagraphs.

Open Source Licensing

MIT - Massachusetts Institute of Technology License

Follow our progress on Github https://github.com/Qoba-ai/MeTTa-KG and ask your questions! There we are also expanding the milestones into a technical roadmap with deadlines.

Proposal Video

DF Spotlight Day - DFR4 - Adam Vandervorst - Scalable Metta Knowledge Graphs

28 May 2024
  • Total Milestones

    5

  • Total Budget

    $120,000 USD

  • Last Updated

    24 Jun 2024

Milestone 1 - Generic Knowledge Graph Platform

Status
😐 Not Started
Description

KG Translation: The foundation for working with Knowledge Graphs in MeTTa. Web Portal Authentication and Data Sharing: Secure and GDPR-compliant providing and retracting data access. Web Editing: Basic manipulation of the data and metadata anywhere.

Deliverables

- Translator of JSON-LD NT and N3 and CSV (edge list) to and from MeTTa. - A web portal with token-based authentication and elementary data viewing. A backend supporting token-based permission to view data. Management to provide and retract access. - Search for specific objects and predicates. Add remove and change data and metadata. Bulk rename. Safe and restore results.

Budget

$24,000 USD

Link URL

Milestone 2 - Advanced Data Management and Control

Status
😐 Not Started
Description

KG Examples: Documentation and examples for users. Web Querying: Use a subset of SPARQL to query the knowledge graph from the web. Report on the query semantics performance and limitations. Data Management: Cleaning and enriching data and exposing views to different users. Reproducible event logging undo and redo and viewing access and modifications.

Deliverables

- End-to-end example of importing a knowledge graph in a standard format filtering and editing it in MeTTa and exposing a protected query view to a group of users. - Import export and viewing of meta-information from the Web Editor. - Support scalable SPARQL queries on real-world data.

Budget

$55,000 USD

Link URL

Milestone 3 - Real World Use and Insights

Status
😐 Not Started
Description

DeepFunding: Start live testing. Knowledge Transfer: Final documentation. Engaging with the community. Hosting: Setup.

Deliverables

- Load verify process and expose real-world DeepFunding data. - Support application teams collaborate on ontology. - Document full functionality address user questions and clarify examples. - Server has been set up with a publicly available demo.

Budget

$20,000 USD

Link URL

Milestone 4 - Integration with the AI Marketplace

Status
😐 Not Started
Description

Integration with SingularityNET's AI platform as a 'knowledge node'.

Deliverables

- Expose data views to the AI platform. - Expose queries as an on-demand service.

Budget

$12,000 USD

Link URL

Milestone 5 - Maintenance and guidance

Status
😐 Not Started
Description

Maintenance plan for ongoing support and updates of the Knowledge Graph for 1 year. Hosting services. Technical support available to expand the KG’s ontology and/or support the integration of new data sources at 95 USD/h.

Deliverables

- Mattermost customer support to Application teams. - Fixing of any bugs or gaps in the technical functioning of the KG and APIs. - Hosting the servers for the DeepFunding data.

Budget

$9,000 USD

Link URL

Join the Discussion (3)

Sort by

3 Comments
  • 1
    commentator-avatar
    CLEMENT
    Jun 1, 2024 | 4:01 PM

    Hey Adam. This is indeed a masterpiece. I wan to ask. I guess there are possible concerns associated with this project. The few I have have identified include challenges related to data privacy, security, and governance. What mitigation strategies have you put in place to ensure this ?

    • 1
      commentator-avatar
      Adam Vandervorst
      Jun 3, 2024 | 8:18 PM

      Hi Clement, thanks for your question! There are three parts to this: Logical and practical side: Managing what namespaces data lives in is core to this proposal and Hyperon MORK. Given a namespace private to Bob where he makes local changes to DeepFunding data DeepFunding.ai development (team "Bob Johnson"), and a target public-facing namespace Bob deploys to for the development of his AI app MyCo.ai serve is as simple as (transform (DeepFunding.ai development (team "Bob Johnson") $changes) (MyCo.ai serve $changes)) in MeTTa. Computer security side: Access to a namespace provides access to everything within that namespace. Access is granted and revoked at this granularity and builds on JSON Web Tokens. Governance and social side: This proposal aims to provide the substrate for any governance strategy and the nature of the token holder (a company, individual, or team). Furthermore, while the RFP assumes centralized hosting, we envision decentralization of the data at the Hyperon interpreter level.

      • 0
        commentator-avatar
        CLEMENT
        Jun 3, 2024 | 8:56 PM

        Thanks for the clarity. Kind regards ! Also, you are also welcomed to make comments on our team proposal as well https://deepfunding.ai/proposal/4757/  - AI4M (Enhancing Malaria Predictability using AI) https://deepfunding.ai/proposal/biotek-nexus-next-gen-biodiversity-conservation/  - BIOTEK NEXUS (Blockchain Biodiversity Conservation)

Reviews & Rating

New reviews and ratings are disabled for Awarded Projects

Sort by

9 ratings
  • 2
    user-icon
    Max1524
    Jun 10, 2024 | 7:18 AM

    Overall

    5

    • Feasibility 5
    • Viability 4
    • Desirabilty 5
    • Usefulness 4
    Feasibility & desirability are given top priority

    Adam Vandervorst and Anneline Daggelinckx have fully disclosed their identities and are expected by me to be fully capable and have enough intelligence to successfully implement this complex proposal. The full presentation of technical elements in 5 milestones speaks for itself. I most appreciate the feasibility and desire of the two talented members.

  • 5
    user-icon
    Robert Haas
    Jun 6, 2024 | 9:28 PM

    Overall

    5

    • Feasibility 5
    • Viability 5
    • Desirabilty 5
    • Usefulness 5
    Efficient knowledge graphs directly in Hyperon

    Regarding desirability and usefulness: The strength of this proposal in comparison to all others in this category is that it uses (and extends!) OpenCog Hyperon as a knowledge graph engine, rather than relying on external solutions such as semantic web technologies or Neo4j. This  paves the way for future projects to also use Hyperon for scalable knowledge representation and querying. The value of this proposal therefore goes beyond the mere RFP requirements. It contributes to the OpenCog ecosystem in a foundational way that will help in the adoption of Hyperon as flexible knowledge graph store and MeTTa as an advanced query language.

    Regarding feasibility and viability: Adam and his team are capable and experienced researchers and developers, as demonstrated by their prior successful work on Hyperon implementations, MeTTa language design and creation of MeTTa examples, which can be tracked both on GitHub and in SNET discussion groups. The project milestones seem well designed and I have full trust in their ability to deliver them as specified.

  • 3
    user-icon
    HenriqC
    Jun 10, 2024 | 11:19 AM

    Overall

    5

    • Feasibility 5
    • Viability 5
    • Desirabilty 5
    • Usefulness 5
    A powerful novel approach from a pioneering team

    It is beneficial to have the data in the natural format for OpenCog Hyperon instead of some other structure. I would bet this as the solution that has the highest optionality going forward and thus being the most solid foundation to expand and build on top of.

    On the other hand, the proposal is much less about immediate follow-ups and applications. Of course, it meets the RFP’s requirements that enable practical integrations and utilization by other entities.

    It is clear that the technical complexity of the system as a whole is high but there could not be much more qualified team working on it. I was pretty impressed by Adam’s presentations already in the AGI 2021 conference and happily entrust this project to his management.

     

  • 1
    user-icon
    Ayo OluAyoola
    Jun 9, 2024 | 1:11 PM

    Overall

    4

    • Feasibility 4
    • Viability 4
    • Desirabilty 4
    • Usefulness 4
    Knowledge Graphs for All Solutions

    Desirability
    The project to use Hyperon's MORK backend for hosting knowledge graphs is highly desirable within the SingularityNET ecosystem. It offers a unified solution for storing and managing large knowledge bases, supporting both SPARQL and MeTTa queries, which are crucial for advanced data manipulation and retrieval. The rich online MeTTa editor enhances user experience by providing a comprehensive interface for interacting with knowledge graphs. This project aligns well with the ecosystem's goals, promoting interoperability and efficient data management.

    Usefulness
    The usefulness of this project is significant, as it facilitates the integration of Hyperon's metagraphs into SingularityNET, enabling seamless data transformation and communication between services. By embedding knowledge graphs in Atom Space and using MeTTa for data transformations, the project enhances the functionality and scalability of the ecosystem. The ability to convert various data formats to graphs, edit knowledge graphs through an intuitive interface, and program custom APIs ensures that developers and companies can leverage this platform for diverse applications. This project ultimately supports more efficient and effective data management and utilization.

    Feasibility
    The feasibility of the project is supported by the strong expertise and experience of the Qoba.ai team, which includes AI researchers, mathematicians, UI specialists, and full-stack developers. The team's prior work with MeTTa and Hyperon, along with successful collaborations on related projects, indicates a solid foundation for implementing the proposed solution. While technical challenges exist, such as ensuring security and scalability, the project's reliance on existing robust frameworks and tools like Hyperon and MeTTa mitigates many potential risks. The project's detailed plan for data augmentation, training, and deployment further enhances its feasibility.

    Viability
    The viability of the project is promising, given its potential to become a cornerstone of the SingularityNET ecosystem. By offering a performant and scalable knowledge graph platform, the project addresses a critical need within the ecosystem and promotes the broader adoption of AI and knowledge graph technologies. The integration with existing infrastructure and the focus on providing a user-friendly experience support long-term sustainability and growth. However, the project will need to ensure continuous funding, effective marketing, and user adoption to achieve its full potential.

    Summary
    This Hyperon-based solution is all about providing a robust solution for managing and querying large knowledge bases within the SingularityNET ecosystem. Its feasibility is backed by the skilled Qoba.ai team and their experience with the core technologies involved. The project's viability is strong, with the potential to significantly enhance the ecosystem's capabilities and user experience. Ensuring continuous support and adoption will be key to its success. Overall, this project promises to be a valuable addition to the SingularityNET platform, promoting efficient data management and interoperability.

     

  • 3
    user-icon
    CLEMENT
    Jun 1, 2024 | 4:08 PM

    Overall

    4

    • Feasibility 4
    • Viability 4
    • Desirabilty 4
    • Usefulness 4
    This can enable comprehensive version control

    I find it interesting that this project is keen on utilizing Hyperon's capabilities. In doing this, the project aims to enhance the efficiency and accessibility of knowledge graphs core to SingularityNET, facilitating seamless integration with existing platforms and file formats, supporting diverse query methods, and enabling comprehensive version control and documentation within a unified representation.

    Additionally,within the SNET AI Marketplace, Scalable MeTTa Knowledge Graphs offer a robust infrastructure for hosting and querying knowledge relevant to AI applications. This infrastructure supports the diverse needs of AI developers and researchers and also fosters collaboration. 

    user-icon
    Adam Vandervorst
    Jun 3, 2024 | 8:20 PM
    Project Owner

    Hi Clement, I can speak for our whole team when I say that we are excited about the comprehensive version control this proposal provides. Version control and access control are critical to a collaborative platform, and by re-using Hyperon's unified representation and data "re-exporting" transformation capabilities, it will be a breeze to use.

  • 2
    user-icon
    Tu Nguyen
    Jun 3, 2024 | 3:37 AM

    Overall

    4

    • Feasibility 4
    • Viability 4
    • Desirabilty 4
    • Usefulness 4
    Scalable MeTTa Knowledge Graphs

    Adam is a former employee of SingularityNET, I believe he has the qualifications and skills to complete this proposal. They have teams that demonstrate high levels of trust.
    The total budget for this Pool is 120,000 USD, but the budget of this proposal is also 120,000 USD. I think the proposal's budget is a bit high. I need to make it clear that this is not against the rules, it's just my personal opinion.

    user-icon
    Adam Vandervorst
    Jun 3, 2024 | 8:25 PM
    Project Owner

    Hi Tu Nguyen, thank you for your trust in our team's ability to deliver. We believe the RFP's budget is well-placed in this part of the ecosystem, and we strive to deliver a correspondingly complete and robust solution. Note that the RFP has a separate pool, and only one project is being awarded. Moreover, we believe that our "dogfooding" approach of fulfilling the RFP's goals by leveraging and expanding SNET's own MeTTa/Hyperon-based tech stack is unique in its ability to provide compounding value across the ecosystem. 

  • 1
    user-icon
    digiRuds
    Jun 9, 2024 | 8:42 PM

    Overall

    4

    • Feasibility 4
    • Viability 4
    • Desirabilty 4
    • Usefulness 4
    knowledge graph of the future

    The proposal is well-structured and comprehensive, addressing critical aspects of developing a robust knowledge graph platform using Hyperon and MeTTa. The team’s expertise aligns well with the project requirements, ensuring a high likelihood of successful implementation. The deliverables for each milestone are clear and achievable within the specified budgets, providing a solid roadmap for enhancing the SingularityNET ecosystem's data management capabilities.
    hoping that Github which looks interesting can be build upon 

  • -4
    user-icon
    Joseph Gastoni
    May 22, 2024 | 9:25 AM

    Overall

    3

    • Feasibility 3
    • Viability 3
    • Desirabilty 3
    • Usefulness 4
    the Hyperon backend for hosting knowledge graphs

    This proposal outlines using the Hyperon backend with MeTTa for hosting knowledge graphs in the SingularityNET ecosystem. Here's a breakdown of its strengths and weaknesses:

    Feasibility:

    • High: The proposal leverages existing SingularityNET technology (Hyperon, MeTTa) with a focus on integration and user experience.
      • Strengths: Builds on established technologies and focuses on scalability and user-friendliness.
      • Weaknesses: Limited details on potential challenges in integrating Hyperon with existing knowledge graph infrastructure.

    Viability:

    • Moderate: Success depends on the adoption by developers and the overall growth of the SingularityNET ecosystem.
      • Strengths: The proposal offers a potentially valuable platform for managing and accessing knowledge graphs within SingularityNET.
      • Weaknesses: The proposal lacks details on the target user base and the competitive landscape for knowledge graph solutions.

    Desirability:

    • Moderate: For developers seeking a user-friendly platform for knowledge graph management within SingularityNET, this could be desirable.
      • Strengths: The proposal offers a familiar environment (Hyperon, MeTTa) for interacting with knowledge graphs.
      • Weaknesses: The proposal needs to clearly demonstrate the advantages of this approach compared to existing SingularityNET tools for knowledge graph management.

    Usefulness:

    • Moderate-High: The project has the potential to improve the usability and accessibility of knowledge graphs within SingularityNET, but its impact depends on user adoption and integration with existing workflows.
      • Strengths: The proposal offers a scalable and user-friendly platform for storing, editing, and querying knowledge graphs.
      • Weaknesses: The proposal lacks details on how the platform will integrate with existing AI services and workflows within SingularityNET.

    Overall, this Hyperon Knowledge Graph Platform has a promising approach, but focus on:

    • Integration Challenges: Providing a clearer picture of potential challenges and solutions for integrating Hyperon with existing knowledge graph infrastructure in SingularityNET.
    • Target User Base: Defining the specific user groups (developers, researchers) who would benefit most from this platform.
    • Competitive Advantage: Clearly articulating the unique value proposition of this platform compared to existing tools for knowledge graph management within SingularityNET.
    • Integration with AI Services: Explaining how the platform will integrate with other AI services and functionalities within the SingularityNET ecosystem.

    By addressing these considerations, this Hyperon Knowledge Graph Platform can increase its chances of success and become a valuable tool for developers working with knowledge graphs in SingularityNET.

    Here are some strengths of this project:

    • Leverages existing SingularityNET technology (Hyperon, MeTTa) for a familiar development environment.
    • Focuses on user-friendliness and scalability for managing and accessing knowledge graphs.
    • Offers a rich online MeTTa editor for manipulating complex data and automating tasks.

    user-icon
    Adam Vandervorst
    Jun 3, 2024 | 8:46 PM
    Project Owner

    With our proposal, we can take the DeepFunding comments as they come in, cross-correlate them with other responses from a particular user, and integrate AI detectors like https://www.scribbr.com/ai-detector/ and https://gptzero.me/. This solution can flag users like Joseph Gastoni who spam AI-generated responses on the platform, as well as provide more nuanced insights into the community. 

  • -3
    user-icon
    JeyGarg23
    Jun 3, 2024 | 7:17 PM

    Overall

    2

    • Feasibility 2
    • Viability 1
    • Desirabilty 1
    • Usefulness 3
    Bla bla bla

    I was expecting a bit more from the presentation. What I heard was bla-bla-bla, I'm a former singularity employee, this is why I know best.

    This ruined the whole proposal for me, but overal planning is good

    user-icon
    Adam Vandervorst
    Jun 3, 2024 | 8:46 PM
    Project Owner

    Hi! I understand that you have concerns about the spotlight presentation. My coverage of our proposal was abstract and did not go into technical depth or paint the full context. Please engage in the discussion section if we can shed light on parts you especially missed. 

Summary

Overall Community

4

from 9 reviews
  • 5
    3
  • 4
    4
  • 3
    1
  • 2
    1
  • 1
    0

Feasibility

4

from 9 reviews

Viability

3.8

from 9 reviews

Desirabilty

3.9

from 9 reviews

Usefulness

4.1

from 9 reviews

Get Involved

Contribute your talents by joining your dream team and project. Visit the job board at Freelance DAO for opportunites today!

View Job Board