Community-Driven Knowledge (graphing) Platform

chevron-icon
Back
project-presentation-img
Presentation
Expert Review🌟
Daniel Ospina
Project Owner

Community-Driven Knowledge (graphing) Platform

Funding Requested

$110,000 USD

Expert Review
Star Filled Image Star Filled Image Star Filled Image Star Filled Image Star Filled Image 5
Community
Star Filled Image Star Filled Image Star Filled Image Star Filled Image Star Filled Image 4.5 (8)

Overview

Our proposal develops a foundational Knowledge Graph (KG) for the SingularityNET community to structure its diverse data resources. This KG will enhance data accessibility for both human and machine processes. Our solution includes a robust Knowledge Platform with data ingestion, automated data structuring (powered by open-source LLMs), and tools to tag, filter, and validate structured knowledge. Our platform design is guided by leading practices in KG creation, as well as by a goal to represent the perspectives of all stakeholders. In this way, we ensure data integrity and accuracy; while also providing a high degree of scalability, performance, and security.

Proposal Description

Our Team

Our team has deep expertise in AI, data science, network analysis, and building applications to empower Web3 communities with their data. Additional background in AI and neuroscience enriches the project's foundation with cutting-edge research in computational neurology. We have focused for years on complex systems, graph theory, and data reliability and governance.

View Team

Company Name (if applicable)

TogetherCrew

Our specific solution to this problem

  • A Knowledge Platform (the Platform) designed to revolutionize the way organizations harness collective knowledge for innovation and discovery. The Platform is structured as a user-driven system; wherein routines for data ingestion and translation turn raw and semi-structured data into a Knowledge Graph (KG).

  • The KG serves as the cornerstone of our proposed solution. The very structure of the KG maximizes the value of the community’s data by interlinking data into webs of shared meaning: data structure grounds User Applications (UA) in the dynamic co-production of a reference database.

  • Feeding the Knowledge Graph is the Data Translation (DT) sub-system: facilitates the extraction and transformation of unstructured data, from the Data Ingestion (DI) pipeline, into structured knowledge. The DT sub-system also contains tools to validate structured knowledge, ensuring data integrity and accuracy through interactive user-to-data visual analytics.

  • End User Applications (UA) built atop the KG, such as the existing Hivemind FAQ Bot, or future tools for AI-enabled proposal revision and review, can offer tailored solutions to address specific SingularityNet needs, further enhancing the Platform’s value proposition.

  • architecture is designed for scalability, performance, and security, leveraging industry-standard technologies and best practices to ensure the integrity and confidentiality of user data.

  • Knowledge Platform

    • Knowledge Graph

    • Data Ingestion

    • Data Translation

    • User Applications

Project details

See the comprehensive proposal including technical specifications here: https://docs.google.com/document/d/1QaYe5zAUkVPN8gnSEf09w4nUQ0IWqDUOJJPL9IEytO4/edit?usp=sharing

TDLR:

Solution Overview

Our solution consists of four components: a reference Knowledge Graph (KG), Data Ingestion (DI) pipelines, elaboratory Data Translation (DT) co-routines, and end-user applications. These components collectively create a  scalable framework for the SingularityNET ecosystem.

Rationale

As a design goal we believe that the reference KG should look to be as complete a representation of community knowledge, and user outlook, as the network state may sustain. That is, the KG should represent the complete epistemic range of ideas held by the community and its user base. 

Our technical specifications leverages “The FAIR Guiding Principles for [Scientific] Data Management and Stewardship.”1 Standing for Findable, Accessible, Interoperable and Reusable, the FAIR principles put specific emphasis on enhancing the ability of machines to automatically find and use data, while also supporting that data’s reuse by individuals. Specifically, we adopt Global Open FAIR’s developer framework for creating “FAIR Data Points (FDP).” The FDP framework articulates KG design specifications for long-term machine-actionable data management. 

To demonstrate the utility of our solution, our proposal develops sample machine-assisted data ingestion and visualization tools to assist users in the process of adding to, enriching, and validating the reference KG. 

Knowledge Graph

  • Graph Database:

    • Stores KG data using RDF (Resource Description Framework) standard. 

    • Structures data as logical triples (subject -> predicate -> object). 

    • Data serialized in formats like JSON-LD, turtle, RDFa, and RDF/XML. 

  • Schema:

    • Utilizes RDF Schema (RDFS) to formalize common terms. 

    • Supports bespoke vocabularies for diverse use cases. 

    • Requires validation against Shapes Constraint Language (SHACL) for user interface building, code generation, and data integration. 

  • Interface:

    • API: Grants and revokes access, supports REST HATEOAS guidelines. 

    • Data Processing Methods: Includes methods for raw data ingestion and translation into KG format. 

    • Knowledge Validation Tools: Offers modeling and visualization tools for interpreting high-dimensional data. 

    • Web Interface: Data processing and data validation tools are made available to users via a friendly web interface.

  • Support Plan:

    • Includes documentation, a sandbox environment, and support for database maintenance. 

  • Knowledge Graph

    • Graph Database

    • Schema

    • Support

    • Interface

    • Resource Description Format (RDF)

    • Data ingestion & translation methods

    • Documentation

    • Web interface

    • Backups & monitoring

    • 1-on-1 support

    • Upgrades & patches

    • RDF Schema (RDFS)

    • Knowledge validation tools

    • {subject -> predicate-> object}

    • API

    • User contributed schemata

Figure 2: The knowledge graph, developed over four parts: a knowledge database, a database schema, user interface methods, and a support plan. 

Data Ingestion

  • Builds pipelines to introduce knowledge into the KG. 

  • Supports popular data sources (Discord, Discourse, Telegram, Github, MediaWiki, Notion, Google Drive). 

  • Extracts and inserts semi-structured data from Deep Funding into the KG. 

  • Incorporates a Knowledge Generator Engine to create a multi-layer KG with social and informational connections. 

  • Allows any data provider to create ingestion pipelines using access tokens from the API. 

Data Translation

  • Infers knowledge from unstructured data.

  • Transforms data into structured formats suitable for the KG.

  • Improves data quality in the KG through Object-relational Mappings (ORMs). 

  • Provides an evolving library of knowledge translation co-routines for users to customize. 

  • Offers tools for both manual and machine-assisted (e.g. (Large Language Model based) data translation. 

  • Supports data translation onto multiple KG scales, especially coarse-grained data tags, and also fine-grained semantic descriptions of data contents.

  • Retains metadata regarding data provenance to facilitate knowledge filtration.

Knowledge Validation Library

  • Offers recursive feedback tools for users to review and improve KG content. 

  • Feedback presented syntactically (e.g., through LLM queries), and also visually (e.g., through graph embedding).

  • User suggestions continuously improve what feedback tools developers offer.

  • New feedback tools added to an expanding library.

End-User Applications

  • Leverage KG for specific purposes like Hivemind (FAQ Bot), Proposal Comparison, Collaboration Gaps Analysis, Talent Matchmaking, and Knowledge Gap Analysis. 

  • Provide a 360 view of knowledge within the ecosystem. 

  • Builders can create access tokens for the KG and develop applications using the knowledge assets, such as Hivemind and tools for Proposal Comparison and Collaboration Gaps Analysis.

 

Existing resources

We leverage our existing interactions Knowledge Graph and data ingestion pipelines including Discord, Discourse (forum), Telegram, Wikimedia, Github, Google Drive, Twitter, and Notion. This provides us with a wealth of data to quickly test the different components of the design.
We count on robust DevOps infrastructure, already serving many Web3 communities including Optimism, PocketNetwork, Celo, Aave, and more. Thus ensuring efficient operation and significantly improving the cost-for-money of this proposal.
Finally, we leverage Hivemind (already funded by SingulairtyNET) as a test User Application to ensure the designs of the Knowledge Graph serve real user needs from the start. And we're actively collaborating with others interested in developing other User Applications.

Open Source Licensing

GNU GPL - GNU General Public License

We use a standard GNU 3.0 license

Proposal Video

DF Spotlight Day - DFR4 - Daniel Ospina - Community-Driven Knowledge (graphing) Platform

3 June 2024
  • Total Milestones

    8

  • Total Budget

    $110,000 USD

  • Last Updated

    3 Jun 2024

Milestone 1 - Knowledge Graph Documentation

Description

Detailed conceptual and technical plan for the project with in-depth documentation and technical choices backed by research

Deliverables

Knowledge Graph Documentation Knowledge Graph API Documentation Knowledge Generator Engine Documentation Knowledge Validation Engine Documentation

Budget

$6,000 USD

Milestone 2 - Deep Funding Data Pipeline Documentation

Description

Align with DeepFunding team to map the requirements of the DeepFunding voting platform data pipeline and produce detailed documentation

Deliverables

Technical agreement on data exchange with the Deep Funding service Deep Funding Data Pipeline Documentation

Budget

$1,500 USD

Milestone 3 - Knowledge Graph & Infrastructure

Description

Creation of the main knowledge graph and covering cost for a year of the infrastructure costs

Deliverables

Build Knowledge Graph Controls Deploy production Knowledge Graph Sandbox docker environment

Budget

$29,500 USD

Milestone 4 - Knowledge Graph API

Description

Development of the API for end-user applications

Deliverables

Build Knowledge Graph API Deploy Knowledge Graph API Sandbox docker environment

Budget

$6,000 USD

Milestone 5 - Knowledge Generator Engine

Description

creation of the engine for structuring data using an LLM

Deliverables

Build Knowledge Generator Engine Deploy Knowledge Generator Engine

Budget

$24,900 USD

Milestone 6 - Knowledge Validation Engine

Description

Develop the system for users to validate and verify the translation from unstructured data to structured data

Deliverables

Build Knowledge Validation Engine Deploy Knowledge Validation Engine

Budget

$24,200 USD

Milestone 7 - Deep Funding Data Pipeline

Description

Creation of the pipeline to take data from the deep funding platform API and move it into the knowledge graph

Deliverables

Build Deep Funding Data Pipeline Deploy Deep Funding Data Pipeline

Budget

$11,200 USD

Milestone 8 - Support and Hot Fixes

Description

Payment to fund the provision of support and hot fixes

Deliverables

Mattermost support setup technical support setup and capacity for hotfixes.

Budget

$6,700 USD

Join the Discussion (3)

Sort by

3 Comments
  • 1
    commentator-avatar
    CLEMENT
    Jun 1, 2024 | 4:59 PM

    Hey Daniel. Great initiative. I would like to ask. Are you keen on addressing potential biases in the automated data structuring process within this RFR ?. I believe this is crucial for maintaining fairness and accuracy in the representation of knowledge within the SNET platform. Thanks

    • 1
      commentator-avatar
      Daniel Ospina
      Jun 3, 2024 | 6:37 AM

      Thanks for the question Clement. We have added a mechanism for community members to be able to review the "translation". This way biases can be spotted and addressed in an ongoing manner.We're also very open to considering other ideas and complimentary suggestions.

      • 0
        commentator-avatar
        CLEMENT
        Jun 3, 2024 | 9:14 AM

        Thanks for your comments. They are helpful and have provided me with much needed clarity. Kind regards !

Expert Review

Overall

5

user-icon
  • Feasibility 4
  • Viability 4
  • Desirabilty 4
  • Usefulness 5
Clear path to a real product and community utility

Strengths: Clear path to a real product with relevant community utility and potential use for other teams. Like the focus on data sources and flows with consideration for risks with a pre-existing working product.

Weaknesses/unclarities: Security and performance approach not clearly addressed. Do they have DevOps? LLM usage risk/concerns with increased potential system maintenance costs (need to understand costs vs. budget)."

Sort by

8 ratings
  • 0
    user-icon
    HenriqC
    Jun 10, 2024 | 11:15 AM

    Overall

    5

    • Feasibility 5
    • Viability 5
    • Desirabilty 5
    • Usefulness 5
    Rich proposal that serves well the RFP’s goals

    I really like and appreciate that a highly trusted and merited team takes time and effort to provide such a comprehensive and detailed, even educational, explanation of what they are going to build and how they are going to do it. I would definitely take the bet even though I’m trying to think of what to say about the 6 month deadline.

    As this RFP is primarily about community knowledge, it is obvious to me that TogetherCrew’s history and expertise regarding social systems plays well together with the nature of the content that is to be facilitated by this knowledge graph. I can’t tell how the RDF will serve over a longer term future but at least it seems like a fit for purpose in this context and demand here today. 

     

  • 0
    user-icon
    Max1524
    Jun 10, 2024 | 7:22 AM

    Overall

    4

    • Feasibility 3
    • Viability 5
    • Desirabilty 4
    • Usefulness 5
    Human resources need clarification

    I know about Daniel Ospina's transparency, I trust his abilities, but because Daniel is working on a number of other proposals in the same DFR4, I am afraid that the proposal will be difficult to complete because of time and staff. limited (remaining member unknown). Therefore, I asked Daniel to clarify more about feasibility and human resources to give me and the community more confidence.

  • 0
    user-icon
    Duke Peter
    Jun 9, 2024 | 1:47 PM

    Overall

    5

    • Feasibility 5
    • Viability 5
    • Desirabilty 5
    • Usefulness 5
    Building a Knowledge Graph for SingularityNET

    This project has the potential to revolutionize the way the SingularityNET community harnesses collective knowledge. By making data more accessible, it could enhance both human and machine processes, leading to innovation and discovery. Specific applications that could be built using the knowledge base include tools for proposal comparison, collaboration gap analysis, talent matchmaking, and knowledge gap analysis. Kudos to the team.

  • 0
    user-icon
    Onize Olie
    Jun 8, 2024 | 9:45 PM

    Overall

    5

    • Feasibility 4
    • Viability 5
    • Desirabilty 5
    • Usefulness 4
    Transformative Knowledge Graph for SingularityNET

    Reviewing this proposal, I am thoroughly impressed by the team's comprehensive and well-structured approach to creating a foundational Knowledge Graph (KG) for the SingularityNET community. The proposed Knowledge Platform, which includes data ingestion, automated data structuring powered by open-source large language models (LLMs), and tools for tagging, filtering, and validating structured knowledge, is particularly noteworthy. The emphasis on incorporating best practices in KG creation and ensuring data integrity and accuracy by representing all stakeholders’ perspectives demonstrates a deep commitment to building a robust and scalable solution.

    The team's extensive expertise in AI, data science, and complex systems significantly enhances the project's credibility. Their background in computational neurology and focus on data reliability and governance add a unique and valuable dimension to the proposal. The solution's four-component structure—Knowledge Graph (KG), Data Ingestion (DI) pipelines, Data Translation (DT) co-routines, and end-user applications—creates a scalable and efficient framework for the SingularityNET ecosystem. Each component is meticulously detailed, with thoughtful touches such as using RDF Schema (RDFS) for data representation and validation against Shapes Constraint Language (SHACL) to ensure accuracy and usability.

    I am particularly excited about the vision for end-user applications leveraging the KG, such as Hivemind (FAQ Bot), Proposal Comparison, and Talent Matchmaking, which offer tailored solutions that significantly enhance the platform's value. The proposal’s strategic use of existing resources and robust DevOps infrastructure, combined with active collaboration with other developers, demonstrates a commitment to efficiency and continuous improvement. Overall, this proposal is a promising and transformative initiative that has the potential to greatly benefit the SingularityNET community through enhanced data accessibility, scalability, and innovative applications.

  • 0
    user-icon
    Tu Nguyen
    Jun 3, 2024 | 7:29 AM

    Overall

    4

    • Feasibility 4
    • Viability 4
    • Desirabilty 4
    • Usefulness 5
    Community-Driven Knowledge Platform

    This is a good and useful suggestion. They shared a lot of information. I only have 2 comments. First, in the milestones section, they should identify the start and end times of each milestone. Second, in the project group section, there are some accounts still in pending status, they should handle this status.

  • 0
    user-icon
    CLEMENT
    Jun 1, 2024 | 5:05 PM

    Overall

    4

    • Feasibility 4
    • Viability 4
    • Desirabilty 4
    • Usefulness 4
    Will birth an automated data structuring platform

    From my own assessmentt, I believe this RFP holds tremendous potential for enhancing data accessibility and collaboration within the SingularityNET community. I also understand that by structuring diverse data resources into a cohesive Knowledge Graph (KG), your proposed RFP will enable both human and machine processes to access and utilize information more effectively. 

    I also believe this holds immense value for SNET. This will enriche the SNET ecosystem by offering a centralized hub for accessing structured knowledge resources, allowing users gain access to a wealth of structured data that can enhance the development and deployment of AI solutions. 

    Kudos to your team !

  • -2
    user-icon
    Joseph Gastoni
    May 23, 2024 | 5:11 PM

    Overall

    4

    • Feasibility 4
    • Viability 3
    • Desirabilty 3
    • Usefulness 4
    This proposal outlines the development of KG

    This proposal outlines the development of a Knowledge Graph (KG) for the SingularityNET community, aiming to improve data accessibility and foster innovation. Here's a breakdown of its strengths and weaknesses:

    Feasibility:

    • Moderate-High: The project leverages existing open-source technologies for knowledge graphs and machine learning.
      • Strengths: The proposal builds upon established concepts and can be developed in stages.
      • Weaknesses: The proposal lacks details on the complexity of data ingestion from various sources and user adoption for data contribution and validation.

    Viability:

    • Moderate: Success depends on user adoption, data contribution, and development of valuable user applications built upon the KG.
      • Strengths: The proposal addresses a need for improved data organization and accessibility within SingularityNET.
      • Weaknesses: The proposal lacks a clear strategy for user incentives to contribute data and for attracting developers to build applications on the platform.

    Desirability:

    • High (for a specific audience): For researchers, developers, and active SingularityNET community members, this could be highly desirable.
      • Strengths: The proposal caters to a need for better knowledge sharing and discovery within the SingularityNET ecosystem.
      • Weaknesses: The proposal needs to demonstrate the value proposition for a broader audience who might not be familiar with knowledge graphs.

    Usefulness:

    • High Potential: The project has the potential to significantly improve knowledge sharing and collaboration within SingularityNET, but hinges on successful development, user adoption, and building valuable applications.
      • Strengths: The proposal offers a platform for structured knowledge storage, retrieval, and analysis.
      • Weaknesses: The proposal lacks details on how the platform will ensure data quality and address potential biases within the data.

    Overall, the proposal is promising, but focus on:

    • User Adoption Strategy: Develop a clear plan for encouraging community members to contribute data and validate existing information.
    • Data Quality Management: Outline a plan for ensuring data accuracy, addressing potential biases, and managing data provenance.
    • Incentive Structure: Consider implementing incentives for data contribution and for developers to build applications on the platform.
    • Value Proposition for All Users: Clearly demonstrate the benefits of the KG for different user groups within SingularityNET.

  • 0
    user-icon
    GraceDAO
    May 19, 2024 | 10:44 AM

    Overall

    5

    • Feasibility 5
    • Viability 3
    • Desirabilty 4
    • Usefulness 5
    Novel approach to social graph interpretation

    Having a reputation graph is a part of understanding reputation. Rather than looking just at the activity of a person, this approach allows a more nuanced approach towards what it means to have a reputation. It also considers the fact that people use different communications platform. It will be interesting to try to understand the relationshipo between reputation on different chat platforms. Translating this into qualitative rather than quantitative data is always a challenge, so it will be interesting to see how the first iterations evolve and how the product can continue to provide better ways of understanding people's participation in a decentralized process.

Summary

Overall Community

4.5

from 8 reviews
  • 5
    4
  • 4
    4
  • 3
    0
  • 2
    0
  • 1
    0

Feasibility

4.3

from 8 reviews

Viability

4.3

from 8 reviews

Desirabilty

4.3

from 8 reviews

Usefulness

4.6

from 8 reviews

Get Involved

Contribute your talents by joining your dream team and project. Visit the job board at Freelance DAO for opportunites today!

View Job Board