Simulating the Quality of Community Decisions

chevron-icon
Back
project-presentation-img
Kenric Nelson
Project Owner

Simulating the Quality of Community Decisions

Funding Awarded

$50,000 USD

Expert Review
Star Filled Image Star Filled Image Star Filled Image Star Filled Image Star Filled Image 0
Community
Star Filled Image Star Filled Image Star Filled Image Star Filled Image Star Filled Image 4.7 (3)

Status

  • Overall Status

    🛠️ In Progress

  • Funding Transfered

    $24,450 USD

  • Max Funding Amount

    $50,000 USD

Funding Schedule

View Milestones
Milestone Release 1
$2,400 USD Transfer Complete 16 May 2024
Milestone Release 2
$10,970 USD Transfer Complete 14 Jun 2024
Milestone Release 3
$11,080 USD Transfer Complete 19 Jul 2024
Milestone Release 4
$11,660 USD Pending TBD
Milestone Release 5
$13,890 USD Pending TBD

Status Reports

Aug. 23, 2024

Status
🙂 Pretty good
Summary

We simulated how people vote on several voting systems, including the one currently used by sNET, 1c1v and other promising alternatives. We also considered how strategic and extreme voting impact the results. This research is critical to designing even better voting systems that optimize the aggregate voter's welfare and provide better results to the community.

Full Report

Project AI Services

No Service Available

Overview

SingularityNET is seeking to advance its high-quality decision process for its decentralized Deep Funding program. In support of this aim, Photrek proposes to develop an open-source simulator which models how the complex network of community sentiments impacts voting outcomes. The project will both lay a foundation for voting simulation capability and demonstrate an initial result. This first investigation will evaluate the claim of Plural Voting that applying a quadratic cost to the strengths of opinion assures a) truthful voting, and b) mitigation of controlling factions. A roadmap of future capabilities including simulation of collusion detection and reputation scores will be provided.

Proposal Description

How Our Project Will Contribute To The Growth Of The Decentralized AI Platform

Photrek will deliver a high-quality governance simulator to assure the long-term integrity of the Deep Funding program. Situational awareness regarding how proposed improvements to the voting, correlation detection, and reputation regards will allow designs to be tested prior to implementation. The Photrek team are international leaders in the simulation of complex community sentiment which can be used to drive understanding of the characteristics of different voting systems.

Company Name (if applicable)

Photrek

The core problem we are aiming to solve

Public treasury decision-making is an exceptionally difficult problem. Each member of a community needs to be able to express their preferences in a manner that contributes to maximizing community value. A variety of problems arise from poorly designed voting systems, including:

  1. splitting of communities into majority and minority factions;

  2. diversion of public resources toward private gains;

  3. funding of low impact projects.

SingularityNET has made a commitment to address these problems through the use of quadratic voting, graded ranking of proposals, and reputation scores; however, the current implementation does not match evidenced-based designs and thus could be improved.

Our specific solution to this problem

Photrek proposes the development of Monte Carlo simulation of agents that model the actions and interactions of multiple individuals forming opinions within a complex social community. This computational simulation foundation will be used to design and analyze voting methods for decentralized communities. The critical aspect of the complex network approach relies on the patterns of connection between their elements that are neither purely regular nor purely random, as the real-world networks of people are structured [9-10].

Photrek will accomplish the following objectives.

  1. Establish a foundation for simulating the interaction between community members that establishes clusters of sentiment regarding opinions about proposals.

  2. Simulate community members with a distribution of sentiments and AGIX holdings voting positive, negative, or abstain on sets of 2 to 10 proposals.

  3. Utilize the complex network of sentiments to simulate voters expressing their strength of opinion on projects with and without quadratic costs on the strength of opinion.

  4. Develop a proposal plan for how correlation measurements can be incorporated into the simulation for collusion mitigation.

  5. Develop a roadmap that includes incorporation of reputation scores into the simulation capability.

In order to assure the simulation results have high integrity and community alignment, Photrek will set aside $5000 for a red team review of the results.

Project details

Objective

Photrek will develop a Monte Carlo simulation [1-2] capability to determine the ability of voting systems to maximize the quality of community decisions. Photrek will compare two ‘strength of conviction’ methods for ranking the quality of proposals. Both methods will distribute voting credits based on the square-root of AGIX holdings. Quadratic or Plural Voting [3-4] requires the voter to rank their opinion on each proposal by distributing their credits. The votes applied to a proposal is the square-root of the credits applied. “No-cost Grading” applies the voter’s credits equally to all proposals and requires the voter to separately grade the proposal on a scale of 1-10. 

We seek to evaluate the claims of Plural Voting that the application of the quadratic cost to preferences incentivizes individual voters to express the truth about their convictions rather than strategically using extreme opinions. A second claim that will be evaluated is whether Plural Voting mitigates the formation of dominant factions across a series of decisions. The goal is to encourage pluralistic values within decentralized communities by assuring that both minorities with strong convictions and majorities with broad support can influence outcomes without a particular faction gaining permanent control.

Approach

Andre Vilela and a graduate student will develop statistical agent-based models to simulate community sentiments [5] about a group of proposals. The proposals will have a distribution of value (the quality returned to the community) and cost (the funding requested). Vilela utilizes majority-vote dynamics to model how gossip within a community leads to dynamic variation in community sentiment [6-10]. The Monte Carlo simulation foundation will be applied to Plural Voting and No-Cost Grading for the purpose of determining the ability of these voting methods to maximize the expected community profit (value - cost). A roadmap incorporating reputation will be provided. Vilela has successfully applied these methods to a variety of socioeconomic systems including social influence, network structures, information diffusion, and consensus dynamic. Photrek with Vilela completed contracts with IOTA on blockchain consensus and Cardano Catalyst on diversified voting [11-12]. 

Nelson and Attieh will work with the SingularityNET community to apply simulation results to a requirements roadmap. To assure an independent critique of the results, Photrek will provide $5000 for 2-3 experts selected from the SingularityNET community to provide a technical review.  Starting with our second report, the reviewers will provide an independent critique and recommendation in each report.

References

[1] Short-time Monte Carlo simulation of the majority-vote model on cubic lattices. K. P. do Nascimento, L. C. de Souza, A. J. F. de Souza, André L. M. Vilela, H. Eugene Stanley. Physica A - Statistical Mechanics and its Applications, 2021.

[2] Three-state Majority-vote Model on Small-world Networks. Bernardo J. Zubillaga, André L. M. Vilela, Minggang Wang, Ruijin Du, Gaogao Dong, H. Eugene Stanley. Scientific Reports, 2022.

[3] Quadratic Voting. Steven P. Lalley and E. Glen Weyl. Available at SSRN, 2014.

[4] Posner, Eric A, and E Glen Weyl. 2015. “Voting Squared: Quadratic Voting in Democratic Politics.” Vanderbilt Law Review 68 (2). 

[5] Effect of Strong Opinions on the Dynamics of the Majority-Vote Model. André L. M. Vilela and H. Eugene Stanley. Scientific Reports, 2018.

[6] Opinion dynamics in financial markets via random networks. Mateus F. B. Granha, André L. M. Vilela, Chao Wang, Kenric P. Nelson and H. Eugene Stanley. PNAS, 2022.

[7] A Three-state Opinion Formation Model for Financial Markets. Bernardo J. Zubillaga, André L. M. Vilela, Chao Wang, Kenric P. Nelson, H. Eugene Stanley. Physica A - Statistical Mechanics and its Applications, 2021.

[8] Majority-vote model with limited visibility: An investigation into filter bubbles. André L.M. Vilela, Luiz Felipe C. Pereira, Laércio Dias, H. Eugene Stanley, Luciano R. da Silva, Physica A - Statistical Mechanics and its Applications, 2021.

[9] Three-state Majority-Vote Model on Barabási-Albert and Cubic Networks and the Unitary Relation for Critical Exponents. André L. M. Vilela, Bernardo J. Zubillaga, Chao Wang, Minggang Wang, Ruijin Du, H. Eugene Stanley. Scientific Reports, 2020.

[10] Majority-vote model for financial markets. André L. M. Vilela, Chao Wang, Kenric P. Nelson, H. Eugene Stanley. Physica A - Statistical Mechanics and its Applications, 2019.

[11]  IOTA Majority Vote Final Report. André L.M. Vilela and Kenric P. Nelson, 2020. https://docs.google.com/presentation/d/1Iu47WIhV2AX7uP0pJxG3uW1G0R8vmggkz-l3vFz8OQM. Accessed February 4, 2024.

[12] Diversify Voting Influence Close Out Report. Photrek, LLC. Kenric P. Nelson, André L.M. Vilela, and Megan Hess, 2021. https://docs.google.com/document/d/1ibAZBDgj3cQGEcAePQ_Qjl5wJvMICLqGEU65CyskDr4/edit?usp=drive_web&ouid=106499308022823206372&usp=embed_facebook.

Open Source Licensing

GNU GPL - GNU General Public License

Additional videos

Presentations of Cardano Catalyst Diversify Voting Infuence:

DVI Video Final Report - 5 minutes

DVI Video Final Report - 50 minutes

 

Proposal Video

Simulating The Quality - Kenric Nelson

9 February 2024
  • Total Milestones

    5

  • Total Budget

    $50,000 USD

  • Last Updated

    10 Sep 2024

Milestone 1 - Contract Signing

Status
😀 Completed
Description

Funding put in reserve to address unforeseen requirements

Deliverables

Contract Signature

Budget

$2,400 USD

Milestone 2 - Prototyping of Monte Carlo Simulations

Status
😀 Completed
Description

* Literature review for informed research design. * Prototyping Plural Voting: initial design a model to simulate multi-agent interactions aiming to analyze social phenomena and voting patterns in groups. * Monte Carlo computational modeling: simulate decisive social complex phenomena. * Selection of 2-3 experts by the community to form the ‘Red Team’

Deliverables

* A comprehensive literature review to guide the research design focusing on voting dynamics and individual preferences. * An Initial prototype of a Plural Voting model designed to uncover voting patterns and facilitate expressions in interactive groups. * A Monte Carlo computational model using random number sequences to model social voting dynamics. * Selected expert reviewers to form the ‘Red Team’.

Budget

$10,970 USD

Milestone 3 - Simulation of Voting Methods

Status
😀 Completed
Description

* Quadratic Voting and No-Cost Grading implementated in computer simulations. * Comparison of the voting dynamics and outcomes between with and without quadratic costs for conviction voting. * Red team review of Photrek’s M2 deliverables.

Deliverables

* Fully implemented simulations of quadratic voting and No-cost Grading. * Complete and detailed design of the plural voting model capable of realistically simulating multi-agent interactions and social. * Comparison of Plural Voting with No-Cost Grading. * Comprehensive evaluation by red team experts of the plural voting prototype design and Monte Carlo modeling as presented in Milestone 2.

Budget

$11,080 USD

Milestone 4 - Analysis of the Voting Dynamics

Status
🧐 In Progress
Description

* Analysis of voting design modeling and simulation outputs utilizing careful statistical analysis including statistical moments (e.g. second moment kurtosis) * Provide summary of the relative performance outcomes of quadratic and no-cost grading. * Develop a proposal plan for how correlation measurements can be incorporated into the simulation for collusion mitigation. * Community Engagement: Red team review of Photrek’s M3 deliverables.

Deliverables

* Extensive data for knowledge extraction conclusions and decision-making support for the SNET community. * A comprehensive report featuring in-depth statistical analysis valuable insights and conclusions along with a detailed examination of complex system statistics. * A proposal plan for integrating correlation measurements into collusion mitigation simulations. * Comprehensive evaluation by red team experts of the quadratic voting implementation and complete plural voting design as presented in Milestone 3.

Budget

$11,660 USD

Link URL

Milestone 5 - Final Report & Roadmap

Status
😐 Not Started
Description

* Present a combination of visual analytical and statistical analyses. * Communicate key aspects of our design and research findings to the SNET community. * Offer insights and a potential roadmap for voting design improvements based on the simulation results. * Create a strategy that involves incorporating reputation scores into the simulation framework. Red team will provide independent recommendations based on the simulation results.

Deliverables

* Presentation of visual analytical and statistical research conclusions engaging the SingularityNET community with key findings and a roadmap for enhancements to SingularityNET voting platform. * Presentation of independent evaluations and recommendations from the red team.

Budget

$13,890 USD

Link URL

Join the Discussion (1)

Sort by

1 Comment
  • 0
    commentator-avatar
    HenriqC
    Feb 10, 2024 | 3:42 PM

    In my view, in the web 3 ecosystems, the real discussion about decision-making resources has barely even started. Yet the dynamic stability and evolution of these collective agents will be completely dependent on how the decision-making power is distributed among the subagents for each decision and over time. Objective metrics and fact based analysis are the only genuine guidance one can have to tackle such questions. Any error correction that can be done through this simulated ‘virtual reality’ means less cost on the actual process. I find this as a super important challenge and your proposal as an equally valuable endeavor.

Reviews & Rating

New reviews and ratings are disabled for Awarded Projects

Sort by

3 ratings
  • 0
    user-icon
    HenriqC
    Feb 10, 2024 | 3:59 PM

    Overall

    5

    • Feasibility 5
    • Viability 5
    • Desirabilty 5
    • Usefulness 5
    Foundational element of collective decision-making

    The proposal targets a highly undervalued research area that is crucial for the survival of any intelligent self-organizing system including Deep Funding and SingularityNET. For this ecosystem, it is not a must-have project quite yet but it will be at some point in the near future and it is definitely not too early to start working on it.

    These pretty straightforward simulations won’t yet produce any too complex insights. However, the most important output would be to tangibly demonstrate the importance of the topic and lay some solid foundations for future work. 

    Photrek has proven its capability to deliver projects in this domain as well as in this funding format, and there are good reasons to believe that the persons with the assigned responsibilities are able to implement the plan also from a technical viewpoint. 

  • 0
    user-icon
    Rafiatu07
    Feb 11, 2024 | 10:34 PM

    Overall

    5

    • Feasibility 4
    • Viability 4
    • Desirabilty 4
    • Usefulness 5
    Good initiative with potential for social impact

    Desirability

    The proposal addresses a critical need within SingularityNet's decentralized Deep Funding program by developing an open-source simulator to model the complex network of community sentiments and its impact on voting outcomes. The project's focus on evaluating different voting methods, such as plural voting and no-cost grading, to maximize the quality of community decisions is highly desirable. Additionally, the incorporation of Monte Carlo simulation techniques and statistical analysis adds rigor to the evaluation process. The proposal outlines clear objectives and milestones, indicating a strong commitment to advancing SingularityNet's governance capabilities. Further clarification on the specific methodologies for modeling community sentiments and implementing voting simulations would enhance the desirability of the project.

     

    Viability

    The project's viability for SingularityNet AI is evident in its potential to enhance the decision-making process within the Deep Funding program. By developing a simulation tool to evaluate different voting methods and their impact on community decisions, the project aligns with SingularityNet's mission of democratizing AI. The proposed milestones and deliverables align well with the project's objectives, demonstrating a practical approach to implementation. However, the proposal could provide more insights into the scalability of the simulation tool and its long-term impact on SingularityNet's AI ecosystem. 

     

    Feasibilty

    The proposal demonstrates feasibility through a well-structured plan and budget allocation across five milestones. Milestones such as literature review, prototyping of Monte Carlo simulations, and analysis of voting dynamics provide a clear roadmap for project execution. The team's expertise in statistical modeling and simulation, as evidenced by their previous contracts with IOTA and Cardano Catalyst, adds credibility to their ability to carry out the project successfully. However, the feasibility could be further enhanced by providing more details on the technical specifications of the simulation tool and the integration process with SingularityNet's platform. 

     

    Usefulness

    The proposal offers significant usefulness to SingularityNet AI by addressing critical issues related to decision-making and governance within the Deep Funding program. The project's focus on evaluating different voting methods, incorporating community sentiments, and providing insights into voting dynamics is invaluable for improving the quality and fairness of community decisions. The open-source nature of the simulator also allows for transparency and collaboration within the SingularityNet community. The proposal demonstrates a clear understanding of SingularityNet's objectives and presents a valuable initiative to support its growth and development.

  • 0
    user-icon
    Walter WaKa
    Feb 9, 2024 | 8:13 AM

    Overall

    4

    • Feasibility 5
    • Viability 4
    • Desirabilty 5
    • Usefulness 4
    Excellent team with direct track record

    GNU OSS license is great!

    The team is uniquely qualified to implement a set of tooling for further study of alternative voting methods. However, as part of their expertise and prior insights they may also have pre-conceived notions about the range of possible algorithmic choices, and strong preferences.

    I would prefer to relax the quadratic [square root] assumption and make it a tunable parameter rather than to hard code code it into the system.  After many dozens of individual votes and over a year of conversations at Gitcoin I find the extra cognitive load distracting, while not necessarily producing desired outcomes. Meanwhile, a desire for uncorrelated votes may paint as nafarious collusion the outcomes of natural coordination and communication requisite in any functiouning organization.

    A majority of token-based group decisions are of a binary approval, as can be seen on Snapshot.  While aggregating the scores of 1-10 is a frequent enough method, and currently in use at Deep Funding, perhaps we should not be hard code that either.

    While the averaged DF votes apparently center somewhere around 6 or 7, the frequency of the choices of 1 - 10 is somewhat similar, with incremental increases around the extremes. Yet, the instances of voting mostly 1 and only a single 10 is strongly frowned upon as 'strategic'. Not a question of a stochastic simulation, but can we look into normalizing the votes from a given reviewer toward whatever distribution is desirable, and perhaps decriminalize a choice of some voting distribution?  However, current rules institute a minimum passing score, which is not immediately compatible with a simplistic renormalization. 

    The proof of the results of a simulation is backtesting the strategy suggested and/or making predictions and measuring the outcomes. Deep Funding does not yet have quantitative measures of past proposal and grant outcomes. When variously adjusting the ways to aggregate the votes, how do we compare different sets of parameters?

    Incremental weighting adjustments, including taking a square root of some factor, will often enough leave the rankings of multiple choices largely intact. The data set of the votes and the outcomes of maybe a hundred proposals and under a hundred reviewers is unlikely to distambiguate fully the impact of a particular value of some parameter. A voting system proves its worth when the stakes are high, the decisions are complex, and the issues impactful and controversial. From my standpoint, so far the process has been polite and placid, so a study of its outcomes may not be fully informative.

    The 'square root' thinking comes from the desire to depower the 'whales' and make decisions more democratic. However, the founders or early investors often have non-trivial token positions. Some Reputation factor aims to give additional voting power to the long-term participants and perhaps positive contributors. Already highly visible and engaged, they have outsize popularity and influence. Amplifying their input even further is a strong centralizing influence, which may be directly at odds with the democratizing aim of the 'square root' adjustment.

    My apologies for having turned this review into a long exposition of personal opinions.

Summary

Overall Community

4.7

from 3 reviews
  • 5
    2
  • 4
    1
  • 3
    0
  • 2
    0
  • 1
    0

Feasibility

4.7

from 3 reviews

Viability

4.3

from 3 reviews

Desirabilty

4.7

from 3 reviews

Usefulness

4.7

from 3 reviews

Get Involved

Contribute your talents by joining your dream team and project. Visit the job board at Freelance DAO for opportunites today!

View Job Board