DFR1 – Voting Analysis Part 2 – improvement options
After analyzing the voting behavior in part 1 of this series, the next question is, “What parts do we want to improve?” And after that, “How can we best accomplish this?”.
While there is a range of things to improve, before, during, and after the voting process, this article focuses on measures that aim to make the voting process itself efficient, fair, and balanced.
Let’s wrap up the main conclusions of part one first:
Grade voting effects
The voting behavior was different per wallet. Some wallets vote only by the extremes (1 or 10), and others vote more differentiated, (all numbers in between 1 and 10)
- When simply removing the extremes the conclusion was that the overall impact was limited.
- If we bucket the votes in smaller sections, 1-5 or 1-2 the results are different but not completely transformed either.
Wallet token balance effects
There is a huge disbalance in tokens per wallet, with especially the top 3 wallets having an uncommonly large weight in the overall results
- If we would count all wallets as one vote instead of each token, the outcome would have been a very different one, with a totally new top 3 in Pool A.
- If we focus on the subset of wallets ordered by size, the results are different in any of the sections we reviewed.
Individual wallet behaviors.
There is a large subset of 38 wallets that only voted for 1 project. In 30 of these cases, this vote was a 10. I am assuming that in many of these cases the wallet owners voted in favor of a project they are somehow involved in, which would be acceptable as long as it does not take an excessive form and is sufficiently balanced by unbiased token holders. (‘Fun’ fact is that 21 of the thirty votes came from smaller wallets in support of 2 projects, which seems to imply that these voters may not have interpreted the 1 token = 1 vote rule correctly.) A bit reassuring here is also that with 2 exceptions most single-vote wallet sizes were relatively small.
There are a few large wallets that voted mainly a 1 or a 2 to all participants, and a single 10. This may be a concerned community member with a strong preference. Or a proposer with very strategic voting behavior, which is definitely something we would like to discourage!
UX related issues
Some voters voted on the proposer portal, during the official voting week, without connecting a wallet. Those votes were basically lost in the overall result. This will be improved by better labeling options in the portal and eventually by enabling wallet connections to the portal and integrating the official voting process itself. This is more or less an isolated case and therefore not further discussed in the article below.
Overall conclusion
Perhaps it’s important to add, that while we were analyzing all the different scenarios above, there were no projects that clearly would have won in all relevant scenarios. There are no ‘moral winners’ that would have benefitted in all ‘fair’ scenarios, except for the current ones. It all depends on the scenarios chosen. And for each alternative scenario, we can argue that this would also have impacted the voting behavior itself. Therefore, in our opinion, this analysis does not reduce or undermine the validity of the current results in any way and we are very happy with the elected projects. (Although, there are also numerous projects that were not awarded that we would love to see appear in our ecosystem nevertheless!)
In case you missed it in the previous article, you can get a good intuitive understanding of the voting behavior by this great interactive visual made by Robert Haas.
Preferred behavior
So with these results in place, What is the kind of behavior we would like to avoid or promote?
Strategic voting
We would prefer to see voting behavior that is based on the perceived value of the project, not primarily based on outweighing the votes of others. Especially downvoting with the objective of reducing the competition is undesired.
This means that:
- We prefer to see wallets with somewhat differentiated voting behavior. In the current configuration of 1-10:
- A few 1’s or 10’s are perfectly OK, but nine 1’s against one 10 would be undesired.
- Five 10’s and five 1’s with nothing in between would also be undesired but to a lesser extent.
Voting bias by personal involvement
It is likely that some people’s voting behavior is steered by their personal involvement in a project. Ideally, the impact of such votes is limited. Most importantly, we should avoid that proposers have the chance to significantly influence the outcomes by temporarily purchasing large quantities of tokens. If the awards are high enough and the chance of positively influencing the voting result is also realistic, proposers will have an attractive business case, even at the risk of losing some value as a consequence of transactions and price volatility.
Such calculated behavior of temporarily buying large amounts of tokens just for voting purposes is perhaps the most important pattern of all we should aim to avoid!
Extreme differences in voting weight.
When comparing the smallest 20% of wallets to the largest 20%, the largest 20% would have a voting weight of 612 times the smallest 20%!
This ratio is a bit distorted by a larger number of very small and a few very large wallets.
- If we ignore the extremes and look at the 10-30 percentile and compare this to 70-90 percentile the ratio is reduced to 37.
- If we look at the 20-40 percentile and compare this to 60-80 percentile the ratio is further reduced to 10.
While there is a good case for giving large token holders also a larger say in the outcomes, the huge differences we see in the extremes are undesired. It would be an improvement to flatten the curve, or at least drastically reduce the weights of the extreme wallet sizes (especially on the high end, due to their large overall impact).
Potential measures to take
With all this background let’s dive into the core of the matter: What tools do we have or do we need in our toolkit to get to the desired end state, where there is a good balance in voting weight per wallet and votes that are based on the merit of the projects do have the highest weights.
Measure 1: Attracting more voters
While 186 Million votes sound impressive, the reality is that there were ‘only’ 158 wallets involved. On a total number of 30,718 wallets that means there was only a small percentage of token holders that felt inspired to vote. Of course, part of this group of token holders are entities that cannot vote, such as exchanges, and for a larger part, traders and speculators that will be hard to motivate and are arguably also not the best judges to assess the quality of the projects. Nevertheless, having more involvement from motivated community members will reduce the impact of any isolated undesired voting strategy and result in a better representation of the community’s preferences. So how can we increase the number of active voters?
1) Give some kind of reward for voting.
While we would prefer to see intrinsic motivation, a bit of extra incentive may give our community members that little push to take action. There are plenty of options on this theme, varying from a small AGIX reward, to a small chance on a higher reward, to NFTs or other creative options.
2) Removing barriers to voting.
Properly assessing all, or even a subset of proposals, takes time and effort. People may not always have enough time or motivation to do so. Even the act of voting itself (connecting a wallet, clicking through the proposals) takes some dedication. A popular approach in DAOs is to enable people to vote on other people’s behalf. This can take different forms:
- A representative election committee, where a subset of elected people executes the actual voting, similar to how many countries are organized.
- A liquid democracy, where everyone can assign their voting power to anyone else, creating a network of voting delegates, where ultimate voting power can be distributed over a few or many token holders.
- It could also be a mix where a selection of voting proxies are elected and people can assign their votes to one of the proxies, OR decide to vote by themselves. (Or even decide to vote for 2 projects themselves and have the rest be managed by their preferred proxy.)
3 Making the assessment process of projects easier.
Going through a number of proposals, and assessing their (relative) merit, based on different structures, levels of detail, and sometimes in-depth technical definitions takes time and effort. We can make this process more seamless in 2 ways: First, helped by new functionality from our portal partner (Swae.io), we will add more restrictions to the layout of the proposals, so they are easier to scan.
Secondly, and more importantly, there is the option to form a committee of experts that will assess the projects on predefined criteria. First and foremost on their technical feasibility. Other criteria could be; realism of budget and scope, team seniority and capability, business viability (market and competition), and quality of the overall plan (development roadmap, milestones, marketing plan).
Once the proposals have been graded, community members can use this to guide and support their own preferences and they may zoom in on a few projects that look promising. In all cases, the expert rating is only advisory and offers no restrictions to the community’s voting preferences.
There are a number of variations possible here as well. The experts can come from the community, can be internal to SingularityNET, can be third-party domain experts, previous winners, or a combination of these.
Perhaps we can have an expert score and an overall score, similar to IMDB, whether we can somehow select a subset of projects to be assessed by the experts (e.g. the top 10 projects by requested amount, prefiltered by unofficial community ratings). An intriguing idea would be to create a live show (!), where the proposers can offer a short presentation and experts can interview them, and then give their final ratings together with their reasoning.
Measure 2: Creating a better weight-balance between individuals and their token balance.
Conceptually there are enough tools to accomplish this: Creating an algorithm that somehow reduces the extremes, giving a minimum voting power per wallet, in combination with a minimum AGIX quantity as a voting threshold. While we can have lengthy discussions on the details and the ‘right’ balance between individuals and their token balances, the math should be rather easy. The harder part is to recognize individuals to avoid strategic wallet splitting. But there are a number of ways to accomplish this:
- Hard KYC. Using official documents and biometric characteristics by integrating a KYC service. While this would be very adequate, it is not very popular and will add a substantial extra threshold for token holders to participate.
- ‘Soft’ KYC. Using face detection when connecting a wallet. While not as secure as full KYC, it is still a good measure with a lower but still significant threshold to participate.
- Using ‘wallet reputations’. We can simply assign a higher value to wallets that have been participating in multiple rounds and have demonstrated overall ‘constructive voting behavior’ (definition TBD). This would become more effective after more rounds have been executed. There is no extra threshold at all, but it needs a few rounds to become effective. The upside is that with sufficiently low weights for first-timers, the impact of proposer’s votes that are not yet involved in the program can be sufficiently reduced.
It would take a longer lead time and more dedication from a proposer to successfully impact the voting results. We can further improve this metric by giving extra value to long-time token-holding wallets!
Measure 3: Incentivize or give extra weight to voters that display consistent and constructive behavior.
If we can define what desired behavior is and we can relate that behavior to certain wallets, there is an option to reward that behavior and give wallets with a proven ‘good reputation’ a higher weight in the overall results. There are some privacy issues to consider here. Do we want this connection to be public, so everyone can see the amount of tokens that are related to certain profiles? Or should that be restricted to a few admins and developers? Or is this a bridge to far and do the downsides outweigh the upsides? These are questions to be explored and discussed with the community. There are however significant benefits to this connection; with this mechanism added to our toolbox we can:
- Give a higher weight to individuals that have been active in multiple rounds (which makes it harder for participants to game the system in a single round.
- Disincentivize strategic voting behavior – give less weight to individuals that only vote on the extreme ends of the scale.
- Give more weight to individuals that are more active in the program, e.g. by commenting on proposals, by giving expert support, being active on the social media channels, participating in meetings, etc. The tricky thing here will be to reward only constructive behavior and not reward trolls and extremists that are just the loudest.
Overview
Possible measures
Attracting more voters
1
Giving rewards for voting
2
Removing barriers by proxies
2a
A representative election committee
2b
Liquid democracy
2c
Limited voting proxies
3
Expert assessments
Balancing weights of wallets and token balance
4
Avoiding wallet spitting as a strategy
4a
Algorithm based on Hard KYC
4b
Algorithm based on Soft KYC
4c
Algorithm based on Wallet reputation
Reward consistent positive behavior
5
Reputation-based measures
5a
By adding or multiplying rewards to voters with a high reputation
5b
By giving extra weight to votes from people with a high reputation
After analyzing the voting behavior in part 1 of this series, the next question is, “What parts do we want to improve?” And after that, “How can we best accomplish this?”.
While there is a range of things to improve, before, during, and after the voting process, this article focuses on measures that aim to make the voting process itself efficient, fair, and balanced.
Let’s wrap up the main conclusions of part one first:
Grade voting effects
The voting behavior was different per wallet. Some wallets vote only by the extremes (1 or 10), and others vote more differentiated, (all numbers in between 1 and 10)
- When simply removing the extremes the conclusion was that the overall impact was limited.
- If we bucket the votes in smaller sections, 1-5 or 1-2 the results are different but not completely transformed either.
Wallet token balance effects
There is a huge disbalance in tokens per wallet, with especially the top 3 wallets having an uncommonly large weight in the overall results
- If we would count all wallets as one vote instead of each token, the outcome would have been a very different one, with a totally new top 3 in Pool A.
- If we focus on the subset of wallets ordered by size, the results are different in any of the sections we reviewed.
Individual wallet behaviors.
There is a large subset of 38 wallets that only voted for 1 project. In 30 of these cases, this vote was a 10. I am assuming that in many of these cases the wallet owners voted in favor of a project they are somehow involved in, which would be acceptable as long as it does not take an excessive form and is sufficiently balanced by unbiased token holders. (‘Fun’ fact is that 21 of the thirty votes came from smaller wallets in support of 2 projects, which seems to imply that these voters may not have interpreted the 1 token = 1 vote rule correctly.) A bit reassuring here is also that with 2 exceptions most single-vote wallet sizes were relatively small.
There are a few large wallets that voted mainly a 1 or a 2 to all participants, and a single 10. This may be a concerned community member with a strong preference. Or a proposer with very strategic voting behavior, which is definitely something we would like to discourage!
UX related issues
Some voters voted on the proposer portal, during the official voting week, without connecting a wallet. Those votes were basically lost in the overall result. This will be improved by better labeling options in the portal and eventually by enabling wallet connections to the portal and integrating the official voting process itself. This is more or less an isolated case and therefore not further discussed in the article below.
Overall conclusion
Perhaps it’s important to add, that while we were analyzing all the different scenarios above, there were no projects that clearly would have won in all relevant scenarios. There are no ‘moral winners’ that would have benefitted in all ‘fair’ scenarios, except for the current ones. It all depends on the scenarios chosen. And for each alternative scenario, we can argue that this would also have impacted the voting behavior itself. Therefore, in our opinion, this analysis does not reduce or undermine the validity of the current results in any way and we are very happy with the elected projects. (Although, there are also numerous projects that were not awarded that we would love to see appear in our ecosystem nevertheless!)
In case you missed it in the previous article, you can get a good intuitive understanding of the voting behavior by this great interactive visual made by Robert Haas.
Preferred behavior
So with these results in place, What is the kind of behavior we would like to avoid or promote?
Strategic voting
We would prefer to see voting behavior that is based on the perceived value of the project, not primarily based on outweighing the votes of others. Especially downvoting with the objective of reducing the competition is undesired.
This means that:
- We prefer to see wallets with somewhat differentiated voting behavior. In the current configuration of 1-10:
- A few 1’s or 10’s are perfectly OK, but nine 1’s against one 10 would be undesired.
- Five 10’s and five 1’s with nothing in between would also be undesired but to a lesser extent.
Voting bias by personal involvement
It is likely that some people’s voting behavior is steered by their personal involvement in a project. Ideally, the impact of such votes is limited. Most importantly, we should avoid that proposers have the chance to significantly influence the outcomes by temporarily purchasing large quantities of tokens. If the awards are high enough and the chance of positively influencing the voting result is also realistic, proposers will have an attractive business case, even at the risk of losing some value as a consequence of transactions and price volatility.
Such calculated behavior of temporarily buying large amounts of tokens just for voting purposes is perhaps the most important pattern of all we should aim to avoid!
Extreme differences in voting weight.
When comparing the smallest 20% of wallets to the largest 20%, the largest 20% would have a voting weight of 612 times the smallest 20%!
This ratio is a bit distorted by a larger number of very small and a few very large wallets.
- If we ignore the extremes and look at the 10-30 percentile and compare this to 70-90 percentile the ratio is reduced to 37.
- If we look at the 20-40 percentile and compare this to 60-80 percentile the ratio is further reduced to 10.
While there is a good case for giving large token holders also a larger say in the outcomes, the huge differences we see in the extremes are undesired. It would be an improvement to flatten the curve, or at least drastically reduce the weights of the extreme wallet sizes (especially on the high end, due to their large overall impact).
Potential measures to take
With all this background let’s dive into the core of the matter: What tools do we have or do we need in our toolkit to get to the desired end state, where there is a good balance in voting weight per wallet and votes that are based on the merit of the projects do have the highest weights.
Measure 1: Attracting more voters
While 186 Million votes sound impressive, the reality is that there were ‘only’ 158 wallets involved. On a total number of 30,718 wallets that means there was only a small percentage of token holders that felt inspired to vote. Of course, part of this group of token holders are entities that cannot vote, such as exchanges, and for a larger part, traders and speculators that will be hard to motivate and are arguably also not the best judges to assess the quality of the projects. Nevertheless, having more involvement from motivated community members will reduce the impact of any isolated undesired voting strategy and result in a better representation of the community’s preferences. So how can we increase the number of active voters?
1) Give some kind of reward for voting.
While we would prefer to see intrinsic motivation, a bit of extra incentive may give our community members that little push to take action. There are plenty of options on this theme, varying from a small AGIX reward, to a small chance on a higher reward, to NFTs or other creative options.
2) Removing barriers to voting.
Properly assessing all, or even a subset of proposals, takes time and effort. People may not always have enough time or motivation to do so. Even the act of voting itself (connecting a wallet, clicking through the proposals) takes some dedication. A popular approach in DAOs is to enable people to vote on other people’s behalf. This can take different forms:
- A representative election committee, where a subset of elected people executes the actual voting, similar to how many countries are organized.
- A liquid democracy, where everyone can assign their voting power to anyone else, creating a network of voting delegates, where ultimate voting power can be distributed over a few or many token holders.
- It could also be a mix where a selection of voting proxies are elected and people can assign their votes to one of the proxies, OR decide to vote by themselves. (Or even decide to vote for 2 projects themselves and have the rest be managed by their preferred proxy.)
3 Making the assessment process of projects easier.
Going through a number of proposals, and assessing their (relative) merit, based on different structures, levels of detail, and sometimes in-depth technical definitions takes time and effort. We can make this process more seamless in 2 ways: First, helped by new functionality from our portal partner (Swae.io), we will add more restrictions to the layout of the proposals, so they are easier to scan.
Secondly, and more importantly, there is the option to form a committee of experts that will assess the projects on predefined criteria. First and foremost on their technical feasibility. Other criteria could be; realism of budget and scope, team seniority and capability, business viability (market and competition), and quality of the overall plan (development roadmap, milestones, marketing plan).
Once the proposals have been graded, community members can use this to guide and support their own preferences and they may zoom in on a few projects that look promising. In all cases, the expert rating is only advisory and offers no restrictions to the community’s voting preferences.
There are a number of variations possible here as well. The experts can come from the community, can be internal to SingularityNET, can be third-party domain experts, previous winners, or a combination of these.
Perhaps we can have an expert score and an overall score, similar to IMDB, whether we can somehow select a subset of projects to be assessed by the experts (e.g. the top 10 projects by requested amount, prefiltered by unofficial community ratings). An intriguing idea would be to create a live show (!), where the proposers can offer a short presentation and experts can interview them, and then give their final ratings together with their reasoning.
Measure 2: Creating a better weight-balance between individuals and their token balance.
Conceptually there are enough tools to accomplish this: Creating an algorithm that somehow reduces the extremes, giving a minimum voting power per wallet, in combination with a minimum AGIX quantity as a voting threshold. While we can have lengthy discussions on the details and the ‘right’ balance between individuals and their token balances, the math should be rather easy. The harder part is to recognize individuals to avoid strategic wallet splitting. But there are a number of ways to accomplish this:
- Hard KYC. Using official documents and biometric characteristics by integrating a KYC service. While this would be very adequate, it is not very popular and will add a substantial extra threshold for token holders to participate.
- ‘Soft’ KYC. Using face detection when connecting a wallet. While not as secure as full KYC, it is still a good measure with a lower but still significant threshold to participate.
- Using ‘wallet reputations’. We can simply assign a higher value to wallets that have been participating in multiple rounds and have demonstrated overall ‘constructive voting behavior’ (definition TBD). This would become more effective after more rounds have been executed. There is no extra threshold at all, but it needs a few rounds to become effective. The upside is that with sufficiently low weights for first-timers, the impact of proposer’s votes that are not yet involved in the program can be sufficiently reduced.
It would take a longer lead time and more dedication from a proposer to successfully impact the voting results. We can further improve this metric by giving extra value to long-time token-holding wallets!
Measure 3: Incentivize or give extra weight to voters that display consistent and constructive behavior.
If we can define what desired behavior is and we can relate that behavior to certain wallets, there is an option to reward that behavior and give wallets with a proven ‘good reputation’ a higher weight in the overall results. There are some privacy issues to consider here. Do we want this connection to be public, so everyone can see the amount of tokens that are related to certain profiles? Or should that be restricted to a few admins and developers? Or is this a bridge to far and do the downsides outweigh the upsides? These are questions to be explored and discussed with the community. There are however significant benefits to this connection; with this mechanism added to our toolbox we can:
- Give a higher weight to individuals that have been active in multiple rounds (which makes it harder for participants to game the system in a single round.
- Disincentivize strategic voting behavior – give less weight to individuals that only vote on the extreme ends of the scale.
- Give more weight to individuals that are more active in the program, e.g. by commenting on proposals, by giving expert support, being active on the social media channels, participating in meetings, etc. The tricky thing here will be to reward only constructive behavior and not reward trolls and extremists that are just the loudest.
Overview
Possible measures |
||
Attracting more voters |
||
1 |
Giving rewards for voting |
|
2 |
Removing barriers by proxies |
|
2a |
A representative election committee |
|
2b |
Liquid democracy |
|
2c |
Limited voting proxies |
|
3 |
Expert assessments |
|
Balancing weights of wallets and token balance |
||
4 |
Avoiding wallet spitting as a strategy |
|
4a |
Algorithm based on Hard KYC |
|
4b |
Algorithm based on Soft KYC |
|
4c |
Algorithm based on Wallet reputation |
|
Reward consistent positive behavior |
||
5 |
Reputation-based measures |
|
5a |
By adding or multiplying rewards to voters with a high reputation |
|
5b |
By giving extra weight to votes from people with a high reputation |
Join the Discussion (0)
Please create account or login to post comments.