Understanding
Principle #11: Systems can be Understood
The eleventh principle of systems science states that systems can be understood.
When we understand a system, it means that we can make good predictions or useful projections about the system’s behavior.
If the mental model we hold of a system correlates well with the relevant features of that system in the real world, our predictions will be accurate. If it isn’t, then the model can be refined and improved, or abandoned if necessary. Understanding a system allows us to function successfully through the guidance of accurate models.
Science is one of the most useful methods humans have devised for understanding systems. This piece will review how BlockScience, a firm specializing in complex systems engineering, used the methods of network science to gain deeper understanding of problems faced by Gitcoin.
Gitcoin and Quadratic Funding
Gitcoin is an organization that creates blockchain-based open-source software tools that help communities fund public goods. The Gitcoin Grants program is a quarterly initiative run by Gitcoin in which projects for people to donate to are curated by the organization.
The grants rounds rely on a mechanism called “Quadratic Funding” to raise money from grantors and distribute them to grantees. First, a large number of small donations are pooled together from the community. Next, these funds are matched with a larger amount of funding from a centralized pool funded by sponsors. Finally, a special mathematical formula is applied which rewards grantees based on the number of people who have donated rather than the donation size.
If one person donates $100 to a project, that project gets minimal funds from the large matching pool. If instead 100 people each donate $1 to the same project, it gets matched with a large amount of funds from the pool. The idea is that this form of funding better captures the wants and needs of the community.
Over the years Gitcoin has evolved from a small fairly centralized platform focused on the Ethereum system into a large open-source decentralized protocol embraced by organizations from the United Nations to Shell.
The Problem
Like any other system, Quadratic Funding isn’t perfect, it has its strengths and weaknesses. Since there are often millions of dollars up for grabs during Gitcoin Grants rounds, it is a target for people who want to exploit the system.
Quadratic Funding is sensitive to two specific forms of exploitation — sybil attacks and collusion strategies. A sybil attack is a form of fraud in which a single person misrepresents themself as many different people. Collusion involves “secret or illegal cooperation or conspiracy, especially in order to cheat or deceive others.”
The Science
I’m just going to give a high-level overview of the process BlockScience went through to gain a better understanding of the ways in which Gitcoin was vulnerable to groups attempting to collude to game the system. For a deeper dive, check out the article “Colluding Communities or New Markets?”
Hypothesis
After determining that there is an underlying “shape” or structure, of communities who use Gitcoin grants, BlockScience outlined the following hypothesis.
“We can interpret these shapes as a “fingerprint” to pattern-match organic community behaviors to better identify exploitive behavior and reduce the ability to siphon away matching funds from the Gitcoin community at large.”
Assumptions
Given the subjective nature of the analysis, BlockScience recognized that it was important to clearly state some of their key assumptions. For example, all contributions from the round were compressed to their USD value regardless of the currency that they were made in.
Methodology
The following methodology was used for the scientific analysis.
Donation data from Gitcoin Grants round 8 was collected.
The data was fed into an open-source model to perform data analysis.
The model was used to generate a graph (pictured below) representing the interconnections between grants and donors in the Gitcoin Grants contributions network. A “Fluid Community algorithm” was applied to the overall graph which returned a list of detected communities.
The team visualized the community subgraphs associated with three distinctive grants. These communities are composed of both grants and contributors as nodes, and the contribution flows between them as edges.
Analysis
Visualizations of the three community subgraphs are pictured below. They are the cadCAD subgraph, the MetaGame subgraph, and the DAppChaser subgraph. Blue nodes represent grants, orange nodes represent contributors, and the size of each node is determined by the size of total contributions made or received. The edges between nodes represent the donations flowing from contributors to grants.
The cadCAD subgraph has an organic appearance.
The Metagame subgraph also has an organic structure.
The DAppChaser subgraph does not look as organic and there is much less connectivity between donors. It warrants further investigation.
Interpretation
BlockScience’s interpretation of the data led them to believe that an algorithm might be able to flag potentially collusive behavior in the DAppChaser subgraph. However, there is a risk of algorithms being overly aggressive which might lead to mistaking legitimate new entrants in a community for colluders, which would result in unjust punishment.
A potential path forward would involve “building specialized data collection and community governance tools designed to help with crafting policy around colluding-behavior and interpreting collusion-policy on a grant by grant basis in the event of a dispute about colluding behavior.”
The Conclusions
BlockScience concluded that there is no objective “correct policy” that will address all the challenges in Gitcoin Grants matching.
Subjective choices regarding what trade-offs should be made to preserve fairness, as defined by Gitcoin, are necessary. This analysis aimed to provide clarity about those trade-offs and provide decision support to the Gitcoin team and community.
BlockScience identified three important questions to ask that would help hone in on potentially exploitive behavior in the Gitcoin Grants system and help make policy choices accordingly:
How many funds are allocated from more organic communities towards more collusive contribution patterns?
Examining current Gitcoin Grant policies, how much money would grants get with or without each policy?
What are the early warning signs of collusion, and how do we escalate those signals to the governance layer of the system?
Once we understand the problems facing a system, we can make necessary adjustments to improve them. The next and final piece in this series will explore how the scientific research performed by Gitcoin and BlockScience informed specific engineering improvements.