A clear sign that the crypto industry should define and align on diligence best practices
by Doug Schwenk, CEO, Digital Asset Research, January 2023
As a firm that has conducted diligence on centralized crypto exchanges and other counterparties for 5 years for institutional clients, we want to reflect and learn in this moment of industry uncertainty after the collapse of FTX. At Digital Asset Research (DAR), we’re evaluating what went right, what we overlooked, and how we can press the industry for better practices and standards. There will be lots of stories, told by Monday morning quarterbacks, that they “knew” the depths of FTX’s problems, but we hope that honest introspection will help us improve how we work for clients and champion even better practices going forward.
Two Measures of Risk
We started measuring exchange quality in 2017 for clients who were concerned about the reputational risk, price risk, and counterparty risks associated with crypto exchanges. Over the last 5 years, we’ve developed and continually improved risk assessment frameworks that incorporate significant feedback from clients and regulators. On behalf of clients, we perform our assessments at least quarterly and measure the results. These aren’t perfect systems, but we’ve tried to raise the bar each year to push the industry forward and provide our clients with a better understanding of exchange quality.
Historically, we’ve assessed the risk of crypto exchanges and counterparties using two different lenses, depending on the need of our institutional client:
1. Exchange Vetting for Price Quality: In this assessment, we want to know that exchange transactions are between a real economic buyer and a real economic seller.
To determine this, we look at whether the exchange exhibits any signs of price data manipulation (e.g., faking trades) or persistent market manipulation (e.g., wash trading) and whether the exchange is doing the right work to prevent manipulation in the future.
In the quantitative portion of this assessment, we examine trade and order book data, using fraud detection and market manipulation detection techniques, for any signs that patterns don’t match what happens in legitimate capital markets and to confirm the exchange is generally in sync with other legitimate crypto markets.
In the qualitative portion of this assessment, we look at information about the business across a number of categories, such as: regulatory compliance, policies and procedures, management expertise, trade surveillance, KYC procedures, business continuity, security, and other important institutional quality factors.
2. Exchange Counterparty Diligence: In this assessment, we want to know if our institutional client bears significant risk for holding assets on the counterparty’s platform, which can help our client size its exposure.
To perform this assessment, we look further at the exchange’s structure and financial strength as a counterparty. We want to know what the client’s exposure and potential loss scenarios are during a time of stress. We also attempt to dig deeper into the exchange’s balance sheet, governance, margin practices, risk management, depth of compliance, insurance, segregation of client assets, and a host of other factors important in counterparty health. This assessment requires a more active and willing participation by the exchange and communication of much more sensitive information that is typically available in traditional finance.
Our Incentives and Perspective
Our clients are hedge funds, trading firms, asset managers, banks, index providers, regulators, and institutional players who are customers of the exchanges or have an interest in the exchange’s activity. At DAR, we do not get paid by exchanges for the evaluations that we provide, nor do we have any exchanges as customers in other business lines. While we’ve given informal feedback to crypto exchanges after our evaluations, to allow them to note areas for improvement, we think it’s important that our incentives are with the customer.
We remain independent and unbiased, which we think is important to the diligence process.
How We Scored FTX.com and FTX US
1. On Exchange Vetting for Price Quality:
a. FTX.com: We never passed FTX.com. They were on our Watchlist as an exchange with numerous qualitative red flags, but offered no clear signs of data manipulation or market manipulation in publicly available trade and order book data.
b. FTX US & Liquid: We recently passed FTX US and Liquid as Vetted Exchanges. We became convinced during communication with insiders that they were being operated as independent entities with sufficient governance and autonomy, as well as appropriate policies and transparency to be independent of FTX.com. Their responses indicated that they were doing the right work to prevent manipulation on their venues.
2. On Exchange Counterparty Diligence:
a. We attempted several times, on behalf of significant crypto hedge funds, to assess FTX.com and FTX US. They did not provide the necessary transparency, thus we were not able to provide a meaningful assessment other than the red flags we identified during Exchange Vetting for Price Quality.
What We (DAR) Got Right – Top 5 Red Flags
We identified a number of red flags regarding FTX.com and communicated those to clients. Our clients did not include FTX.com prices in calculating best quality market prices (Tier 1) and were able to manage their counterparty exposure. To our knowledge, none of our clients who depended on our risk assessments have any material funds at risk in the bankruptcy for FTX.com, FTX US, or Liquid.
The top issues we raised included:
1. Loose KYC and AML policies. A lack of appropriate policies leads to the potential for market manipulation and money laundering that may be undetectable using trade surveillance systems.
2. Alameda Research as an affiliate without any disclosure as to how conflicts of interest were addressed. Management and control over both organizations by the same people could (and in hindsight did) create opportunities to disadvantage clients of FTX.com in favor of Alameda.
3. FTT token and its risks. It is important to understand the risks with any token and risks presented to the sponsor. FTT has hallmarks of an unregistered security, which created significant risks to FTX and its customers.
4. Unwillingness by FTX and its entities to provide confirmation of certain public diligence information. Confirming or correcting information in the public domain ensures clients are not misled.
5. Lack of transparency and tone of FTX.com as a counterparty. Reasonable counterparties will answer at least some key diligence questions from institutional clients. Trivial responses indicate a lack of maturity or other issues that should significantly concern a fiduciary client.
In the category of what we also got right, we’re pleased to see that Liquid should be returning funds to clients soon; we had passed them as a Vetted Exchange for Price Quality.
What We (DAR) Missed – Top 5 Oversights
With hindsight, we’ve also identified some missed opportunities related to our evaluation of FTX:
1. We underweighted the tone of FTX.com and FTX US diligence responses. We thought the tone was poor, but after reviewing some past correspondence, it was in fact clear that their unwillingness to answer questions went beyond immaturity. We should have seen that they were purposely trying to avoid scrutiny.
2. We were too slow to respond to the news of Brett Harrison’s departure as CEO of FTX US. Rather than acting immediately, we waited to see how FTX US would reorganize and for further information.. Change in leadership should trigger an immediate reassessment.
3. We were too slow to respond to the freezing of client withdrawals. It took some time for us to remove FTX exchanges as pricing sources. We wanted to be deliberative and not hasty, and had some presumption that a deal with Binance would restore order.
4. We underestimated the risk of the FTX.com and FTX US affiliation. While we had some comfort that they were independently managed, we should have been more skeptical of that conclusion and pressed harder to understand what might happen in the event of FTX.com’s collapse.
5. We overweighted the importance of FTX’s market share and strength of market presence. We gave FTX exchanges an undue benefit of doubt by arguing that we could not ignore such a significant player by market share.
As part of our continuous improvement efforts, our assessment frameworks have been updated to more clearly identify similar red flags in the future.
Where We Go From Here
The collapse of FTX and its ripple effects across the industry emphasize the importance of strong exchange diligence practices. For better or worse, this moment offers a pivotal opportunity to define industry best practices related to exchanges and other counterparties. At DAR, we’re continuing to evaluate and improve our evaluation frameworks to better serve clients in their risk evaluation processes.
For more information, please contact us.