New ways to measure the success of decentralized applications
Contents
Key takeaways
- The most popular blockchain games sustain monthly retention rates greater than 75%.
- Most games demonstrated their ability to maintain retention rates above 60%.
- But some game dapps experienced months with retention rates less than 50%, which means they had a declining user base.
- While some of these trends are blockchain-specific, the design of individual game features is the main factor dictating retention rates.
Introduction
The game sector is the largest single dapp category with 19,310 daily active unique wallets recorded across all the games running on the four main smart contract blockchains – Ethereum, EOS, TRON, and Ontology – on 31 December 2019.
During the year, cumulative user growth across all games was 118%, with Ethereum ending 2019 as the leading blockchain with 44% of the category’s total wallets. EOS accounted for 41% of wallets, with TRON and Ontology trailing with 9% and 4% respectively.
Drilling deeper into such overview data reveals more interesting trends.
Notably, the successes (and failures) of each game to build an active player base is clearly driven more by the behavior of individual audiences than blockchain-scale events, although they can also have an impact.
In that context, this report focuses on the importance of month-on-month player retention as a key measurement of success.
How we calculate user retention rate for dapps
There are many ways of measuring retention but for the sake of clarity and timeliness, in this report, we’re considering the monthly retention of unique active wallets on a per-game basis.
This metric breaks down each game’s monthly audience into two parts; new users and retained users, each compared from the previous month’s unique wallets. This provides a snapshot of whether players are enjoying the experience and continuing to play month-on-month, or quickly churning out.
Note, this methodology doesn’t attempt to measure the medium or long term re-engagement of audiences over two or more months.
Such analysis throws up a wide spectrum of retention rates. Some of these are due to the game genre, and some are linked to blockchain characteristics.
But the strongest trends are due to the interaction between the player base and in-game features.
In this report, we consider the performance of 10 blockchain games; five running on the Ethereum blockchain, four on EOS, and one on Ontology. TRON was excluded because although it has plenty of live games, no single game on TRON sustained a large enough user base during 2019 to provide useful data.
As can be clearly seen, each individual game generated a wide range of monthly retention rates during 2019.
Some games sustained monthly retention rates of over 75%, and most had multiple months of retention greater than 50%. In contrast, all games bar one also had months in which more players churned out than were retained i.e. they relied on new players for headline growth.
Retention rates of the top Ethereum games
The first games considered are the five games running on the Ethereum blockchain, which mainly use the blockchain for NFT ownership, although a couple of games also use it for payments and subscriptions.
My Crypto Heroes
In terms of overall retention, the best performing blockchain game in 2019 was My Crypto Heroes, developed by Japanese studio Double Jump.Tokyo. It was also one of the most popular Ethereum games in terms of audience and growth, ending the year with almost 5,000 monthly active unique wallets.
One element of this was the game’s exceptional retention rate, which grew steadily in the second half of 2019 when it continually surpassed 75%, ending the year at 89%. This sort of retention isn’t an accident but derives from a strong focus on iterative product improvement and community development.
In particular, it’s worth noting the impact of My Crypto Heroes’ Prime monthly subscription which engages users through its daily log-in rewards.
Looking at the absolute monthly wallet figures, My Crypto Heroes’ improvement during 2019 is apparent.
However, the increase in retention rate can also be seen as a byproduct of fewer new players onboarding. During the first quarter, over 1,500 new players started playing the game every month, but this then fell to under 1,000 as the year progressed.
Nevertheless, because My Crypto Heroes became so much better at retaining its player base, it still managed to steadily grow its retained monthly active unique wallet total to almost 4,000 by December.
0xUniverse
0xGames’ 0xUniverse demonstrated a solid monthly retention rate of above 60% for the first half of 2019, but the second half of the year saw much more mixed results.
There were two main reasons. As a fairly decentralized game in which the majority of player actions are Ethereum transactions, 0xUniverse was heavily impacted by the doubling of gas fees during September. This resulted in the collapse of its retention rate to 33%. In absolute terms, more than 2,000 monthly active unique wallets were lost that month.
When the gas price went back to normal in October, many of these ‘lost’ players returned. But the retained player base has not yet rebounded to its summer peak, even if the monthly total including new players ended the year on a 2019 peak of 4,260.
The other main reason for these trends is that 0xUniverse is a game lacking strong return engagement features, especially in terms of the first time user experience. This means that when it experiences a rise in new player numbers, as happened at the end of 2019, the retention rate tends to drop as new players find the game difficult to understand and churn out.
Blockchain Cuties
Blockchain Cuties, from Latvian developer We Can Games, displays one of the tightest ranges of retention rates of the games surveyed. For the first five months of 2019, it maintained a high level of retention, peaking at over 75% in May.
The second half of the year was less successful, however, partly because in May and June, a lot more new players started playing Blockchain Cuties. Such influxes tend to reduce retention unless a game offers a very polished user experience that keeps players coming back month-on-month.
Nevertheless, Blockchain Cuties was the only game in the survey that maintained a positive retention rate greater than 50% during 2019.
It also experienced a spike in new users during December when it launched its Forge feature; something that would have re-engaged players who had churned out more than a month ago. They account for the strong rise in ‘new wallets’ for that month.
CryptoDozer
As a casual arcade collection game, it’s very easy for new users to understand how CryptoDozer works. In addition, its gifting of free tokens every 30 seconds also drives daily play sessions. Getting players to spend ETH for in-game items and return week-on-week and month-on-month is more valuable.
It is interesting to see how CryptoDozer launched with a poor retention rate of less than 50%. However, as South Korean developer PlayDapp focused on metagame elements, including those employing its ERC20 PLA token, there was a steady increase in retention to over 75%.
This can also be seen by tracking the flow of retained, lost and new wallets.
CryptoDozer built its retained audience from August onwards but, for such as a casual game, it added a surprisingly small number of new players a month during this period; well under 500 compared to almost 1,300 new wallets in July.
HyperSnakes
Given our previous analysis, the monthly retention graph for Mixmarvel’s HyperSnakes is something of a shock. Of the 10 games analyzed for this report, it demonstrates by far the lowest retention rate, barely scraping 25% at best.
Partly, this is a byproduct of the game’s incredible launch period, which saw it attracting tens of thousands of new wallets on a monthly basis, thanks to its promise of simple gameplay, discounted items, and speculative rewards.
Indeed, HyperSnakes attracted an amazing total of 47,336 new wallets during July 2019 but only retained 102 of them the following month. This is a warning for developers that powerful incentives during launch do not automatically build a long term audience, especially given the inherent complexity of blockchain dapps.
Post-launch, the monthly inflow of new wallets dropped to a couple of thousand and the game’s retention rate picked up. But it remains the case that the vast majority of new wallets do not play HyperSnakes after their initial experience.
Retention rate of the top Ontology game
Ontology is an enterprise-focused smart contract blockchain, which also supports a small number of games.
HyperDragons Go
Operating on the Ontology blockchain, it’s clear that 2019 was a year of two halves for Mixmarvel’s HyperDragons Go, although this isn’t immediately apparent by looking at its monthly retention rate.
Nevertheless, the wide range of this rate – peaking at 84% in October but as low as 30% in September – suggests something significant was happening.
To understand this we have to look at wallet activity. This shows very high audience flows, both in terms of new and retained wallets, up until July. Indeed, apart from HyperSnakes’ launch months, no other game in this report records anything as large as the ten of thousands of monthly active unique wallets that HyperDragons Go does.
However, post-July, this activity drops away sharply. From September onwards, it’s measured in the thousands of wallets monthly, which is broadly similar with most of the other blockchain games covered.
In this context, the large decline in retention rate in August and September is clearly the result of the previously high volume flows churning out of a game experience which doesn’t appear to have the features to attract long term retention.
This is also reflected in HyperDragon Go’s falling retention rate in the final three months of 2019, when it drops from 84% in October to 59% in December.
Retention rates of the top EOS games
Here we consider four of the top games that run on the EOS blockchain.
EOS Knights
For the first half of 2019, South Korean developer Biscuit Labs’ EOS Knights was the most popular game on EOS blockchain in terms of monthly active unique wallets. It peaked at over 10,000 during February and March, something which reflected its accessibility as a free-to-play mobile idle RPG.
This status changed rapidly, however; the game’s retention rate dropped from over 80% to 28% in April.
It’s not clear why this occurred, although idle games tend to lack long term retention features. Equally, blockchains such as EOS (and TRON) which use proof-of-stake consensus are known to suffer from higher rates of bot activity, which can account for highly volatile month-on-month user activities.
It’s worth noting April was the peak month for new users – over 5,500 wallets.
And from that point, EOS Knights continued to steadily lose more wallets than it added or retained, although its monthly retention rate remained solid until November.
By this stage, though, the entire EOS ecosystem was suffering from the network congestion created by the EIDOS token airdrop, which meant many wallets with only small amounts of EOS staked could no longer access the EOS blockchain.
A combination of these factors saw EOS Knights end 2019 with less than 1,000 monthly active unique wallets.
EOS Dynasty
Launched in the summer, EOS Dynasty is effectively a version of EOS Knights for a Chinese audience using a Three Kingdoms art theme and setting. It also has its own in-game token, which is used for its item marketplace as well as dividend rewards.
These features saw EOS Dynasty maintaining solid retention in the range 71 to 54% for most months. Then, it declined sharply in December to 36%.
The reason for this was a large influx – more than 7,500 – new wallets in November. These weren’t retained however with more than 7,000 wallets lost in December. Again, this is a reminder that attracting large numbers of wallets either through promotions or marketing isn’t a sensible opinion for developers unless they are certain their game is accessible and can onboard players into longer-term meta game features.
Despite this, EOS Dynasty ended 2019 having grown its retained wallet total to almost 4,000, and boasting an overall active wallet total of over 6,000, making it the most popular blockchain game.
Prospectors
Launched by a Ukrainian development team in July, Prospectors is an economic simulator game set in the California Gold Rush. Like EOS Dynasty, it uses its own in-game token to capture value. But while this potentially provides a strong long term retention feature, it also adds complexity in the early stages of play.
And this is one of the underlying reasons for Prospectors’ relatively weak retention, which aside from its post-launch month, trended down, ending 2019 less than 50%. It is also likely that EOS’ network congestion in November and December would have had an impact, especially in terms of month-on-month retained audience.
On a more positive note, Prospectors did generate plenty of new wallets every month. It attracted over 6,500 wallets at launch and over 1,500 in every subsequent month, including over 6,000 in November.
Crypto Sword & Magic
Crypto Sword & Magic’s retention rate in 2019 seems to be typical for many games running on EOS. It launched strongly, then dropped and built back up before being hit by the general congestion on the EOS blockchain in the final two months of 2019.
This situation wasn’t helped as Crypto Sword & Magic is a mobile RPG, which is accessible in the early stage but which has relatively weak meta game features.
However given its monthly active unique wallet total peaked in November and its retained total peaked in October, the biggest issue for Crypto Sword & Magic appears to have been EOS’s congestion.
Conclusions
While every game is different, considering the retention trends across 2019 we reach the following conclusions.
On a blockchain level, it was easier for games running on Ethereum to build their retention rates during the year. In part, this was due to the fact these games had been live products for longer and how they interacted with the Ethereum blockchain.
In contrast, games running on the EOS blockchain were less mature in terms of their features and also had to deal with the systemic EIDOS airdrop, which reduced all activity on the blockchain for all dapps.
More generally, though, with a couple of notable exceptions, few blockchain games have learned the lessons from non-blockchain games in terms of building strong regular retention features such as daily, weekly and monthly rewards, and VIP subscription models.
In that context, the good news is these features are relatively easy to implement. Hence we expect the sector to see a strong uptick in retention during 2020.
Certainly, all developers operating games with a retention rate of less than 50% need to think very seriously about their products. Even those with rates in the mid-60s need to have a coherent plan to increase this above 75%, as demonstrated by the likes of My Crypto Heroes and CryptoDozer.
