Our new article in „IEEE Transactions on Games“ shows how game data mining, based on data from Blade & Soul, can be used to improve marketing results.

We are proud to share our latest publication on churn prediction with you. It has been published in the designated journal for Computational Intelligence and AI in Games IEEE Transactions on Games“. The publication describes the results of a competition whose goal was to develop an accurate AI-based churn prediction using game data mining. The data for this project came from the popular online game Blade & Soul. We worked on this article as part of a highly qualified team and as experts in game analytics.

Why is churn prediction so important to make marketing more successful?

AI-based churn prediction forecasts precisely whether and when a potential client would churn. Provided with this knowledge, the marketers can gain great benefits and implement actions to prevent churn.

This allows to develop an optimized churn prevention model, for example an individualized promotion campaign or an outstanding service-update.

Applied to the most valuable players of a game studio, a churn prediction model in combination with a churn prevention campaign can increase sales substantially. Such machine learning enhanced campaigns can greatly advance the state of marketing and allow personalization at a much greater detail.

The churn prediction was applied to data from Blade & Soul. Games benefit from churn prediction as this helps to improve marketing and retain valuable players.

What makes this game data mining project so interesting for business and marketing?

There have been some other approaches in the past that tried to use game data mining to improve business planing and marketing success. The problem the developers often faced until now, was the small amount of available data. Big game companies typically were not willing to share their inside gamer information with the outside world.

NCSOFT changed this difficult situation allowing researchers and developers to use data of one of their most popular action role-playing games – Blade & Soul. The information included player’s actions, interactions with other players, and game progress. The data came with an additional twist and included two different test scenarios. NCSOFT changed their business model from subscription-based to free-to-play in December 2016. Therefore, two test data-sets were provided. One before the change of the business model and one from players afterwards.

With that data, a competition was organized at the 2017 CIG conference. Out of the 300 registrations for the competition, just 13 managed to deliver a valid submission to predict whether a player would churn and only five to predict when the player would churn.

The submissions included Yokozuna Data from Japan, goedle.io from Germany, and other teams from South Korea and Finland. It was an international competition where highly qualified experts from different countries developed new algorithms to compare their approaches and results.

You will find more details about our work in this project in our blog post: Churn Prediction in NCSOFT’s Blade & Soul

The concept drift in Test Set 2 made the churn prediction challenge particularly challenging.

Goedle.io – among the best in the entire competition

The main goal of the competition was to predict the churn for already engaged players which is more difficult than that for all players. We participated with the engine of our live system and were, with a score of 0.58, among the best in the entire competition.

However, in this setting some of the obvious churners have been removed already. For example, a player with only a single session would typically be labeled as a churner by the algorithm and this prediction has high accuracy for most cases. Removing these players will lower the overall score in comparison to other settings because one tackles a particularly difficult instance of the problem.

Achieving this result, we developed new techniques and methods that will help companies and marketing agencies to keep their loyal customers happy and secure their valuable business.

If you wish to know further information about this work or receive a digital copy of the publication, send us an email to info@goedle.io. And don’t forget to subscribe to our newsletter and follow us on Twitter, Facebook, LinkedIn, and XING.