animated one armed bandit slots machine gif

Search for the best email /content / landing page or the best banner This example can of course also be extended to the sending of emails or product recommendations. Assuming that the previous content does not influence the user’s reaction, this use case is a direct application of the one-armed bandit theory. We can therefore imagine that we have the choice between several contents/banners to propose to a user each time he connects to a site, and that we are rewarded by a click (or another KPI to be defined) as soon as the user is satisfied by the proposal. The choice of a slot machine in the above example can be replaced by any type of reward action.

The one-armed bandit model is extremely versatile, since it can be applied whenever there is a sequential choice between several actions, and one can rely on the observation of rewards to make an informed choice.

The problem of the one-armed bandits thus comes down to solving an exploration/exploitation trade-off that consists in choosing when to exploit and when to explore. In this example, the player will have to allocate a certain amount of money to find the most profitable slots (this is called exploration) and the rest of the budget to invest in these slots (this is called exploitation). One can consider that in each round rewards are drawn from the slot machines that were not chosen, but are neither observed nor collected by the player, see the figure below, where the observed rewards are shown in color. The reward given by the chosen slot machine is a random variable drawn from a certain probability law. The goal is to collect as much money as possible after N draws. The player has a certain budget, for example N coins of 1 dollar, and has to play these coins one by one on the different slot machines. It is this ignorance of profitability that makes it a learning problem.

The context is as follows: a player has a choice of several slot machines (also called arms) whose average profitability is not known in advance. It is a statistical learning model, with the aim to make a sequential choice between several actions based on the rewards they generate.Īn example of the application (which gave its name to the model) is the choice between several slot machines, “slot machine” being also called “one-armed bandit” in English.

Animated one armed bandit slots machine gif