Faulty dice: frequentists vs. Bayesians

If you’ve read a few blogs or articles on machine learning, data, analytics, or any field related to statistics, you may have come across the terms “frequentist” and “Bayesian”. These terms refer to two different approaches to statistics that, on the surface, seem to be in conflict with one another.

Even after doing a bit of research, it may not be clear to you what the difference is. It wasn’t clear to me. I started to suspect it’s just another mathematicians’ nerd-war over nit-picky details.

Turns out, the difference between the two approaches goes deep, and reflects the assumptions behind the decision we make in everyday life. I’ll try to explain this difference with an example that you might be familiar with.

The normal die

dice

(Note: “die” singular -> “dice” plural)

Suppose you rolled a normal six-sided die. How likely are you to roll a 2?

Most of us would answer this question by looking at the number of sides on the die (in this case, six) and assume it has an equal chance of turning up any of the sides. There are six sides, one of the sides is a 2, so roughly one out of every six times you rolled you’d get a 2.

This thinking seems intuitively correct. If I had to bet money on these dice rolls, I’d have a rough idea of how much money I’d lose or win over time based on the number of sides the die had. If 5 of my friends and I were betting on the outcome of the roll, and we each had to pick a side to bet on, unless I were superstitious (or suspected my friends of cheating) I’d have no practical reason to pick one number over any of the others.

The faulty die

dice
A faulty die. Looks the same, doesn’t it?

What if I told you now that when the die was being made in the factory, it was made faulty? Not intentionally, mind you; these aren’t cheater’s dice. They’re just weird, irregular.

Now, if I asked you what the odds are that any roll would turn up a 2, what would your answer be?

This question feels harder to answer than the first one. The die may be weighted to one side or another, so some sides may never turn up, and others may turn up more than half the time. The assumptions we made when talking about the normal die no longer apply.

If I pushed you for an answer, the only way you could guess is by rolling the die a few (hundred?) times, and keeping a record of how often each number comes up. Then you could give me a good sense of how likely I am to roll a 2.

The catch

Here’s the catch. If I gave you a die, how could you know if it was faulty or not?

In other words, given any unknown die, how could you decide if it was well made, or if it was imbalanced?

There’s no easy way to answer this. Our common intuition is to roll the die a few hundred times and keep a record of how many times each number is rolled. From the results we can see if the die is weighted towards one number or another.

But there’s a problem with that approach. Even if you roll a perfectly balanced die you won’t roll all 6 numbers evenly. You could easily roll five 6s in a row.

I have a die next to me. These are my first 10 rolls:

1, 5, 2, 1, 3, 5, 6, 6, 5, 3

I didn’t roll any 4s, and I rolled 5 three times. You can try this experiment yourself at this link.

Some would argue that ten rolls isn’t enough to decide if a die is faulty. You’d need at least a few dozen rolls, if not more, to feel confident.

But no matter how many times you roll the die, you will likely never get a perfect 1/6th split. It may get very close, but it won’t be exact. This is a natural result of the fact that the world has randomness in it. Given that’s the case, how can you know if these variations were because the die was faulty, or if it was due to luck?

Even a tiny margin, say 1%, can make a difference over the long run. For example, a gambler who is playing with weighted dice, where the number 5 is slightly more likely to be rolled than other numbers, could use that knowledge to her advantage and make a profit. Casinos use this information to give them a small edge, which, in the long run, adds up to massive (and legal) profits.

Frequentists and Bayesians

So we’re stuck in a dilemma. There is no real way to differentiate between results that are caused due to randomness, or results that are actually a consequence of how the die was made.

In cases like these, we tend to go with our common sense. Consider the following two scenarios:

If you get a die out of a brand new board-game box, you have no reason to believe it is fraudulent or faulty. If you rolled one a few times, and 6s came up more often than 3s, you’d attribute it to luck. This is the Bayesian approach, which says that you can guess beforehand what the outcomes (“probabilities”) are going to be based on certain rational criteria and experiences. In this case, new dice always roll each of the six numbers evenly, with a bit of randomness thrown in to spice things up. It’s only when you get new information, such as news of a fault in the factory, that you readjust your predictions.

On other other hand, if you hang out with gamblers and magicians, and one of them handed you a die, you’d probably want to check it beforehand. If you rolled it a few times, and 6s came up more often than 3s, you might accuse the person who handed it to you of cheating. This is closer to the frequentists position; it says that every new die is a new experiment, and must be experimented with on its own terms. Even after the experiment, the frequentist will only talk about the history the dice has shown thus far, and leaves open the possibility that it will change in the future.

Ultimately, there is no universal way to know what caused the dice to roll one way or another, so neither approach is applicable in all cases. It’s up to you to decide in a given situation which seems most appropriate.

When this difference matters

If you had to guess which of your country’s political parties was going to win the next election, how would you make that guess?

One approach is to go by past success. If a particular fringe party has never won, it seems reasonable to assume that they won’t win this time either. This is the Bayesian approach; if every new election is like rolling a new die, you have no reason to suspect any of them is faulty until new information (like a shift in political zeitgeist) is added to the mix.

Consider, however, there haven’t been that many elections in any given country’s history. The political parties who have won may have done the equivalent of rolling 5 sixes in a row, and this assumption is not impossible. Your decision that a certain fringe party is unlikely to win is a judgement call you make based on past experiences, and a gut feeling.

Alternately, you may decide to poll pedestrians on the street and generalize your sample to the entire population. This is the frequentist approach. Every new election, like every new die from a gambler, has to be experimented with separately, to see what the outcome is.

You could even imagine yourself taking both approaches in different contexts, and that is the point. Though the two approaches contradict each other in some ways, in other ways they can be seen as complimentary. They serve different use cases. You have to decide in each case if you’re going to start from prior probabilities, or if you’re going to do a tally of the current use case, and only go by the result.

How we enable in-store purchases using crypto

Background: STACK has partnered with STK to provide a cryptocurrency wallet inside the existing STACK app.

STACK + STK (1).png

STK lets you make instant payments at points of sale directly from your cryptocurrency wallet. Now you can add your cryptocurrency wallet alongside your other currency wallets in STACK. When you tap to pay through the STACK app, you can make purchases at any retail location that supports credit or debit cards. STK opens a bridge between the Ethereum blockchain and traditional credit card payment rails.

Almost a decade after the introduction of cryptocurrencies, and despite their promise of immense profit, none of the established payment providers have rolled out a convenient way to pay at stores or online with Bitcoin or Ether. In this article I’m going to explain the challenges of creating a cryptocurrency payment protocol at point of sale, and how we resolved them.

Traditional payment rails

Most of the world’s retail transactions (shopping), as well as banking go through a handful of core payment rails. The best known of these rails are run by major credit card companies. They enable banks and other financial institutions to transmit money to each other, securely and reliably.

Yervant Diagram 1.png

At STK we focused on consumer-to-retail payments, like those you make at grocery stores, convenience stores, restaurants and other retail locations.

Continue reading “How we enable in-store purchases using crypto”

Understanding Blockchains Intuitively, part 2

This is part two of an ongoing set of articles, designed to help you understand the motivations and concepts behind blockchains, in an intuitive and non-technical way. You can read the first part here.

To briefly recap, there is an island in the Pacific called Yap, whose inhabitants, rather than holding their money in their pocket, simply agree amongst each other how much money each person has, and update these records every time a sale or transaction is made. This provides a good analogy for blockchains. In modern blockchains, computers the network connections between them stand in place of the islanders of Yap.

Contracts

No discussion of blockchains would be complete without explaining Smart Contracts. The most popular Smart Contract blockchain in use right now is Ethereum, a blockchain similar to Bitcoin but with an added twist. To help explain this twist, let’s return once again to our inhabitants of Yap.

Peering into the brain of one of the Yapese we can see a long history of transfers of stone coins. These go in order, starting from the oldest that he or she remembers up till the most recent.

SmartContract (3)

One day, this islander is chatting with a friend of hers named Peter. Peter is worried about the damaging effect of annual hurricanes on his crops. He wishes to buy crop insurance to protect himself in case of disaster. Peter has therefore decided to enter into an insurance contract with another islander, Sumesh. Here’s how that contract goes:

Continue reading “Understanding Blockchains Intuitively, part 2”

Technological disruptions vs societal disruptions

The fairy-tale of technological disruption is a familiar one to us all. It is the recurring parable of our modern society. A plucky entrepreneur, filled with determination and the tiny but invincible seed of a vision, sets out to overturn our assumptions, confronts and overcomes the guardians of the decaying establishment, and brings his new gift to the world. The invention of the iPod and the iPhone, the birth of Google and AirBnB, and countless other entrepreneurial stories are enshrined this way in our culture.

If you’re involved in tech, or even peripheral to it, you’re likely familiar with the idea of technological disruption. Ever since Christensen first wrote about it in The Innovator’s Dilemma, it has been the dominant model used for analyzing new companies and emergent trends.

The pattern is typically as follows: a novel technology facilitates or makes more convenient what was previously tedious and expensive. Germinating in niche or low-end markets, and spreading though unorthodox sales channels, it eventually overtakes established industries and becomes the new norm or mode of life. Uber and AirBnb are poster children of this wave of technological revolution; but by no means the only ones.

Continue reading “Technological disruptions vs societal disruptions”

Understanding Blockchains Intuitively, part 1

This is the first part of a two part series. This part explains the basics of the blockchain through an analogy. The second part builds on this and discusses the idea of smart contracts.

There is an island in the Pacific called Yap. The people of this island have a strange and unintuitive type of money.

Yap_Islands.png

Their money consists of large stone coins called rai. Many of these coins are taller than a person. They are heavy and difficult to move. Nevertheless, these large stones are what they use as money.*

immagine-rai31.jpg

*In recent decades they have gradually moved away from using rai to using the US dollar, and the stone coins are reserved for ceremonial occasions.

How do you get your hands on, or even spend one of these coins? Your first guess might be that when you get a coin it gets shipped to your house and put in your front yard. That way you can know who owns what coin and, as an added bonus, you can make your neighbours jealous.

Continue reading “Understanding Blockchains Intuitively, part 1”

Intuitions in Machine Learning: Bias and overfitting

We are constantly trying to predict the future. Understanding how our actions affect what will happen helps us make better decisions. Since none of us can actually see the future, we use our past experience to make connections between two or more events, then use that to predict what will happen.

For instance, the open screen door seems to be connected to flying bugs coming into my house:

Screen Shot 2017-06-18 at 10.40.45 AM.png

“Hmmm… When I leave the front door open, bugs tend to fly in.”

Continue reading “Intuitions in Machine Learning: Bias and overfitting”