G™Decision Theory

Decision theory solves the extremely hard* problem of telling you what you should do if you have a consistent and mathematically formalized model of the world and a utility function telling you exactly how good any particular world-state is. The most common decision theories are evidential decision theory and causal decision theory; various accursed decision theories exist but are not in wide use.

Evidential decision theory

EDT says, informally, that you should take whatever action has the highest expected utility (i.e. the sum of utility for each outcome weighted by the probability of that outcome conditional on you taking the action).

Causal decision theory

Informally, CDT says that you should take whatever action causes the highest expected utility, i.e. the conditional probability is replaced with a counterfactual (there is some debate about how to operationalize this, but roughly speaking, it's the probability if you replaced the action you took with another one).

Limitations

Neither theory provides a satisfying answer to all problems.

Toxoplasmosis Dilemma

decision_theory/toxoplasmosis_dilemma

Imagine that you have the opportunity to pet a cat. In this hypothetical world cats, in general, might carry toxoplasmosis, which causes you to be more likely to pet cats, as well as having many other negative effects you disvalue greatly. The cat you may pet has been tested for toxoplasmosis, and you are very confident it is not infected and could not infect you. However, people with toxoplasmosis are twice as likely to pet a cat given the opportunity than people without it.

XOR Blackmail

decision_theory/xor_blackmail

The XOR blackmail problem allows Decision Theory agents to be money-pumped. XOR blackmail is incorrectly named, as it is actually not blackmail. However, the terminology remains persistent.

Newcomb's Paradox

decision_theory/newcombs_paradox

Imagine Omega, a perfectly honest being which is also known to be a historically very accurate predictor of people's actions in this situation, sets up two boxes containing things, and gives you a choice between taking box A and B and box B alone. At the time this is set to you, the contents of the boxes are fixed.

Further reading

Subpages