# Interference effects of choice on confidence: Quantum characteristics of evidence accumulation

See allHide authors and affiliations

Edited by James L. McClelland, Stanford University, Stanford, CA, and approved July 10, 2015 (received for review January 13, 2015)

## Significance

Most cognitive and neural decision-making models—owing to their roots in classical probability theory—assume that decisions are read out of a definite state of accumulated evidence. This assumption contradicts the view held by many behavioral scientists that decisions construct rather than reveal beliefs and preferences. We present a quantum random walk model of decision-making that treats judgments and decisions as a constructive measurement process, and we report the results of an experiment showing that making a decision changes subsequent distributions of confidence relative to when no decision is made. This finding provides strong empirical support for a parameter-free prediction of the quantum model.

## Abstract

Decision-making relies on a process of evidence accumulation which generates support for possible hypotheses. Models of this process derived from classical stochastic theories assume that information accumulates by moving across definite levels of evidence, carving out a single trajectory across these levels over time. In contrast, quantum decision models assume that evidence develops over time in a superposition state analogous to a wavelike pattern and that judgments and decisions are constructed by a measurement process by which a definite state of evidence is created from this indefinite state. This constructive process implies that interference effects should arise when multiple responses (measurements) are elicited over time. We report such an interference effect during a motion direction discrimination task. Decisions during the task interfered with subsequent confidence judgments, resulting in less extreme and more accurate judgments than when no decision was elicited. These results provide qualitative and quantitative support for a quantum random walk model of evidence accumulation over the popular Markov random walk model. We discuss the cognitive and neural implications of modeling evidence accumulation as a quantum dynamic system.

Decisions in a wide range of tasks (e.g., inferring the presence or absence of a disease, the guilt or innocence of a suspect, and the left or right direction of enemy movement) require evidence to be accumulated in support of different hypotheses. Arguably, the most successful theory of evidence accumulation in humans and other animals is Markov random walk (MRW) theory (and diffusion models, their continuous space extensions) (1, 2). MRWs can be viewed as psychological implementations of a first-order Bayesian inference process that assigns a posterior probability to each hypothesis (3). MRWs can account for choices, response times, and confidence for a variety of different decision types (2, 4). Moreover, these models of the accumulation process have been connected to neural activity during decision-making (5, 6).

According to MRW models, when deciding between two hypotheses, the cumulative evidence for or against each hypothesis realizes different levels at different times to generate a single particle-like trajectory of evidence levels across time (Fig. 1). At any point in time, the decision-maker has a definite level of evidence, and choices are made by comparing the existing level of evidence against a criterion. Evidence above the criterion favors one option, and evidence below it favors the alternative. Other responses are modeled in a similar manner; for example, confidence ratings are modeled by mapping evidence states onto one or more ratings (4). However, this idea that judgments and decisions are simply read out from the existing level of evidence—henceforth referred to as the “read-out” assumption—is inconsistent with the well-established idea that preferences and beliefs are constructed rather than revealed by judgments and decisions (7).

We present an alternative model of choice and judgment based on quantum random walk (QRW) theory (8⇓⇓–11), which posits that preferences and beliefs are constructed when a judgment or decision is made. Note that this work does not make the assumption that the brain is a quantum computer; instead, we simply use the mathematics of quantum theory to explain and predict human behavior. According to QRW theory, at any point in time before a decision, the decision-maker is in a superposition state that is not located at a single level of evidence. Instead, each level of evidence has a potential to be expressed, formalized as a probability amplitude (Fig. 1). New information changes the amplitudes, producing a wavelike process that moves the amplitude distribution across time.

In some ways the QRW is like a second-order Bayesian model (12). According to the latter, the decision-maker assigns a probability (rather than an amplitude) to each level of evidence for each hypothesis. However, like the MRW model, second-order Bayesian models are perfectly compatible with the read-out assumption, and as an optimal model, this would suggest that a decision should not change the probability assigned to each evidence level. In contrast, a QRW, like all quantum models of cognition (13), treats a judgment or decision as a measurement process that constructs a definite state from an indefinite (superposition) state. When a decision is made, the indefinite state collapses onto a set of evidence levels that correspond to the observed choice, producing a definite choice state. Confidence ratings work similarly, with the indefinite state collapsing onto a more specific set of levels corresponding to the observed rating.

These different theories of choice and judgment have strong implications for sequences of responses. Consider the situation when decision-makers have to make a choice (e.g., decide that hypothesis *A* or *B* is true) and later rate their confidence that a given (usually the chosen) hypothesis is true. According to the read-out assumption, a choice is reported on the basis of existing evidence that does not change the internal state of evidence itself. This applies to the MRW, a second-order Bayesian model, and many other accumulation models as well. Thus, after pooling across a person’s choices, the distribution of confidence ratings should be identical to conditions in which the person makes no choice at all. By contrast, the state of the system in a QRW is changed when a choice creates a definite state. Subsequent processing starts from the definite state, and the amplitudes spread out again. Thus, if information processing continues after the initial stage, the QRW predicts an interference effect where the marginal distribution of confidence judgments following a choice will differ from a condition in which no choice is made.

A proof of the predicted interference effect for QRWs is in *SI Appendix*. The proof shows that the interference effect of choice on confidence is the result of the interaction between the creation of a definite state and subsequent evidence accumulation after making a choice. Subsequent or second-stage processing is a necessary condition for the effect. Critically, second-stage processing occurs when people are asked to report a confidence rating following a choice, giving rise to response reversals (14) and other properties (15). We also provide a proof that MRWs predict no difference between the marginal distributions of confidence ratings (i.e., no interference) regardless of the presence of second-stage processing. This proof holds for a large range of MRWs, including ones with decay (16), leakage of evidence (17), and trial-by-trial variability in the decision process (18).

## Empirical Test of Predicted Interference Effect

We tested these opposing predictions concerning interference effects using a perceptual task that requires participants to judge the direction of motion in a dynamic dot display (Fig. 2). Specifically, nine participants completed 112 blocks of 24 trials each over five 1-h experimental sessions, a total of 2,688 trials per person (*SI Appendix*). During each trial, participants viewed a random dot motion stimulus that consisted of moving white dots in a circular aperture on a black background (19). A percentage of the dots moved coherently in one direction (left or right), and the rest moved randomly. Difficulty was manipulated between trials by changing the percentage of coherently moving dots (2%, 4%, 8%, or 16%). In the choice condition—half of the randomly ordered blocks—participants were prompted 0.5 s from stimulus onset via a low-frequency beep (400 Hz) to decide whether the coherently moving dots were moving left or right and entered their choice by clicking the corresponding mouse button. In the no-choice condition—the other half of the blocks—participants were prompted 0.5 s from stimulus onset via a high-frequency beep (800 Hz) to make a motor response (click the left or right mouse button as instructed). In all trials, the stimulus remained on screen for a second stage of processing after the choice or click. After an additional 0.05, 0.75, or 1.5 s following the first response, participants were prompted via a second beep (400 Hz) to rate their confidence that the coherently moving dots were moving right on a semicircular scale that appeared at the time of the prompt, ranging from 0 (certain left) to 100% (certain right) in unit steps. Note that to match the overall processing time of the stimulus across conditions, the confidence prompt was time-locked to the initial choice or click entry.

For the behavioral analyses, we collapsed confidence responses across the dot motion direction, recoding confidence onto a half scale (50% guess to 100% certain). All behavioral analyses were conducted using hierarchical Bayesian general linear models (20). The coefficient *b* is the linear effect of a predictor on the criterion. We also report the highest density interval (HDI) for all estimates, which specifies the range covering the 95% most credible values of the posterior estimates. A normal link was used for confidence judgments after transforming them to log odds, and a logistic link was used for choices.

On average, confidence increased with motion coherence (*SI Appendix*, Table S1). *SI Appendix*, Figs. S1 and S6 also show this effect at the distribution level. This effect of choice on confidence provides evidence against the read-out assumption and is consistent with the quantum claim that choice changes the state of the cognitive system.

We also examined the QRW prediction that the interference effect does not occur with choice alone but that the interaction between choice and subsequent processing creates interference. To gauge whether participants were sampling information during the second stage, we examined whether there was an interaction between coherence and duration of the second stage when confidence was predicted on a full scale from 0 (completely sure, incorrect direction) to 100 (completely sure, correct direction). The magnitude of this coefficient indicates second-stage processing under the following logic: on average, people sample information in favor of the correct answer during this second stage, and this evidence should be stronger with higher coherence (drift), resulting in an interaction between coherence and the duration of the second stage.

The value of this second-stage processing interaction coefficient is reported in Table 1. With the exception of participant 3, there is a 1:1 correspondence between credible second-stage processing and a credible interference effect. Note that participant 3 is by several measures an anomaly: this participant was unable to distinguish between dot motion directions in most conditions and in fact had lower than 50% correct choices in some conditions (*SI Appendix*, Fig. S4).

Further evidence for the requirement of second-stage processing comes from a study in which we failed to obtain interference when no second-stage processing was induced. This study was almost identical to the one described above, with two differences: first, there was no trial-by-trial feedback, and second, the decision time was 0.8 s rather than 0.5 s. The result of these differences is that participants did not pay attention to the stimulus after giving their initial choice or click response. This is evidenced by the lack of credible second-stage processing in this experiment (*SI Appendix*).

Finally, we examined how the accuracy of the confidence ratings changed as a result of this interference. To this end, we coded whether the confidence rating fell on the correct side of the scale relative to the actual left/right motion direction. In the choice and no-choice conditions, confidence ratings were on the correct side of the scale in 76.36% and 76.25% of cases, respectively. The accuracy of the confidence ratings in these conditions was credibly the same when we fit each one with a one-parameter Bernoulli distribution (

## Direct Comparison of QRW and MRW

The interference effect of choice on subsequent confidence provides empirical support for a QRW theory of evidence accumulation over an MRW theory. However, the question remains if the QRW can provide a parsimonious account of the choice and confidence data. Therefore, we compared the QRW to a matched MRW using Bayesian model comparison methods. Both models used drift, diffusion, starting point variability, and attenuation parameters. Versions of the MRW with these parameters or more restricted ones (e.g., without attenuation) have been shown to account well for choice, response time, and confidence data, so superior performance would indicate that the QRW is a particularly viable model (2, 4).

Below we describe each model in mathematical detail (see also *SI Appendix*, Fig. S3, for a visual walk-through). In line with the established mathematical principles governing both the MRW and QRW theories, each one updates the state following a choice at time

### MRW.

The MRW used

According to the MRW, a decision-maker is in exactly one evidence state at any given time. However, the decision-maker’s state is unknown to the observer, and a probability distribution across evidence states is therefore used to represent the state. This distribution is defined by a mixed state vector *x* at time *t*,

The probability distribution *w* is a free parameter indexing trial-by-trial variability in the initial state.

As the decision-maker considers information, the process moves from state to state. An *t* is

Choice probability and confidence are determined as follows. Define a response operator

For confidence ratings, define *y* and zeros otherwise. In the choice condition, the probability of choosing confidence level *y* at time *y* at time

The transition matrix is constructed from an *γ* is a parameter describing the proportion of time spent processing information up to time *t*. Consistent with recent work in modeling postdecisional processing (15), this was set to

The entries *δ* determines the probability that the process steps toward the true dot motion direction. We scaled the drift rate directly from the percentage of coherently *c* moving dots so that*c* is negative. The parameter *μ* is a free parameter indexing sensitivity to the coherence. The parameter

### QRW.

The QRW also used

According to the QRW, a decision-maker is not necessarily in any one evidence state at any given time. This uncertainty on the part of the decision-maker is modeled with a superposition state vector *x*th evidence level at time *t*. The probability of observing state *x* at time *t* is the squared length of the amplitude in the corresponding row:

The state vector *w* of this distribution is a free parameter representing initial uncertainty.

As information is processed, the superposition state drifts over time until a response is elicited. The

Choice probability and confidence are determined as follows. We define

Subsequent processing starts from this new state so that in the choice condition the probability of choosing confidence level *y* at time *y* at time

The unitary matrix is constructed from a Hamiltonian matrix *γ* operates in the same way as in the MRW. The entries

This definition of the Hamiltonian matrix was chosen so that the discrete state quantum process closely approximates the continuous state Schrödinger process (9). The *δ* and *δ* determines the rate at which probability amplitude flows in. The drift rate *δ* was set to be a multiplicative function of coherence (Eq. **5**). [Eq. **9** is a linear potential function in the diagonal of the Hamiltonian (multiplying drift by the state index) so there is a constant positive force pushing evidence toward the correct direction. However, other potential functions (e.g., quadratic) should be investigated in the future.]

The interference effect arises because the amplitudes in states 0–50 interact with those in 50–100 in the no-choice condition, pushing each other outward toward more extreme evidence states. This pressure is not present in the choice condition, leading to less extreme evidence and hence confidence ratings. One consequence of these less extreme confidence ratings is less overconfidence in the choice condition.

### Model Comparison.

Each model has four free parameters: a parameter that sets the drift as a scalar function of motion direction coherence (*μ*), a diffusion parameter (*γ*), and a parameter that determines the width of the initial state distribution (starting point variability) (*w*) (*SI Appendix*, Table S3). Non-decision time parameters, accounting for components of the response time exogenous to the evidence accumulation process, had limited influence on model fits and were dropped in order to facilitate model estimation.

The models were compared at the individual level: for each participant and each model, four parameters were used to account for 2,688 trials across 24 experimental conditions. Despite having the same number of parameters, the QRW may be functionally more complex, allowing it to produce good fits to the data without necessarily bearing any relationship to the underlying process. To account for this, we compared the Bayes factor between the two models for each participant (22).

The Bayes factor was calculated using a fine-grid approximation across all possible combinations of the four parameters to compute the likelihood function and uniform priors over their values. The results are summarized in Table 1; the log Bayes factor indicates the log odds of the QRW model over the MRW given the data (see *SI Appendix*, Tables S4 and S5, for the maximum likelihoods and parameter estimates).

The Bayes factor for seven out of nine participants and the group level factor decisively favored the QRW (maximum likelihoods yield the same conclusion). Participant 7 did not show second-stage processing or an interference effect, so the MRW may well describe the behavior of this participant. Participant 3 was unable to distinguish between dot motion directions in many conditions, which caused difficulty in fitting both models (*SI Appendix*, Fig. S4). Future model development incorporating methods for mapping evidence to confidence (e.g., using only 0/10/20% or 0/50/100% ratings) could potentially improve fits, but this does not affect interference so we favor simpler, more parsimonious models here.

Fig. 3 illustrates the fit of each model to the choice proportions for each coherence condition and the distribution of confidence in the choice and no-choice conditions for one participant and coherence level [all participants across conditions are given in *SI Appendix*, Fig. S4 (see also *SI Appendix*, Figs. S5 and S6)]. There are several reasons that the QRW gives a better account of the data than the MRW. First, the MRW predicts identical marginal distributions of confidence ratings between choice and no-choice conditions, whereas the quantum model picks up the slight rightward shift of these ratings in the no-choice condition; this phenomenon is the interference effect we described (see *SI Appendix*, Fig. S6; the QRW posterior predictions yield a group mean shift in confidence of +0.66%, compared to +1.19% in the data). Second, the QRW was often better able to simultaneously capture choices along with confidence ratings across the various conditions, whereas the MRW often had to sacrifice or compromise between the two. Notably, the MRW underestimated choice proportions because higher diffusion more accurately captured confidence distributions but at the cost of predicting lower choice accuracy. Finally, the observed confidence distributions are frequently multimodal and discontinuous. The MRW again does not account for these properties. By contrast, the QRW accounts for all of these characteristics in a parsimonious way, operating only on its first principles to earn a superior Bayes factor.

Although this MRW and similar versions have been used to model a wide range of choice and judgment data, it may struggle to account for this data simply because it cannot account for the interference effect. To examine this possibility, we tested a second model—the MRW-E—which assumes that additional evidence may have been accumulated in the no-choice condition, producing more extreme confidence ratings and thus an interference effect. Despite the added ability to produce interference, the QRW still outperformed the MRW-E. The Bayes factor for seven out of nine participants and the group level factor again decisively favored the QRW over the MRW-E. In comparison with the MRW, the MRW-E provides a largely equivalent or often poorer fit in terms of Bayes factors (*SI Appendix*, Table S6 and Fig. S7). Part of the reason the MRW-E does poorly, in addition to the characteristics it inherits from the MRW, is that it assumes more evidence is accumulated in the no-choice condition producing a change in the accuracy of the confidence ratings as well as the mean shift in confidence. Recall, however, that there is no credible change in the accuracy of the confidence ratings in the data.

## Discussion

In this paper, we have developed a model of evidence accumulation during judgment and decision-making based on quantum random walk theory. The QRW represents a point of departure in modeling evidence accumulation from the more typical classical probability approach. In the classical case, evidence evolves over time, but judgments and decisions are simply read out from an existing state without changing the internal state of evidence. In the quantum case, evidence also evolves over time, but judgments and decisions are measurements that create a new definite state from an indefinite (superposition) state. This quantum perspective reconceptualizes how we model uncertainty and formalizes a long-held hypothesis that judgments and decisions create rather than reveal preferences and beliefs. The different approaches make competing a priori predictions for the effect of sequences of responses, and we have shown strong empirical support for the quantum prediction that choices interfere with subsequent confidence judgments. Moreover, we have shown for the first time to our knowledge that the QRW is a viable competitor to the MRW in quantitatively fitting choice and confidence distributions. Note that the QRW can also account for response time distributions (8) and can outperform Markov models in this area as well (10).

A pertinent question is whether the MRW can be adapted to account for the phenomena we observed. This is certainly possible but may prove difficult: as we have shown, our results provide several constraints on potential adaptations. The interference effect itself is a strong constraint: many versions of the MRW that commonly give good accounts of choice and confidence data do not predict any interference.

A second constraint is how the interference effect occurred. In particular, confidence was less extreme following a choice. This poses a problem for explanations like the confirmation bias, where people focus on evidence that justifies their decision after making a choice, meaning they should be more confident in the choice condition (23, 24). Moreover, confidence accuracy also did not change. This poses a problem for models like the MRW-E that assume different amounts of processing between the choice and no-choice conditions.

A third constraint is that the interference effect only occurred when there was second-stage processing. This result poses problems for explanations based on differences in the mapping of evidence onto confidence (25) and explanations assuming that the act of making a choice introduces error into the cognitive system. Both explanations would fail to explain why choice alone (without second-stage processing) does not interfere with confidence. Alternatively, on some trials during the choice condition, participants reversed their initial choice (14) and could have reported unexpectedly low confidence on these trials, producing the interference effect. However, reversals during the choice condition happened infrequently (6.1%), and confidence on reversal trials was only slightly lower than on consistent trials. We discuss this and other alternative models in more detail in *SI Appendix*, section F.

Although an alternative MRW may be found to account for our results, this does not diminish the QRW’s significance in highlighting and challenging important assumptions regarding the judgment and decision-making process. In this paper, we have shown that a common assumption of cognitive and neural theories of decision-making—the read-out assumption—is violated even in a simple perceptual task. An interference effect occurred when participants were asked to make a decision about the leftward or rightward motion of a stimulus. Specifically, their subsequent confidence estimates were more conservative than when no earlier decision was made, and they were consequently less overconfident. This result, along with quantitatively superior model fits, lends strong support to the modeling of choice and confidence as a quantum random walk process, a model which describes decision-making as a constructive process wherein a definite state is created from an indefinite superposition. In addition to the cognitive implications, a QRW model of evidence accumulation potentially sidesteps the problem of how a group of neurons can produce observed behavior that is consistent with a single evidence accumulation trajectory (26). The QRW suggests that the mismatch might lie in the cognitive representation of evidence accumulation: instead of treating evidence accumulation as a single trajectory, it may be more accurate to conceptualize it as a wavelike superposition state. In fact, populations of interacting neurons processing evidence in parallel can give rise to a quantum random walk like the one presented here (10), and similar population coding models would certainly be capable of carrying out the necessary operations (27). Hence, quantum random walk theory provides a previously unexamined perspective on the nature of the evidence accumulation process that underlies both cognitive and neural theories of decision-making.

## Acknowledgments

This work was supported by grants from the National Science Foundation (NSF) (0955140) (to T.J.P.) and Air Force Office of Scientific Research (FA9550-12-1-0397) (to J.R.B.). P.D.K. was supported by a graduate fellowship from the NSF (1424871).

## Footnotes

- ↵
^{1}To whom correspondence may be addressed. Email: kvam.peter{at}gmail.com or pleskac{at}mpib-berlin.mpg.de.

Author contributions: P.D.K., T.J.P., and J.R.B. conceived of the study; P.D.K., T.J.P., .S.Y., and J.R.B. designed the research; P.D.K. ran the study; P.D.K. and T.J.P. analyzed data; P.D.K., T.J.P., S.Y., and J.R.B. wrote the paper; and T.J.P. supervised all aspects of the work.

The authors declare no conflict of interest.

This article is a PNAS Direct Submission.

This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1500688112/-/DCSupplemental.

Freely available online through the PNAS open access option.

## References

- ↵
- ↵
- ↵
- ↵
- ↵.
- Hanes DP,
- Schall JD

- ↵.
- Shadlen MN,
- Newsome WT

- ↵.
- Lichtenstein S,
- Slovic P

- ↵
- ↵.
- Feynman RP,
- Leighton RB,
- Sands M

- ↵
- ↵
- ↵
- ↵.
- Busemeyer JR,
- Bruza P

- ↵
- ↵
- ↵
- ↵
- ↵.
- Ratcliff R,
- Rouder JN

- ↵
- ↵.
- Kruschke J

- ↵
- ↵
- ↵
- ↵
- ↵.
- Green DM,
- Swets JA

- ↵.
- Zandbelt B,
- Purcell BA,
- Palmeri TJ,
- Logan GD,
- Schall JD

- ↵

## Citation Manager Formats

## Article Classifications

- Social Sciences
- Psychological and Cognitive Sciences

- Physical Sciences
- Applied Mathematics