The role of intolerance of uncertainty in the acquisition and extinction of reward

Individuals, who score high in self-reported intolerance of uncertainty (IU), tend to find uncertainty anxiety-provoking. IU has been reliably associated with disrupted threat extinction. However, it remains unclear whether IU would be related to disrupted extinction to other arousing stimuli that are not threatening (i.e., rewarding). We addressed this question by conducting a reward associative learning task with acquisition and extinction training phases (n = 58). Throughout the associative learning task, we recorded valence ratings (i.e. liking), skin conductance response (SCR) (i.e. sweating), and corrugator supercilii activity (i.e. brow muscle indicative or negative and positive affect) to learned reward and neutral cues. During acquisition training with partial reward reinforcement, higher IU was associated with greater corrugator supercilii activity to neutral compared to reward cues. IU was not related to valence ratings or SCR’s during the acquisition or extinction training phases. These preliminary results suggest that IU-related deficits during extinction may be limited to situations with threat. The findings further our conceptual understanding of IU’s role in the associative learning and extinction of reward, and in relation to the processing of threat and reward more generally.


Procedure
On the day of the experiment, participants were informed about the experimental procedures upon arrival at the laboratory. Participants were then seated in the testing booth and asked to complete a consent form and a series of questionnaires (see section 2.4 below) on the computer screen.
After the questionnaires, physiological sensors were attached to the participants left hand and left corrugator supercilii. Before the task began, participants were played the sound stimulus through the headphones, so they knew what to expect. Participants were instructed: (1) to maintain attention to the task by looking at the geometrical shapes, (2) to use the keyboard for the ratings, (3) that the '£' symbol and sound represented a value of £1, and (4) that the £5 from taking part was separate from the money acquired during the task, and that they would receive the total amount of money at the end of the experiment. The conditioning task (see section 2.3 below for details) was presented on a computer screen whilst skin conductance response, corrugator supercilii and valence ratings were recorded.
To maintain uncertainty, participants were not instructed about the CS-US contingency or the total amount of money that could be acquired during the task. The total amount of money that could be won during the task was fixed at £5. Therefore, all participants received a total £10 (£5 from taking part and £5 acquired from the task) at the end of the experiment.

Conditioning task
The conditioning task was designed using E-Prime 2.0 software (Psychology Software Tools Ltd, Pittsburgh, PA). Participants were sat approximately 60 cm from the screen. Visual stimuli were yellow and blue squares displayed on a black background. To represent monetary reward, the presentation of a '£' symbol and a 1000 ms 70 dB sound of coins dropping served as the US (Kruse, Klein, Tapia León, Stark, & Klucken, 2020;Kruse et al., 2018;Kruse et al., 2017;Tapia León, Kruse, Stark, & Klucken, 2019).
The task comprised of two phases: acquisition and an immediate extinction training. During acquisition training, one of the squares was paired with the US 50% of the time (CS+), whilst the other square was presented alone (CS-). During extinction training, the squares were presented in the absence of the US. There was no break between the acquisition and extinction training phases. Conditioning contingencies were counterbalanced.
The acquisition training phase consisted of 16 trials (4 CS+ paired, 4 CS+ unpaired, 8 CS-). The extinction training phase comprised of 32 trials (16 CS+ and 16 CS-), where early is defined as the first 8 CS+/CS-trials and late is defined as the last 8 CS+/CS-trials. Experimental trials were pseudo-randomised such that the first trial of acquisition training was always paired and then after each trial type could only be played 3 times in a row. The squares were presented for a total of 6 seconds.
After this, a blank black screen was presented for 8-12 seconds. During reinforced trials, presentation of the square coterminated exactly with the presentation of the US.
Participants were presented with two other 9-point Likert scales at the end of the experiment. Participants were asked to rate: (1) the valence and (2) the arousal of the stimuli (i.e. the '£' symbol and sound stimulus combined). These scales ranged from 1 (Valence: very negative; Arousal: calm) to 9 (Valence: very positive; Arousal: excited).

Questionnaires
To assess intolerance of uncertainty and trait anxiety, we administered the

Skin conductance acquisition and scoring
Identical to previous work (Morriss, 2019), physiological recordings were obtained using AD Instruments (AD Instruments Ltd, Chalgrove, Oxfordshire) hardware and software. Electrodermal activity was measured with dry MLT116F silver/silver chloride bipolar finger electrodes that were attached to the distal phalanges of the index and middle fingers of the left hand. A low constant-voltage AC excitation of 22 mVrms at 75 Hz was passed through the electrodes, which was connected to a ML116 GSR Amp, and converted to DC before being digitized and stored. A PowerLab 26T Unit Model was used to amplify the skin conductance signal, which was digitized through a 16-bit A/D converter at 1000 Hz. The electrodermal signal was converted from volts to microSiemens using AD Instruments software (AD Instruments Ltd, Chalgrove, Oxfordshire).
Skin conductance response onsets and offsets were marked using ADinstruments software (AD Instruments Ltd, Chalgrove, Oxfordshire) and extracted using Matlab R2017a software (The MathWorks, Inc., Natick, Massachusetts, United States). Skin conductance response onsets and offsets were assigned using a macro in ADinstruments and then visually inspected to ensure the onsets and offsets were assigned correctly. Skin conductance responses (SCR) were scored when there was an increase of skin conductance level exceeding 0.03 microSiemens (Dawson, Schell, & Filion, 2000). The amplitude of each response was scored as the difference between the onset and the maximum deflection prior to the signal flattening out or decreasing. SCR onsets and respective peaks were counted if the SCR onset was within 0.5-4 seconds (CS response) following CS onset (Bauer et al., 2020;Morriss, Macdonald, & van Reekum, 2016). Trials with no discernible SCRs were scored as zero. SCR magnitudes were then square root transformed to reduce skew and z-scored to control for interindividual differences in skin conductance responsiveness (Ben-Shakhar, 1985). SCR magnitudes were calculated from remaining trials by averaging SCR-transformed values for each condition (Acquisition CS+; Acquisition CS-; Extinction Learning CS+ Early; Extinction Learning CS-Early; Extinction Learning CS+ Late; Extinction Learning CS-Late).
Non-responders were defined as those who responded to 10% or less of the total CS+ and CS-trials across acquisition and extinction training (48 trials in total) (Morriss, Chapman, Tomlinson, & Van Reekum, 2018;Xia, Dymond, Lloyd, & Vervliet, 2017). Using this criterion, 2 non-responders were excluded from the SCR analyses, leaving 56 participants with useable SCR data.

Corrugator supercilii acquisition and scoring
The protocol for corrugator acquisition and scoring was in line with previous research Amp. Before placing the sensors, the skin site was slightly abraded with isopropyl alcohol skin prep pads to reduce skin impedance to an acceptable level (below 20 kΩ). EMG was sampled at 1000 Hz. A high-pass filter of 20 Hz was applied to the raw EMG online (Solnik, DeVita, Rider, Long, & Hortobágyi, 2008). The EMG were root mean squared offline (Fridlund & Cacioppo, 1986). For the corrugator supercilii data, there was a recording error for 1 participant, thus, leaving 57 participants with useable corrugator supercilii data.

Analyses
The analyses were conducted using the mixed procedure in SPSS 24.0 (SPSS, Inc; Chicago, Illinois). We conducted separate MLMs for SCR magnitude, corrugator supercilii activity and valence ratings during acquisition and extinction training. For SCR magnitude and valence ratings during the acquisition training phase we entered Stimulus (CS+, CS-) at level 1 and individual subjects at level 2. For SCR magnitude and valence ratings during the extinction training phase we entered Stimulus (CS+, CS-) and Time (Early: first 8 CS+/CS-trials, Late: last 8 CS+/CS-trials) at level 1 and individual subjects at level 2. For corrugator supercilii activity, an additional factor of Second (time bins: 1,2,3,4,5,6) at level 1 was included in the MLMs. We included individual difference predictor variables (IU and STICSA) in the MLMs. In all models, we used a diagonal covariance matrix for level 1. Random effects included a random intercept for each individual subject, where a variance components covariance structure was used. Fixed effects included Stimulus and Time. We used a maximum likelihood estimator for the MLMs.
The IUS and STICSA covariates were entered separately. If there was a significant interaction with one of the predictor variables (IUS, STICSA), then we conducted a further MLM with both predictor variables entered to test specificity.

Results
For descriptive statistics see Table 1.
The hypothesis was that if IU-related effects during extinction are driven by the arousingness of an uncertain stimulus, then higher IU, relative to lower IU, would be associated with larger conditioned responding to reward vs. neutral cues during extinction training. However, IU was not associated with any of the rating or physiological measures during reward extinction training. In relation to the broader literature on IU and threat extinction, the results from the current experiment suggest that IU-related deficits in extinction are limited to situations with uncertain threat (Dunsmoor et al., 2015;Morriss, 2019;Morriss & van Reekum, 2019) and may not occur in the same way for situations with uncertain reward, despite potential similarity in arousingness. Importantly, however, this is only a single study and further evidence is warranted to make stronger conclusions on the role of IU in reward extinction. For example, perhaps, we would observe IU-related effects during reward extinction if the reward was more arousing (i.e. larger monetary rewards) or motivationally relevant in relation to a threat or loss to the self (i.e. money when poorer, food when hungry, and substances such as nicotine or alcohol when addicted).
In the current study we show that individuals with high IU are able to acquire and extinguish uncertain reward. These results may have implications for researchers examining counterconditioning (i.e. replacing threatening outcomes with rewarding outcomes) as an alternative to threat extinction training (i.e. replacing threatening outcomes with nothing) (Keller, Hennings, & Dunsmoor, 2020;Pittig, 2019) more broadly, and in relation to IU. Counterconditioning may be more effective than standard extinction training for individuals with high IU because during counterconditioning the new reward association is explicitly reinforced (i.e. a reward is given), whereas during standard extinction training the new association is less obvious (i.e. nothing happens). On this basis, counterconditioning vs. standard extinction training may be less distressing and therefore lead to more effective removal of old threat associations in individuals with high IU.
In conclusion, higher IU was associated with greater corrugator supercilii activity to neutral vs. reward cues during acquisition training. However, IU was not related to any other measure during the reward acquisition or extinction training phases. These initial results further our conceptual understanding of IU in reward associative learning, and in relation to the processing of reward and threat more generally. In order to assess the generalisability and reliability of the results reported here, further research is needed to examine how individual differences in IU modulate reward acquisition and extinction (i.e. vary levels of uncertainty and reward).  Expectancy ratings, 1 = dislike, 9 = like. Square root transformed and z-transformed SCR magnitude (μS), skin conductance magnitude measured in microSiemens. Zscored corrugator supercilii activity (μV), measured in microVolts. Figure 2. Bar graphs depicting IU estimated at + or -1 SD of mean IU (controlling for STICSA) from the multilevel model analysis for corrugator supercilii activity during the acquisition phase of the associative reward learning task (A and B). Higher IU, relative to lower IU, was associated with greater corrugator supercilii activity to the CS-, compared to the CS+ during reward acquisition. Bars represent standard error at + or -1 SD of mean IU. Z-scored corrugator supercilii activity (μV), measured in microVolts. Table 1. Summary of means (SD) for each dependent measure as a function of condition (CS+ and CS-), separately for acquisition, early extinction and late extinction.