Self-Report Methods for Assessing Emotions: Understanding Structure and Accuracy of Measurement
Onur Çalışkan, Aycan KapucuEmotion is structurally multifaceted with different components. One of these components, subjective experience, is pivotal in defining and measuring emotions. Self-report methods are used to ascertain the subjective experience of emotion. Unlike the neural and behavioral components of emotion, the qualities in subjective experience have been analyzed relatively less. However, for a more accurate and valid determination, the relevant structure should be well conceptualized. In this sense, the first part of this review precisely refers to the concept of emotional experience. This section concerns the distinctive meaning of subjective experience and details the correlation of subjective experience with other components of emotion. Next, the methods of accessing and expressing this experience will be discussed. Finally, the various scales in literature for measuring subjective experience are discussed. The status of these scales in Turkish and international literature is summarized. After conceptualizing subjective experience, the second part includes crucial aspects that may be useful for a more accurate and valid measurement. For this purpose, temporal issues that are important in measurement are addressed. First, the results of the temporal position of measurement, which may be in the past, future, or online; the results of the differences in the temporal emphasis in the instruction and the mechanism by which it affects the determination are mentioned. Subsequently, systematic explanations of individual differences in the experience of emotion are emphasized. Emotional granularity, referring to the extent to which an emotion is expressed in detail, and emotional dialecticism, indicating the state of experiencing different emotions together, are explained. Individual differences and their potential contribution to measuring and understanding emotion were addressed. The different views on the subjective experience of emotion, the temporal aspects, individual differences that can matter for measurement, and the mechanisms by which they can contribute to determination practices were discussed.
Duygu Ölçmede Kullanılan Öz Bildirim Yöntemleri: Neyi Ölçer? Nasıl Doğru Ölçer?
Onur Çalışkan, Aycan KapucuDuygu farklı bileşenleri olan, çok yönlü bir yapıya sahiptir. Bu bileşenlerden biri olan öznel deneyim hem duygunun tanımında hem de ölçülmesinde önemli bir yer tutmaktadır. Duygunun öznel deneyiminin ölçümünde ise öz bildirim yöntemleri kullanılmaktadır. Fakat literatüre bakıldığında duygunun nöral ve davranışsal bileşenlerinin aksine öznel deneyimindeki niteliklerin görece daha az incelendiği görülmektedir. Oysa ölçümün daha doğru ve geçerli yapılabilmesi için ilgili yapının önce iyi kavramsallaştırılmış olması gerekmektedir. Bu anlamda derlemenin birinci kısmı duygu deneyiminin içeriğinin tam olarak neye tekabül ettiğine ilişkindir. Bu kısımda ilk olarak öznel deneyimin diğer duygu bileşenleriyle olan ilişkileri ve tek başına öznel deneyimin ne ifade ettiği detaylandırılmıştır. Sonrasında ise deneyime erişimin ve bu deneyimin ifadesinin nasıl mümkün olduğundan bahsedilmiştir. Son olarak öznel deneyimin ölçümünü için geliştirilmiş olan literatürde yaygın şekilde kullanılan farklı ölçekler üzerinde durulmuştur. Hem ulusal hem uluslararası literatürdeki ölçeklerin güncel durumları özetlenmiştir. Öznel deneyimin kavramsallaştırılmasının ardından ikinci kısım, daha doğru ve geçerli bir ölçüm için fayda sağlayabilecek önemli hususları içermektedir. Bu amaçla derlemenin ikinci kısmında öncelikle ölçümde dikkat edilmesi gereken zamansal hususlara yer verilmiştir. Önce ölçümün zamansal pozisyonunun geçmişte, gelecekte veya anlık olmasının sonuçlarına, sonrasında yönergede zamansal vurgudaki farkların sonuçlarına ve ölçümü nasıl etkilediğine dair çalışmalara değinilmiştir. Ardından duygunun deneyiminde yaşanan bireysel farklılıkların sistematik açıklamaları üzerinde durulmuştur. Duygunun ne ölçüde detaylı bir şekilde ifade edildiğine işaret eden duygusal detaycılık, farklı duyguların beraber yaşanma durumunu ifade eden duygusal diyalektisizm kavramları açıklanmıştır. Bu bireysel farklılıkların duygunun hem ölçümü hem de anlaşılmasına nasıl bir katkı sağlayabileceği ele alınmıştır. Tartışmada, duygunun öznel deneyimine dair farklı görüşler, zamansal hususların ve bireysel farklılıkların ölçüm için ne ifade edebileceği, ölçüm pratiklerinde nasıl katkılar sağlayabileceği tartışılmıştır.
Scientists define a neural loop, a neurobiological process, a phenomenological experience and feeling, and a perceptual-cognitive process (Izard, 2010). Emphasizing such different aspects of emotion leads to various techniques used to measure and evaluate these aspects in research.
Third-person techniques involve behavioral, neural, and physiological determination of emotion. These techniques provide objective data that are equally accessible. On the other hand, first-person methods are based on the subjective experiences of people. These methods are based on self-reports by the person experiencing the phenomenon. The selfreport method assesses the phenomenological aspect of emotion based on subjective experience.
Searle’s biological naturalism (Searle, 1992, 2000, 2004) provides a framework for scientifically examining emotional experience (Barrett et al., 2007). The first principle expounds on the reasons and distinct features that differentiate the content of emotional experiences from those of other experiences. The second principle emphasizes that the content of emotional experience cannot be entirely reduced to causes, as it is associated with specific brain systems. The third principle underscores that conscious experiences occur solely from an individual’s perspective and are inherently subjective, making them inaccessible to evaluation through third-person techniques. The purpose of science is to evaluate these subjective experiences in an epistemologically objective way (Barrett et al., 2007).
Understanding emotional experience requires distinguishing between the raw phenomenological and the reflective consciousness aspects (Nielsen & Kaszniak 2006). James (1890, 1983) characterized this distinction as the “feltness of a feeling” and its “perception by a subsequent reflective action,” while Lambie and Marcel (2002) frame it as “primary-order phenomenological experience” and “secondary-order awareness” (as cited in Nielsen & Kaszniak, 2006, p. 364). ” The raw phenomenological aspect deals with the part that defines the natural form of an emotion. Reflective consciousness enables the collection of essential information to form the experience in a cognitive layer that includes the environmental and situational information, thoughts, and evaluations at the time of the experience.
The dimensional approach to emotions focuses on two primary components: valence and arousal (Russell, 1980; Barrett & Russell, 1999). Valence determines the positive and negative qualities of an emotion by reflecting the pleasurable tone of an experience. This dimension is related to raw phenomenology (Barrett et al., 2007). Arousal represents the change in physiological activity; however, the extent to which this dimension corresponds to natural phenomenology is unclear, as cross-cultural studies have revealed that this dimension is represented in different languages (Barrett et al., 2007). Based on appraisal theories, emotions arise from evaluations, and subjective experience includes their reflection (Moors, 2020). The contents reflected in consciousness need not be labeled with a specific category of emotions. While emotional experience includes evaluative dimensions and a reflection of outcomes into consciousness, it can also be experienced without emotion category labeling.
Emotional experience has various neural and behavioral correlations. Using thirdperson methods for measuring experiences does not fully reflect the experience content of emotions since experience can exist independently of its other correlates. For example, in one study, Parkinson’s Disease patients with problems forming facial expressions reported experiences involving emotional valence and intensity (Reid, 2000). In addition, the content of the experience can only be accessed by the person concerned (Barrett et al., 2007). Because of this ontological reason, accessing this knowledge is possible via the self-reports of the experiencer.
Self-report methods in emotion-related research are grounded in two fundamental theoretical perspectives: the dimensional approach, exemplified by the circumplex model (Russell, 1980), and the discrete emotions perspective (Ekman, 1999; Izard, 2007; Panksepp, 2007). The former defines emotions along valence and arousal dimensions, as observed in measures like the Self-Assessment Manikin (Bradley & Lang, 1994) and Affect-Grid (Russell et al., 1998). The latter posits specific emotions as distinct categories, as reflected in instruments like the Differential Emotions Scale and the more recent Discrete Emotions Questionnaire (Harmon-Jones et al., 2016a).
Positive and Negative Affect Schedule [PANAS] (Watson et al., 1988) is widely used for measuring emotions by assessing the intensity of positive and negative feelings through Likert scales. An updated version, PANAS-X (Watson & Clark, 1994), expands the evaluated expressions to include discrete emotions along with valence. Profile of Mood States [POMS] (McNair et al., 1992) and the Multiple Emotion Adjective Checklist [MAACL] (Zuckerman & Lubin, 1965) are scales that can be associated with the discrete emotion view and are primarily used in the field of psychiatry.
Different practices stand out during the measurement of emotional experience. In this direction, various research findings on the mechanisms by which the process of measuring temporal issues and individual differences can increase validity are discussed in the present article.
Temporal Issues
Robinson and Clore (2002) argue that in the accessibility model, in addition to experiential knowledge, beliefs about emotion also play a role in reporting emotions. Four different sources of information about emotional experience are used in self-reports: a) online assessments of feelings, b) episodic information, c) semantic information including situation-specific beliefs, and d) semantic information including identity-related beliefs (p. 937). According to the accessibility model, the source of information for online emotion reports is experiential emotions. Conversely, episodic information presents non-experiential information due to peak bias or recency effect. The model states that the missing details are complemented by semantic information if episodic information is inaccessible. In summary, biases in episodic memory and information from semantic memory cause different types of information to be reported rather than the original experiential information in retrospective reports.
Harmon-Jones and colleagues (2016b) investigated the effects of the change in temporal emphasis in the instructions given to the participants before the scale could make a difference in self-reports. In this study, the participants were asked what they felt at that moment or during the event that evoked an emotion, and it was found that the participants reported their emotions more intensely in the “during” condition than in the “right now” condition.
Individual Differences
Emotional granularity (Tugade et al., 2004) or emotional differentiation (Barrett et al., 2001) is the ability to characterize emotional experiences verbally. While someone with a high level of detail may prefer various representations with different qualities referring to the related emotion when reporting their experience (e.g., for fear: “shudder,” “terrified,” and “frightened”), someone with low detail may prefer more comprehensive and general expressions (e.g., for fear: “feeling so bad”).
Emotional dialecticism refers to the co-occurrence and temporal relations between positive and negative emotional experiences (Lindquist & Barrett, 2008). Differences in affective dialecticism are associated with the degree to which a culture indicates an interdependent or independent structure (Bagozzi et al., 1999). Resilience to stress (Rafaeli et al., 2007), cognitive complexity (Reich et al., 2001), age (Carstensen et al., 2000), and gender (Bagozzi et al., 1999) are also factors that impact dialecticism to varying extent.
Discussion
Subjective experiences, including neural and behavioral components, play a significant role in explaining emotions. While the neural and behavioral correlates of experience provide insights, they cannot capture the rich phenomenological content of emotions, as individuals experience these first-hand. Despite the epistemic advantage of direct access to mental experience through self-report methods, the literature suggests that third-person methods can also be employed to measure mental experience (Pauen & Haynes, 2021). The argument posits that third-person methods are currently in an early stage of development for this purpose; but their maturity in the future will be achieved through methods like triangulation (Pauen & Haynes, 2021).
In terms of measurement, two aspects of emotional experience need attention. First, considering existing standardized measurement methods, self-reporting appears to be the singular and optimal approach, which aligns with the perspective of Pauen and Haynes (2021). Second, Pauen and Haynes (2021) suggested that if third-person methods undergo sufficient development, they could enhance the accuracy and validity of the measurement, possibly providing better measurements than self-report methods. However, if the essence of the measurements concerns the “experience of emotion,” they would lack construct validity.
This highly precise measurement provides information other than the phenomenology of experiencing green color when observing a leaf. While spectroscopy accurately measures the biocomponent triggering the experience, it differs from quantitatively determining the wavelength of light reflected from the leaf. In this context, emotional experience is more nuanced because it is richer in content than color experience. Although third-person methods offer precise measurements, they will miss a significant part of the picture.
Measuring the time interval of the emotional experience seems to be the most accurate way to access “raw” information. Participants should be clearly instructed about the state of the range of emotional experiences measured in the study. Creating a framework for the scales in use or which need to be constructed based on the information provided by individual differences would contribute uniquely to their differences outside of their theoretical foundation.
A clear conceptualization of the subjective experience measured by these methods and an exploration of measurement procedures can be further improved, which is essential. A refined conceptualization of the subjective experience of emotion would also enhance the contribution of third-party methods to the measurement process. Future studies should focus on integrating the prevalent third-party techniques with self-report methods to improve the validity of emotional measurement.
