 |
 |
 |
 |
|
|
|
|
|
Measuring emotional intelligence: Victor Dulewicz and Malcolm Higgs Mark Slaski Abstract Many authors claim there is a paucity of evidence for
the validity of measures of emotional intelligence (El). This paper summarises
existing information on the reliability and validity of two measures of
El, the Dulewicz and Higgs ElQ and the Bar-on EQ-i. It also reports the
results of a study on middle managers which investigated the degree to
which these two El instruments measure the same constructs: their concurrent/criterion-related
validity; and the relationship between El and morale and stress at work
Correlations between the two instruments showed content and construct
validity, with 16 out of the 20 hypothesised relationships between scales
being significant. Correlations between various measures of morale and
stress at work and ElQ demonstrated construct validity. Significant relationships
were also found between ElQ and current job performance, thus providing
further evidence of concurrent/criterion-related validity. Introduction Great interest has been shown in recent years in the topic of emotional intelligence (EI), stimulated by Goleman's (1996) book, and in particular the assertion that EI explains a higher proportion of variance in individual success than IQ. As long ago as the 1920s, Thorndike (1920) reviewed the predictive power of IQ and subsequently developed the concept of "social intelligences" to explain aspects of success which could not be accounted for by IQ. However, it was not until the early 1980s that Gardner (1993) resurrected interest in factors other than IQ which may influence individual success. In an educational context, he developed and explored the concept of multiple intelligences. In particular, his "personal intelligence" included inter-personal, self-awareness and emotional traits. Gardner's (1993) work was cited by Goleman (1996), as was Salovey and Mayer's (1990). The latter mapped the way in which we can bring intelligence to our emotions and first coined the term emotional intelligence. While Goleman (1996) debated the feasibility of operationalising the construct, Bar-On (1997a, b), Mayer et al (1999) and Dulewicz and Higgs (2000a) produced questionnaires which are widely used in the USA and UK to measure the construct. There appears from the literature to be some debate about what constitutes the domain of emotional intelligence, about terminology used to describe the construct and about methods used to measure it. Maver et al (1999) desiImed a questionnaire to measure abilities, a feature which distinguished their work from others. "Our focus is on ability measures of EI because we view these instruments as the most promising" (Mayer et al, 2000). Goleman (1998) presented a model with 25 competencies, derived from the Hay-McBer consultants' competencies framework, arrayed in 12 clusters and these have since been reduced empirically to 20 competencies to form the Emotional Competencies Inventory (Boyatzis et al, 2000). Other authors (e.g. Bar-On, 1997a; Dulewicz and Higgs, 2000a) have produced questionnaires and models derived from empirical research into personal factors related to EI, and particularly into "emotionally and socially competent behaviour" (Bar-On, 2000, p. 364). Henceforth, the authors will refer to this as the "EI personal factors model" as distinct from the emotional intelligence ability model. Some authors claim EI is a marketing term which is impossible to measure. For example, Steiner (1997, p. 23) comments that:
Woodruffe (2001) believes that the construct is not new but simply a new brand name for a set of long-established competencies. Furthermore, some claim there is no evidence for its validity. Robertson and Smith (2001) state that:
Luthans (2002) points out the relatively weak theory development,
research and measures of EI but also stresses its potential importance
for leadership effectiveness, human resource (HR) performance improvement
and other applications. Davies et al (1998) suggest that emotional intelligence
is not a true intelligence and is best viewed as a cluster of personality
traits. This view provides support for the EI personal factors model. The basis for the concept of emotional intelligence I is by no means a new concept (see Thorndike, 1920; Gardner, 1993). While there are a number of precursors to the concept of emotional intelligence the broad psychological concept was first described by Salovey and Mayer (1990). In general they proposed that individuals vary in their capacity to process information of an emotional nature and their ability to relate these to a wider cognition. This ability is seen to manifest itself in certain adaptive behaviours (Mayer et aL, 2000). In exploring this concept the presence of psychological disorders such as alexithymia (Sifneos, 1991; Bagby et aL, 1994) are presented as supportive evidence. A significant element of this assertion is supported by current research into the functioning of the human brain (e.g. LeDoux, 1994, 1998). Goleman (1998) proposed that high levels of EI were associated with success in a business context It is suggested that "emotionally intelligent" individuals can perceive, understand and regulate the emotions of others, thus making emotional intelligence a significant factor in the success of interpersonal interaction in a work context (Mayer et al, 2000). In exploring this construct Bar-On (2000) saw emotional intelligence as being a:
In broad terms Salovey and Mayer (1990) and subsequent researchers in the field (e.g. Bar-On and Parker, 2000; Goleman, 1996; Dulewicz and Higgs, 2(XX)a, b, c) have somewhat different conceptualisations of emotional intelligence. These result in different approaches to the operationalisation of the construct while acknowledging a shared view of the roots of its place within an organisational context (Mayer et al, 2000; Dulewicz and Higgs, 2000a ,b, c). These differences have led to an important debate (e.g. Mayer et al, 2000; Woodruffe, 2001; Dulewicz & Higgs, 2000a, b, c). The authors contend that EI elements are personal, not ability factors, based in part on the results of studies reported in the next section showing clear links between many EI elements and personality factors as measured by personality questionnaires. Dulewicz and Higgs (1999) define emotional intelligence as being concerned with:
The study reported is one part of a much larger investigation
into the quality of working life and stress of managers in a large retail
organisation (Slaski and Cartwright, 2002, 2003) in which some participants
completed two different measures of emotional intelligence, the Dulewicz
and Higgs EIQ and the Bar-On EQ- i. Before presenting the study, the two
instruments used to measure emotional intelligence will be described and
existing evidence of their reliability and validity will be presented,
in response to claims cited above that such evidence does not exist The two EI instruments The EIQ In an initial exploratory study, Dulewicz and Higgs (2000b) found that, on a sample of general managers, an EI scale based on 16 relevant competencies showed promising reliability and predictive validity over a seven-year period. Building on this study and on an extensive literature review, and in order to move on from competencies' assessment, a tailored questionnaire (the EIQ) was designed to specifically assess through self-report seven elements of an individual's emotional intelligence (Dulewicz and Higgs. 1999: 2000a): (1) self-awareness: being aware of one's feelings and
managing them; Full definitions are provided by Higgs and Dulewicz (2002); Dulewicz and Higgs (2000a). In view of the nature of the EI construct, a 3600 version of the EIQ was constructed in a similar way to the original version of the questionnaire (Dulewicz and Higgs, 2000a, c). As part of the development of the questionnaire, Cronbach alpha reliability co-efficients for each of the element scales ranged from 0.6 to 0.8. The alpha for the overall EIQ score derived from the seven elements was 0.77 (Dulewicz and Higgs, 2000a) Before designing the EIQ, its authors conducted a comprehensive review of the existing literature from significant contributors in the field (e.g. Gardner, 1993; Gardner and Hatch, 1989; Salovey and Mayer, 1990; Goleman, 1996, 1997; Steiner, 1997; Cooper and Sawaf, 1997). This extensive review of 72 books and articles, which is summarised by Dulewicz and Higgs (2000b), was undertaken to ensure that all of the critical elements of Emotional Intelligence were represented so that the questionnaire had acceptable content validity. The development study of EIQ demonstrated its construct validity in relation to the occupational personality questionnaire; the 16PF personality questionnaire; Belbin team roles derived from the 16PF; and the Myers Briggs type inventory (Dulewicz and Higgs, 1999; 2000a). A study of team leaders (Dulewicz and Higgs, 2000a, c) provided clear evidence of the concurrentJcriterion-related validity of the EIQ against measures of current performance. The total EIQ score was highly significantly related to the performance measures, as were six of the elements (except interpersonal sensitivity). The results from the 360° version of EIQ (Dulewicz and Higgs, 2000a, c) provided further support. In a study of sales staff, (Dulewicz and Higgs, 2000a, c), self assessments on two elements were related to job performance, but 3600 assessment showed that four elements and the overall EIQ score were significantly correlated with performance. Indirect links to the original predictive validity study on general managers reported above was provided in the development study of the EIQ (Dulewicz and Higgs, 1999). The overall EIQ score correlated highly significantly with the overall EI competencies scale used in the original study. In the team leaders study reported above, the EIQ total score was highly significantly correlated with the EI competencies total score (r = 0.66; P < 0.01). Furthermore, the EI Competencies scale itself was highly significantly related (r = 0.80; P < 0.01). to overall job performance (Dulewicz and Higgs, 2000a). These findings provide the critical link with the original study of general managers, reported above, and so produced indirect evidence of predictive validity over a seven-year period. The EQ-i instrument The structure of the EQ-i is based on the literature and its author's research experience as a clinical psychologist (Bar-On, 1997a). The concept was theoretically developed from logically clustering variables and identifying underlying key factors purported to determine effective and successful functioning as well as positive emotional health (Bar-On, 1997b). The EQ-i produces a total EQ score, five composite scale scores, and 15 sub-scale scores, defined by Bar-On (1997a). This structure is presented in Table I. A number of factor analyses were performed and provided empirical support for the 1-5-15 structure of the EQ-i. Therefore, the EQ-i presents a hierarchical structure of emotional intelligence (Bar-On, 1997a).
The internal reliability of the EQ-i was examined using Cronbach's alpha (Bar-On, 1997a). The internal consistency coefficients for the EQ-i sub-scales, based on seven different samples ranged from 0.70 to 0.89, and thus demonstrated good reliability. Test-retest reliability has also been examined with two groups. Reliabilities for a one-month study ranged from 0.78 to 0.92, and for a four-month study from 0.55 to 0.87 (Bar-On, 1997a). Construct validity for the EQ-i has been extensively examined by correlating the inventory's sub-scale scores with various scale scores of other personality, mental health and job satisfaction measures. According to its author, the coefficients are high enough to give ample support that the EQ-i sub-scales are measuring the constructs that they were intended to measure and yet not so high as to suggest that the EQ-i is a duplication of existing inventories (Bar-On, 1997b). In a study examining the relationship between emotional intelligence (using the EQ-i), stress, well-being and performance on a sample of 224 managers, those with higher scores on EQ-i reported significantly lower stress and distress, significantly higher morale and quality of working life, and significantly better health and work performance than managers with lower EI (Slaski and Cartwright, 2002). Concurrent/criterion-related validity of the EQ-i was
demonstrated in the preliminary stage of the study reported below, derived
from statistically significant correlations between measures of job performance,
described below, and EQ-i scores (Slaski and Cartwright, 2002). Relationship between the EIQ and EQ-i As noted above, while the Mayer et at. (1999) questionnaire is designed to measure abilities, and Goleman's emotional competencies inventory (Boyatzis et at., 2000) underlying competencies, the EIQ and EQ-i measure constructs which are more akin to those personal factors associated with emotionally and socially competent behaviour. Although these two instruments have been developed from different conceptual positions, there does appear to be some degree of relationship between them in terms of the factors based on a content analysis. These hypothesised relationships are presented in Table I and suggest a reasonably high conceptual overlap between the two questionnaires. In particular, the first two components of the EQ-i are intra- and inter-personal factors and the EIQ could also be seen to cover intra- and inter-personal domains. The intra-personal cluster in the EIQ includes the elements self-awareness, emotional resilience, motivation, intuitiveness and conscientiousness; the inter-personal cluster includes inter-personal sensitivity and influence. A strong relationship between these elements of EIQ and the intra- and inter-personal clusters of the EQ-i is hypothesised. The study reported below explores whether existing EIQ validity results would be replicated on middle-management; tests the hypothesis that EIQ is related to morale and stress at work; and investigates whether two differentinstruments measure the same constructs/elements. Results from con-elations between the two instruments (content and construct validity), and between the EI measures and stress (construct validity) and current job perfomlance (concurrent/criterion-related validity) are reported and discussed. Method Sample Psychological health, stress and morale measures A subjective measure of stress asked respondents to rate "how stressed their life is at this time" on a scale of 1 to 10. This subjective measure reflects the phenomenological nature of the study and is supported by definitions of stress that include the individual's "perception" of stress as a central issue in a stress process (e.g. McGrath, 1976). In addition, it is hypothesised that stress is experienced as a collection of negative emotional and cognitive experiences such as frustration, anger, fear and anxiety. Consequently, the measures of distress and morale were considered to be appropriate. A widely-used measure of general health, the general health questionnaire (GHQ 28) (Goldberg, 1978), was also administered. This measure has proven reliability and validity and is specifically concerned with the "hinterland between psychological sickness and psychological health" (Goldberg, 1978, p. 6). It was constructed to focus on an individual's inability to carry out normal "healthy" functions, rather than the more severe disorders of psychological functioning such as psychotic depression or acute schizophrenia. The GHQ 28 comprises 28 items that ask respondents about recently experienced symptoms or aspects of behaviour. Response options are presented on a four-point scale ranging from "less than usual" to "much more than usual". Management performance measures Analysis Results From the correlations presented in Table ill, a large number (16) of the hypothesised links are supported while, in contrast, relatively few (four) were not borne out. These hypothesised links are reported in full in Table IV. An important distinction was noted above between the intra- and inter-personal elements of both insb"uments. Looking specifically at the correlations between the two EIQ inter-personal elements, sensitivity and influence, we find that all but one of the eight correlations with the EQ-i inter-personal factor are significantly correlated with each other. The exception is the relationship between social responsibility and sensitivity (not significant; p < 0.07). Interestingly, social responsibility is correlated at an even lower level with EIQ conscientiousness. Overall, three EIQ elements are statistically significantly correlated with five or fewer EQ-i elements. These are sensitivity, intuitiveness and conscientiousness, with the last of which appearing to be the one element that is not effectively measured by the EQ-i elements. Turning to the EQ-i elements, emotional self-awareness is correlated with only one element of EIQ and social responsibility with none. Looking finally at the overall EI level, we find that the total EIQ score is significantly correlated with the total EQ-i score (r = 0.633, sig. at 0.001 level) and with 11 of the 15 EQ-i elements. A multiple regression analysis with total EIQ score as the dependent variable and the 15 EQ-i elements as independent variables produced anR = 0.749 and anR2 = 0.559; and a multiple regression analysis with the five EQ-i main elements as independent variables produced an R = 0.671 and an R2 = 0.450. The EQ-i total is significantly correlated with all seven EIQ elements, four of which, self-awareness, resilience, motivation and influence, are at a very high level (0.001) of significance. A multiple regression analysis with total EQ-i score as the dependent variable and the seven EIQ elements as independent variables produced an R = 0.647 and an R2 = 0.419. The standardised beta co-efficients were all positive. Taking these results together, there appears to be a high overlap between the domains of EI measured by the two different instruments. Psychological health, stress and morale Correlations between emotional intelligence, as measured by the EIQ, and five different measures of psychological health, stress and morale are presented in Table V. The hypothesis that EI is related to such measures is borne out Total score on EIQ is significantly correlated at the 0.01 level with all five measures (GHQ, r = 0.400; stress, r = -0.604; distress, r = -0.380; morale, r = 0.505; QWL, r = 0.437). Looking beyond the total EIQ score, one finds that three of the EIQ elements, self-awareness, emotional resilience and motivation, make a major contribution to all five measures. All correlations between these elements and the five scales are statistically significant bar one (motivation and QOWL). The other four EIQ elements are not significantly correlated with psychological health (GHQ28), distress and stress (apart from conscientiousness). On the other hand, three elements -sensitivity, influence and conscientiousness - are significantly related to morale, which together with stress are the two characteristics most highly related to, and predicted by EIQ.
The final column in Table V shows the correlations between managerial perfornlance and EIQ total and element scores. The managers' bosses rated their perfornlance using the organisation's "critical success factors model" as the criteria. Once again, total EIQ score (r = 0.321) and three elements - self-awareness (r = 0.353), emotional resilience (r = 0.342) and motivation (r = 0.387) - were significantly related to job perfornlance. It is interesting to note that the two EIQ Inter-personal elements, sensitivity and influence, and two other elements, intuitiveness and conscientiousness, were not related to perfornlance. A multiple regression analysis, with the managerial perfornlance rating as the dependent variable and the seven EIQ elements as independent variables, produced an R = 0.548 and an R2 = 0.301. This suggests that self-perceptions of emotional intelligence alone account for 30 per cent of the variance in management performance.
The two EI instruments were developed in different ways and for different purposes. According to Bar-On and Parker (2000, p. 363), the EQ-i was:
In contrast, the EIQ was designed to measure the main components of EI, based on a literature review of the work of nine leading authors on EI, but not Bar-On (1997a, b, 2000), for use primarily in the world of work (Dulewicz and Higgs, 1999, 2000b). In view of these differences, it is perhaps surprising that the two instruments measure so much common variance although some of this could be due to "common method bias". The 15 EQ-i factor scores account for 56 per cent of the variance on the EIQ total and the five components' scores 45 per cent, while the seven EIQ elements account for 42 per cent of the variance on the EQ-i total. If one bears in mind the limit set by the overall reliabilities of the two instruments (0.77 for EIQ; 0.85 for EQ-i), they appear to be measuring very similar constructs, although a few elements appear to be unique. EIQ conscientiousness is not picked up by any of the hypothesised equivalent EQ-i elements while EQ-i social responsibility and emotional self-awareness do not appear to be measured by any equivalent EIQ elements. While social responsibility, defined as "the ability to demonstrate oneself as a co-operative, contributing and constructive member of one's social group" (Bar-On and Parker, 2000), is clearly a different construct from conscientiousness - being consistent in one's words and actions and behaving according to prevailing ethical standards (Dulewicz and Higgs, 2000a) - the two self-awareness constructs would be expected to be correlated. Perhaps the reason for the difference is that, while both self-awareness elements cover recognising and understanding one's feelings, the EIQ element also covers managing those feelings and emotions. In contrast, the inter-personal domains measured by the two instruments show significant correlations on five out of the six pairings. Furthermore, the overall inter-personal component on EQ-i appears to be highly related to the aggregate score of the two inter-personal elements (influence and sensibility) on EIQ (r = 0.38, significant at the 0.001 level). In view of the importance of the inter-personal elements for management, in terms of achieving goals through other people, these results are particularly important. As noted above, the two instruments are different in terms of their development and intended purposes. While the EQ-i was designed to measure social and emotional constructs in all possible situations, this version of the EIQ was designed to focus on the core EI constructs for managers in the world of work. This perhaps helps to explain why the former has 15 primary elements and the latter only seven. Nevertheless, despite the aforementioned, these results do suggest that the two are measuring similar core EI constructs. The EIQ total score is significantly related to all five measures of psychological health and morale, but particularly so with the single item scale of current stress and the morale scale which measures feelings of pride, happiness and energy. The latter is also significantly related to all but one, intuitiveness, of the seven EIQ elements and the former with four elements, the most important of which are emotional resilience and self-awareness (the two elements which are specifically concerned with feelings and emotions). These two intra-personal elements are also significantly related to the other three measures, general psychological health, distress and quality of working life, and along with motivation appear to be the main contributors to psychological well-being. It is perhaps not surprising that the two inter-personal elements have less affect on what is actually an internal state. Finally, it is interesting to note that conscientiousness is highly related to morale and, to a slightly lesser extent, to lack of stress, but not to distress, psychological health (GHQ) and quality of working life. Conscientious individuals are indeed likely to feel pride in their work, to be happier and more energetic and to feel less stressed, but the element is probably not relevant to broader psychological health, self -actualisation and feelings of anxiety or fright. Similar findings emerged from the preliminary study. Slaski and Cartwright (2002) found that these measures of psychological health, stress and morale were significantly correlated with EQ-i total and most of the sub-scales. Their findings provide further support for the construct validity of the broader concept of emotional intelligence. The form used by the bosses to rate managers' performance contained 16 critical success factors, only four of which broadly appear to be EI factors. Most of the others could be seen to be "intellectual" competencies with a smaller number of "managerial" competencies making up the total. It is, therefore, perhaps surprising that as much as 30 per cent of the variance on managerial performance is accounted for by the EIQ elements alone. The overall EIQ score is highly correlated with performance, as well as the elements self-awareness, emotional resilience and motivation. The last two elements were also found in previous studies to be related to team leader performance and sales staff performance. Self-awareness (and three other elements) was also correlated with team leader performance (Dulewicz and Higgs, 2000a, c). Therefore, these three intra-personal elements appear so far to be the ones most relevant for explaining performance. It is surprising that the inter-personal elements do not playa larger part (nor do the equivalent EQ-i elements, see below) but perhaps, as in the current study's measure of performance, these elements do not have a high weighting in the respective organisation's critical competencies frameworks. Further support for the concurrent/criterion-related validity of the broader concept of emotional intelligence was provided by Slaski and Cartwright (2002) who found that these measures of job performance were significantly correlated with EQ-i total and with all second-order sub-scales except for the inter-personal scales. Conclusion The main conclusion from this study is that there is clear evidence for the content and construct validity of the two instruments and therefore for the "the EI personal factors" model of emotional intelligence. In addition, further evidence is produced for the conCUlTent/criterion-related validity of the EIQ, on middle-managers. On the evidence reported here, the EIQ appears to be a more parsimonious measure of EI and has more job-related validity. It is therefore, perhaps a more appropriate tool for assessing EI in the world of work. For staff, and managers in particular, time is usually at a premium and "face validity" is of paramount importance - constructs assessed must be perceived as being relevant and meaningful. Inevitably with studies involving managers at this level, there are some limitations but none seem to be of major concern. The sample size of 59, with a minimum of 53 for one part of analysis, could be seen as being relatively small but nevertheless quite acceptable compared to many other similar studies involving managers. The study was also conducted in a single, private sector, retail organisation. Further research on EI and on the validity of the instruments used to measure it should include managers from other sectors, including the public sector, and organisations with different organisational cultures; and also involve non-managerial groups. References
|
||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|