VALIDITY AND RELIABILITY

Background
Validity and Reliability
SPQ*GOLD® Validity
Criterion Validity
Content Validity
Construct Validity
Predictive Validity
Reliability
Norms
Universal Application
Global Acceptance
Not About Personality
ReMap Evaluation

Background

The SPQ*GOLD® is specifically designed to assess behaviours associated with the process of selling; whether directly (salespeople selling products) or indirectly (for instance those who are required to ‘sell’ concepts and ideas, and who need to influence, persuade and negotiate.

Therefore, it is appropriate for anyone in a contact-dependent work setting, including experienced salespeople, new salespeople, sales managers, Directors, Marketers, trainers, consultants, psychologists, human resources practitioners and others who may influence the attitudes or behaviors of individuals in direct sales.

When a salesperson is hesitant to initiate contact with prospective buyers they are less likely to realize their full potential for success in sales. This hesitancy is referred to as Sales Call Reluctance® (SCR) and is measured by the SPQ*GOLD®.

The SPQ*GOLD® is a special purpose diagnostic instrument that exclusively measures the twelve types of SCR, three SCR ‘Impostors’, and three scales used to detect nonstandard attitudes toward completing the instrument.

This instrument was first developed in 1980 and has since been the subject of extensive research (over 900 studies) aimed at establishing its effectiveness when used as intended.

These studies make SPQ*GOLD® one of the most validated applications in use today for assessing salespeople.

The Intellectual Property Rights are owned by Behavioral Sciences Research Press, Inc., Dallas, Texas. ALL RIGHTS RESERVED

Validity and Reliability

The Relationship of Reliability and Validity.

Test validity is requisite to test reliability. If a test is not valid, then reliability is moot (i.e. of little or no practical value or meaning)

In other words, if a test is not valid there is no point in discussing reliability because test validity is required before reliability can be considered in any meaningful way.

Likewise, if a test is not reliable it is also not valid.

The SPQ*GOLD® is both Valid and Reliable.

SPQ*GOLD® Validity

Test validity refers to the degree to which the test actually measures what it claims to measure. For example a test of intelligence should measure intelligence and not something else (such as memory).

Test validity is also the extent to which inferences, conclusions, and decisions made on the basis of test scores are appropriate and meaningful.

An extensive body of research results has established good concurrent validity in a variety of disparate organizational settings, correlating significantly with objective outcome data (almost always measured
in pounds, euros or dollars).

SPQ*GOLD® has also demonstrated excellent discriminant validity between groups such as successful salespeople, university students, and technical non-sales personnel.

Criterion Validity

Criterion validity refers to the extent to which one measure estimates or predicts the values of another measure or quality.

Research on the validity of SPQ*GOLD® focuses on criterion referenced studies based on objective sales performance criteria, which is typically measured in pounds/euros/dollars or units sold.

These types of objective measurements represent hard criteria, which impact profitability directly. Research conducted with SPQ*GOLD® in various industries has repeatedly shown a systematic link between lower levels of SCR and higher sales productivity.

Content Validity

Content validity refers to how well a test measures the behaviour for which it is intended.

Cross-cultural studies have shown that the majority of individuals completing SPQ*GOLD® (approximately 90%) find that the test items are related to a career in sales. These results are statistically significant (p<.0001) and indicate that SPQ*GOLD® has good content validity.

These studies further supplement the criterion validity studies that indicate a link between SPQ*GOLD® scales and sales production.

Construct Validity

Construct Validity refers to the ability of a measurement tool (e.g., a survey, test, etc) to actually measure the psychological concept being studied. In other words, does it properly measure what it’s supposed to measure?

Construct validity studies have shown that SPQ*GOLD® subscale scores correlate significantly in expected directions with scales on established mainstream psychological instruments such as the 16PF, NEO, MBTI, CPI, MMPI and others. Additionally, SPQ*GOLD® also has demonstrated significant discriminant validity between groups such as successful salespeople, university students, and technical non-sales personnel.

Predictive Validity

Predictive validity can be defined as the extent to which the measure being used will allow you to predict future behaviors that this measure should be able to predict.

SPQ*GOLD® Global predictive validity studies on the profiling tool have shown an  extremely high 80% correlation between a non Sales Call Reluctant score and High Producers.  Similarly, studies have discovered a 75% correlation between a Sales Call Reluctant score and marginal producers.

ReMap’s own evaluation study of 445 candidates has shown very similar results at 82% correlation for High Producers.

Reliability

Test reliability refers to the degree to which a test is consistent and stable in measuring what it is intended to measure.

The SPQ*GOLD® has good test-retest stability, averaging r=.75 over all scales (18 of 21 scales were significant at p<.01, the remainder p<.05).

The stability of the Brake score, the overall diagnostic measure, is r=.91 (p<.01). Cronbach’s alpha, one estimate of internal consistency, for the Brake score is r=.84. Coefficient Alpha can vary somewhat because research has shown that the sales profession is not one-dimensional.

Radically dissimilar settings can exist within branch offices of the same company, across companies within the same industry and across industries.

In each case, the correlation of customer contact initiation to objective outcome measures can vary from extremely important (certain financial services companies) to considerably less important (some forms of retail sales.)

Norms

SPQ*GOLD® has been used in a series of studies with samples representative of many organizations, industries and countries.

The result is an array of normative data from 24 countries that include the U.S., Australia, New Zealand, Singapore, Hong Kong, Indonesia, China, Benelux, Malaysia, Sweden, Norway, Denmark, the U.K., Italy, Spain, South Africa, Finland, France, Japan, Korea, India, Ireland, Canada and South / East Asia

Universal Application

SPQ*GOLD® is particularly useful in providing assessment based support to supplement selection, training, and career development applications.

This measurement tool has universal applications across industries and countries, regardless of the sales environment because success invariably requires generation of new business from prospective and/or existing clients.

Other factors become equally, if not more, important once first contact is made. However, contact initiation comes first. Research studies have shown that contact initiation behaviours, as measured by SPQ*GOLD®, serve as a leading indicator of future success in sales.

Clearly, since other factors beside contact initiation also affect success in a sales career, it is not recommended that SPQ*GOLD® be the sole source of information from which the selection decision is made.

Global Acceptance

The SPQ*GOLD® has always met, and exceeded, the development rigors required by government organizations in all countries around the world, including the USA and UK. Indeed, it was also passed by the Psych Board in South Africa whose sensitivities re discrimination and appropriateness was predictably intense.

The Behavioural Science Research Press have Dr. Clive Fletcher as a member of their Scientific Advisory Board; he is a former chair of the occupational psychology section of the British Psychological Society, a former Professor of Occupational Psychology at Goldsmiths’ College, University of London, and a Visiting Professor at Henley Management College.

As a Chartered Occupational Psychologist, he is also one of the relatively few to be elected to Fellowship of the British Psychological Society (BPS).

His knowledge and experience in the assessment field is recognised internationally. He has acted in an advisory capacity to many private sector organisations and to Government departments, and in particular to the UK Cabinet Office.

Not About Personality

The vast majority of instruments submitted for the psychological Societies to test, are for
those which fundamentally measure “what people ARE” (i.e. their personality) and are very different to the SPQ which measures “what people DO” (their role competency: how they will actually perform in the sales environment)

Most of the popular and traditionally used psychometric tests look to ascertain a whole series of general personality measures: Relationships (Sociable, Empathic etc) Thinking style (Traditional, Conceptual, Innovative etc)  Emotions (Relaxed, Optimistic etc) Energies (Active, Decisive etc), whereas the SPQ*GOLD® has the ability to measure and predict the use of sales behaviours of that person (regardless as to whether they have any of the identified types of personality)

In this way the SPQ has an advantage over personality or intelligence focused tools by being able to prove its ‘validity and appropriateness’, by consistently demonstrating tangible sales results.

The SPQ has been subjected to over 900 studies in the last 30 or so years, making it one of the most validated applications in use today around the world for assessing sales people, and their robustness to actually sell for businesses

ReMap Evaluation

As described elsewhere on this site, we put a lot of resources into evaluating each and every one of the ReMap 4 week Programmes, (to Kirkpatrick level’s 3 and 4), create long terms case studies and track individual performance in terms of recruitment support. Results are available here:

Sales Growth Case Studies

Recruitment Evaluation.