At Comms Multilingual, we understand the significance of providing accurate and culturally-relevant translations to the assessment and certification industries.
Our commitment to providing the highest quality services means that we aim to provide services not generally offered by generalist translation agencies. Read on to explore the importance of normative data collection, its role in the psychometric assessment translation process, and how our expertise can benefit your organisation.
What is Normative Data Collection?
Normative data collection is carried out to address issues of validity and to evaluate the equivalence of interpretations across countries. This step helps to enhance cross-cultural comparability.
Our clients provide us with their own demographic specifications for their particular assessment. We then select between 200 and 600 people, who take the assessment either online or on paper.
The results from this can then be used as norms against which future takers of the assessment in that country can be measured.
The Role of Normative Data Collection in Assessment Translation Projects
When translating assessments for use in multiple languages and cultures, it’s essential to account for the cultural and linguistic variations that may influence test scores. Normative data collection plays a crucial role in this process by:
- Ensuring Fairness
Normative data collection helps identify potential bias in assessments when administered to different cultural or linguistic groups. It ensures that test scores accurately reflect the abilities and competencies of candidates from diverse backgrounds. - Maintaining Validity
Validity is a fundamental criterion for any assessment. Normative data helps in maintaining the validity of assessments across various language versions through quantitive analysis. It provides evidence that the test is measuring what it’s designed to measure across different cultural contexts. - Adapting Scoring Scales
Normative data allows for the adaptation of scoring scales to accommodate variations in language, culture, and educational systems. This adaptation ensures that candidates receive scores that are meaningful within their cultural context.
Sample Normative Data Collection Projects
We were asked by one of our clients, a psychometric test publisher, to translate an error-checking test into multiple European languages. As part of the process, they asked us to provide 250 responses from a representative sample of the population in several of those countries to assist in the norming and validation of the assessment. This required closely working with a panel provider to recruit suitable respondents and code in the interface between the client’s test delivery system and the panel provider’s reward system.
Dynamic Linking and Data Quality
There are several aspects that will assist greatly in terms of assuring the quality of normative data collected. One is working with a high-quality data panel provider; they should track the data quality submitted by respondents, not just for your own assessment, but across all submissions made by the same person in other studies. This helps to ensure that respondents are answering seriously and honestly.
It is worth ensuring that your assessment also contains checks and balances for gauging “speedsters”, i.e. those who go through an assessment and answer in repetitive patterns, or without spending enough time on questions to warrant their answers being taken into account when creating the norm groups.
The use of a “dynamic” link to the data collection platform means that only completions that meet the criteria for good-quality data are rewarded. How this may work in reality:
- A participant is sent to the test delivery platform, along with the panel provider’s unique participant ID in the URL (a dynamic URL).
- The participant’s ID is retained by the delivery platform throughout administration of the assessment.
- Once complete, or when the participant’s session is determined to be invalid (e.g. if they are a “speedster” or a “straight liner”), the participant is returned to a URL hosted by the data panel provider along with the unique participant ID and a status code, e.g. “speed” or “accept”.
- Those who provided valid completes of the assessment get rewarded, while those deemed speedsters or straight liners are not incentivised.
Normative Data Collection Requirements for Translated Assessments
If you are interested in finding out how we can help you to create norms for your translated assessment, please do get in touch.
We will need to know the typical completion time, the sample size and composition that you require, and how you plan to deliver the assessment (i.e. does it need to be hosted separately, or are you able to implement redirect links). We will usually arrange a fact-finding call to understand your requirements before assessing which is our most suitable data panel provider and providing estimated pricing.
