Verbal Reasoning tests (VRTs) are among the most commonly used aptitude tests for measuring people’s ability to accurately understand business-related written materials and draw logical conclusions from them.
There are many different types of VRT, from the relatively simple through to the very complex. The translation of verbal reasoning tests into other languages is among the most challenging of any psychometric test, but also the most interesting. It is not just straightforward translation: localisation and adaptation are vitally important.
If there is any chance that such a test (or indeed any other psychometric test) may need to be put into a different language at some stage, then these tests need to be written in English (or any other original language) with translation in mind. We have come across many original items that are psychometrically valid and very cleverly written in the original language. However, when you try to put them into a different language, you realise just what a challenge it is.
With that in mind, we have produced a 30-minute webinar for item writers that addresses these issues and teaches them what to look out for. If you would like to find out more about our upcoming webinar dates, please let us know.
One of the main things to look out for with multiple choice answers is to ensure the same level of difficulty among the distractors in another language. The programme “Who wants to be a millionaire?” is a classic example of distractor difficulty. The questions at the beginning have distractors that are quite often obviously wrong. As the prize money increases, so the distractors become much closer to the right answer and introduce a lot of doubt.
It is very important to ensure that the distractor length in a different language does not lead to bias. For example, 1 word in English may need 2-3 words in Spanish to convey the same meaning. However, if the right answer and the other 2 distractors only comprise one word each in Spanish and 1 distractor has 3 words, this may well lead to bias.
This is something that English item writers obviously won’t know when writing the items, but there are times when we have advised our clients that an item just won’t work in another language because of the above. We have then worked with our clients to come up with an item that avoids such bias in the target language.
For this reason, it is also good practice to provide us with more items for translation than will actually be used. The English items will have been validated in English and you don’t want to have to go back and do that all over again just for 1 item that can’t function properly in another language.
We have had VRTs that require the candidate to choose a synonymous word to one in the body text. English is a very rich language and a large number of words have synonyms. However, either the original word or the synonym may be at a level of difficulty above those used in every day parlance to represent the same concept.
We were translating a verbal reasoning test into German, and we spent hours trying to get words in German that were at the same level of difficulty as in the original English. In some cases, we had to give up as there was only one word that could be used in the translation and that word was obviously in everyday use, and one which everyone would understand.
We have had to translate verbal reasoning tests that included questions on grammar, for example completing a sentence with the right word grammatically. This caused a lot of difficulty in languages such as Chinese, which are constructed completely differently to English. We had to work with a Chinese grammar expert to come up with workable equivalents for Chinese.
We have had examples where one of the distractors started with the same 5 letters in English as the word in the main text. This was a very good distractor in English, but it was impossible to achieve the same first 5 letters in Finnish for example.
One of the things that we are always very cognisant of is ensuring that the correct answer can be obtained in the translated version. The answers must be translated in such a way that the correct answer is not impossible to arrive at.
Sentence completion and analogies tend to be the easier forms of VRT, and these measure a person’s ability to use the source language at work. However, analogies can be very difficult to put into another language.
For example, “Kitten is to Cat as Puppy is to ______” is an example of an analogy item.
This appears to be fairly straightforward, but we have come across many such analogies that have had to be changed, as the words would have no meaning culturally in another language or they may be such obscure things that people wouldn’t have heard of or be familiar with.
In this case, for some languages, the word for “puppy” can only be translated by using the equivalent of “baby dog”, which would give away the answer! A change of animal may therefore be in order…
The above are just some of the issues that we come across when translating, localising and adapting verbal reasoning tests into other languages and for other cultures. However, they must all be taken into consideration if you wish to offer your VRT successfully in another language and country. Please talk to us if you are considering localising a verbal reasoning test into another language and we will be happy to advise you.
