Why is reliability important?

5 Dec

Reliability is very important when considering experiments carried out in psychology, in which, results are used to provide evidence for a theory.

Reliability is a quality of measurement i.e. the extent to which you can measure a participant’s true score. It refers to the consistency and repeatability of a measurement across time, settings and individuals. A reliable measurement instrument will give similar scores when it is repeated.

Alternate forms of reliability include developing multiple versions of the test, administering each form to the same participants at different times, correlating the results; higher scores will indicate a stronger reliability. However, a problem that arises from this is that it is hard to know if both forms are really the same.

To test for reliability, you can use the coefficient alpha and the inter-rater agreement.

A test cannot be valid if it has poor reliability, but just because the test is reliable, it doesn’t mean that it is valid.

Validity refers to the accuracy of the measurement, i.e. the degree to which the test measures the construct you intend it to measure.

The degree to which you can draw conclusions from your test is determined by its validity. If your test doesn’t really measure what you want it to, your conclusions won’t be accurate or valid.



12 Responses to “Why is reliability important?”

  1. rtanti December 5, 2011 at 8:30 pm #

    Reliability is very important 🙂
    But, you are right to conclude that it is not important on its own, but when paired with validity it becomes increasingly important.
    If something is only reliable, the same results are repeated time and time again… But if this is the amount of time it takes you to put on your shoe, but you’re looking at reaction times to a stimulus, then this reliability won’t account for much. However, if the results are reliable and valid then you would get repeatable results, of a variable which you intended to measure.
    Therefore, I think you have argued your point very well in this blog and that I agree with you about the important role validity plays in the importance of reliability.

  2. statisticsbyrachel December 8, 2011 at 10:56 am #

    As you said, reliability is a very important factor for researchers to take into account when conducting psychological studies. With poor reliability, it is difficult to generalise findings from research to a wider population. Therefore several methods have been developed in order to strengthen reliability. One is the test-retest method, in which participants complete the interview, questionnaire or test on one occasion and then completes the same one again after a suitable interval (a week or a month maybe). If their outcome is the same every time, then the measure is reliable.
    Also, reliability can be measured in terms of whether two independent assessors give similar score. This is called inter-rater reliability and is often used in a clinical setting to assess mental disorders. For example, Kendler et al. (1999) found low reliability for their interview study into the assessment of phobias. They believed this low reliability may be due to different clinicians providing different diagnoses for the same phobic symptoms, therefore the study lacked inter-rater reliability.

  3. psychjs1 December 9, 2011 at 3:31 pm #

    I agree that reliability is important in research, if not essential! Reliability is a criterion to meet the standards of the scientific process. There are many variables that can reduce reliability in research such as observer error, environmental changes and participant changes. However, there are many measures of reliability to avoid such threats. As you noted these are coefficient alpha and the inter-rater agreement. However, there are also many more such as the split half technique that measures the internal consistency of research and the successive measure of test-retest reliability. The purpose of test-retest reliability is to compare the test scores collected in two successive measurements testing the same sample of participants. Researchers might employ the same measurement for the same participants at two differential intervals or may use modified versions of certain aspects such as using different questions in an IQ test.

  4. psychmja1 December 9, 2011 at 6:31 pm #

    I agree with you that reliability is extremely important in psychology 🙂 and as mentioned above, test-retest reliability is used to establish if a measure is consistent and reliable. In general, it is best used to establish whether something is consistently reliable over time, such as intelligence. For example, if we administered an IQ test today and again in 3 months time we would expect the results to be about the same each time. This would then suggest that the measure is reliable. Whilst test-retest reliability is a good measure of reliability it is important to consider that it only tells us the consistency of results but does not always show us that the test is valid… For example, if we measured intelligence by measuring the circumference of a person’s head with a tape measure it would be reliable but not valid.

  5. suuzblog December 9, 2011 at 7:12 pm #

    I agree with you on how reliability is important, it is essential to be honest!! If you repeat an experiment or study that keeps getting completely different results all the time, you’ve got to worry about it. For a study to be as good as possible, it has to show good reliability and validity. If it doesn’t show that it measures what it intended to measure, or that it is unreliable, it can easily be disregarded. But, like you said, just because something’s reliable doesn’t mean it’s valid, and just because something’s valid it doesn’t necessarily mean it will be reliable. And reliability on its own is not nearly as strong as reliability and validity together.

  6. lisaoliver1613 December 9, 2011 at 8:19 pm #

    The quality of a research report is usually shown through validity and reliability, without these the research seems pointless (as without them your results aren’t consistent and didn’t measure what they were supposed to). There are many forms of validity:

    Face Validity, Construct Validity, Convergent Validity, Divergent Validity, Content Validity, Predictive Validity, Factorial Validity, Concurrent Validity, External Validity, Internal Validity and Ecological Validity.

    Violating some of these are more detrimental than others and some can indicate that potentially your research methods where ineffective, for instance if your study doesn’t have face validity then just from looking at it you can tell that it hasn’t done what it was supposed to (which isn’t good). External validity can indicate that your research wasn’t effective as you may not be able to generalise to the population. However external validity is violated in many studies and this isn’t an issue as long as you know when you come to generalising. External validity cannot always be satisfied as it is not always possible to gain a sample that is representative of everyone. Regardless of this however for both reliability and validity it is important to satisfy and not violate them as best possible, because this means you have produced a strong research design and are supported by what you thought when originally beginning your study as well as when comparing to previous investigations. Reliability and validity provide confidence in a research report that what they found was accurate and that those variables are linked (or potenially linked as you can’t prove.. only support).

  7. psucb0 February 22, 2012 at 10:59 pm #

    I think that way you define reliability in the first instance is a little bit confusing- when you say it is the extent to which you can measure a participants true score. I feel this is more relevant to validity as this is when you are sure that the method you are using is measuring what you want it to measure. However I agree with what you say about it being involved with consistency and repeatability across time. The extent to which an experiment is reliable will be due to its replicability, if an experiment is highly replicable there is a higher chance that it will be repeated which will then increase its reliability if the results are replicated.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: