Table of Contents

assessment - a method of evaluation

cerebellar tonsils - portion of the cerebellum located at the bottom, so named because of their shape

cerebellum - part of the brain located at the bottom of the skull, near the opening to the spinal area; important for muscle control, movement, and balance

Chiari malformation - condition where the cerebellar tonsils are displaced out of the skull area into the spinal area, causing compression of brain tissue and disruption of CSF flow

cognitive - having to do with thinking, reasoning, memory, etc.

disease - commonly refers to any departure from the normal health and functioning of the body and its organs

disability - that which limits a person's ability to perform one or more normal, daily activities

mobility - the ability to move from one place to another

pain - an unpleasant experience usually associated with an injury or disease

quantitative research - scientific research which uses numbers as data and statistical techniques to analyze results

reliability - in testing, a measure of the consistency of the test

test - a formalized procedure for measuring something

syringomyelia - neurological condition where a fluid filled cyst forms in the spinal cord

validity - the degree to which a test, or assessment, measures what it is supposed to measure

     Pain, disease, and disability; in a general sense we all know what these terms mean.  Pain is what you feel when you're hurt or sick.  Disease is a virus, disorder, or neurological condition.  Disability is when something prevents you from doing what you should be able to do.  Easy to define, right?  In English, yes; but how do you define these things in such a way that they can be used effectively in medical research?  Most of medical research is what is known as quantitative research, which means it uses numbers.  In order to do quantitative studies involving pain, disability, and disease impact, these words need to be translated into numbers.  In other words, an effective method of measuring - or testing for -  items like pain and disability is required to produce useful, scientific research.
     If you take a moment to think about the various types of tests there are in everyday life, it is mind-boggling.  From the slew of standardized tests that our children take every year, to personality tests that many employers now require, to product safety tests, to the hundreds of medical tests, tests are everywhere.   Given this, it is not surprising that an entire of science of testing has evolved.  Some researchers specialize in the theory of test development and implementation.  Out of this science has emerged a couple of key concepts for evaluating the usefulness of tests:  reliability and validity.
    Reliability refers to whether a test consistently produces the same results on the same subjects.  In other words, if Joe takes an IQ test, his score shouldn't change depending on who administers the test.  A blood test is reliable if it gives the same - or very close - results when run several times on the same sample.  Reliability is an important measure of a test, because if test results vary because a machine is inconsistent or the test is administered by different people, the test results are not very useful.
     Validity refers to whether a test measures what it is supposed to measure.  Validity can be harder to determine than reliability and is often the subject of controversy.  One of the most controversial tests of all time is the IQ test.  The controversy surrounds it's validity.  Does the IQ test really measure intelligence?  Some would argue it does, many would argue it doesn't.  When dealing with issues like pain and disability, determining a test's validity can be challenging.  Often a patient's view of disability and disease impact differs sharply from a doctor's and people experience pain differently.  One way to determine a test's validity is to compare it's results to another measure.  For example, a new pain measure might be compared to existing pain measures, or the results for a test of how outgoing someone is would be expected to be inversely related to a test for how introverted someone is.
     While there are a number of tests that have been developed to measure pain, disability, and disease impact, it is important to realize that every test has it's strengths, weaknesses, and limitations.  Just as scientific research must undergo peer review before publication, we as patients should examine research results through a skeptical filter and decide for ourselves whether the tools used in the study were reliable and valid enough to apply to us. 
      Following is a very brief overview of three scales:  the Karnofsky Performance Scale, the McGill Pain Questionnaire (MPQ), and the SF-36 Health Survey.  These assessments are by no means the only ones  used to measure pain, disease, and disability; but they are illustrative of the importance of understanding a test when evaluating the results it produced.  These tests were selected because they have been used in research reviewed by this publication relevant to the Chiari and syringomyelia community.

Karnofsky Performance Index (KPI)

The Scale

• Health care professional assigns patient a score along 11 descriptions.  Most relevant criteria are selected.  Overall score ranges from 0 (Dead) to 100 (Normal):
  100 - Normal, no complaints, no evidence of disease
  70 - Requires occasional assistance from others but able to care for most needs
  40 - Disabled, requires special care and assistance
  10 - Dying, near death
  0 - Dead

• In use for many years, by many types of medical professional

• Not designed as a quality of life measure, but most often used as such

• Originally designed to assess nursing work loads (how much help each patient needed from a nurse)

Strengths

• Widely used and recognized

• Research shows valid measure of physical functioning

• Predictive value in cancer and transplant survival

• Quick and easy to use and administer

Weaknesses

• Very crude measure

• Relies solely on doctor assessment, yet research shows there is a marked difference between patient and doctor assessments on quality of life

• Narrow focus on the physical aspects of quality of life

• Overemphasizes mobility and assumes mobility leads to a higher quality of life

• Numeric scale has not been adequately tested given the measure's wide adoption

• Some research suggests the scale is not very reliable

Conclusion

     The Karnofsky Performance Index appears to be a valid measure of physical functioning but there is no good basis for it's use as a measure of overall quality of life.  It is very crude and relies solely on physician assessment and does not take into account social support, psychological well-being, and a patient's own point of view.  When used with other quality of life measures, the KPI may be beneficial, but  unfortunately this measure is often the sole one used in studies of Chiari and syringomyelia.

McGill Pain Questionnaire

The Scale

• One of - if not the most - widely used measure of chronic pain

• Developed by Melzack at McGill University, Canada

• Uses 78 pain adjectives (throbbing, beating, pounding, e.g.) organized in 20 groups

• Subjects select words that describe their pain

• A group of doctors, patients, and students assigned numerical values to each word in developing the test

• Results are based on a Pain Rating Index (uses the assigned numerical values), number of words chosen, and Present Pain Intensity (1-5)

Strengths

• Can be self-administered or by an interviewer

• Effectively quantifies pain

• Research shows the measure can differentiate between diagnostic groups of patients

• Research shows validity when compared to simple one-number pain scales

• Short version is available

• Has been widely used for many years

Weaknesses

• Takes 15-20 minutes to complete (repeat tests take only 5-10 minutes)

• Person must be familiar with the words being used

• Pain descriptors may not adequately cover words often used by people with joint pain

• Limited data on reliability

Conclusion

     While acute pain can be quickly and easily measured using a one number scale (0-100), chronic pain can take many forms.  The MPQ has shown over the years to be probably the best available scale for measuring chronic pain. 

SF-36 Health Survey

The Scale

• 36 question survey used to assess general health status

• Answers to each question contribute to score along one of 8 sub-scales:  Physical Functioning, Role Physical, Bodily Pain, General Health, Vitality, Social Functioning, Role-Emotional, Mental Health

• Two summary measures are available based on sub-scales:  Physical Health and Mental Health

• Most questions have been in use since the 70's and 80's

• 8 sub-scales represent the most frequently measured concepts in widely used health surveys

• Disease impact scores can be measured by subtracting "normal" scores for a population from the respondent's actual score

Strengths

• Can be self-administered or given by an interviewer or computer

• Widely used in many countries for many purposes

• Fairly strong reliability and validity data

• If norms are established can assess disease impact

Weaknesses

• Generic type health survey, not specific to a disease

• Does not try to measure items such as sleep, cognitive functioning, sexual functioning, family functioning, eating, recreation, and other common measures

• Does not measure symptoms or problems specific to a disease

• Norms must be established (data gathered from a large group of people) to assess impact

Conclusion

     Widely used, the SF-36 is a solid, respected general health survey.  But because of it's generic nature, it may miss issues critical to patients of a specific disease. 

A Chiari/Syringomyelia Scale

     As the descriptions above demonstrate, there are many scales to measure pain, disease, and disability.  Each scale has it's own strengths and weaknesses and like any tool, it's effectiveness is determined largely by how it is used and who is using it.  Ideally, a scale specific to the symptoms, treatments, and neurological deficits of Chiari and syringomyelia patients should be developed.  The development of such a scale would benefit research into surgical (and non-surgical) treatments and provide a tool to measure how much an individual is being affected by their condition. 
     Until such a measure is developed, it is important to keep in mind the limitations of what is being used today.

Sources: 

• Psychological Testing and Assessment:  An Introduction To Tests and Measurement By Ronald Cohen & Mark Swerdlik, Mayfield Publishing, 1999

• www.sf-36.org

• Measuring disease : a review of disease-specific quality of life measurement scales By Ann Bowling, Open University Press, 2001

Back to Table of Contents

Key Points

1. Quantitative research requires a way to measure - with numbers - such things as pain, disability, and disease impact

2. Tests and measurements have been designed for almost every imaginable aspect of life

3. Reliability and validity are two key measures of how good a specific test is

4. The Karnofsky scale, the MPQ, and the SF-36 are examples of assessment tools that can be used

5. Every tool has advantages and disadvantages that should be understood when interpreting the results it produces

6. An ideal scale would be developed specifically for the Chiari/SM patient group and take into account symptoms and quality of life issues common these people