Course: Introduction to Methods for Health Service Research and Evaluation

  • Course Home
  • Syllabus
  • Schedule
  • Lecture Materials
  • Readings
  • Assignments
  • Labs
  • Want to stay in touch?

    4 + 2 =  


Where possible, links to readings are provided below; however, JHSPH OpenCourseWare does not furnish users with full-text access

Topic Readings
Session 1: Introduction to Health Services Research


Lohr KN, Steinwachs DM (2002). Health Services research: an evolving definition of the field, Health Services Research, 37: 15-17.

Scott I, Campbell D (2002). Health Services Research: What is it and what does it offer? Internal Medicine Journal, 32:91-99.


Bensing JM, Caris-Verhallen WMCM, Dekker J, Delnoij DMJ, Groenewegen PP (2004). Doing the right thing and doing it right: toward a framework for assessing the policy relevance of health services research. International Journal of Technology Assessment in Health Care 19:604-612.

Lavis JA, Ross SE, Hurley JE (2002). Examining the role of health services research in public policymaking. The Milbank Quarterly, 80: 125-154.

Ginsburg E (ed). (1991). Health Services Research: Key to Health Policy. Cambridge MA: Harvard University Press.

Luft H (1986). Health services research as a scientific process: The metamorphosis of an empirical research project from grant proposal to final report. Health Services Research 21:563-584.

Mechanic D (2001). Lessons from the Unexpected: The Importance of Data Infrastructure, Conceptual Models and Serendipity in Health Services Research. The Milbank Quarterly 79:459-477.
Session 2: Introduction to Health Program Evaluation


Shortell S, Richardson W (1978). The Evaluation Process. In: Health Program Evaluation. St. Louis: CV Mosby Company, Chapter 2, pp. 16-37.

Morbidity and Mortality Weekly Report (1999). Framework for Program Evaluation in Public Health. MMWR 48(RR11);1-40


Gliner JA, Morgan GA (2000). Research Design and Analysis in Applied Settings: An Integrated Approach to Design and Analysis. Mahwah NJ: Lawrence Erlbaum Associates, Inc. Chapters 1 and 2, pp 3-29.

Grembowski D (2001). The Practice of Health Program Evaluation. Newbury Park, CA: Sage Publications

Rossi PH, Freeman HE, Lipsey MW (1999). Evaluation: A Systematic Approach. 6th ed. Newbury Park, CA: Sage Publications

Session 3: Concepts, Hypothesis, and Theoretical Frameworks


Gliner JA, Morgan GA (2000). Research Problems, Variables and Hypotheses. In: Research Methods in Applied Settings: An Integrated Approach to Design and Analysis. Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Chapter 4, pp 43-57.

Herek G (1990). Development of a Theoretical Framework and Rationale for Your Research Proposal. In: NIMH Handbook on Developing a Successful Research Application. DHHS: National Institutes of Health.


Kerlinger FN (1973). Problems and Hypotheses. In: Foundations of Behavioral Research. 2nd ed. New York: Holt, Rinehrt and Winston, Inc. Chapter 2, pp 66-27.   

Weinstein ND (1993). Testing four competing theories of health-protective behavior. Health Psychology 12:324-333.

Research Design
Session 4: Research Design: Campbell and Stanley


Shortell S, Richardson W (1978). Evaluation Designs. In: Health Program Evaluation. St. Louis: CV Mosby and Company. Chapter 3, pp.38-73.


Campbell DT, Stanley JC (1966). Experimental and Quasi-Experimental Designs for Research. Chicago: Houghton-Mifflin College

Cook T, Campbell D (1979). Quasi-Experimentation: Design and Analysis Issues for Field Settings. Chicago: Houghton-Mifflin Co.

Gliner JA, Morgan GA (2000). Research Design and Analysis in Applied Settings: An Integrated Approach to Design and Analysis. Mahwah NJ: Lawrence Erlbaum Associates, Inc. Chapters 6 to 8 (pp 81d- 123) and Chapters 10 (pp 143-164).

Jefferson T, Demicheli V (1999). Relation between experimental and non-experimental study designs. HB vaccines: A case study. J Epidemiol Community Health 53:51-54.

Newhouse J (1974). A design for a Health Insurance Experiment. Inquiry 11:5-27.

Session 5: Research Design: Wasson


Wasson J, et al. (1984). Continuity of Outpatient Care in Elderly Men: A Randomized Trial. JAMA 252:2413-2417

Tarlov AR, Ware JE, Greenfield S, Nelson EC, Perrin E, Zubkoff M (1989). The Medical Outcomes Study: An application of methods for monitoring the results of medical care. JAMA 262:925-930.

Safran DG, Tarlov AR , Rogers WH (1994). Primary care performance in fee-for-service and prepaid health care systems. JAMA 271:1579-1586.

Session 6: Research Design: Other Examples

Carey RS, Garrett J, Jackman A, et al. (1995). The outcomes and costs of care for acute low back pain among patients seen by primary care practitioners, chiropractors, and orthopedic surgeons. N Engl J Med, 333:913-917.

Brook RH, Ware JE, Rogers WH, et al (1983). Does free care improve adults' health? N Engl J Med 309:1426-1434.

Newhouse JP, Manning WG, Morris CN, et al. (1981). Some interim results from a controlled trial of cost sharing in health insurance. N Engl J Med, 305:1501-1507.

Session 7: Measurement: Reliability and Validity Measures


Gliner JA, Morgan GA (2000). Research Design and Analysis in Applied Settings: An Integrated Approach to Design and Analysis. Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Chapter 7 (pp 128-134) and Chapter 20 (pp 309-327).

McDowell I, Newell C (1996). The quality of a measurement: validity and reliability. In: Measuring Health: A Guide to Rating Scales and Questionnaires. 2nd ed. New York: Oxford University Press, pp 29-43.

Clancy CM, Eisenberg JM, (1998). Outcomes research: measuring the end results of health care. Science 282: 245-6.


Carmines E, Zeller R (1979). Reliability and Validity Assessment. Beverly Hills: Sage Publications, pp 9-27.

Nachmais D, Nachmais C (1981). Chapter 2: Conceptual Foundations of Research; and Chapter 6: Measurement. In: Research Methods in the Social Sciences. New York: St. Martin 's Press. pp 29-53 and pp 131-151.

Aday LA (1996). Defining and clarifying the survey variables. In: Designing and Conducting Health Surveys. 2nd Edition. San Francisco: Jossey-Bass. Chapter 3, pp 44-74.

Juniper EA, et al (1996). How to develop and validate a new health-related quality of life instrument. In: B. Spilker (ed). Quality of Life and Pharmacoeconomics in Clinical Trials. Philadelphia: Lippincott-Raven, pp 49-56.

Kozinetz CA, Warren RW, Berseth CL, Aday LA, Sachdeva R, Kirkland RT (1999). Health status of children with special health care needs: Measurement issues and instruments. Clinical Pediatrics (Phila) 38:525-533.

Lee Y, Kasper JD (1998). Assessment of medical care by elderly people: General and physician quality. Health Services Research 32: 741-758.

Shortell SM, Richardson WC (1978). Health Program Evaluation. St. Louis: CV Mosby Company. Chapter 4, pp 74-97.

Hendriks AAK, Vrielink MR, Smets EMA, van Es SQ, De Haes, JCJM (2001). Improving the assessment of (in)patients' satisfaction with hospital care. Medical Care 39:270-283.

Session 8: Measurement Examples


Andersen R (1995). Revisiting the Behavioral Model and Access to Medical Care. Journal of Health and Social Behavior 36,1-10.

Ware JE, Snyder MK, Wright R, Davies AR (1983). Defining and Measuring Patient Satisfaction with Medical Care. Evaluation and Program Planning 6:247-263.

Survey Application and Survey Design
Session 9: Proposal Writing


Locke LF, Spirduso WW, Silverman SJ (1999). Proposals That Work: A Guide of Planning Dissertaions and Grant Proposals. Thousand Oaks, CA: Sage Publications, Chapter 1, pp. 3-24.


NIAID Funding and Council News Center. �All About Grants� Tutorials.

Session 10: Quality of Care and Its Measurement in the HSRE Context


Donabedian A (1988). The quality of care: How can it be assessed? JAMA 260:1743-1748.

Blumenthal D (1996). Part 1: Quality of Care�What is it? NEJM 335(12): 891-894.

Brook RH, McGlynn EA, Cleary PD (996). Part 2: Measuring Quality of care. NEJM 335(13): 966-970.

Chassin MR (1996). Part 3: Improving the quality of care. NEJM 335(14): 1060-1063.

Blumenthal D (1996). Part 4: The origins of the quality-of-care debate. NEJM 335(15): 1146-1149.

Berwick DM (1996). Part 5: Payment by capitation and the quality of care. NEJM 335(16): 1227-1231.

Blumenthal D, Epstein AM (1996). Part 6: The role of physicians in the future of quality management. NEJM 335 (17): 1328-1331.


Brook RH, McGlynn EA, Shekelle PG (2000). Defining and measuring quality of care: A perspective from US researchers. Int J Qual Health Care, 12(4):281-295.

Clancy CM, Eisenberg JM (1998). Outcome research: Measuring the end results of health care. Science 282:245-246.

Donabedian A (1980, 1983, 1985). Explorations in Quality Assessment and Monitoring , Vols I, II, and III. Ann Arbor, MI: Health Administration Press.

Institute of Medicine (2001). Crossing the Quality Chasm: A New Health System for the 21 st Century. Washington DC: National Academy Press.
Session 11: Survey Research: Choice of Instrument, Sample


Aday LA (1996). Chapter 3: Defining and clarifying the survey variables; Chapter 8: General Principles for formulating questions. In: Designing and Conducting Health Surveys: A Comprehensive Guide. Second Edition. San Francisco: Jossey-Bass. pp 44-74 and pp 177-199.


Medical Expenditure Panel Survey (MEPS), Agency for Healthcare Research and Quality

Frankel M, Banks M (1979). Adjusting For Non-response To Specific Questions. In: Total Survey Error. Andersen R, Kasper J, Frankel MR (eds). San Francisco: Jossey-Bass, pp 97-103.

Session 12: Survey Research: Designing an Instrument


Dillman DA (2000). Chapter 1: Introduction to tailored design; Chapter 11: Internet and Interactive Voice Response Surveys. In: Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley & Sons. pp 3-31 and pp 352-361.

Foddy W (1993). Chapter 1: An initial statement of the Problem; Chapter 2: A theoretical framework. In: Constructing Questions for Interviews and Questionnaires. New York: Cambridge University Press. pp 1-24.


Fowler FJ (1995). Improving Survey Questions: Design and Evaluation. Newbury Park, CA: Sage Publications, Chapters 1 to 3.

Mangione TW (1995). Mail Surveys: Improving the Quality. Newbury Park, CA Sage Publications, Chapters 2 and 3.

Schuman H, Presser S (1996). Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording and Context. Thousand Oaks, CA: Sage Publications.

Schwarz N, Sudman S (1996). Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research. San Fransico: Jossey-Bass.

Sudman S, Bradburn NM , Schwarz N (1996). Thinking About Answers: The Application of Cognitive Processes to Survey Methodology. San Francisco: Jossey-Bass.

Tanur JM (1991). Questions about Questions: Inquiries into the Cognitive Bases of Surveys. New York: Russell Sage Foundation.

Session 13: Cost-Benefit and Cost-Effectiveness Analysis


Russell LB, Gold MR, Siegel JE, Daniels N, Weinstein MC (1996). The role of cost-effectiveness analysis in health and medicine. Panel on Cost-Effectiveness in Health and Medicine. JAMA 276:1172-1177.

Weinstein MC, Siegel JE, Gold MR, Kamlet MS, Russell LB (1996). Recommendations of the Panel of Cost-Effectiveness in Health and Medicine. JAMA 276: 1253-1258.

Siegel JE, Weinstein MC, Russell LB, Gold MR (1996). Recommendations for reporting cost-effectiveness analyses. Panel on Cost-Effectiveness in Health and Medicine. JAMA 276:1339-1341.

Krahn M, et al (1998). Costs and cost-effectiveness of a universal, school-based hepatitis B vaccination program. Am J Public Health 88:1638-1644.


Bunker J, Barnes B, Mosteller F (eds) (1977). Costs, Risks and Benefits of Surgery. New York: Oxford University Press.

Chang R, Pellissier JM, Hazen GB (1996). A cost-effectiveness anlaysis of toal hip arthroplasty for osteoarthritis of the hip. JAMA 275:858-865.

Detsky AS, Naglie IG (1990). A clinician's guide to cost effectiveness analysis. Annals of Internal Medicine 113:147-154.

Levin H (1983). Cost-Effectiveness: A Primer. Newbury Park, CA: Sage Publications.

Warner K, Luce B (1982). Cost Benefit and Cost Effectiveness in Health Care. Ann Arbor MI: Health Administration Press.

Maynard A, McDaid D, (2003) Evaluating health interventions: exploiting the potential. Health Policy 63:215-226

Session 14: Application of Information Systems and Secondary Data


Kasper JD (1998) Asking about access: challenges for surveys in a changing healcare environment. Health Services Research 33: 715-39.

Medical Expenditure Panel Survey (MEPS), Agency for Healthcare Research and Quality

Surveys and Data Collections Systems of the National Center for Health Statistics (National Health Interview Survey, National Health and Nutrition Examination Survey, National Health Care Survey, National Vital Statistics System, etc.)


National Center for Health Statistics (2004). Health US, 2004. Washington DC: US Government Printing Office. Appendix 1, Data Sources, pp. 393-440

Weiner JP, et al (1990). Applying insurance claims data to assess quality of care: A compilation of potential indicators. Quality Review Bulletin 16:424-238.

Session 15: Ethics in Health Services Research


Gliner JA, Morgan GA (2000). Research Design and Analysis in Applied Settings: An Integrated Approach to Design and Analysis. Mahwah NJ: Lawrence Erlbaum Associates, Inc. Chapter 3, pp 31-41.


Brett A, Grodin M (1991). Ethical aspects of human experimentation in health services research. JAMA 265:1854-57