Evaluating Search Engines


Contents:

  1. Evaluation 1: overview
  2. Evaluation 2: research hypotheses
  3. Evaluation 3: effectiveness vs. efficiency
  4. Evaluation 4: Cranfield paradigm
  5. Evaluation 5: relevance judgments
  6. Evaluation 6: precision and recall
  7. Evaluation 7: why we can't use accuracy
  8. Evaluation 8: F-measure
  9. Evaluation 9: when recall/precision is misleading
  10. Evaluation 10: recall and precision over ranks
  11. Evaluation 11: interpolated recall-precision plot
  12. Evaluation 12: mean average precision
  13. Evaluation 13: MAP vs NDCG
  14. Evaluation 14: query logs and click deviation
  15. Evaluation 15: binary preference and Kendall tau
  16. Evaluation 16: hypothesis testing
  17. Evaluation 17: statistical significance test
  18. Evaluation 18: the sign test
  19. Evaluation 19: training / testing splits