John P. Lalor

John P. Lalor

Assistant Professor of IT, Analytics, and Operations

I am an Assistant Professor at the Mendoza College of Business at the University of Notre Dame. I recently defended my Ph.D. dissertation at the University of Massachusetts Amherst in the College of Information and Computer Science. At UMass I was a member of the Bio-NLP group, working with Dr. Hong Yu. My research interests are in Machine Learning and Natural Language Processing. I am particularly interested in model evaluation and quantifying uncertainty, as well as applications in biomedical informatics.

Prior to UMass, I worked as a software developer at Eze Software in Chicago and as an IT Audit Associate for KPMG. I received my Master’s Degree in Computer Science at DePaul University, where I worked on projects in Computer Science Education and Recommender Systems.

Recent Updates

  1. May 2021: Evaluation Examples Are Not Equally Informative: How Should That Change NLP Leaderboards? accepted to ACL 2021.
  2. April 2021: Evaluating the Effectiveness of NoteAid in a Community Hospital Setting: Randomized Trial of Electronic Health Record Note Comprehension Interventions With Patients accepted to JMIR.
  3. October 2020: An Empirical Analysis of Human-Bot Interaction on Reddit accepted at the 2020 Workshop on Noisy User-generated Text.
  4. September 2020: Towards Measuring Algorithmic Interpretability accepted at the 2020 INFORMS Workshop on Data Science.
  5. September 2020: Dynamic Data Selection for Curriculum Learning via Ability Estimation accepted to appear in Findings of EMNLP.

Research Publications

Journal

  • J.P. Lalor, W. Hu, M. Tran, H. Wu, K. Mazor, H. Yu. Evaluating the Effectiveness of NoteAid in a Community Hospital Setting: Randomized Control Trial. J Med Internet Res [paper]
  • J. Chen, J.P. Lalor, W. Liu, E. Druhl, E. Granillo, V. Vimalananda, H. Yu. Detecting Hypoglycemia Incidents Reported in Patients Secure Messages: Using Cost-sensitive Learning and Oversampling to Reduce Data Imbalance. J Med Internet Res 2019;21(3):e11990. doi:10.2196/11990. [paper]
  • J.P. Lalor, B. Woolf, H. Yu, Improving EHR Note Comprehension with NoteAid: A Randomized Trial of EHR Note Comprehension Interventions with Crowdsourced Workers, J Med Internet Res 2019;21(1):e10793. [paper]
  • J.P. Lalor, H. Wu, L. Chen, K. Mazor, H. Yu, ComprehENotes, an Instrument for Assessing Patient Electronic Health Record Note Reading Comprehension: Development and Validation, J Med Internet Res 2018;20(4):e139. [paper]

Conference

  • P. Rodriguez, J. Barrow, A.M. Hoyle, J.P. Lalor, R. Jia and J. Boyd-Graber. Rethinking NLP Leaderboards with Methods from Educational Testing. ACL 2021 (forthcoming).
  • J.P. Lalor, H. Yu. Dynamic Data Selection for Curriculum Learning via Ability Estimation. Findings of ACL: EMNLP 2020. [paper]
  • J.P. Lalor, H. Wu, H. Yu. Learning Latent Parameters without Human Response Patterns: Item Response Theory with Artificial Crowds. EMNLP 2019 [paper, code]
  • J.P. Lalor, H. Wu, T. Munkhdalai, H. Yu. Understanding Deep Learning Performance through an Examination of Test Set Difficulty: A Psychometric Case Study. In EMNLP 2018. [paper, youtube link for presentation]
  • J.P. Lalor, H. Wu, H. Yu. Building an Evaluation Scale using Item Response Theory, In EMNLP 2016. [paper]
  • C. Miller, A. Settle, and J.P. Lalor. Learning Object-Oriented Programming in Python: Towards an Inventory of Difficulties and Testing Pitfalls, SIGITE 2015. [paper]
  • A. Settle, J.P. Lalor, and T. Steinbach. Evaluating a Linked-courses Learning Community for Development Majors. In SIGITE 2015. [paper]
  • A. Settle, J.P. Lalor, and T. Steinbach. A Computer Science Linked-courses Learning Community, In ITiCSE 2015. [paper]
  • A. Settle, J.P. Lalor, and T. Steinbach. Reconsidering the Impact of CS1 on Novice Attitudes. In SIGCSE 2015. [paper]

Workshop

  • M. Ma, J.P. Lalor. An Empirical Analysis of Human-Bot Interaction on Reddit. Workshop on Noisy User-generated Text (W-NUT), 2020. [paper]
  • J.P. Lalor, H. Guo. Towards Measuring Algorithmic Interpretability. INFORMS Workshop on Data Science, 2020.
  • E. Cho, H. Xie, J.P. Lalor, V. Kumar, W.M. Campbell. Efficient Semi-Supervised Learning for Natural Language Understanding by Optimizing Diversity. ASRU 2019 [paper]
  • J.P. Lalor, H. Wu, H. Yu. Learning Latent Parameters without Human Response Patterns: Item Response Theory with Artificial Crowds. NAACL Workshop on Shortcomings in Vision and Language (SiVL) 2019 [poster]
  • J.P. Lalor, H. Wu, H. Yu. Comparing Human and DNN-Ensemble Response Patterns for Item Response Theory Model Fitting. NAACL Workshop on Cognitive Modeling and Computational Linguistics (CMCL) 2019 [poster]
  • J.P. Lalor, H. Wu, H. Yu. Soft Label Memorization-Generalization for Natural Language Inference. UAI Workshop on Uncertainty in Deep Learning., 2018. [arxiv pre-print]
  • J. Chen, J.P. Lalor, H. Yu. Detecting Hypoglycemia Incidents from Patients' Secure Messages. American Medical Informatics Association (AMIA) Annual Symposium Poster, 2018
  • J.P. Lalor, H. Wu, H. Yu, Modeling Difficulty to Understand Deep Learning Performance. Northern Lights Deep Learning Workshop (NLDL), 2018 [slides]
  • J.P. Lalor, H. Wu, L. Chen, K. Mazor, H. Yu, Generating a Test of Electronic Health Narrative Comprehension with Item Response Theory. American Medical Informatics Association (AMIA) Annual Symposium Podium Abstract, 2017. [slides]
  • J.P. Lalor, H. Wu, H. Yu. CIFT: Crowd-Informed Fine-Tuning to Improve Machine Learning Ability, HCOMP 2017 Works in Progress [poster]
  • T. Munkhdalai, J.P. Lalor, H. Yu. Citation Analysis with Neural Attention Models, In LOUHI 2016 EMNLP Workshop. [pdf]

Teaching

University of Notre Dame

  • ITAO 40250: Unstructured Data Analytics
  • ITAO 70810: Data Wrangling with R
  • ITAO 70800: Integrated Analytics Deep Dive

University of Massachusetts Amherst

  • UMass First Year Seminar: Artificial Intelligence and Healthcare