publications

2023

1. AISTATS

   Domain Adaptation under Missingness Shift

   Helen Zhou, Sivaraman Balakrishnan, and Zachary Lipton

   In International Conference on Artificial Intelligence and Statistics, 2023

   Abs Bib HTML PDF Poster Slides

   Rates of missing data often depend on record-keeping policies and thus may change across times and locations, even when the underlying features are comparatively stable. In this paper, we introduce the problem of Domain Adaptation under Missingness Shift (DAMS). Here, (labeled) source data and (unlabeled) target data would be exchangeable but for different missing data mechanisms. We show that if missing data indicators are available, DAMS reduces to covariate shift. Addressing cases where such indicators are absent, we establish the following theoretical results for underreporting completely at random: (i) covariate shift is violated (adaptation is required); (ii) the optimal linear source predictor can perform arbitrarily worse on the target domain than always predicting the mean; (iii) the optimal target predictor can be identified, even when the missingness rates themselves are not; and (iv) for linear models, a simple analytic adjustment yields consistent estimates of the optimal target parameters. In experiments on synthetic and semi-synthetic data, we demonstrate the promise of our methods when assumptions hold. Finally, we discuss a rich family of future extensions.

   @inproceedings{zhou2023domain,
     title = {Domain Adaptation under Missingness Shift},
     author = {Zhou, Helen and Balakrishnan, Sivaraman and Lipton, Zachary},
     booktitle = {International Conference on Artificial Intelligence and Statistics},
     pages = {9577--9606},
     year = {2023},
     organization = {PMLR},
   }

2. CHIL

   Model Evaluation in Medical Datasets Over Time

   Helen Zhou*, Yuwen Chen*, and Zachary C Lipton

   Machine Learning for Healthcare (ML4H) Symposium 2022 (Abstract). To be presented at Conference on Health, Inference, and Learning (CHIL), 2023

   Abs PDF Supp Poster

   Machine learning (ML) models deployed in healthcare systems must face data drawn from continually evolving environments. However, researchers proposing such models typically evaluate them in a time-agnostic manner, splitting datasets according to patients sampled randomly throughout the entire study time period. This work proposes the Evaluation on Medical Datasets Over Time (EMDOT) framework, which evaluates the performance of a model class across time. Inspired by the concept of backtesting, EMDOT simulates possible training procedures that practitioners might have been able to execute at each point in time and evaluates the resulting models on all future time points. Evaluating both linear and more complex models on six distinct medical data sources (tabular and imaging), we show how depending on the dataset, using all historical data may be ideal in many cases, whereas using a window of the most recent data could be advantageous in others. In datasets where models suffer from sudden degradations in performance, we investigate plausible explanations for these shocks. We release the EMDOT package to help facilitate further works in deployment-oriented evaluation over time.

2022

1. AMIA

   Learning Clinical Concepts for Predicting Risk of Progression to Severe COVID-19

   Helen Zhou, Cheng Cheng, Kelly J Shields, Gursimran Kochhar, Tariq Cheema, Zachary C Lipton, and Jeremy C Weiss

   AMIA Annual Symposium Proceedings, 2022

   Abs Bib HTML PDF Slides

   With COVID-19 now pervasive, identification of high-risk individuals is crucial. Using data from a major healthcare provider in Southwestern Pennsylvania, we develop survival models predicting severe COVID-19 progression. In this endeavor, we face a tradeoff between more accurate models relying on many features and less accurate models relying on a few features aligned with clinician intuition. Complicating matters, many EHR features tend to be under-coded degrading the accuracy of smaller models. In this study we develop two sets of high-performance risk scores: (i) an unconstrained model built from all available features; and (ii) a pipeline that learns a small set of clinical concepts before training a risk predictor. Learned concepts boost performance over the corresponding features (C-index 0.858 vs. 0.844) and demonstrate improvements over (i) when evaluated out-of-sample (subsequent time periods). Our models outperform previous works (C-index 0.844-0.872 vs. 0.598-0.810).

   @article{Zhou2022-zn,
     title = {Learning Clinical Concepts for Predicting Risk of Progression to
                   Severe {COVID-19}},
     author = {Zhou, Helen and Cheng, Cheng and Shields, Kelly J and Kochhar, Gursimran and Cheema, Tariq and Lipton, Zachary C and Weiss, Jeremy C},
     journal = {AMIA Annual Symposium Proceedings},
     volume = {2022},
     pages = {1257--1266},
     year = {2022},
     language = {en},
   }

2. ML4H

   Machine Learning for Health (ML4H) 2022

   Antonio Parziale, Monica Agrawal, Shengpu Tang, Kristen Severson, Luis Oala, Adarsh Subbaswamy, Sayantan Kumar, Elora Schoerverth, Stefan Hegselmann, Helen Zhou, and  others

   In Machine Learning for Health, 2022

   Bib HTML PDF

   @inproceedings{parziale2022machine,
     title = {Machine Learning for Health (ML4H) 2022},
     author = {Parziale, Antonio and Agrawal, Monica and Tang, Shengpu and Severson, Kristen and Oala, Luis and Subbaswamy, Adarsh and Kumar, Sayantan and Schoerverth, Elora and Hegselmann, Stefan and Zhou, Helen and others},
     booktitle = {Machine Learning for Health},
     pages = {1--11},
     year = {2022},
     organization = {PMLR},
   }

2021

1. AMIA

   Unpacking the Drop in COVID-19 Case Fatality Rates: A Study of National and Florida Line-Level Data

   Helen Zhou*, Cheng Cheng*, Jeremy C Weiss, and Zachary C Lipton

   In AMIA Annual Symposium Proceedings, 2021

   Abs Bib HTML PDF Slides

   Since the COVID-19 pandemic began, the United States’s case fatality rate (CFR) has plummeted. Using national and Florida data, we unpack the drop in CFR between April and December 2020, accounting for such confounders as expanded testing, age distribution shift, and detection-to-death lags. Guided by the insight that treatment improvements in this period should correspond to decreases in hospitalization fatality rate (HFR), and using a block-bootstrapping procedure to quantify uncertainty, we find that although treatment improvements do not follow the same trajectory in Florida and nationally (with Florida undergoing a comparatively severe second peak), by December, significant improvements are observed both in Florida and nationally (at least 17% and 55% respectively). These estimates paint a more realistic picture of improvements than the drop in aggregate CFR (70.8%–91.1%). We publish a website where users can apply our analyses to selected demographics, regions, and dates of interest.

   @inproceedings{cheng2021unpacking,
     title = {Unpacking the Drop in COVID-19 Case Fatality Rates: A Study of National and Florida Line-Level Data},
     author = {Zhou*, Helen and Cheng*, Cheng and Weiss, Jeremy C and Lipton, Zachary C},
     booktitle = {AMIA Annual Symposium Proceedings},
     volume = {2021},
     pages = {285},
     year = {2021},
     organization = {American Medical Informatics Association},
   }

2. ICML IMLH

   Do You See What I See? A Comparison of Radiologist Eye Gaze to Computer Vision Saliency Maps for Chest X-ray Classification

   Jesse Kim, Helen Zhou, and Zachary Lipton

   Workshop in Interpretable Machine Learning for Healthcare (IMLH) at International Conference ML 2021, 2021

   Abs Bib HTML PDF Poster

   We qualitatively and quantitatively compare saliency maps generated from state-of-the-art deep learning chest X-ray classification models to radiologist eye gaze data. We find that across several saliency methods, correct predictions have saliency maps more similar to the corresponding eye gaze data than the same for incorrect predictions. To incorporate eye gaze into the model training process, we create DenseNet-Aug, a simple augmentation of DenseNet which performs comparably to the state-of-the-art. Finally, we extract salient annotated regions for each label class, thereby characterizing model attribution at the dataset level. While sample-level saliency maps visibly vary, these dataset-level regional comparisons indicate that across most class labels, radiologist eye gaze, DenseNet, and DenseNet-Aug often identify similar salient regions.

   @article{kim2020do,
     title = {Do You See What I See? A Comparison of Radiologist Eye Gaze to Computer Vision Saliency Maps for Chest X-ray Classification},
     author = {Kim, Jesse and Zhou, Helen and Lipton, Zachary},
     journal = {Workshop in Interpretable Machine Learning for Healthcare (IMLH) at International Conference ML 2021},
     year = {2021},
   }

2020

1. Science

   A decision algorithm to promote outpatient antimicrobial stewardship for uncomplicated urinary tract infection

   Sanjat Kanjilal, Michael Oberst, Sooraj Boominathan, Helen Zhou, David C. Hooper, and David Sontag

   Science Translational Medicine, 2020

   Abs Bib HTML PDF

   Machine learning models for predicting antibiotic resistance could reduce the use of broad-spectrum antibiotics in urinary tract infection. Use of broad-spectrum, second-line antibiotics in treating urinary tract infections (UTIs) is increasing, likely due to the prevalence of antibiotic resistance. Kanjilal et al. applied a machine learning approach calibrated to local hospital electronic health record data to predict the probability of resistance to first- and second-line antibiotic therapies for uncomplicated UTI. The algorithm then recommended the least broad-spectrum antibiotic to which a given isolate was predicted nonresistant. Use of the pipeline reduced both broad-spectrum and ineffective antibiotic prescription for UTI in the patient cohort relative to clinicians, suggesting the clinical potential of the approach. Antibiotic resistance is a major cause of treatment failure and leads to increased use of broad-spectrum agents, which begets further resistance. This vicious cycle is epitomized by uncomplicated urinary tract infection (UTI), which affects one in two women during their life and is associated with increasing antibiotic resistance and high rates of prescription for broad-spectrum second-line agents. To address this, we developed machine learning models to predict antibiotic susceptibility using electronic health record data and built a decision algorithm for recommending the narrowest possible antibiotic to which a specimen is susceptible. When applied to a test cohort of 3629 patients presenting between 2014 and 2016, the algorithm achieved a 67% reduction in the use of second-line antibiotics relative to clinicians. At the same time, it reduced inappropriate antibiotic therapy, defined as the choice of a treatment to which a specimen is resistant, by 18% relative to clinicians. For specimens where clinicians chose a second-line drug but the algorithm chose a first-line drug, 92% (1066 of 1157) of decisions ended up being susceptible to the first-line drug. When clinicians chose an inappropriate first-line drug, the algorithm chose an appropriate first-line drug 47% (183 of 392) of the time. Our machine learning decision algorithm provides antibiotic stewardship for a common infectious syndrome by maximizing reductions in broad-spectrum antibiotic use while maintaining optimal treatment outcomes. Further work is necessary to improve generalizability by training models in more diverse populations.

   @article{doi:10.1126/scitranslmed.aay5067,
     author = {Kanjilal, Sanjat and Oberst, Michael and Boominathan, Sooraj and Zhou, Helen and Hooper, David C. and Sontag, David},
     title = {A decision algorithm to promote outpatient antimicrobial stewardship for uncomplicated urinary tract infection},
     journal = {Science Translational Medicine},
     volume = {12},
     number = {568},
     pages = {eaay5067},
     year = {2020},
     doi = {10.1126/scitranslmed.aay5067},
     url = {https://www.science.org/doi/abs/10.1126/scitranslmed.aay5067},
   }

2. AIME

   Mortality Risk Score for Critically Ill Patients with Viral or Unspecified Pneumonia: Assisting Clinicians with COVID-19 ECMO Planning

   Helen Zhou*, Cheng Cheng*, Zachary C Lipton, George H Chen, and Jeremy C Weiss

   In Artificial Intelligence in Medicine: 18th International Conference on Artificial Intelligence in Medicine, AIME 2020, Minneapolis, MN, USA, August 25–28, 2020, Proceedings 18, 2020

   Bib HTML PDF Poster Slides

   @inproceedings{zhou2020mortality,
     title = {Mortality Risk Score for Critically Ill Patients with Viral or Unspecified Pneumonia: Assisting Clinicians with COVID-19 ECMO Planning},
     author = {Zhou*, Helen and Cheng*, Cheng and Lipton, Zachary C and Chen, George H and Weiss, Jeremy C},
     booktitle = {Artificial Intelligence in Medicine: 18th International Conference on Artificial Intelligence in Medicine, AIME 2020, Minneapolis, MN, USA, August 25--28, 2020, Proceedings 18},
     pages = {336--347},
     year = {2020},
     organization = {Springer},
   }

3. KDD

   Treatment policy learning in multiobjective settings with fully observed outcomes

   Soorajnath Boominathan, Michael Oberst, Helen Zhou, Sanjat Kanjilal, and David Sontag

   In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020

   Abs Bib HTML PDF

   In several medical decision-making problems, such as antibiotic prescription, laboratory testing can provide precise indications for how a patient will respond to different treatment options. This enables us to "fully observe" all potential treatment outcomes, but while present in historical data, these results are infeasible to produce in real-time at the point of the initial treatment decision. Moreover, treatment policies in these settings often need to trade off between multiple competing objectives, such as effectiveness of treatment and harmful side effects. We present, compare, and evaluate three approaches for learning individualized treatment policies in this setting: First, we consider two indirect approaches, which use predictive models of treatment response to construct policies optimal for different trade-offs between objectives. Second, we consider a direct approach that constructs such a set of policies without intermediate models of outcomes. Using a medical dataset of Urinary Tract Infection (UTI) patients, we show that all approaches learn policies that achieve strictly better performance on all outcomes than clinicians, while also trading off between different objectives. We demonstrate additional benefits of the direct approach, including flexibly incorporating other goals such as deferral to physicians on simple cases.

   @inproceedings{boominathan2020treatment,
     title = {Treatment policy learning in multiobjective settings with fully observed outcomes},
     author = {Boominathan, Soorajnath and Oberst, Michael and Zhou, Helen and Kanjilal, Sanjat and Sontag, David},
     booktitle = {Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery \& Data Mining},
     pages = {1937--1947},
     year = {2020},
   }

4. Physionet

   AMR-UTI: antimicrobial resistance in urinary tract infections (version 1.0.0)

   M Oberst, S Boominathan, H Zhou, S Kanjilal, and D Sontag

   2020

   Bib HTML

   @misc{oberst2020amr,
     title = {AMR-UTI: antimicrobial resistance in urinary tract infections (version 1.0.0)},
     author = {Oberst, M and Boominathan, S and Zhou, H and Kanjilal, S and Sontag, D},
     year = {2020},
     publisher = {Physionet},
   }

2019

1. Course Project

   Post-nonlinear Causal Model with Deep Neural Networks

   Youngseog Chung, Joon Kim, Tom Yan, and Helen Zhou

   2019

   Abs Bib PDF

   Causal discovery is a process of discovering causal information from observed data. Previous work in the field has focused on discovering causal directions in two variables, with extra effort on constructing assumptions for the data distribution and proving identifiability. However they lack flexibility in scaling up causal discovery for multiple variables and a systematic endto-end pipeline. We extend two existing work in the literature (LiNGAM and PNL), and by integrating them with deep neural networks, we propose an end-to-end learning scheme for training a model for causal discovery. We evaluate our method on several real and simulated datasets.

   @article{chung2019post,
     title = {Post-nonlinear Causal Model with Deep Neural Networks},
     author = {Chung, Youngseog and Kim, Joon and Yan, Tom and Zhou, Helen},
     year = {2019},
   }

2016

1. Course Project

   Fast-Fi: Designing Large Throughput-Optimized Wireless Networks

   Brandon Carter, Geronimo Mirano, and Helen Zhou

   2016

   Abs Bib PDF

   Large wireless networks often have difficulty handling situations in which many clients in close proximity join the network in a short time. Performance can also degrade when serving clients that have different usage patterns. We propose Fast-Fi, a system that seeks to maximize throughput for clients and provide a better network experience. Fast-Fi employs traffic control mechanisms on both the client and access point (AP) machines, helping clients to choose APs such that their usage is distributed optimally across in-range APs. Additionally, it suggests nearby networks for users if no networks in-range can provide sufficient throughput. Moreover, the system collects usage patterns from all access points into a centralized server database for network administrators. This system is both distributed and scalable such that it can be used in networks even larger than that at MIT. We show that this model improves upon the current MIT network in a variety of common usage scenarios.

   @article{carter2016fast,
     title = {Fast-Fi: Designing Large Throughput-Optimized Wireless Networks},
     author = {Carter, Brandon and Mirano, Geronimo and Zhou, Helen},
     year = {2016},
   }

2015

1. ACL WASSA

   Enhanced Twitter Sentiment Classification Using Contextual Information

   Soroush Vosoughi, Helen Zhou, and Deb Roy

   In Proceedings of the 6th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, Sep 2015

   Abs Bib HTML PDF

   The rise in popularity and ubiquity of Twitter has made sentiment analysis of tweets an important and well-covered area of research. However, the 140 character limit imposed on tweets makes it hard to use standard linguistic methods for sentiment classification. On the other hand, what tweets lack in structure they make up with sheer volume and rich metadata. This metadata includes geolocation, temporal and author information. We hypothesize that sentiment is dependent on all these contextual factors. Different locations, times and authors have different emotional valences. In this paper, we explored this hypothesis by utilizing distant supervision to collect millions of labelled tweets from different locations, times and authors. We used this data to analyse the variation of tweet sentiments across different authors, times and locations. Once we explored and understood the relationship between these variables and sentiment, we used a Bayesian approach to combine these variables with more standard linguistic features such as n-grams to create a Twitter sentiment classifier. This combined classifier outperforms the purely linguistic classifier, showing that integrating the rich contextual information available on Twitter into sentiment classification is a promising direction of research.

   @inproceedings{vosoughi-etal-2015-enhanced,
     title = {Enhanced {T}witter Sentiment Classification Using Contextual Information},
     author = {Vosoughi, Soroush and Zhou, Helen and Roy, Deb},
     booktitle = {Proceedings of the 6th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis},
     month = sep,
     year = {2015},
     address = {Lisboa, Portugal},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/W15-2904},
     doi = {10.18653/v1/W15-2904},
     pages = {16--24},
   }

2. SocInfo

   Digital stylometry: Linking profiles across social networks

   Soroush Vosoughi, Helen Zhou, and Deb Roy

   In Social Informatics: 7th International Conference, SocInfo 2015, Beijing, China, December 9-12, 2015, Proceedings 7, Sep 2015

   Abs Bib HTML PDF

   There is an ever growing number of users with accounts on multiple social media and networking sites. Consequently, there is increasing interest in matching user accounts and profiles across different social networks in order to create aggregate profiles of users. In this paper, we present models for Digital Stylometry, which is a method for matching users through stylometry inspired techniques. We experimented with linguistic, temporal, and combined temporal-linguistic models for matching user accounts, using standard and novel techniques. Using publicly available data, our best model, a combined temporal-linguistic one, was able to correctly match the accounts of 31% of 5,612 distinct users across Twitter and Facebook.

   @inproceedings{vosoughi2015digital,
     title = {Digital stylometry: Linking profiles across social networks},
     author = {Vosoughi, Soroush and Zhou, Helen and Roy, Deb},
     booktitle = {Social Informatics: 7th International Conference, SocInfo 2015, Beijing, China, December 9-12, 2015, Proceedings 7},
     pages = {164--177},
     year = {2015},
     organization = {Springer},
   }