Professional Experience

  • Sept 2024

    HAI Postdoc Fellow

    Stanford University, Institute for Human-Centered Artificial Intelligence (HAI), CA, USA



  • 2024 Jun 2018 Sept

    Research Assistant

    Dartmouth College, NH, USA



  • 2023 Sep 2023 Jun

    Research Intern

    Microsoft Research, Cambridge, MA



  • 2022 Sep 2022 Jun

    Research Intern

    Microsoft Research, Redmond, WA

    Remote Internship



  • 2018 2015

    Co-founder and CTO

    TechLekh Services Pvt. Ltd., Kathmandu, Nepal

Education

  • Ph.D. 2018 - 2024

    Ph.D. in Computer Science

    Dartmouth College, USA

  • B.Sc.2017

    Bachelors in Computer Science & Information Technology

    Tribhuwan University, Nepal

Updates

  • I'll be presenting the Burnout in Cybersecurity Incident Responders paper at CSCW 2024 in San Jose, Costa Rica in November 2024.

  • The final chapter of my PhD thesis just got accepted at UbiComp 2025. MindScape Study is the first to combine mobile phone sensing + LLM to generate contextual journaling prompts to improve student wellbeing.

  • I attended UbiComp 2024 at Melbourne, Australia presenting the College Experience Study paper. Fun fact: my PhD advisor Andrew Campbell and co-authors of the seminal StudentLife paper won a 10-year impact award in this event.

  • I joined Stanford Institute for Human-Centered AI as a postdoc. I'll be mentored by Dr. Gabriella Harari.

  • I attended CHI 2024 at Hawaii to present our LBW paper on Contextual AI journaling.

  • I successfully defended my PhD thesis (without any revisions) Toward the Integration of Behavioral Sensing and Artificial Intelligence paper. Thankful to my advisor and my committee, Dr. Mary Czerwinski (Microsoft Research), Dr. Nicholas Jacobson (Dartmouth) and Dr. Sarah Preum (Dartmouth).

  • My final dissertation work's study/system design was accepted in CHI 2024 as a Late Breaking Work. MindScape Study leverages AI and Sensing to improve student wellbeing.

  • The College Experience Study paper was accepted at UbiComp 2024. This study tracks 200+ Dartmouth students over 4 years using mobile sensing, self-reports, surveys, and brain imaging. This is the longest mobile sensing study to date. I have also led the effort to publicly release the dataset. I hope this dataset and study will be instrumental in advancing the study of college students mental health.

  • My first time on-air! MoodCapture study received considerable media attention and I also got to speak about it on iHeartRadio.

Filter by type:

Sort by year:

MindScape Study: Integrating LLM and Behavioral Sensing for Personalized AI-Driven Journaling Experiences

Subigya Nepal, Arvind Pillai, William Campbell, Talie Massachi, Michael V. Heinz, Ashmita Kunwar, Eunsol Soul Choi, Orson Xu, Joanna Kuc, Jeremy Huckins, Jason Holden, Sarah M. Preum, Colin Depp, Nicholas Jacobson, Mary Czerwinski, Eric Granholm, Andrew T. Campbell
Conference PapersProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2025 (In-Press)

Abstract

Mental health concerns are prevalent among college students, highlighting the need for effective interventions that promote self-awareness and holistic well-being. MindScape pioneers a novel approach to AI-powered journaling by integrating passively collected behavioral patterns such as conversational engagement, sleep, and location with Large Language Models (LLMs). This integration creates a highly personalized and context-aware journaling experience, enhancing self-awareness and well-being by embedding behavioral intelligence into AI. We present an 8-week exploratory study with 20 college students, demonstrating the MindScape app's efficacy in enhancing positive affect (7%), reducing negative affect (11%), loneliness (6%), and anxiety and depression, with a significant week-over-week decrease in PHQ-4 scores (-0.25 coefficient), alongside improvements in mindfulness (7%) and self-reflection (6%). The study highlights the advantages of contextual AI journaling, with participants particularly appreciating the tailored prompts and insights provided by the MindScape app. Our analysis also includes a comparison of responses to AI-driven contextual versus generic prompts, participant feedback insights, and proposed strategies for leveraging contextual AI journaling to improve well-being on college campuses. By showcasing the potential of contextual AI journaling to support mental health, we provide a foundation for further investigation into the effects of contextual AI journaling on mental health and well-being.

From User Surveys to Telemetry-Driven AI Agents: Exploring the Potential of Personalized Productivity Solutions

Subigya Nepal, Javier Hernandez, Talie Massachi, Kael Rowan, Judith Amores, Jina Suh, Gonzalo Ramos, Brian Houck, Shamsi T. Iqbal, Mary Czerwinski
Conference Papers Proceedings of the ACM on Human-Computer Interaction (PACM HCI) CSCW. 2025. (Under Review).

Abstract

We present a comprehensive, user-centric approach to understand preferences in AI-based productivity agents and develop personalized solutions tailored to users' needs. Utilizing a two-phase method, we first conducted a survey with 363 participants, exploring various aspects of productivity, communication style, agent approach, personality traits, personalization, and privacy. Drawing on the survey insights, we developed a GPT-4 powered personalized productivity agent that utilizes telemetry data gathered via Viva Insights from information workers to provide tailored assistance. We compared its performance with alternative productivity-assistive tools, such as dashboard and narrative, in a study involving 40 participants. Our findings highlight the importance of user-centric design, adaptability, and the balance between personalization and privacy in AI-assisted productivity tools. By building on the insights distilled from our study, we believe that our work can enable and guide future research to further enhance productivity solutions, ultimately leading to optimized efficiency and user experiences for information workers.

Burnout in Cybersecurity Incident Responders: Exploring the Factors that Light the Fire

Subigya Nepal, Javier Hernandez, Robert Lewis, Ahad Chaudhry, Brian Houck, Eric Knudsen, Raul Rojas, Ben Tankus, Hemma Prafullchandra, Mary P Czerwinski
Conference Papers Proceedings of the ACM on Human-Computer Interaction (PACM HCI) CSCW. September 2024.

Abstract

As concerns about employee burnout and skilled staff shortages in cybersecurity grow, our study aims to better understand the contributing factors to burnout in this field. Utilizing a mixed-methods approach, we analyze self-reported job and personal characteristics, along with digital activity data from 35 incident responders, identifying several factors such as high workload, time pressure, and lack of support from management. Our findings reveal that over half of the participants experience burnout (N=19), which is linked to increased workload, limited control, poor teamwork, and inadequate recognition. Burned-out responders often work more than 40 hours per week, have poor sleep quality, and engage in more email activities, meetings, and after-hour collaborations. Through our research, we also identify coping strategies individuals use to mitigate these stressors. Based on our findings, we provide practical recommendations to help organizations better support their cybersecurity incident response teams. While our study acknowledges limitations and suggests future research directions, it contributes significantly to understanding the challenges faced by cybersecurity incident responders. Our insights offer a comprehensive understanding of burnout factors in this domain and have broader implications for other high-stress work environments consistent with the interdisciplinary nature of CSCW.

Social Isolation and Serious Mental Illness: The Role of Context-Aware Mobile Interventions

Subigya Nepal, Arvind Pillai, Emma M Parrish, Jason Holden, Colin Depp, Andrew T Campbell, Eric Granholm
Magazine Paper IEEE Pervasive Computing, Special issue on Future of Work, March 2024 | Impact Factor: 3.175

Abstract

Social isolation is a common problem faced by individuals with serious mental illness (SMI), and current intervention approaches have limited effectiveness. This paper presents a blended intervention approach, called mobile Social Interaction Therapy by Exposure (mSITE), to address social isolation in individuals with serious mental illness. The approach combines brief in-person cognitive-behavioral therapy (CBT) with context-triggered mobile CBT interventions that are personalized using mobile sensing data. Our approach targets social behavior and is the first context-aware intervention for improving social outcomes in serious mental illness.

Capturing the College Experience: A Four-Year Mobile Sensing Study of Mental Health, Resilience and Behavior of College Students during the Pandemic

Subigya Nepal, Wenjun Liu, Arvind Pillai, Weichen Wang, Vlado Vojdanovski, Jeremy F Huckins, Courtney Rogers, Meghan Meyer, Andrew Campbell.
Conference PapersProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 8, Issue 1, March 2024

Abstract

Understanding the dynamics of mental health among undergraduate students across the college years is of critical importance, particularly during a global pandemic. In our study, we track two cohorts of first-year students at Dartmouth College for four years, both on and off campus, creating the longest longitudinal mobile sensing study to date. Using passive sensor data, surveys, and interviews, we capture changing behaviors before, during, and after the COVID-19 pandemic subsides. Our findings reveal the pandemic's impact on students' mental health, gender based behavioral differences, impact of changing living conditions and evidence of persistent behavioral patterns as the pandemic subsides. We observe that while some behaviors return to normal, others remain elevated. Tracking over 200 undergraduate students from high school to graduation, our study provides invaluable insights into changing behaviors, resilience and mental health in college life. Conducting a long-term study with frequent phone OS updates poses significant challenges for mobile sensing apps, data completeness and compliance. Our results offer new insights for Human-Computer Interaction researchers, educators and administrators regarding college life pressures. We also detail the public release of the de-identified College Experience Study dataset used in this paper and discuss a number of open research questions that could be studied using the public dataset.

Contextual AI Journaling: Integrating LLM and Time Series Behavioral Sensing Technology to Promote Self-Reflection and Well-being using the MindScape App

Subigya Nepal, Arvind Pillai, William Campbell, Talie Massachi, Eunsol Soul Choi, Xuhai Xu, Joanna Kuc, Jeremy F Huckins, Jason Holden, Colin Depp, Nicholas Jacobson, Mary P Czerwinski, Eric Granholm, Andrew T Campbell
Conference PapersExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems, May 2024

Abstract

MindScape aims to study the benefits of integrating time series behavioral patterns (e.g., conversational engagement, sleep, location) with Large Language Models (LLMs) to create a new form of contextual AI journaling, promoting self-reflection and well-being. We argue that integrating behavioral sensing in LLMs will likely lead to a new frontier in AI. In this Late-Breaking Work paper, we discuss the MindScape contextual journal App design that uses LLMs and behavioral sensing to generate contextual and personalized journaling prompts crafted to encourage self-reflection and emotional development. We also discuss the MindScape study of college students based on a preliminary user study and our upcoming study to assess the effectiveness of contextual AI journaling in promoting better well-being on college campuses. MindScape represents a new application class that embeds behavioral intelligence in AI.

A Survey of Passive Sensing in the Workplace

Subigya Nepal, Gonzalo J. Martinez, Arvind Pillai, Koustuv Saha, Shayan Mirjafari, Vedant Das Swain, Xuhai Xu, Pino G. Audia, Munmun De Choudhury, Anind K. Dey, Aaron Striegel, Andrew T. Campbell
arXivAvailable on arXiv, 2024

Abstract

As emerging technologies increasingly integrate into all facets of our lives, the workplace stands at the forefront of potential transformative changes. A notable development in this realm is the advent of passive sensing technology, designed to enhance both cognitive and physical capabilities by monitoring human behavior. This paper reviews current research on the application of passive sensing technology in the workplace, focusing on its impact on employee wellbeing and productivity. Additionally, we explore unresolved issues and outline prospective pathways for the incorporation of passive sensing in future workplaces.

MoodCapture: Depression Detection Using In-the-Wild Smartphone Images

Subigya Nepal, Arvind Pillai, Weichen Wang, Tess Griffin, Amanda C. Collins, Michael Heinz, Damien Lekkas, Shayan Mirjafari, Matthew Nemesure, George Price, Nicholas C. Jacobson, Andrew T. Campbell
Conference PapersProceedings of the 2024 CHI Conference on Human Factors in Computing Systems, May 2024

Abstract

MoodCapture presents a novel approach that assesses depression based on images automatically captured from the front-facing camera of smartphones as people go about their daily lives. We collect over 125,000 photos in the wild from N=177 participants diagnosed with major depressive disorder for 90 days. Images are captured naturalistically while participants respond to the PHQ-8 depression survey question: "I have felt down, depressed, or hopeless"". Our analysis explores important image attributes, such as angle, dominant colors, location, objects, and lighting. We show that a random forest trained with face landmarks can classify samples as depressed or non-depressed and predict raw PHQ-8 scores effectively. Our post-hoc analysis provides several insights through an ablation study, feature importance analysis, and bias assessment. Importantly, we evaluate user concerns about using MoodCapture to detect depression based on sharing photos, providing critical insights into privacy concerns that inform the future design of in-the-wild image-based mental health assessment tools.

Investigating Generalizability of Speech-based Suicidal Ideation Detection Using Mobile Phones

Arvind Pillai, Subigya Nepal, Weichen Wang, Matthew Nemesure, Michael Heinz, George Price, Damien Lekkas, Amanda C Collins, Tess Griffin, Benjamin Buck, Sarah Masud Preum, Trevor Cohen, Nicholas C Jacobson, Dror Ben-Zeev, Andrew Campbell
Conference PapersProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 7, Issue 4, January 2024

Abstract

Speech-based diaries from mobile phones can capture paralinguistic patterns that help detect mental illness symptoms such as suicidal ideation. However, previous studies have primarily evaluated machine learning models on a single dataset, making their performance unknown under distribution shifts. In this paper, we investigate the generalizability of speech-based suicidal ideation detection using mobile phones through cross-dataset experiments using four datasets with N=786 individuals experiencing major depressive disorder, auditory verbal hallucinations, persecutory thoughts, and students with suicidal thoughts. Our results show that machine and deep learning methods generalize poorly in many cases. Thus, we evaluate unsupervised domain adaptation (UDA) and semi-supervised domain adaptation (SSDA) to mitigate performance decreases owing to distribution shifts. While SSDA approaches showed superior performance, they are often ineffective, requiring large target datasets with limited labels for adversarial and contrastive training. Therefore, we propose sinusoidal similarity sub-sampling (S3), a method that selects optimal source subsets for the target domain by computing pair-wise scores using sinusoids. Compared to prior approaches, S3 does not use labeled target data or transform features. Fine-tuning using S3 improves the cross-dataset performance of deep models across the datasets, thus having implications in ubiquitous technology, mental health, and machine learning.

The Power of Speech in the Wild: Discriminative Power of Daily Voice Diaries in Understanding Auditory Verbal Hallucinations Using Deep Learning

Weichen Wang, Weizhe Xu, Ayesha Chander, Subigya Nepal, Benjamin Buck, Serguei Pakhomov, Trevor Cohen, Dror Ben-Zeev, Andrew Campbell
Conference PapersProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 7, Issue 3, September 2023

Abstract

Mobile phone sensing is increasingly being used in clinical research studies to assess a variety of mental health conditions (e.g., depression, psychosis). However, in-the-wild speech analysis -- beyond conversation detecting -- is a missing component of these mobile sensing platforms and studies. We augment an existing mobile sensing platform with a daily voice diary to assess and predict the severity of auditory verbal hallucinations (i.e., hearing sounds or voices in the absence of any speaker), a condition that affects people with and without psychiatric or neurological diagnoses. We collect 4809 audio diaries from N=384 subjects over a one-month-long study period. We investigate the performance of various deep-learning architectures using different combinations of sensor behavioral streams (e.g., voice, sleep, mobility, phone usage, etc.) and show the discriminative power of solely using audio recordings of speech as well as automatically generated transcripts of the recordings; specifically, our deep learning model achieves a weighted f-1 score of 0.78 solely from daily voice diaries. Our results surprisingly indicate that a simple periodic voice diary combined with deep learning is sufficient enough of a signal to assess complex psychiatric symptoms (e.g., auditory verbal hallucinations) collected from people in the wild as they go about their daily lives.

Rare Life Event Detection via Mobile Sensing Using Multi-Task Learning

Arvind Pillai, Subigya Nepal, Andrew Campbell
Conference PapersConference on Health, Inference, and Learning (CHIL), 2023

Abstract

Rare life events significantly impact mental health, and their detection in behavioral studies is a crucial step towards health-based interven- tions. We envision that mobile sensing data can be used to detect these anomalies. However, the human-centered nature of the problem, com- bined with the infrequency and uniqueness of these events makes it challenging for unsuper- vised machine learning methods. In this pa- per, we first investigate granger-causality be- tween life events and human behavior using sensing data. Next, we propose a multi-task framework with an unsupervised autoencoder to capture irregular behavior, and an auxil- iary sequence predictor that identifies transi- tions in workplace performance to contextu- alize events. We perform experiments using data from a mobile sensing study comprising N=126 information workers from multiple in- dustries, spanning 10106 days with 198 rare events (< 2%). Through personalized inference, we detect the exact day of a rare event with an F1 of 0.34, demonstrating that our method out- performs several baselines. Finally, we discuss the implications of our work from the context of real-world deployment.

Dissociation of Cognitive Effort–Based Decision Making and Its Associations With Symptoms, Cognition, and Everyday Life Function Across Schizophrenia, Bipolar Disorder, and Depression

Deanna M. Barch, Adam J. Culbreth, Dror Ben Zeev, Andrew Campbell, Subigya Nepal, Erin K. Moran
Journal Paper Biological Psychiatry, Volume 94, Issue 6, September 2023 | Impact Factor: 10.6

Abstract

Anhedonia and amotivation are symptoms of many different mental health disorders that are frequently associated with functional disability, but it is not clear whether the same processes contribute to motivational impairments across disorders. This study focused on one possible factor, the willingness to exert cognitive effort, referred to as cognitive effort–cost decision making. We examined performance on the deck choice task as a measure of cognitive effort–cost decision making, in which people choose to complete an easy task for a small monetary reward or a harder task for larger rewards, in 5 groups: healthy control (n = 80), schizophrenia/schizoaffective disorder (n = 50), bipolar disorder with psychosis (n = 58), current major depression (n = 60), and past major depression (n = 51). We examined cognitive effort–cost decision making in relation to clinician and self-reported motivation symptoms, working memory and cognitive control performance, and life function measured by ecological momentary assessment and passive sensing.

Workplace Rhythm Variability and Emotional Distress in Information Workers

Subigya Nepal, Javier Hernandez, Judith Amores Fernandez, Mehrab Bin Morshed, Robert Lewis, Hemma Prafullchandra, Mary P Czerwinski
Conference PapersExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, April 2023

Abstract

Regularity in daily activities has been linked to positive well-being outcomes, but previous studies have mostly focused on clinical populations and traditional daily activities such as sleep and exercise. This research extends prior work by examining the regularity of both self-reported and digital activities of 49 information workers in a 4-week naturalistic study. The data was analyzed in conjunction with self-reported stress, anxiety, and depression scores. The findings suggest that greater variability in self-reported mood, job demands, lunch time, and sleep quality may be associated with increased stress, anxiety, and depression. However, for digital activity-based measures, we find that the opposite is true---greater variability in digital rhythm is related with reduced emotional distress. This study expands our understanding of workers and the potential insights that can be gained from analyzing technology interactions and well-being.

GLOBEM: Cross-Dataset Generalization of Longitudinal Human Behavior Modeling

Xuhai Xu, Xin Liu, Han Zhang, Weichen Wang, Subigya Nepal, Yasaman Sefidgar, Woosuk Seo, Kevin S Kuehn, Jeremy F Huckins, Margaret E Morris, Paula S Nurius, Eve A Riskin, Shwetak Patel, Tim Althoff, Andrew Campbell, Anind K Dey, Jennifer Mankoff
Conference PapersProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 6, Issue 4, January 2023 | Distinguished Paper Award

Abstract

There is a growing body of research revealing that longitudinal passive sensing data from smartphones and wearable devices can capture daily behavior signals for human behavior modeling, such as depression detection. Most prior studies build and evaluate machine learning models using data collected from a single population. However, to ensure that a behavior model can work for a larger group of users, its generalizability needs to be verified on multiple datasets from different populations. We present the first work evaluating cross-dataset generalizability of longitudinal behavior models, using depression detection as an application. We collect multiple longitudinal passive mobile sensing datasets with over 500 users from two institutes over a two-year span, leading to four institute-year datasets. Using the datasets, we closely re-implement and evaluated nine prior depression detection algorithms. Our experiment reveals the lack of model generalizability of these methods. We also implement eight recently popular domain generalization algorithms from the machine learning community. Our results indicate that these methods also do not generalize well on our datasets, with barely any advantage over the naive baseline of guessing the majority. We then present two new algorithms with better generalizability. Our new algorithm, Reorder, significantly and consistently outperforms existing methods on most cross-dataset generalization setups. However, the overall advantage is incremental and still has great room for improvement. Our analysis reveals that the individual differences (both within and between populations) may play the most important role in the cross-dataset generalization challenge. Finally, we provide an open-source benchmark platform GLOBEM- short for Generalization of Longitudinal BEhavior Modeling - to consolidate all 19 algorithms. GLOBEM can support researchers in using, developing, and evaluating different longitudinal behavior modeling methods. We call for researchers' attention to model generalizability evaluation for future longitudinal human behavior modeling studies.

First-Gen Lens: Assessing Mental Health of First-Generation Students across Their First Year at College Using Mobile Sensing

Weichen Wang, Subigya Nepal, Jeremy F Huckins, Lessley Hernandez, Vlado Vojdanovski, Dante Mack, Jane Plomp, Arvind Pillai, Mikio Obuchi, Alex daSilva, Eilis Murphy, Elin Hedlund, Courtney Rogers, Meghan Meyer, Andrew Campbell
Conference PapersProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 6, Issue 2, July 2022

Abstract

The transition from high school to college is a taxing time for young adults. New students arriving on campus navigate a myriad of challenges centered around adapting to new living situations, financial needs, academic pressures and social demands. First-year students need to gain new skills and strategies to cope with these new demands in order to make good decisions, ease their transition to independent living and ultimately succeed. In general, first-generation students are less prepared when they enter college in comparison to non-first-generation students. This presents additional challenges for first-generation students to overcome and be successful during their college years. We study first-year students through the lens of mobile phone sensing across their first year at college, including all academic terms and breaks. We collect longitudinal mobile sensing data for N=180 first-year college students, where 27 of the students are first-generation, representing 15% of the study cohort and representative of the number of first-generation students admitted each year at the study institution, Dartmouth College. We discuss risk factors, behavioral patterns and mental health of first-generation and non-first-generation students. We propose a deep learning model that accurately predicts the mental health of first-generation students by taking into account important distinguishing behavioral factors of first-generation students. Our study, which uses the StudentLife app, offers data-informed insights that could be used to identify struggling students and provide new forms of phone-based interventions with the goal of keeping students on track.

COVID student study: A year in the life of college students during the COVID-19 pandemic through the lens of mobile phone sensing

Subigya Nepal, Weichen Wang, Vlado Vojdanovski, Jeremy F Huckins, Alex daSilva, Meghan Meyer, and Andrew Campbell
Conference PapersProceedings of the 2022 CHI Conference on Human Factors in Computing Systems, April 2022

Abstract

The COVID-19 pandemic continues to affect the daily life of college students, impacting their social life, education, stress levels and overall mental well-being. We study and assess behavioral changes of N=180 undergraduate college students one year prior to the pandemic as a baseline and then during the first year of the pandemic using mobile phone sensing and behavioral inference. We observe that certain groups of students experience the pandemic very differently. Furthermore, we explore the association of self-reported COVID-19 concern with students’ behavior and mental health. We find that heightened COVID-19 concern is correlated with increased depression, anxiety and stress. We evaluate the performance of different deep learning models to classify student COVID-19 concerns with an AUROC and F1 score of 0.70 and 0.71, respectively. Our study spans a two-year period and provides a number of important insights into the life of college students during this period.

A Smartphone Intervention for People With Serious Mental Illness: Fully Remote Randomized Controlled Trial of CORE

Dror Ben-Zeev, Ayesha Chander, Justin Tauscher, Benjamin Buck, Subigya Nepal, Andrew Campbell, Guy Doron
Journal Paper Journal of Medical Internet Research, Volume 23, Issue 11, November 2021 | Impact Factor: 7.08

Abstract

People with serious mental illness (SMI) have significant unmet mental health needs. Development and testing of digital interventions that can alleviate the suffering of people with SMI is a public health priority. The aim of this study is to conduct a fully remote randomized waitlist-controlled trial of CORE, a smartphone intervention that comprises daily exercises designed to promote reassessment of dysfunctional beliefs in multiple domains.

Predicting Job Performance Using Mobile Sensing

Shayan Mirjafari, Hessam Bagherinezhad, Subigya Nepal, Gonzalo J. Martinez, Koustuv Saha, Mikio Obuchi, Pino G. Audia, Nitesh V. Chawla, Anind K. Dey, Aaron Striegel, Andrew T Campbell
Magazine Paper IEEE Pervasive Computing, Special issue on Future of Work, November 2021 | Impact Factor: 3.175

Abstract

We hypothesize that behavioral patterns of people are reflected in how they interact with their mobile devices and that continuous sensor data passively collected from their phones and wearables can infer their job performance. Specifically, we study day-today job performance (improvement, no change, decline) of N=298 information workers using mobile sensing data and offer data-driven insights into what data patterns may lead to a high-performing day. Through analyzing workers' mobile sensing data, we predict their performance on a handful of job performance questionnaires with an F-1 score of 75%. In addition, through numerical analysis of the model, we get insights into how individuals must change their behavior so that the model predicts improvements in their job performance. For instance, one worker may benefit if they put their phone down and reduce their screen time, while another worker may benefit from getting more sleep.

On the Transition of Social Interaction from In-Person to Online: Predicting Changes in Social Media Usage of College Students during the COVID-19 Pandemic based on Pre-COVID-19 On-Campus Colocation

Weichen Wang, Jialing Wu, Subigya Kumar Nepal, Alex daSilva, Elin Hedlund, Eilis Murphy, Courtney Rogers, Jeremy F Huckins
Conference Paper Proceedings of the 2021 International Conference on Multimodal Interaction, October 2021

Abstract

Pandemics significantly impact human daily life. People throughout the world adhere to safety protocols (e.g., social distancing and self-quarantining). As a result, they willingly keep distance from workplace, friends and even family. In such circumstances, in-person social interactions may be substituted with virtual ones via online channels, such as, Instagram and Snapchat. To get insights into this phenomenon, we study a group of undergraduate students before and after the start of COVID-19 pandemic. Specifically, we track N=102 undergraduate students on a small college campus prior to the pandemic using mobile sensing from phones and assign semantic labels to each location they visit on campus where they study, socialize and live. By leveraging their colocation network at these various semantically labeled places on campus, we find that colocations at certain places that possibly proxy higher in-person social interactions (e.g., dormitories, gyms and Greek houses) show significant predictive capability in identifying the individuals’ change in social media usage during the pandemic period. We show that we can predict student’s change in social media usage during COVID-19 with an F1 score of 0.73 purely from the in-person colocation data generated prior to the pandemic.

Assessing the Impact of Commuting on Workplace Performance Using Mobile Sensing

Subigya Nepal, Gonzalo J Martinez, Shayan Mirjafari, Stephen Mattingly, Vedant Das Swain, Aaron Striegel, Pino G Audia, Andrew T Campbell
Magazine Paper IEEE Pervasive Computing, Special issue on Future of Work, October 2021 | Impact Factor: 3.175

Abstract

Commuting to and from work presents daily stressors for most workers. It is typically demanding in terms of time and cost, and can impact people’s mental health, job performance, and, broadly speaking, personal life. We use mobile phones and wearable sensing to capture location-related context, physiology, and behavioral patterns of N=275 information workers while they commute, mainly by driving, between home and work locations spread across the United States for a one-year period. We assess the impact of commuting on participant’s workplace performance, showing that we can predict self-reported workplace performance metrics based on passively collected mobile-sensing features captured during commute periods.

Current practices in mental health sensing

Subigya Nepal, Weichen Wang, Bishal Sharma, Prabesh Paudel
Magazine Paper XRDS: Crossroads, The ACM Magazine for Students, Volume 28, Issue 1, September 2021

Abstract

The ubiquity of smartphones and wearables makes it an attractive option to passively study human behavior. We explore the current practices of using passive sensing devices to assess mental health and wellbeing, including the limitations and future directions.

Mental Health and Behavior of College Students During the COVID-19 Pandemic: Longitudinal Mobile Smartphone and Ecological Momentary Assessment Study, Part II

Dante L Mack, Alex W DaSilva, Courtney Rogers, Elin Hedlund, Eilis I Murphy, Vlado Vojdanovski, Jane Plomp, Weichen Wang, Subigya K Nepal, Paul E Holtzheimer, Dylan D Wagner, Nicholas C Jacobson, Meghan L Meyer, Andrew T Campbell, Jeremy F Huckins
Journal Paper Journal of Medical Internet Research, Volume 23, Issue 6, June 2021 | Impact Factor: 7.08

Abstract

In the initial lockdown phase of the COVID-19 pandemic, people spent more time on their phones, were more sedentary, visited fewer locations, and exhibited increased symptoms of anxiety and depression. As the pandemic persisted through the spring, people continued to exhibit very similar changes in both mental health and behaviors. Although these large-scale shifts in mental health and behaviors are unsurprising, understanding them is critical in disrupting the negative consequences to mental health during the ongoing pandemic.

Detecting Job Promotion in Information Workers Using Mobile Sensing

Subigya Nepal, Shayan Mirjafari, Gonzalo J. Martinez, Pino Audia, Aaron Striegel, and Andrew T. Campbell
Conference Papers Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 4, Issue 3, September 2020

Abstract

Most people desire promotions in the workplace. Typically, rising through the ranks comes with increased demands, better salary and higher status among peers. However, promoted workers have to deal with new challenges, such as, adjusting to new roles and responsibilities, which can in turn impact their physical and mental wellbeing. In this year long study, we use mobile sensing to track physiological and behavioral patterns of N=141 information workers who are promoted. We show that the workers experience a change in their physiological and behavioral patterns after promotion captured by passive sensing from phones, wearables and Bluetooth beacons. Furthermore, we use a random convolutions based approach to extract patterns from multivariate time series signals and evaluate the performance of different models to classify a worker’s mobile sensing data as belonging to a promoted or non-promoted period with an AUC of 0.72. As a result, we report for the first time that mobile sensing can detect job promotion events by modeling physiological and behavioral changes of information workers in an objective manner.

Differentiating Higher and Lower Job Performers in the Workplace Using Mobile Sensing

Shayan Mirjafari, Kizito Masaba, Ted Grover, Weichen Wang, Pino Audia, Andrew T Campbell, Nitesh V Chawla, Vedant Das Swain, Munmun De Choudhury, Anind K Dey, Sidney K D'Mello, Ge Gao, Julie M Gregg, Krithika Jagannath, Kaifeng Jiang, Suwen Lin, Qiang Liu, Gloria Mark, Gonzalo J Martinez, Stephen M Mattingly, Edward Moskal, Raghu Mulukutla, Subigya Nepal, Kari Nies, Manikanta D Reddy, Pablo Robles-Granda, Koustuv Saha, Anusha Sirigiri, Aaron Striegel
Conference Papers Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 3, Issue 2, June 2019

Abstract

Assessing performance in the workplace typically relies on subjective evaluations, such as, peer ratings, supervisor ratings and self assessments, which are manual, burdensome and potentially biased. We use objective mobile sensing data from phones, wearables and beacons to study workplace performance and offer new insights into behavioral patterns that distinguish higher and lower performers when considering roles in companies (i.e., supervisors and non-supervisors) and different types of companies (i.e., high tech and consultancy). We present initial results from an ongoing year-long study of N=554 information workers collected over a period ranging from 2-8.5 months. We train a gradient boosting classifier that can classify workers as higher or lower performers with AUROC of 0.83. Our work opens the way to new forms of passive objective assessment and feedback to workers to potentially provide week by week or quarter by quarter guidance in the workplace.

A Multisensor Person-Centered Approach to Understand the Role of Daily Activities in Job Performance with Organizational Personas

Vedant Das Swain, Koustuv Saha, Hemang Rajvanshy, Anusha Sirigiri, Julie M Gregg, Suwen Lin, Gonzalo J Martinez, Stephen M Mattingly, Shayan Mirjafari, Raghu Mulukutla, Subigya Nepal, Kari Nies, Manikanta D Reddy, Pablo Robles-Granda, Andrew T Campbell, Nitesh V Chawla, Sidney D'Mello, Anind K Dey, Kaifeng Jiang, Qiang Liu, Gloria Mark, Edward Moskal, Aaron Striegel, Louis Tay, Gregory D Abowd, Munmun De Choudhury
Conference Papers Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 3, Issue 4, December 2019

Abstract

Several psychologists posit that performance is not only a function of personality but also of situational contexts, such as day-level activities. Yet in practice, since only personality assessments are used to infer job performance, they provide a limited perspective by ignoring activity. However, multi-modal sensing has the potential to characterize these daily activities. This paper illustrates how empirically measured activity data complements traditional effects of personality to explain a worker’s performance. We leverage sensors in commodity devices to quantify the activity context of 603 information workers. By applying classical clustering methods on this multisensor data, we take a person-centered approach to describe workers in terms of both personality and activity. We encapsulate both these facets into an analytical framework that we call organizational personas. On interpreting these organizational personas we find empirical evidence to support that, independent of a worker’s personality, their activity is associated with job performance. While the effects of personality are consistent with the literature, we find that the activity is equally effective in explaining organizational citizenship behavior and is less but significantly effective for task proficiency and deviant behaviors. Specifically, personas that exhibit a daily-activity pattern with fewer location visits, batched phone-use, shorter desk-sessions and longer sleep duration, tend to perform better on all three performance metrics. Organizational personas are a descriptive framework to identify the testable hypotheses that can disentangle the role of malleable aspects like activity in determining the performance of a worker population.

Mental Health and Behavior of College Students During the Early Phases of the COVID-19 Pandemic: Longitudinal Smartphone and Ecological Momentary Assessment Study

Jeremy F Huckins, Alex W DaSilva, Weichen Wang, Elin Hedlund, Courtney Rogers, Subigya K Nepal, Jialing Wu, Mikio Obuchi, Eilis I Murphy, Meghan L Meyer, Dylan D Wagner, Paul E Holtzheimer, Andrew T Campbell
Journal Paper Journal of Medical Internet Research, Volume 22, Issue 6, June 2020 | Impact Factor: 7.08

Abstract

The vast majority of people worldwide have been impacted by coronavirus disease (COVID-19). In addition to the millions of individuals who have been infected with the disease, billions of individuals have been asked or required by local and national governments to change their behavioral patterns. Previous research on epidemics or traumatic events suggests that this can lead to profound behavioral and mental health changes; however, researchers are rarely able to track these changes with frequent, near-real-time sampling or compare their findings to previous years of data for the same individuals. By combining mobile phone sensing and self-reported mental health data among college students who have been participating in a longitudinal study for the past 2 years, we sought to answer two overarching questions. First, have the behaviors and mental health of the participants changed in response to the COVID-19 pandemic compared to previous time periods? Second, are these behavior and mental health changes associated with the relative news coverage of COVID-19 in the US media?

On the Quality of Real-world Wearable Data in a Longitudinal Study of Information Workers

Gonzalo J Martinez, Stephen M Mattingly, Shayan Mirjafari, Subigya K Nepal, Andrew T Campbell, Anind K Dey, Aaron D Striegel
Conference PapersIEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), August 2020

Abstract

Over the past few years, an incredible diversity of consumer-grade wearables has emerged with tremendous breadth in capabilities, form factor, and cost. These wearable devices show significant promise for researchers to conduct expansive research studies in terms of scale, scope, and duration. Unfortunately, there is limited public data shared with respect to the data quality, device longevity, and scaling issues that emerge when trying to execute such studies. To that end, we share real-world results with respect to data quality, participant compliance, and device efficacy on a large scale, longitudinal study involving over seven hundred and fifty working professionals over the period of an entire year. In this paper, we present analyses with respect to the different types of data being collected including sleep, heart rate, physical activity, and stress. Furthermore, we explore participants behavior regarding charging frequency, and device robustness to further aid researchers considering large scale wearable studies.

Improved Sleep Detection Through the Fusion of Phone Agent and Wearable Data Streams

Gonzalo J Martinez, Stephen M Mattingly, Jessica Young, Louis Faust, Anind K Dey, Andrew T Campbell, Munmun De Choudhury, Shayan Mirjafari, Subigya K Nepal, Pablo Robles-Granda, Koustuv Saha, Aaron D Striegel
Conference PapersIEEE Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Aug 2020 | Best Paper Honorable Mention

Abstract

Commercial grade activity trackers and phone agents are increasingly being deployed as sensors for sleep in large scale, longitudinal designs. In general, wearables detect sleep through diminished movement and decreased heart rate (HR), while phone agents look for lack of user input, movement, sound or light. However, recent literature suggests that commercial-grade wearables and phone apps vary greatly in the accuracy of sleep predictions. Constant innovation in wearables and proprietary algorithms further make it difficult to evaluate their efficacy for scientific study, especially outside of the laboratory. In a longitudinal study, we find that wearables cannot detect when a person is laying still but using their phones, a common behavior, overestimating sleep when compared to self-reports. Therefore, we propose that fusing wearables and phone sensors allows for more accurate sleep detection by capitalizing on the benefits of both streams: combining the movement detection of wearables with the technology usage detected by cell phones. We determine that fusing phone activity to wearables can generate better models of self-reported sleep than either stream alone, and test models in two separate datasets.

Social Sensing: Assessing Social Functioning of Patients Living with Schizophrenia using Mobile Phone Sensing

Weichen Wang, Shayan Mirjafari, Gabriella Harari, Dror Ben-Zeev, Rachel Brian, Tanzeem Choudhury, Marta Hauser, John Kane, Kizito Masaba, Subigya Nepal, Akane Sano, Emily Scherer, Vincent Tseng, Rui Wang, Hongyi Wen, Jialing Wu, Andrew Campbell
Conference PapersProceedings of the 2020 CHI Conference on Human Factors in Computing Systems, April 2020

Abstract

Impaired social functioning is a symptom of mental illness (e.g., depression, schizophrenia) and a wide range of other conditions (e.g., cognitive decline in the elderly, dementia). Today, assessing social functioning relies on subjective evaluations and self assessments. We propose a different approach and collect detailed social functioning measures and objective mobile sensing data from N=55 outpatients living with schizophrenia to study new methods of passively accessing social functioning. We identify a number of behavioral patterns from sensing data, and discuss important correlations between social function sub-scales and mobile sensing features. We show we can accurately predict the social functioning of outpatients in our study including the following sub-scales: prosocial activities (MAE = 7.79, r = 0.53), which indicates engagement in common social activities; interpersonal behavior (MAE = 3.39, r = 0.57), which represents the number of friends and quality of communications; and employment/occupation (MAE = 2.17, r = 0.62), which relates to engagement in productive employment or a structured program of daily activity. Our work on automatically inferring social functioning opens the way to new forms of assessment and intervention across a number of areas including mental health and aging in place.

TA Experience