Personalizing Hearing-Aids
Mentor: Dr. Octav Chipara
Domain Expert: Dr. Yu-Hsiang Wu
Approximately 35 - 50% of Americans over 65 years old report having an age-related hearing impairment treated primarily with hearing aids. Regular use of hearing aids has been shown to improve communication and avoid the harmful effects of hearing loss, including an increased risk of social isolation and depression. Over the last decade, researchers have successfully improved the performance of hearing aids by leveraging the increasing capabilities of hardware platforms, which enabled the deployment of sophisticated signal processing algorithms such as those for noise suppression, directional microphones, and auditory scene classification. In contrast, future internet-enabled hearing aids will overcome limited hardware resources by accessing cloud services and edge devices' significant computational and storage capabilities. As part of this development project, you will learn how to design, train, and deploy machine learning algorithms that enhance the capabilities of hearing aids. These algorithms will run either in the cloud or on resource-constrained edge devices.
Using Smart Glasses and Watches to Access Hearing Aids
Mentor: Dr. Octav Chipara
Domain Expert: Dr. Yu-Hsiang Wu
For any hearing aids to work, they must do the right things (e.g., enable directional microphones), for the right people, in the right places (when speech and noise are spatially separated). To this end, hearing aids must accurately identify challenging and critical listening situations, the characteristics of these situations, and the listening goals of listeners. One commonly used method to assess the performance of hearing aids in the real world is to use Ecological Momentary Assessment (EMA) self-reports. EMA is a methodology that asks respondents to repeatedly report their experiences during or shortly after the experiences in their natural environments. Although EMA in audiology research is increasing, EMA is not a tool without limitations. For example, it is generally tricky for EMA to capture challenging listening situations when EMA surveys are delivered at random as they occur infrequently. In addition, the association between discrete EMA data and continuous sensor data (i.e., how listeners integrate the experience of the past several minutes to answer an EMA survey question) is unknown. This project aims to use smartwatches and smart glasses to (1) capture more challenging listening situations with higher fidelity than previously possible and (2) determine the association between EMA self-reports and continuous sensor data is using novel machine learning algorithms.
Ethics of Emerging Technologies for Children
Mentor: Dr. Juan Pablo Hourcade
We are studying the ethics of emerging technologies, such as extended reality and generative artificial intelligence, with respect to children as part of a national consortium. Our goal is to develop detailed ethical guidelines that arise from informed stakeholder (children, parents, teachers, pediatricians, therapists) views. Each site in our consortium is working with a focus on different groups of children (urban, rural, young children, elementary school children, teenagers, neurodivergent children). Conducting the research involves various tasks, such as, identifying ethically salient characteristics of emerging technologies, compiling potential use cases, understanding how to introduce and enable use of emerging technologies by stakeholders, integrating results from multiple research sites, and developing outreach materials.
Measuring the Exploitation of Data Voids
Mentor: Dr. Rishab Nithyanand
Data voids are obscure terms or queries that result in few results on search engines associated with online platforms such as Google, YouTube, Facebook, and Reddit. There has been recent evidence to suggest that these data voids are exploited by malicious actors in order to promote problematic content such as disinformation and propaganda. One common form of exploitation involves the production of problematic content to populate the search results associated with data voids, promoing the term/query in discourse that occurs on mainstream platforms (e.g., Twitter, Reddit, YouTube, etc.), and therefore directing unsuspecting users searching for these terms to the problematic content. The terms "migrant caravan" and "crisis actor" were believed to have been promoted into mainstream discourse with malintent in this way. In this REU project, students will work in the SPARTA lab to build a platform that measures this form of data void exploitation. The end goal is to produce a tool that enables the identification of such manipulation and provides resources for journalists and common citizens to recognize when they are being manipulated by these exploited data voids.
Multi-Device Health Tracking and Coaching Integration
Mentor: Dr. Lucas Silva
This Human-Computer Interaction (HCI) project focuses on developing health tracking solutions by integrating data from various consumer wearable devices (smart rings, phones, smartwatches) alongside offering AI/LLM-based interactions. We are working on creating a data processing pipeline that enables individuals to better understand their health patterns and behaviors through interactions with multiple devices. These solutions are part of three research threads: supporting physical activity during pregnancy, helping older adults reflect on parameters that impact their brain health, and supporting health tracking collaboration within families. There are several opportunities to contribute to this research under each of the threads:
1. Developing apps for collecting and synchronizing data streams from different devices,
typically through different platform's API.
2. Developing apps for processing and combining various health parameters
(e.g., heart rate, sleep patterns, activity levels) into databases for multiple users.
3. Exploring visualizations and conversational interactions with LLM to make health data accessible for users.
Students will gain experience in working with consumer devices (e.g., smartwatches) and developing data pipelines for conversational agents (e.g., using LLMs) under a person-centered research approach.
Improving LLM-based Audio Understanding via Hallucination Reduction
Mentor: Dr. Weiran Wang
Large Audio-Language Models (LALMs) can be used in health care to provide useful analysis of health-related human sounds (e.g., heatbeat and respiration), and in smart homes where they can provide advanced security monitoring. We would like to improve the audio understanding capability of LALMs and reduce hallucination, with novel training objectives and decoding strategies. For training, we go beyond the standard next token prediction task, and apply reinforcement learning with human feedback, to better align audio with text. Regarding inference, we will use contrastive decoding to boost the confidence of factual responses, with the help of informative negative samples. We will also compile a new hallucination benchmark for training and evaluating models developed under the new framework.
Algorithm Analysis course is required. The preferred qualification includes courses in Artificial Intelligence and/or machine learning, proficiency in python, and experience in ML frameworks like pytorch or tensorflow.
Collecting Medical Data over 5G
Mentor: Dr. Tianyu Zhang
Over the last decade, we have observed the emergence of new devices capable of collecting health data ranging from simple step counts, vital signs, or blood glucose levels. A key challenge for these types of devices is to efficiently use the cellular network to collect data. As part of this project, we aim to conduct experimental research on 5G New Radio (NR) using the NSF-supported ARA platform. ARA is a remotely accessible "living laboratory" and provides open-source 5G testbeds, combining software-defined radio (SDR) units equipped with custom-designed RF frontends and implementation of the open-source OAI (OpenAirInterface) 5G software stack. ARA provides a detailed manual and some ready-to-use experiment configurations called ‘profiles.’ We plan to proceed with the project following two steps.
1. Using some of the handy profiles provided by ARA to evaluate the 5G
PHY layer techniques, e.g., mixed-numerology, by measuring the network throughput and latency.
2. Implementing well-designed scheduling algorithms into the platform,
creating our own profile, and performing performance evaluation.