Tech + Research Workshop

The world of home conversational AI (intelligent home voice assistants like Amazon Alexa and Google Home) spans multiple areas. With this project, we hope to design a working VoiceFlow (skill/application) that serves those who are part of the accessibility community and have needs in their home that could be supported by voice technology. This could include supporting the home ecosystem environment through Voice UI, developing a skill that supports a particular population (aging adults, or cognitively impaired adults) or we ask that students bring in their own perspective in what is needed to be developed for the Voice platforms.

Researchers

Ramita Shrestha
Graduate Student

Galina Reitz
Senior Lecturer and Faculty Director, BS in Information Science at Shady Grove

Recent advances in self-driving platforms, from food delivery robots to autonomous vehicles (AV), highlight the urgent need for strong safety standards. AVs to operate in a mixed autonomy, where there will be human-driven and self-driving cars operating side by side on the road for the foreseeable future. AVs require the capability to anticipate human-driven vehicle behaviors that will likely be different and less predictable than a self-driving car, thereby making it especially critical to capture large amounts of human driving data under diverse pre-crash scenarios. This work aims to bridge this gap by employing a virtual reality (VR) vehicle simulator to immerse participants in various accident scenarios. This approach enables safe collections of human driving data and behaviors in high-risk situations, significantly enhancing our understanding of driving dynamics and safety. The presentation will include background information, related works, methodology, results, and a conclusion, similar to a full research project.

Researchers

Ming Lin
Professor, Computer Science

Sandeep Thalapanane
General Assistant

Sandip Sharan Senthil Kumar
General Assistant

Sourang Sri Hari
General Assistant

Social and behavioral researchers often collect sensitive data about people. Publishing research data is beneficial—enabling replication and meta-analysis, as well as providing transparency for public funds—and is often required by journals and funders. To prevent harms and privacy violations to research participants, data must be de-identified, which is a complex and challenging task. Principled approaches to de-identification, such as differential privacy and k-anonymity, can help ensure that data meets certain standards of privacy. However, researchers understandably have concerns that this gain in privacy will be unacceptably offset by a loss of data utility or fairness.

As a first step towards addressing researchers’ concerns, we aim to establish a baseline understanding of how existing de-identification tools impact data utility. In this project, we will conduct experiments applying de-identification tools on published research data.

Researchers

Michelle Mazurek
Associate Professor, Computer Science

Wentao Guo
Graduate Student

Emma Shroyer
Graduate Student

Deep networks have shown strong capabilities for standard vision tasks such as image classification and object detection. More recently, generative networks have become extremely popular. These models are able to create detailed images that can even fool humans into believing that they are human-generated. Recently, diffusion models have become extremely popular in allowing users to create a highly detailed image using just a text prompt. In this project, students will be introduced to how diffusion networks work and explore their properties such as how different prompts can affect image generation quality. They will also have the opportunity to finetune these networks for their own custom images by introducing new concepts to an existing network. Finally, students can exploit these networks to classify unseen images.

Researchers

Abhinav Shrivastava
Associate Professor, Computer Science

Vatsal Agarwal
Graduate Student

"Your package is delayed; click here or else!" We have all received SMS phishing ("smishing") messages like these. What makes them so effective is that they can seem convincing, and we don't have widespread solutions like email has spam detection. The goal of this project is to collect a sample of smishing messages from public repositories, identify what makes them work, and develop new AI-based techniques to detect and even generate the next phase in smishing evolution.

Researchers

Dave Levin
Associate Professor, Computer Science

Julio Poveda
Graduate Student

Point tracking in videos involves following specific points or features across consecutive frames, allowing for the analysis of motion and trajectory. This technique utilizes algorithms to identify distinctive features in each frame and match them with corresponding points in subsequent frames. In scientific applications, point tracking is crucial for animal behavior studies, enabling researchers to monitor movement patterns, speed, and interactions of individual animals or groups in their natural habitats. For instance, tracking points on a fish's body can reveal intricate swimming patterns, while following points on monkeys can provide insights into their tree-climbing movements, foraging behaviors, or social dynamics. In this project, students can apply any existing point-tracking models to insects like ants, flies, or bees or to animals like fish, rats, or monkeys. The goal is to develop either a re-identification strategy or some form of behavioral analysis. This hands-on approach allows students to apply tracking techniques to real-world animal studies, potentially uncovering new insights into animal behavior or improving existing tracking methods.

Researchers

Abhinav Shrivastava
Associate Professor, Computer Science

Pulkit Agarwal
Graduate Student

Namitha Padmanabhan
Graduate Student

Many countries around the world censor internet traffic in an effort to control information, suppress political opposition, and even restrict access to basic information about reproductive health. The goal of this project will be to use public datasets to learn more about how censors operate. Researchers will pose questions like: What is blocked where? How are users affected?

Researchers

Dave Levin
Associate Professor, Computer Science

Sadia Nourin
Graduate Student

Our research project is centered on evaluating question-answering (QA) systems, with a particular focus on their calibration. Calibration, in this context, refers to how closely a model’s confidence in its predictions matches the actual correctness of those predictions. This is crucial for ensuring that the model’s confidence reflects its reliability in real-world tasks. To measure calibration, we draw on the concept of a "buzz" from Trivia Quiz competitions, where participants buzz in with an answer as soon as they feel confident enough, often before hearing the full question. Similarly, we assess whether a QA model’s confidence aligns with its likelihood of making a correct prediction as the question is incrementally revealed.

A key feature of our approach is that questions are presented in stages, with the model producing a series of guesses and confidence scores at each step. This allows us to track how the model’s confidence evolves as it receives more information. Our evaluation focuses on three main objectives: 1) determining at which point in the question reveals the model becomes confident enough to produce a correct answer, 2) assessing whether the model’s confidence scores accurately reflect the correctness of its guesses, and 3) comparing the alignment between confidence and correctness in models versus human participants.

To quantify these dynamics, we use a novel metric called Average Expected Buzz, which measures the expected confidence level at which the model will likely buzz in with a correct prediction. This provides a comprehensive evaluation of the system's calibration. After submission, we plan to test these models on adversarial questions crafted by human experts, specifically designed to be challenging at the final stage (or "run") of the question in a Trivia human-computer tournament. This will allow us to evaluate whether the submitted QA systems can consistently outperform human experts using our calibration metric.

The overarching goal of this project is to enhance the reliability of QA systems by improving the alignment between their confidence estimates and actual performance, making them more trustworthy for real-world applications that depend on accurate, well-calibrated decision-making under uncertainty.

Researchers

Jordan Boyd-Graber
Professor, Computer Science

Yoo Yeon Sung
Graduate Student

Yu Hou
Graduate Student