By Ash Estevan
Jose Aguilar Escamilla (he/him) is researching some of the big questions about artificial intelligence, specifically safety, reinforcement, and how and what makes AI function. The third-year Ph.D. student is studying at Oregon State University in the AI program and recently passed his qualifying exam, allowing him to resume his complex research on his project, “Towards Theoretically-Grounded Trustworthy Sequential Decision-Making: Safety, Robustness, and Security.”
“I research two sides of AI: theoretical and applied. I study a lot of the mathematics behind AI, and use that to create applied, real-world scenarios,” said Jose. “For example, what makes a problem setting harder or easier to attack?”
Jose is advised by Huazheng Wang, assistant professor in the School of Electrical Engineering and Computer Science. Wang’s research interests focus on reinforcement learning, information retrieval and machine learning in general with a focus on developing provably efficient and trustworthy reinforcement learning. The opportunity to work with Wang is one of the reasons Jose chose Oregon State’s AI program for his doctoral studies.
Jose explains that AI decision-making encompasses self-driving, content recommendation, and even language learning models (LLMs), i.e., ChatGPT. This is the function that allows AI to do what people want it to do, the alignment. Jose explains what that looks like in practice.
“If a hacker is trying to change the observations or the feedback a learner is getting, there are certain characteristics to look out for,” said Jose. “A computer controls a robot, or a self-driving car, or an electrical grid system that's reinforcement learning. The safety is (learning about) what breaks an AI model and makes it volatile.”
Jose, born in Colima, Mexico, moved to Oklahoma when he was 15 years old with one goal: study computer science. From a young age, Jose developed an interest in coding and computer science but realized he had to move to continue his studies. The local schools in Colima don’t offer computer science courses.
“I started high school (in Oklahoma), and during those four years I took computer science classes,” said Jose. “Then I applied to the University of Oklahoma.”
As an undergraduate research assistant at the National Weather Center in Norman, Oklahoma, Jose’s path to study AI came through an unlikely route: weather prediction.
“The National Weather Center was looking for someone with a computer science background to predict hailstorms,” said Jose. “Using that data is how I started getting into machine learning, or how others know it, ‘AI.’”
Jose’s undergraduate research experience awarded him an understanding of the gravity and importance of grasping when computing systems fail or when something mechanical goes rogue. But more than knowing things can go wrong, Jose studies why they do.
“After a hailstorm chase, we were almost hit by a tornado,” said Jose. “Seeing people emerge from their shelters and homes and waving at us, we were in a NOAA chasing vehicle, made me realize the impact of research on people's lives.”
During his junior year of undergrad studies, Jose started honing his specific interest within AI: reinforcement learning and theory. Jose worked on a project with the Air Force Research Lab and Tinker Air Force Base, working on an autopilot project. Jose used this project as the basis for his master's thesis, earning his Bachelor's and Master of Science in computer science through the UO accelerator program.
“I started to shift towards reinforcement learning and theory, and I worked on a project with the Air Force Research Lab and Tinker AFB to make a trustworthy and explainable plane autopilot,” said Jose.
Before starting at Oregon State, Jose interned at the National Renewable Energy Laboratory at the Flatirons campus in Colorado. His job was to use AI to improve fluid simulations for offshore wind turbines. Jose says that his Oregon State offer was competitive, and ultimately, the AI program, advisors, and the research being done at Oregon State felt high caliber to him. The combination of his GEM fellowship and funding aid allows him to continue his research.
Studying the safety and robustness of decision-making is the reinforcement of learning in AI. Jose does this through the lens of theory as a guiding principle for safer and secure AI. Jose’s use of theory is to discover what makes an AI model or algorithm better. Ultimately, Jose knew that Oregon State was the only program that could help him achieve his goals in AI research and transfer what he’s learned into industry work. Jose hopes to gain more professional experience once he completes his program with an intended goal of returning to academia.
“I see myself using my understanding to connect with my students and using my (industry) experience to teach them theory applied to problems companies need to solve,” said Jose.