With a goal of exploring the convergence between virtual reality (VR) and machine learning (ML) and artificial intelligence (AI), Rowan University launched the Machine & Artificial Intelligence Virtual Reality Center (MAVRC) in 2023. The Center aims to create immersive VR environments that seamlessly adapt and respond to user interactions in real-time. Directed by Dr. Nidhal Carla Bouaynaya, MAVRC is pioneering research in AI technologies and exploring innovative ways these systems could help transform a variety of industries. “Our mission is to discover how combining emerging technologies like VR and AI can drive future industries,” says Bouaynaya. “The combination is powerful and each technology benefits from the other. VR generates data and AI needs data to learn. VR provides simulated environments that serve as invaluable training grounds for AI models. This is a powerful tool with great potential for many industries, from manufacturing and transportation to healthcare and marketing.”
Along with heading up MAVRC, Bouaynaya is a professor of Electrical and Computer Engineering (ECE), the director of Rowan’s Artificial Intelligence Lab (RAIL) at Rowan, and Associate Dean for Research and Graduate Studies of the Henry M. Rowan College of Engineering. In addition, she is the co-founder of MRIMath, a startup company that uses AI and ML for radiation oncology and helps doctors make better decisions that improve patients’ lives. Among Bouaynaya’s top research interests is what she calls trustworthy ML, which looks at the issue of how much we should trust ML models.
“There are a few research directions I pursue in parallel,” explains Bouaynaya. “One explores the confidence level of machine learning models. In addition to quantifying that level of confidence, I’m pushing the direction of what are called self-aware machines. Along with providing their predictions, their decision-making process matches a human level of confidence. Another direction is researching the issue of the vulnerability of these models, what I call optical illusions or attacks. I’m trying to understand why the phenomenon happens and how we can make the models more robust against these types of attacks.”
Real-Life Applications of Machine Learning Models
By finding patterns within data, machine learning models can make decisions and predictions based upon this information. “There is published research that shows ML models have better diagnosis rates than radiologists,” says Bouaynaya. “The issue is once you deploy these models in a hospital, for example, all the knowledge they’ve gathered is based on historical data they were trained on, so they cannot continue learning. While they’re trained on potentially millions of scans, X-rays, and MRIs, they cannot incorporate new data as new cases come in. We are looking at how these models can continue to learn after deployment without forgetting what they learned initially. This is called catastrophic forgetting. We’re exploring ways to avoid this from happening in an artificial brain by adding empty artificial neurons that are reserved for future learning.”
Bouaynaya’s work at MRIMath focuses on using AI to assist physicians in brain tumor segmentation and volumetric analysis. “The co-founder of MRIMath is a neuro oncologist and treats patients with brain cancer every day,” explains Bouaynaya. “Fifty percent of brain tumor patients do radiation therapy, but before the procedure is done, doctors must delineate the tumor. This can be very challenging since brain tumors are highly unpredictable and come in all shapes, sizes, and locations. AI models are trained on millions of variations and obtain high quality measurements of brain tumor contours and volumes within minutes. Typically, it takes at least an hour for radiation oncologists to delineate a tumor; or several hours for more difficult cases. The entire process, including doctor sign off and any revisions, reduces that time to three minutes when using MRIMath.”
With the potential for more groundbreaking discoveries ahead, Bouaynaya is excited to continue to explore the intersection of AI and VR. “I believe the combination of these technologies will transform many applications and industries and will allow us to correct processes in real time. While VR is visualizing everything, AI is identifying where issues could be; allowing the user to correct those problems in real time.”
“There is published research that shows ML models have better diagnosis rates than radiologists. The issue is once you deploy these models in a hospital, for example, all the knowledge they’ve gathered is based on historical data they were trained on, so they cannot continue learning. While they’re trained on potentially millions of scans, X-rays, and MRIs, they cannot incorporate new data as new cases come in. We are looking at how these models can continue to learn after deployment without forgetting what they learned initially. This is called catastrophic forgetting. We’re exploring ways to avoid this from happening in an artificial brain by adding empty artificial neurons that are reserved for future learning.”
— Nidhal Carla Bouaynaya, Ph.D.
Associate Dean for Research and Graduate Studies,
Director, Machine & Artificial Intelligence and Virtual Reality Center (MAVRC),
Professor, Electrical and Computer Engineering, Rowan University
Leveraging Emerging Technologies
To help advance Rowan’s research initiatives and further explore the synergy between AI, ML and VR, Bouaynaya wants to recognize the importance of regional networks and institutional collaboration. “The MAVRC launch was a perfect example of how beneficial it can be to bring together higher education institutions, industry partners, and local and federal agencies. We had Homeland Security, Federal Aviation Administration (FAA), and the New Jersey Commission on Science, Innovation and Technology (NJCSIT) among the one hundred guests at the demonstration last September. Collaboration is key to understanding how these emerging technologies can be applied out in the world—this is very interdisciplinary work.”
Dr. Forough Ghahramani, Assistant Vice President for Research, Innovation, and Sponsored Programs for Edge adds, “Edge is thrilled to partner with Rowan University on several NSF projects and work alongside distinguished researchers, such as Dr. Bouaynaya, to help enable collaborations, provide access to cutting-edge resources, and advance discovery by leveraging emerging technologies.”
Through conferences and networking events, Bouaynaya says there are many discussions on how AI and ML are being integrated into the workforce today and in the years ahead. “Many radiologists and neurosurgeons, for example, are well versed in emerging technologies and can share potential issues they’re seeing in deployment and application. Many radiologists are taking basic courses in machine learning as part of their education. I foresee this becoming more prevalent in the manufacturing industry as well.”
“To help prepare our future generations, AI and VR courses will also need to become part of the undergraduate curriculum at colleges and universities,” continues Bouaynaya. “I love being a role model and helping students, especially women and minorities, explore this career path, pursue careers and leadership positions in STEM, and join our mission of pushing the boundaries of what’s possible at the intersection of artificial intelligence and mixed reality.”
To learn more about how MRIMath is transforming medicine with trustworthy AI and aims to improve cancer diagnosis and care, visit mrimath.com.
