Spring 2019 - Capstone Project

ImpulseVR

ImpulseVR is both a research initiative as well as a capstone senior project that seeks to explore the potential interactions from combining both the fields of VR (Virtual Reality) and BCI (Brain Computing Interface). By the end of the semester, my team successfully put out a game demo that made use of both VR and BCI elements, as well as a research paper that attempted to explore a wall we ran into when combining both VR and BCI.

VR + BCI: Does anyone even study this?

During our research and experimental phase, we discovered a paradox that seems to apply only since we're combining VR and BCI, and we couldn't find any works or research related to this problem. VR by itself, is a system that utilizes tracking of the physical body, which is encouraged to freely move. As of right now, this results in the system where head, body, and hand positions are accurately tracked and mapped into VR. BCI by itself, from what I presume, works optimally when a user remains motionless, due to the sensitivity of external brain sensors used to capture and read EEG. Put the two together without major adjustments at first, and you can see how they don't actually play nice with eachother, since major movements like twisting torso or limb movements is the fastest way to add unneeded noise to the BCI system.

Research: Is there a relationship in EEG readings when comparing seated and room-scale VR?

Naturally, the next step was to investigate a potential relationship between the two in order to attempt a compromise or solution in order to clear this problem. Under the guidance of Dr. Santoso from UF Digital Arts and Virtual Worlds Program, we designed a simple research study that attempted to monitor and discern differences between the two common methods of experiencing VR systems, seated VR and room-scale VR. We used the Oculus Rift or HTC Vive in conjunction with OpenBCI's Ganglion Headband Kit in order to let users play in VR while we monitor and record their EEG data. After data collection, we then used OpenViBE Designer to perform simple digital signal processing and attempted to draw conclusions based on that data.


OpenBCI interface and openVIBE designer

Game: Impulsive Cave

All while the research was ongoing at parallel, we also designed a game in VR that would eventually be capable of listening and using EEG data in real-time, hence classifying itself as a VR + BCI game. Called "Impulsive Cave," the core gameplay revolved around using a alternative resource system, called "Focus." In order to generate more focus, the user has to remain in a state of focus we predefined by watching boundaries within a user's current alpha and beta band readings. The philosophy in this design attempts to engage the user to force themself to remain as calm as possible within a VR environment that throws them stimuli that attempts to break their concentration. In return, the user will hopefully unconsciously immerse themselves more into VR.


A flowchart which illustrates how we interpret EEG into a game mechanic, "Focus"

As the user explores the island and is led into the cave by process of elimination, the user is then presented into a floor-by-floor dungeon which houses a boss or trial to be defeated in order to progress to the next floor. Using Focus that's generated by the BCI system, the player in VR has access to a spellbook that lets them engage in a magic combat system, and different spells cost different amounts of Focus.


One of my teammates for this project, Dylan, playtesting one of the bosses

A Post-Graduate Review

Learned a lot in this class, and only furthered my interests and passion in VR due to the many ways we can eventually solve problems that matter more in society. As a summary, things I gained were:

  • Agile development
  • Desktop VR development and design (Oculus Rift)
  • Storytelling for an audience
  • Being a scrum leader

OpenBCI Ganglion Headband alongside Oculus Rift, and OpenBCI GUI running to capture and eventually carry over EEG data to Unity

Links

- To a video demo of Impulsive Cave (VR game)
- To OpenBCI, the most popular open-source BCI hardware and software solutions as of the moment
- To Dr. Santoso's homepage, who I greatly appreciate for mentoring us through this project


more projects!