Visician VR
VR controllers that emulate the affordances of real-world tools for greater precision in virtual training. IDSA IDEA finalist 2024.
context
solo project IDSA IDEA 2024
role
designer developer
timeline
spring 2024 12 weeks
tools
autodesk maya processing 3D printing
Digital tools that closely resemble their analog counterparts play a major role in tasks that require precision and manual dexterity, like art and design—and medicine.
Background
Augmented and Virtual Reality are fast-growing categories of immersive technology used in education. In the medical field, current leading VR surgery companies like Ghost Productions utilize a combination of bare hand-tracking and household VR controllers like the Oculus Quest. However, neither provide the tactile experience of handling a real surgical instrument, reducing simulation accuracy and efficacy.
Research
Research methodologies included interviews and user tests with students from Dell Medical School. The ask: what difficulties do you encounter regarding medical training?
Cost:
Cadavers, lab space and equipment, opportunity cost of practicing on real machines, and instructors are all significant costs.
Accessibility:
Demos occur at scheduled times and locations. Students cannot repeat demos to their own liking. VR can be done anywhere.
Accuracy:
Practice scenarios feel unrealistic or lack nuance. VR, with better controllers, can provide immersion.
Diversity:
Using a "default" patient makes it hard to prepare for different pathologies. VR parameters can be adjusted as needed.
Concept
What if we could improve VR surgical training by creating controllers that emulate the mechanics of real-world instruments?
Instead of creating a new controller for each instrument, a set of three controllers could mimic the major mechanical categories of each tool: stylus (fixed, rod-shaped), hinged (lever mechanism), and looped (pantograph mechanism). This way, the design prioritizes the ergonomics of the different ways each tool is held and operated.
The controller buttons are reprogrammable for different tools: the top button on the stylus can be the action button for a syringe, or the pen button can be the action button for a scalpel in the pencil grip. Controller top surfaces are bevelled for housing IR lights that a VR headset can read easily from different angles.
Prototyping
Low fidelity foam tests focused on ergonomics and size, medium fidelity polymer clay tests focused on aesthetics and usability and were tested by medical students. The high fidelity prototypes were modeled in Maya and 3D printed.
Development
I created an interactive computer-vision based surgical simulation in Processing. Controller tips are colored and tracked as the in-game cursor. For the hinge and loop controllers, each tip has a different color and a "close" mechanism is calculated using the distance between the two tips.
.jpg)
