Skip to content

3D Tablet/ VR Menu

Setting Up a VR Menu in SightLab

Introduction

The 3D Tablet Menu feature in SightLab allows users to use and modify a VR Menu to interact with your scene. This guide will walk you through the setup and use of this feature. Use either a selector/ highlighter tool or your controllers to interact with the menu.

Installation

  1. Locate the following files in the ExampleScripts\- 3D Tablet Menu directory:

    • 3D_tablet_menu.py: Example using the GUI and standard VR scenes
    • 3D_tablet_menu_XR.py: Example using the passthrough camera to show menu in AR
    • VR_Menu.osgb: Required object for the virtual menu functionality

  2. Choose your preferred implementation:

    • If you want to use the existing example: Copy either script and VR_Menu.osgb to your project directory.
    • If you prefer to integrate the functionality into your existing script: Copy only the necessary code snippets and VR_Menu.osgb

Basic Setup

If integrating to your own script, add the following code:

Import the vr_menu and selector modules

from sightlab_utils.vr_menu import VRMenu
from sightlab_utils import selector

Or newer imports for auto highlighter (SightLab 2.5.14 or higher)

from sightlab_utils.vr_menu_auto import VRMenu
from sightlab_utils import selector_auto as selector

Add functions that will be called from the menu buttons

def Action1():
    print('button 1 pressed')

def Action2():
    print('button 2 pressed')

def Action3():
    print('button 3 pressed')

def Action4():
    print('button 4 pressed')

Define which functions are called with the buttons. Can add as many buttons as you have available (default available is 4).
actions = [Action1, Action2, Action3, Action4]

Create VR Menu

vr_menu = VRMenu(actions,tablet_model = 'VR_Menu.osgb',config=sightlab.getConfig())
vr_menu_object = vr_menu.tablet

#Add menu as grabbable
sightlab.addSceneObject('vr_menu_object', vr_menu_object, grab=True)

After sightlab.startTrial() or yield viztask.waitEvent(TRIAL_START), add:

vr_menu.experiment_loop()
selector.setupSelector(sightlab.getConfig())

Using the VR Menu

  • Menu Visibility: Press 'm' key or right thumbstick (controller) to reposition and toggle menu visibility (left trackpad for Vive controllers).
  • Press either the right mouse button or RH grip or LH grip to bring up the highlighter (for desktop should come up automatically)
  • Menu Grabbing: Use either trigger button to grab the menu.
  • Button Interaction: Use either the selector/ highlighter tool or your controllers to interact with the menu.

  • Configuring the Menu

  • Open the "VR_Menu.osgb" model in Inspector
  • Select the text for each button and change the "Value" property on the right to change the name of the button
    For adding new buttons you will need to scale and move the Main-VR_Menu object in Inspector (using the scale and move tools) to make room, then use Shift and the move tools to make copies of the original button images and button(number)_target objects (the targets are what is used for the sensor). Making sure to place them in the same hierarchy in the tree as the others. 

Advanced Configuration

  • Seated Mode: Add seated=True when initializing JumpToLocations.
  • More than 4 Buttons: Modify the VR_Menu.osgb file and update the vr_menu initialization:
vr_menu = VRMenu(
    actions,
    tablet_model="VR_Menu.osgb",
    config=sightlab.getConfig(),
    number_of_buttons=5,
)