Skip to content

Gaze Based Interactions

To create a gaze-based interaction in SightLab, you can follow these steps. SightLab already has built-in support for gaze tracking, so you only need to configure it by setting up gazeable objects and adding optional callbacks for when gaze events occur.

Basic Gaze Interaction Setup

Import SightLab and necessary modules:

import sightlab_utils.sightlab as sl
from sightlab_utils.settings import *
import vizshape  # if using basic shapes

Initialize SightLab:

sightlab = sl.SightLab(gui=False)

Add your environment (if not using GUI):

env = vizfx.addChild('sightlab_resources/environments/complete_scene.osgb')
sightlab.setEnvironment(env)

Add an object and enable gaze interaction:

object1 = vizshape.addCube(size=0.2, color=viz.RED)
object1.setPosition([0, 1.6, 2])
sightlab.addSceneObject('CubeObject', object1, gaze=True)

Define what happens when gaze events are triggered:

def gazeActionStart(e):
    if e.object == sightlab.sceneObjects[GAZE_OBJECTS]['CubeObject']:
        print('Gaze started on CubeObject!')

def gazeActionEnd(e):
    if e.object == sightlab.sceneObjects[GAZE_OBJECTS]['CubeObject']:
        print('Gaze ended on CubeObject.')

vizact.addCallback(sightlab.GAZE_TIME_EVENT, gazeActionStart)
vizact.addCallback(sightlab.GAZE_END_EVENT, gazeActionEnd)

Create the experiment loop:

def sightLabExperiment():
    yield viztask.waitEvent(EXPERIMENT_START)
    for i in range(sightlab.getTrialCount()):
        yield sightlab.startTrial(startTrialText="Look at the red cube to trigger a gaze event.")

     # Allow some time for the gaze to occur
        yield viztask.waitTime(10)

        yield sightlab.endTrial(endTrialText="End of trial. Thanks!")

## Scheduling the experiment
viztask.schedule(sightlab.runExperiment)
viztask.schedule(sightLabExperiment)
viz.callback(viz.getEventID('ResetPosition'), sightlab.resetViewPoint)

GAZE_START_EVENT for your event to be triggered as soon as a gazeObject is seen

GAZE_END_EVENT to trigger something to happen when a user’s gaze point is no longer intersecting with that object

GAZE_TIME_EVENT to trigger something that happens only when an object is being focused on over the set threshold

🔍 Notes You don’t need to code gaze tracking logic manually — just set gaze=True in addSceneObject.

Gaze events are triggered when gaze dwell time meets a threshold (can be modified with code or in the GUI).

Full Code

import sightlab_utils.sightlab as sl
from sightlab_utils.settings import *
import vizshape

# Initialize SightLab without GUI

sightlab = sl.SightLab(gui=False)

# Set environment

env = vizfx.addChild('sightlab_resources/environments/complete_scene.osgb')
sightlab.setEnvironment(env)
sightlab.setStartText(' ')

# Add a gazeable object (a red cube)

cube = vizshape.addCube(size=0.2, color=viz.RED)
cube.setPosition([0, 1.6, 2])
sightlab.addSceneObject('CubeObject', cube, gaze=True)

# Define callback functions for gaze events

def gazeActionStart(e):
    if e.object == sightlab.sceneObjects[GAZE_OBJECTS]['CubeObject']:
        print("👁️ Gaze started on CubeObject!")
        cube.color(viz.BLUE)

def gazeActionEnd(e):
    if e.object == sightlab.sceneObjects[GAZE_OBJECTS]['CubeObject']:
        print("❌ Gaze ended on CubeObject.")
        cube.color(viz.RED)

# Register gaze event callbacks

vizact.addCallback(sightlab.GAZE_TIME_EVENT, gazeActionStart)
vizact.addCallback(sightlab.GAZE_END_EVENT, gazeActionEnd)

# Define the experiment logic

def sightLabExperiment():
    yield viztask.waitEvent(EXPERIMENT_START)
    for i in range(sightlab.getTrialCount()):
        yield sightlab.startTrial(startTrialText="Look at the red cube to begin tracking gaze.")

        # Let the participant gaze around for 10 seconds
        yield viztask.waitTime(10)

        yield sightlab.endTrial(endTrialText="Trial ended. Good job!")

# Schedule experiment
viztask.schedule(sightlab.runExperiment)
viztask.schedule(sightLabExperiment)
viz.callback(viz.getEventID('ResetPosition'), sightlab.resetViewPoint)

Gaze Based Events using the GUI

There is also an end condition in the GUI for gaze based events that allow you to trigger ending the current trial or progressing to the next one using an onGazeTime event. For additional gaze based triggers see below. 

Multi-User

On SightLabVR_Server.py script: 

Add Network events then create a function for gazeActionstart and for gazeActionEnd, add the code for whatever you want to have happen (i.e. a print statement) inside the function. Callback the functions with the network event. 

#Generate IDs for events
def id(name):
    """Generate unique ID for use over viznet"""
    return f'viznet:{name}'

FIXATION_NETWORK_KEY_EVENT = id('FIXATION_NETWORK_KEY_EVENT')
FIXATION_STOPPED_NETWORK_KEY_EVENT= id('FIXATION_STOPPED_NETWORK_KEY_EVENT')

def gazeActionEnd(e):
    if e.name == "creature1":
        creature1.setScale(1,1,1)
        print('stopped looking')

    if e.name == "chick1":
        chick1.setScale(1,1,1)
        print('stopped looking')

def gazeActionStart(e):
    if e.name == "creature1":
        creature1.setScale(1.5,1.5,1.5)
        print('saw creature')

    if e.name == "chick1":
        chick1.setScale(1.5,1.5,1.5)
        print('stopped looking')

vizact.addCallback(NETWORK_GAZE_TIME_EVENT, gazeActionStart)    
vizact.addCallback(NETWORK_GAZE_END_EVENT, gazeActionEnd)

On SightLabVR_Client.py script: 

#Generate IDs for events
def id(name):
    """Generate unique ID for use over viznet"""
    return f'viznet:{name}'

FIXATION_NETWORK_KEY_EVENT = id('FIXATION_NETWORK_KEY_EVENT')
FIXATION_STOPPED_NETWORK_KEY_EVENT= id('FIXATION_STOPPED_NETWORK_KEY_EVENT')

def gazeActionEnd(e):
    if e.gazeData.name == "creature1":
        creature1.setScale(1,1,1)
        print('stopped looking')

    if e.gazeData.name == "chick1":
        chick1.setScale(1,1,1)
        print('stopped looking')


def gazeActionStart(e):
    if e.gazeData.name == "creature1":
        creature1.setScale(1.5,1.5,1.5)
        print('saw creature')

    if e.gazeData.name == "chick1":
        chick1.setScale(1.5,1.5,1.5)
        print('stopped looking')

vizact.addCallback(NETWORK_GAZE_TIME_EVENT, gazeActionStart)    
vizact.addCallback(NETWORK_GAZE_END_EVENT, gazeActionEnd)
Was this page helpful?