Skip to content

Session Replay

To view the tracking data after a simulation is run, start the Session Replay file (also referred to as "SightLab Replay"). You will be presented with a list of options for session playback. Note: You must first run an experiment session for the session replay to show any data.

For 360 video/photo playback, the viewpoint will be from a fixed position, but can look around if uncheck "Follow Mode"

  • Spacebar to start/stop.
  • R to reset.
  • Use the slider along the bottom to scrub through the playback

Session playback can be viewed in either Desktop Mode or in a headset if you choose "Meta," "SteamVR," or "OpenXR."

Note: Session Replay uses a "Free Camera" mode, meaning you can use the WASD and ZX keys to move the camera around, or use the VR controls or walk freely if viewing in a headset.

Features and Options

Trial Number
- Select any trial from your session to replay.

Client ID (Multi- User Only)

  • Show Visualizations for only the selected client

Open Data Folder

  • Opens the folder containing data files associated with the session.

Avatar Visualization
- Toggle the display of the avatar (or avatars in multi-user mode). Head and hands are shown separately.

Gaze Visualizations
- Toggle:
- Gaze Ray: The line of sight from the user (individual or per eye).
- Gaze Point: The intersection of gaze and objects (individual or per eye).
- Dwell Spheres: A sphere related to objects of interest that will change size and color based on the time spent gazing at them, once the dwell time threshold is met.

Fixation Spheres
- Toggle visualization of fixation spheres. Fixations are calculated based on:
- Angular distance (<1 degree).
- Duration (>100 ms, default; configurable).
- Click Static to view all fixations at once.
- First: Shows the time and location of the first fixation
- All: Will show a numbered sequence of all the fixations

Heat Map Visualization
- Provides a dynamic representation of gaze data:
- Dynamic Mode: Displays gaze over time during playback.
- Static Mode: Aggregates all gaze data into a single heatmap.
- Occlusion Setting: Enables heatmaps that respect physical barriers (e.g., walls).
- Note: requires setting fastHeatmap = False and can affect frame rate.
- Gradient: A heatmap that fades over a specified time.
- Adjust parameters using sliders for intensity, decay rate, and transparency.
- Note: If running a trial longer than 2 or 3 minutes you can switch to the faster heatmap if loading times increase or frame rate drops (replay = Replay.SightLabReplay(fastHeatmap = True) )

Scan Path

  • Visualize gaze paths dynamically or statically.
  • Options:
  • Gaze Path: Tracks gaze movement.
  • Dispersion Path: Tracks areas of focus.

Walk Path

  • The walk path will show arrows in the direction that a user walked and will also show velocity based on the spacing between arrows. Walk Path can be shown as static or dynamic.

Console

  • Toggle text overlays for additional session information such as Pupil Diameter and real time views
  • ROIs: Shows the regions of interest for 360 media
  • Tags: Shows the names of the ROIs

Views
- Toggle between:
- First-person view: From the user's perspective.
- Third-person view: Scene-wide view.
- For 360 media: Toggle whether the head position is locked or free.

Reset to Default

  • Resets all settings to their default values.

Controls

If using a headset the default controls apply, except in the case of the Vive Pro/2/Vive Pro Eye, where the system button (one above trackpad) on the right needs to be held down and use the left trackpad to move, and the left system button is used for the arc teleport. For the other headsets, the navigation is their default 

Action Key
Start/Stop Playback Spacebar
Reset Playback R
Scrub Playback C, V (or use slider)
Frame-by-Frame B, N
Move Camera WASD, Arrow Keys
Move Up/Down X, Z
Rotate Camera Q, E, O, L, or Mouse
Hide Menu ., F1
Take Screenshot /
Save Equirectangular Image 3
Video Capture (uncompressed) Ctrl + F12
Full Screen Toggle F2
Rendering Mode Toggle F3
Performance Data Toggle F4
Show Instructions I
Connect to Biopac 1
Connect to Biopac Graph 2
Exit Escape

Advanced Features

Heatmap Aggregation

Found in ExampleScripts/HeatMap_Visualizer

Create an aggregated heatmap for:

  • All session data.

  • Specific conditions.

  • Individual trials.

  • All data from the whole experiment.

Noise Filter

Smooth eye tracking data by averaging the gaze points over a specified window of time. This helps to reduce the impact of random noise or fluctuations in the eye tracking data, resulting in more stable and reliable gaze points. The moving average algorithm works by maintaining a rolling window of the most recent gaze points and calculating the average for each new point as it becomes available. 

replay = Replay.SightLabReplay(noiseFilter=True)

Synchronizing Traditional Video

For synchronizing 2D video playback with the replay (note: first need to define and add video in replay as well):

def reset():
  video.setTime(0)
  video.play()

def play():
  video.play()

def pause():
  video.pause()

def scrub(time):
  video.setTime(time)
viz.callback(REPLAY_RESET, reset)
viz.callback(REPLAY_PLAYING, play)
viz.callback(REPLAY_PAUSED, pause)
viz.callback(REPLAY_SCRUB, scrub)

Custom Replay Flags and Events

Set custom flags during an experiment session and visualize them in the replay (for instance to keep track of the state of an object like color change and play that back in the replay at the correct time). See also ExampleScripts\Replay_Events

Example:

# During Experiment with GUI
def changeColor():
    color_list = [viz.RED, viz.BLUE, viz.GREEN, viz.YELLOW]
    target_color = random.choice(color_list)
    basketball.color(target_color)  
vizact.onkeydown('a', changeColor)

def getColor(): 
    sightlab.setCustomReplayFlag('color', basketball.getColor())

#run sightlab experiment
def sightLabExperiment():
    while True:
        yield viztask.waitKeyDown(' ')
        yield sightlab.startTrial()

        #Add update here for GUI
        vizact.onupdate(0,getColor)

        yield viztask.waitKeyDown(' ')
        yield sightlab.endTrial()

# During Replay
def changeColor():
    basketball = replay.sceneObjects[GAZE_OBJECTS]['basketball']
    flag_data = replay.getCurrFlag()
    if "color" in flag_data:
        print(flag_data)
        basketball.color(flag_data["color"])    
import vizact
vizact.onupdate(0,changeColor)

#Visibility
def changeVisibility():
    Les_Paul.visible(viz.TOGGLE)
vizact.onkeydown("a", changeVisibility)

def getVisibility():
    visibilityFlag = Les_Paul.getVisible()
    sightlab.setCustomReplayFlag("visible", visibilityFlag)
vizact.onupdate(0, getVisibility)

#Replay Side 
def applyReplayFlag():
    Les_Paul = replay.sceneObjects[GAZE_OBJECTS]["Les_Paul"]
    flag_data = replay.getCurrFlag()
    if "visible" in flag_data:
        print(flag_data)
        Les_Paul.visible(flag_data["visible"])
vizact.onupdate(0, applyReplayFlag)

Data Directory

If you have a custom location for your data files give the path to your custom data folder:

replay = Replay.SightLabReplay(dataDirectory='data')

Advanced Configuration: Scan Path Settings

Modify the appearance of scan paths:

replay.replayVisualisations.scanPathConfig = {
    POINT_COLOR: viz.GREEN,
    POINT_SIZE: 2,
    LINE_COLOR: viz.BLUE,
    LINE_WIDTH: 2
}

Environment Access

Access the environment object during replay:

replay.transportNode.setPosition([x, y, z])
env = replay.getEnvironmentObject()

Scene Objects

Access specific scene objects during replay:

screen = replay.sceneObjects[GAZE_OBJECTS]["screen"]

Setting Starting Position of Viewpoint

replay.transportNode.setPosition([x, y, z])

Loading Older Replay Files

Steps for replaying data from older SightLab versions (1.x):

  1. Copy the blank project folder.

  2. Move older data files to a folder named data.

  3. Place environments or media in the appropriate Resources subfolders.

  4. Use this code to enable compatibility:

replay = Replay.SightLabReplay(sightlab_1_9=True)

Trial Loaded Event

Perform an action when trials are changed

def ontrialchanged():
    trial_number = int(replay.currTrial)
    print("change: ", trial_number)       
viz.callback(TRIAL_LOADED_EVENT, ontrialchanged)
Was this page helpful?