Skip to content

3D Models on a 360° Media Backdrop

Overview

This script demonstrates how to overlay standard 3D objects on top of a full-surround (360°) video or image “skybox” in SightLab. It shows:

  • Loading a blank environment
  • Wrapping a 360° media file onto a sphere
  • Adding a 3D crate model that can be both gazed at and grabbed
  • Basic key controls for video playback and trial flow

Note: You may need the K-Lite Codec Pack to play certain video formats.

Requirements

  • Vizard (7 or 8) with Python 3 support
  • SightLab module (import via import sightlab_utils.sightlab as sl)
  • vizfx, vizshape, viztask, vizact

Script Breakdown

import sightlab_utils.sightlab as sl
from sightlab_utils.settings import *
import vizshape
  1. Initialize SightLab
sightlab = sl.SightLab(gui=False, headlight=True)
  • gui=False disables the built-in GUI so the script drives everything.

  • headlight=True ensures the scene is lit.

  • Global Variables

count = 0
countToggle = 0

Used internally for toggling video playback.

1. Environment Setup

env = vizfx.addChild('sightlab_resources/environments/empty.osgb')
sightlab.setEnvironment(env)
  • Loads a completely empty .osgb environment into the world.
  • Registers it as the 360° media backdrop (since no GUI/config is present) .

2. Trial Configuration

sightlab.setTrialCount(1)

Sets the experiment to run exactly one trial.

3. Creating the 360° Backdrop

video = viz.addVideo('resources/skybox_room1.jpg')
sphere = vizshape.addSphere(flipFaces=True)
sphere.setScale(10,10,10)
sphere.texture(video)
sphere.setEuler(180,0,0)
sphere.disable(viz.DEPTH_TEST)
sphere.drawOrder(-100)
  • Load your 360° image or video as a texture.
  • Wrap it onto a large inside-out sphere (flipFaces=True).
  • Disable depth testing so the sphere always renders behind everything else.

4. Adding a Tracked 3D Object

crate = vizfx.addChild('crate.osgb')
crate.setPosition(0,0,2)
sightlab.addSceneObject('crate', crate, gaze=True, grab=True)
  • Places a “crate” model two meters in front of the user.
  • Registers it with SightLab for both gaze dwell and grab tracking .

5. Key Controls

def resetVideo():
    video.setTime(0)
    video.play()
vizact.onkeydown('b', resetVideo)

def ToggleUpdate():
    global countToggle    
    if countToggle == 0:
        video.play()
        countToggle = 1
    else:
        video.pause()
        countToggle = 0    
vizact.onkeydown('v', ToggleUpdate)
  • b: Restart the video from time 0.
  • v: Toggle play/pause.

6. Experiment Loop

def sightLabExperiment():
    while True:
        yield viztask.waitKeyDown(' ')
        yield sightlab.startTrial()
        video.play()
        yield viztask.waitKeyDown(' ')
        yield sightlab.endTrial()
        video.pause()
  • Spacebar starts and ends each trial.
  • Within each trial, the video plays until you press Space again.

7. Scheduling and Cleanup

viztask.schedule(sightlab.runExperiment)
viztask.schedule(sightLabExperiment)
viz.callback(viz.getEventID('ResetPosition'), sightlab.resetViewPoint)
  • sightlab.runExperiment drives all data‐logging and internal state.
  • sightLabExperiment controls your custom trial flow.
  • Pressing r (the “ResetPosition” event) will recenter the user’s view on the origin.

Additional Examples

  • Example 2: Adding a proximity sensor to a 360° scene
  • Example 3: Using that proximity sensor to cycle between multiple videos