Face Tracking

Using Face Tracking with SightLab

SightLab supports face tracking on any headset that uses the OpenXR standard, including Meta (Quest Pro, Quest 3) and HTC Vive Focus Vision and Vive Focus 3 with the Face Tracker add-on

Adding Face Tracking to Your Script

To log facial tracking data, import the relevant module:

# For Meta/Quest
from sightlab_utils import face_tracker_data

# For HTC headsets
from sightlab_utils import face_tracker_data_htc

This module automatically saves face tracking data to your experiment’s /data folder.

Built-In Face Tracking Templates & Examples

SightLab includes several example scripts for leveraging facial tracking:

Mirror Demo Use this to map facial expressions to an animated avatar. ➤ Edit or remove the mirror if you only need the avatar mapping.
FaceTracker_Sliders Demonstrates how facial expressions can drive GUI sliders in real time.
Facial_Expressions_Over_Time Visualizes expression data with a matplotlib chart. Can load data from any face tracking-enabled trial.

Example Script

See Face_Tracking_Saving or Face_Tracking_Saving_HTC in the Face_Tracking_Data folder in ExampleScripts

from sightlab_utils import face_tracker_data
import sightlab_utils.sightlab as sl
from sightlab_utils.settings import *

sightlab = sl.SightLab()
# Set up with custom values
face_tracker_data.setup()

def sightLabExperiment():
    while True:
        yield viztask.waitKeyDown(" ")

        # Start the update process
        vizact.ontimer(0, face_tracker_data.UpdateAvatarFace)

        yield sightlab.startTrial()

        yield viztask.waitKeyDown(" ")
        yield sightlab.endTrial()


viztask.schedule(sightlab.experiment)
viztask.schedule(sightLabExperiment)

Face Tracking Expression Parameters

Here are the parameters available for Face Tracking Expression:

For Facebook (Meta)

EXPRESSION_BROW_LOWERER_L = 0
 EXPRESSION_BROW_LOWERER_R = 1
 EXPRESSION_CHEEK_PUFF_L = 2
 EXPRESSION_CHEEK_PUFF_R = 3
 EXPRESSION_CHEEK_RAISER_L = 4
 EXPRESSION_CHEEK_RAISER_R = 5
 EXPRESSION_CHEEK_SUCK_L = 6
 EXPRESSION_CHEEK_SUCK_R = 7
 EXPRESSION_CHIN_RAISER_B = 8
 EXPRESSION_CHIN_RAISER_T = 9
 EXPRESSION_DIMPLER_L = 10
 EXPRESSION_DIMPLER_R = 11
 EXPRESSION_EYES_CLOSED_L = 12
 EXPRESSION_EYES_CLOSED_R = 13
 EXPRESSION_EYES_LOOK_DOWN_L = 14
 EXPRESSION_EYES_LOOK_DOWN_R = 15
 EXPRESSION_EYES_LOOK_LEFT_L = 16
 EXPRESSION_EYES_LOOK_LEFT_R = 17
 EXPRESSION_EYES_LOOK_RIGHT_L = 18
 EXPRESSION_EYES_LOOK_RIGHT_R = 19
 EXPRESSION_EYES_LOOK_UP_L = 20
 EXPRESSION_EYES_LOOK_UP_R = 21
 EXPRESSION_INNER_BROW_RAISER_L = 22
 EXPRESSION_INNER_BROW_RAISER_R = 23
 EXPRESSION_JAW_DROP = 24
 EXPRESSION_JAW_SIDEWAYS_LEFT = 25
 EXPRESSION_JAW_SIDEWAYS_RIGHT = 26
 EXPRESSION_JAW_THRUST = 27
 EXPRESSION_LID_TIGHTENER_L = 28
 EXPRESSION_LID_TIGHTENER_R = 29
 EXPRESSION_LIP_CORNER_DEPRESSOR_L = 30
 EXPRESSION_LIP_CORNER_DEPRESSOR_R = 31
 EXPRESSION_LIP_CORNER_PULLER_L = 32
 EXPRESSION_LIP_CORNER_PULLER_R = 33
 EXPRESSION_LIP_FUNNELER_LB = 34
 EXPRESSION_LIP_FUNNELER_LT = 35
 EXPRESSION_LIP_FUNNELER_RB = 36
 EXPRESSION_LIP_FUNNELER_RT = 37
 EXPRESSION_LIP_PRESSOR_L = 38
 EXPRESSION_LIP_PRESSOR_R = 39
 EXPRESSION_LIP_PUCKER_L = 40
 EXPRESSION_LIP_PUCKER_R = 41
 EXPRESSION_LIP_STRETCHER_L = 42
 EXPRESSION_LIP_STRETCHER_R = 43
 EXPRESSION_LIP_SUCK_LB = 44
 EXPRESSION_LIP_SUCK_LT = 45
 EXPRESSION_LIP_SUCK_RB = 46
 EXPRESSION_LIP_SUCK_RT = 47
 EXPRESSION_LIP_TIGHTENER_L = 48
 EXPRESSION_LIP_TIGHTENER_R = 49
 EXPRESSION_LIPS_TOWARD = 50
 EXPRESSION_LOWER_LIP_DEPRESSOR_L = 51
 EXPRESSION_LOWER_LIP_DEPRESSOR_R = 52
 EXPRESSION_MOUTH_LEFT = 53
 EXPRESSION_MOUTH_RIGHT = 54
 EXPRESSION_NOSE_WRINKLER_L = 55
 EXPRESSION_NOSE_WRINKLER_R = 56
 EXPRESSION_OUTER_BROW_RAISER_L = 57
 EXPRESSION_OUTER_BROW_RAISER_R = 58
 EXPRESSION_UPPER_LID_RAISER_L = 59
 EXPRESSION_UPPER_LID_RAISER_R = 60
 EXPRESSION_UPPER_LIP_RAISER_L = 61
 EXPRESSION_UPPER_LIP_RAISER_R = 62
 CONFIDENCE_LOWER_FACE = 0
 CONFIDENCE_UPPER_FACE = 1

For HTC:

EYE_LEFT_BLINK = 0
 EYE_LEFT_WIDE = 1
 EYE_RIGHT_BLINK = 2
 EYE_RIGHT_WIDE = 3
 EYE_LEFT_SQUEEZE = 4
 EYE_RIGHT_SQUEEZE = 5
 EYE_LEFT_DOWN = 6
 EYE_RIGHT_DOWN = 7
 EYE_LEFT_OUT = 8
 EYE_RIGHT_IN = 9
 EYE_LEFT_IN = 10
 EYE_RIGHT_OUT = 11
 EYE_LEFT_UP = 12
 EYE_RIGHT_UP = 13
 LIP_JAW_RIGHT = 0
 LIP_JAW_LEFT = 1
 LIP_JAW_FORWARD = 2
 LIP_JAW_OPEN = 3
 LIP_MOUTH_APE_SHAPE = 4
 LIP_MOUTH_UPPER_RIGHT = 5
 LIP_MOUTH_UPPER_LEFT = 6
 LIP_MOUTH_LOWER_RIGHT = 7
 LIP_MOUTH_LOWER_LEFT = 8
 LIP_MOUTH_UPPER_OVERTURN = 9
 LIP_MOUTH_LOWER_OVERTURN = 10
 LIP_MOUTH_POUT = 11
 LIP_MOUTH_SMILE_RIGHT = 12
 LIP_MOUTH_SMILE_LEFT = 13
 LIP_MOUTH_SAD_RIGHT = 14
 LIP_MOUTH_SAD_LEFT = 15
 LIP_CHEEK_PUFF_RIGHT = 16
 LIP_CHEEK_PUFF_LEFT = 17
 LIP_CHEEK_SUCK = 18
 LIP_MOUTH_UPPER_UPRIGHT = 19
 LIP_MOUTH_UPPER_UPLEFT = 20
 LIP_MOUTH_LOWER_DOWNRIGHT = 21
 LIP_MOUTH_LOWER_DOWNLEFT = 22
 LIP_MOUTH_UPPER_INSIDE = 23
 LIP_MOUTH_LOWER_INSIDE = 24
 LIP_MOUTH_LOWER_OVERLAY = 25
 LIP_TONGUE_LONGSTEP1 = 26
 LIP_TONGUE_LEFT = 27
 LIP_TONGUE_RIGHT = 28
 LIP_TONGUE_UP = 29
 LIP_TONGUE_DOWN = 30
 LIP_TONGUE_ROLL = 31
 LIP_TONGUE_LONGSTEP2 = 32
 LIP_TONGUE_UPRIGHT_MORPH = 33
 LIP_TONGUE_UPLEFT_MORPH = 34
 LIP_TONGUE_DOWNRIGHT_MORPH = 35
 LIP_TONGUE_DOWNLEFT_MORPH = 36