Skip to content

External Data Recorder - Data Files Reference

This page describes every file the External Data Recorder writes, and every column in them, starting with the gaze and object data most studies use. For an overview of the folders, see What Gets Recorded.

Every file from a session starts with the session ID, <date>-<time>_<participant ID> (for example 09-26-2026-09-32-04_0), written as <id> below. <n> is the trial number. The CSV files are in data/<id>_experiment_data/trial_data/. All times are in seconds unless the column name says otherwise.

Data you can add, such as physiological signals and ratings, is described in Optional Data.

The External Data Recorder is built on SightLab and also writes SightLab's standard files. A few of them, and some of their columns, measure SightLab's own 3D scene, which external applications don't use. Those are listed separately at the end, in SightLab Data That Doesn't Apply.

AI Object Detection Files

After each trial, postprocessing scans every frame of the recording for objects and matches the gaze point to them (see AI Object Detection). It writes these three files when USE_OBJECT_DETECTION = True. Each detected object gets a tracker ID that follows it from frame to frame, and objects are written as <class> <tracker ID>, such as control panel 1.

Object Detection Summary

<id>_obj_det_summary_trial_<n>.csv has one row per object that was looked at long enough to register a dwell.

  • Tracker ID: the object's tracker ID, the number shown in Gazed Object and All Detected Objects in the timeline
  • Total Dwell Time: the total time spent on the object across all dwells, including the first DWELL_THRESHOLD seconds of each
  • Dwell Count: the number of separate dwells
  • Mean Dwell Time: Total Dwell Time divided by Dwell Count
  • Total Fixation Time and Total Saccade Time: the time spent in fixations and in saccades while dwelling on the object
  • First Dwell Time: when the first dwell started, in seconds from the start of the trial
  • First Dwell Frame: the video frame where the first dwell started
  • Total Dwell Frames: the number of video frames counted while dwelling
  • Mean Confidence: the average detection confidence while dwelling, from 0 to 1
  • Majority Class: the class the object was labeled as for the most time, since a tracked object's label can change from frame to frame

Class Detection Summary

<id>_class_det_summary_trial_<n>.csv has the same measures as the object detection summary, per class instead of per object (for example, all chairs together). It has Class Name in place of Tracker ID, and no Majority Class.

Class totals aren't the sum of the object totals: a class dwell continues when the gaze moves between two objects of the same class, and an object's label can change during a dwell.

Object Detection Timeline

<id>_obj_det_timeline_trial_<n>.csv has one row per video frame.

  • Frame: the frame number in the video, starting from 0
  • Frame PTS Timestamp: the frame's time in seconds from the start of the trial (its time in the video plus the Recording Offset). This matches timestamp (secs) in the trial data.
  • Gazed Object: the object being looked at, as <class> <tracker ID>. When the gaze point is inside several boxes, the smallest one is used.
  • Gazed Object Confidence: that object's detection confidence
  • Object Dwell Flag: Gaze Started when the gaze moves onto a new object, Dwell Registered once it has stayed for DWELL_THRESHOLD seconds, and Dwelling while it continues. Blank on other frames.
  • Class Dwell Flag: the same as Object Dwell Flag, for classes
  • Gaze Flag: FIXATION or SACCADE; blank if there was no gaze sample within MAX_GAP_S of the frame
  • Gaze Screen X Position and Gaze Screen Y Position: the gaze point in pixels of the video, measured from the top-left corner; blank if there was no gaze sample
  • All Detected Objects: each object detected in the frame, as <class> <tracker ID>
  • All Detected Classes: each class detected in the frame, separated by commas
  • All Bounding Boxes: each object's box, as [x1 y1 x2 y2] in pixels of the video, in the same order as All Detected Objects
  • All Confidences: each object's detection confidence, from 0 to 1, in the same order
  • Near Misses Objects: objects whose box the gaze point was just outside of, within the eye tracker's measured accuracy (the validation RMS error). If the gaze point isn't inside any box, the nearest of these becomes the Gazed Object.
  • Near Misses Angular Margins: how far outside each of those boxes the gaze point was, in degrees, in the same order

Videos

recordings/, one set per trial. Videos are .avi by default (RECORDING_TYPE and OUTPUT_VIDEO_TYPE in Postprocess_Config.py).

  • <id>_overlay_trial_<n>.avi: the recording with the detected objects' boxes and labels, and the gaze point, drawn on it
  • <id>_transcoded_trial_<n>.avi: the recording re-encoded for the replay, with no overlays, and cropped if you cropped the recording in postprocessing
  • <id>_experiment_data_trial_<n>.avi: the original OBS recording of the window you chose

The overlay and transcoded videos can both be used in the replay.

Trial Data

<id>_trial_data_<n>.csv has one row per sample, and is the file to use for sample-by-sample eye tracking data. Columns that don't apply to your hardware show None or -.

Eye Tracker Gaze

For Vive Focus Vision, Focus 3, Pro Eye, Meta Quest Pro (over Meta Horizon Link or Steam Link), Steam Frame, and Pimax Dream Air:

  • Tracker Right Eye Yaw/Pitch, Tracker Left Eye Yaw/Pitch, Tracker Both Eye Yaw/Pitch: the direction each eye is looking, as yaw and pitch in degrees, relative to the head
  • Right Eye Tracker Matrix, Left Eye Tracker Matrix, Both Eye Tracker Matrix: the same gaze directions as 4×4 transform matrices. Postprocessing converts the mirror eye's matrix into a point on the recorded window using the calibration.

Headsets that report a single combined gaze, such as the Pimax Dream Air and headsets connected over Steam Link, have the same values for all three eyes.

For hardware without eye tracking, a single column replaces all of these:

  • Stand-in Eye Tracker Matrix: for Meta Quest 3 Recorder and Desktop First Person Game, an identity matrix, meaning the center of the view. For Desktop, the mouse cursor's position in the recorded window, in pixels, stored as the first two values of the matrix's third row.

Fixations and Saccades

  • fixation status: FIXATION or SACCADE, from SightLab's dispersion-based (I-DT) algorithm, using the eye tracker's gaze direction (or the mouse, for Desktop)
  • delta gaze angle (deg): how far the gaze moved since the previous sample
  • delta gaze velocity (deg/sec): how fast the gaze moved since the previous sample

Meta Quest 3 Recorder and Desktop First Person Game have no eye tracking, so for them these don't reflect eye movements.

Pupil Size, Eye Openness, and Heart Rate

  • Eye OpenL, Eye OpenR: how open the left and right eye are, from 0 (closed) to 1 (open), for HP Omnicept, Vive (Focus Vision, Focus 3, and Pro Eye), and the Meta Quest Pro over Steam Link. Over Steam Link, both columns have the same value unless you set up per-eye openness, and the Steam Frame always reads fully open.
  • Eye OpenBoth: how open both eyes are together, from 0 (closed) to 1 (open). Recorded over Steam Link only; None for other hardware. On port 9015 it's the average of Eye OpenL and Eye OpenR, and on port 9000 it's the one value Steam Link sends for both eyes.

The rest of these columns are only recorded over Steam Link. They show where the eye openness values came from, so you can check whether you have separate values for each eye:

  • Eye Open Source: face weights per eye when each eye has its own value (port 9015, with face tracking sharing on), EyesClosedAmount shared when one value for both eyes was copied into Eye OpenL, Eye OpenR, and Eye OpenBoth (port 9000), or none if no eye openness data arrived, in which case those columns are None
  • Steamlink EyesClosedAmount, Steamlink EyesClosedL, Steamlink EyesClosedR, Steamlink LidTightenerL, Steamlink LidTightenerR: the raw eyelid values as Steam Link sent them, from 0 (open) to 1 (closed), the opposite direction to the Eye Open columns. EyesClosedAmount covers both eyes. The per-eye EyesClosed and LidTightener values only arrive on port 9015, and are None on port 9000. On port 9015, each eye's openness is 1 − (EyesClosed + EyesClosed × LidTightener), kept between 0 and 1. On port 9000, openness is 1 − EyesClosedAmount.
  • Pupil Data: pupil diameter, HP Omnicept only. Vive headsets don't provide pupil diameter to the External Data Recorder, a limitation of SRanipal.
  • Heart Rate: HP Omnicept only

Events and Lab Streaming Layer

  • Sync Event: key pressed on the sample where a sync event was sent, with NETWORK_SYNC_KEY or NETWORK_SYNC_EVENT
  • flag: custom flags set during the trial, or - if there are none
  • LSL Data: when SAVE_LSL_DATA = True, the latest sample from the Lab Streaming Layer stream you selected; No data for this timestamp if the stream sent nothing new, or No streams were found if there was no stream
  • LSL Timestamp: the Lab Streaming Layer timestamp of that sample; -1 if there was no stream

Timing and Video Synchronization

  • timestamp (secs): time since the trial started. The fixation and saccade timeline and the object detection files use the same clock, so you can match rows between them by time.
  • unix timestamp: the sample's wall-clock time, in seconds since January 1, 1970 (UTC)
  • fps: the frame rate when the sample was taken
  • Recording Offset: seconds from the start of the trial to the start of the OBS recording, the same on every row. A frame's time in the video plus the Recording Offset gives its time on the trial clock. If it's blank, OBS didn't report when recording started, and postprocessing can't align the video with the gaze data.
  • Trial Start Wall and Record Start Wall: when the trial started and when OBS started recording, as Unix timestamps

Session Information

  • p_id: the participant ID
  • date/time: when the session started, the same as in the session ID
  • trial number and label: the trial and its label (TRIAL_CONDITION)

The trial data also has SightLab's 3D scene gaze and head columns, which don't apply to external applications. See Trial Data Scene Columns.

Fixation and Saccade Timeline

<id>_trial_timeline_fixation_saccade_<n>.csv has one row per fixation or saccade. To find what was looked at during a fixation, match its start and end times to the object detection timeline.

  • Type: FIXATION or SACCADE
  • Start Time (secs), End Time (secs), Duration (secs)
  • Dispersion (deg): how spread out the gaze points were
  • Average Delta Velocity (deg/sec) and Peak Delta Velocity (deg/sec): the average and highest sample-to-sample gaze velocity
  • Peak Delta Amplitude (deg): the largest sample-to-sample gaze movement
  • Total Angular Distance (deg): the total distance the gaze moved
  • Saccadic Amplitude (deg) and Saccadic Velocity (deg/sec): saccades only; - for fixations
  • Participant ID, Trial Date/Time, Last Name, First Name, Trial Number, Trial Label: the session and trial
  • ROI/Object: doesn't apply to external applications; always None

Fixations are detected with the DISPERSION_THRESHOLD and DURATION_THRESHOLD recorded in config.json. See Data Analysis Overview for how SightLab detects fixations.

Calibration Files

calibration_data/<id>_eye_calibration_trial_1.json is written for each new calibration, with the measured accuracy in degrees. It's named after the session that created it, so a calibration you reuse in later sessions keeps its original name.

Calibration file fields
  • metadata:
    • datetime: when the file was written
    • hardware_config: the selected hardware configuration
    • window_title: the title of the window calibrated against, such as SteamVR's "VR View"
    • window_size_px: the window's width and height in pixels
    • mirror_eye: the eye the mirror window shows, and that gaze data is collected from
    • dwell_seconds: how long gaze is sampled on each calibration dot
    • settle_seconds: how long gaze is allowed to settle on a dot before sampling
    • dot_depth_m: the distance of the calibration dots, in meters
    • dot_extent_deg: half the angular size of the calibration grid in the headset view
    • grid: the columns and rows of the calibration grid
    • dot_jitter_frac: the small offset applied so no three dots are in a straight line
    • validation_attempts: each validation attempt, with:
      • attempt: the attempt number
      • n_calibration_points: the number of calibration points
      • passed: whether the validation passed
      • reason: a summary of the errors if it passed, or why it failed
      • summary: n_points, mean_deg (mean angular error), rms_deg (root mean square angular error), max_deg (largest angular error), rms_px and max_px (pixel errors), rms_pct_diag (pixel error as a percentage of the window diagonal, for comparing different window sizes), mean_offset_px and offset_magnitude_px (average offset between gaze and targets), offset_noise_floor_px (the offset expected from random error alone), and mean_precision_deg
    • saved_attempt: which validation attempt was kept
  • points: each calibration point, with its dot_index, target_px (the dot's position in the window), gaze_ab (the sampled gaze direction), dot_world (the dot's 3D position), n_red_target_frames, n_gaze_samples, and precision_deg
  • validation: the validation of the saved attempt, with passed, accepted_despite_failure (whether you pressed A to accept a failed validation), criteria (the max_mean_deg, max_point_deg, and min_points it was tested against), points_frac (where the validation dots were), summary (as above), points, and raw_points

The saved attempt's rms_deg is the accuracy margin postprocessing uses for near misses.

Post-experiment validation: the optional end-of-session validation writes <calibration name>_postvalidation.json next to the calibration it tested. It has the same metadata (plus run_mode, source_calibration, and drift_warn_deg), the new summary, points, and raw_points, and the result of the slippage check: drift, error_grew, mapping_shifted, and drift_exceeds_warning.

Hardware without eye tracking (Meta Quest 3 Recorder, Desktop, and Desktop First Person Game) gets a stand-in calibration instead, with mirror_eye set to Stand-in, the window size and field of view, and four points that map the view directly onto the window.

See the Eye Calibration Guide for how calibration and validation work.

Optional Data

The External Data Recorder can record any sensor data that comes into Vizard and isn't being used by the external application. The external application usually only uses the headset's head and hand tracking, so other devices, such as physiological sensors and EEG and fNIRS systems, are free to record. You can also add SightLab features such as rating scales and audio recording. See Common Metrics for everything SightLab can collect.

None of these are recorded by default. Biopac, Lab Streaming Layer, and face tracking are turned on with a setting in Data_Recorder_Config.py. The others may require some custom code: add them to External_Data_Recorder.py by following the linked examples.

Physiological Data, EEG, and fNIRS

With BIOPAC_ON = True, AcqKnowledge starts acquiring when the recording starts, and the recorder inserts video start and sync event markers so you can line the physiological signals up with the session data. Examples of what you can record with BIOPAC (not a full list):

  • Electrodermal Activity (EDA): skin conductance response
  • Heart Rate (HR): cardiovascular responses
  • Electroencephalography (EEG): brain activity
  • Functional Near-Infrared Spectroscopy (fNIRS): cortical hemodynamic responses

The signals are saved in AcqKnowledge. To also save channel values to the trial data, see Saving Physiological Data to the SightLab Data Files. See Biopac Integration for how the recorder works with AcqKnowledge.

Ratings, Surveys, and Demographics

Audio, Transcription, and Speech Recognition

Record the participant's microphone during the session, and convert the recording to a text transcript with the convert_to_transcript script. A speech recognition example shows how to respond to what the participant says. See Audio Recording, Speech Recognition and Transcription.

Face Tracking

With RECORD_FACE_TRACKER_DATA = True, facial expressions from supported headsets, such as the Meta Quest Pro, are saved to data/expressions_<YYYYMMDD_HHMMSS>.csv. The file is in the data folder itself rather than the session folder, and is named by the time the recorder started. Each row has a TimeStamp and one column per facial expression weight, Expression_0 to Expression_58. Visualize it with facial_expressions_over_time.py from the ExampleScripts/Face_Tracking_Data folder. See Face Tracking.

Other Session Files

Session Configuration

data/<id>_experiment_data/config.json records how the session was set up:

  • hardware_config: the hardware configuration you chose, such as Pimax Dream Air
  • selected_window: the title of the window that was recorded, such as Pimax Mirror - Right or VR View
  • "1": SightLab's settings for the session. The ones that matter for external applications are NUMBER_OF_TRIALS, TRIAL_LABEL, BIOPAC, and the fixation thresholds DISPERSION_THRESHOLD (degrees) and DURATION_THRESHOLD (seconds).

Replay File

data/<id>_experiment_data/replay_data/<id>_replay_data_<n>.rply is SightLab's replay file for the trial. It holds the same samples as the trial data file, in a binary format. External_Data_Replay.py plays it back, and postprocessing reads the gaze data and the Recording Offset from it, so keep it with the session if you want to replay or reprocess it later.

Frame Timestamps

<id>__frame_timestamps_trial_<n>.json (with two underscores) is written at the start of postprocessing. It lists every frame of the OBS recording under frames, each with:

  • pts_time: the frame's time in the video, in seconds
  • width and height: the frame size, in pixels

Postprocessing uses it to match each video frame to the nearest gaze sample. It's written even when object detection is off.

Console Logs

logs/<id>_logfile.txt is everything the console showed during the session, including calibration results, postprocessing progress, and errors. It's saved when you close the recorder.

SightLab Data That Doesn't Apply

These SightLab files and columns measure SightLab's own 3D scene and regions of interest. External applications don't run in that scene, so the data is empty, constant, or not meaningful. They're written because the External Data Recorder uses SightLab's standard data saving.

Experiment Summary

data/<id>_experiment_data/<id>_experiment_summary.csv has one row per trial, for a single region called ENTIRE_SCENE:

  • Dwell Visits, Total Dwell Time (secs), and Avg Dwell Time (secs) are always 0. Dwell on objects is in the AI object detection files.
  • Fixation Count, Time to First Fixation (secs), Total Fixation Duration (sec), Average Fixation Duration (sec), Avg Saccadic Amplitude (deg), Avg Saccadic Velocity (deg/sec), and Saccadic Velocity Peak (deg/sec) are valid: they summarize the fixation and saccade timeline over the whole trial.
  • The other columns are Participant ID, Experiment Date/Time, First Name, Last Name, Trial Number, Trial Length (secs), Label, and ROI/Object.

If you add ratings, survey or demographic questions, or other per-trial values (see Optional Data), they're saved as extra columns in this file.

Dwell Timeline

<id>_trial_timeline_dwell_<n>.csv has only a header row, since dwell is measured on SightLab 3D objects. Its columns are Participant ID, Trial Date/Time, Last Name, First Name, Trial Number, Start Time (secs), End Time (secs), ROI/Object, Dwell Time (secs), Fixation Count, and Flag.

Trial Data Scene Columns

These columns in the trial data measure SightLab's 3D scene, and stay constant:

  • combined eye intersect x/y/z, left eye intersect x/y/z, right eye intersect x/y/z
  • combined eye yaw/pitch/roll, left eye yaw/pitch/roll, right eye yaw/pitch/roll
  • head x/y/z, head yaw/pitch/roll: fixed while LOCK_TRANSPORT = True (the default)
  • view status: always None

Also, Eye Yaw, Eye Pitch, and Eye Roll are only recorded in the first row, when the trial starts. Use the eye tracker gaze columns for eye angles, and Gaze Screen X/Y Position in the object detection timeline for where the participant looked on screen.

Other Columns and Settings

  • ROI/Object in the fixation and saccade timeline is always None.
  • In config.json, the settings under "1" other than those listed in Session Configuration, such as ENVIRONMENT, the avatar settings, and REGIONS_OF_INTEREST, describe SightLab's 3D scene and aren't used.