Dataset Structure

This page describes the layout of the public AVS release – what each directory holds, how files are named, and what is deliberately not included. For how to obtain the data, see Data Access; for data-quality caveats, see Known Issues.

Note

The public release is being prepared. The layout, file names and file formats below are those of the release tree as built and verified internally. One packaging detail is still open – whether small files are grouped into archives for the hosting repository – and is marked as such below.

Top-Level Layout

avs-public/
├── manifest.tsv                       # every file: category, path, size
├── stimuli/
│   ├── images/                        # 4,080 scene JPEGs
│   ├── annotations/
│   │   ├── coco_objects/              # MS-COCO instance masks, per scene
│   │   └── cocostuff/                 # COCO-Stuff masks, per scene
│   └── avs_scenes_all_licenses.parquet # per-image licence/attribution table
├── sub-01/ .. sub-05/
│   ├── anat/                          # T1.mgz (defaced)
│   └── ses-01/ .. ses-10/
│       ├── meg/                       # raw MEG, native names
│       ├── eyetrack/                  # raw EyeLink EDF + Parquet exports
│       └── beh/                       # experiment logs, transcriptions
└── derivatives/
    ├── pyavs/sub-0X/ses-0Y/
    │   ├── meg/                       # per-run SSS, annotations, ICA
    │   ├── epochs/                    # fixation/saccade epochs + metadata
    │   └── eyetrack/                  # preprocessed gaze events and samples
    └── freesurfer/sub-0X/             # a directly usable MNE SUBJECTS_DIR
        ├── bem/  label/  surf/  mri/transforms/

Size and Composition

The release tree as built is 22,488 files, 663.1 GiB. Five categories account for over 99% of the volume:

Category

Size

Fixation/saccade epochs (derivatives/pyavs/.../epochs)

270.9 GiB

Raw MEG (sub-0X/ses-0Y/meg)

254.2 GiB

Maxwell-filtered MEG (derivatives/pyavs/.../meg, SSS)

129.1 GiB

Raw eye tracking (sub-0X/ses-0Y/eyetrack)

5.3 GiB

Preprocessed eye tracking (derivatives/pyavs/.../eyetrack)

1.7 GiB

Everything else – stimuli, annotations, behavioural logs, ICA solutions, the anatomy tier – together comes to roughly 1.8 GiB. If you do not need raw MEG or the precomputed epochs, a usable working subset is small.

The two eye-tracking categories are small because the gaze tables ship as Parquet rather than CSV (see below); as CSV they were 62.4 GiB and 23.6 GiB respectively.

manifest.tsv at the tree root lists every released file with its category, path within the release, and size in bytes; use it to check a download is complete. Per-file checksums are planned for the published version.

Subject, Session and Run Naming

Two naming schemes coexist: the internal one used by the acquisition systems (and still visible in raw filenames), and the BIDS-style one used for directories and derivatives.

Internal / raw filenames

Release / derivatives

Meaning

as01 – as05

sub-01 – sub-05

The 5 participants

session letter a – j

ses-01 – ses-10

The 10 MEG + eye-tracking sessions per participant

(n/a)

anat/

The structural MRI (acquired in a separate session)

as01a03.fif

run-03

One scene-viewing block within a session

as01ab.fif / as01ad.fif

task-noise_recording-b / -d

Empty-room recordings, before and danach (= after) the session

So as03b07.fif is sub-03, ses-02, run-07, and its Maxwell-filtered counterpart is sub-03_ses-02_task-avs_run-07_raw-sss.fif.

pyAVS’s pyavs.utils.derivatives module encodes the derivative naming conventions; see Utilities (pyavs.utils).

BIDS-Inspired, Not BIDS-Valid

The release borrows BIDS’s directory grammar – sub-<label>/ses-<label>/<datatype>/ with a derivatives/ tree – but it is not a BIDS-validated dataset and does not ship the sidecar JSON/TSV files a validator would require. Do not point a BIDS tool at it and expect it to parse.

The main consequence for users is in the file names: raw files keep the names they were written with by the acquisition systems (as01b03.fif, as1_1_0.EDF), while everything pyAVS produced uses BIDS-style entity names (sub-01_ses-02_task-avs_run-03_raw-sss.fif). Directories are reorganized; filenames were deliberately left alone so that released files stay byte-identical to – and traceable to – the originals. BIDS-conformant renaming was considered and deferred.

Session Composition

Runs per session are not constant:

  • ses-01: 10 task runs. ses-02 – ses-10: 14 task runs each.

  • Each run is a block of 30 trials (one scene per trial), so a participant completes 30 x (10 + 9 x 14) = 4,080 trials – one per scene in the stimulus set.

  • Every session additionally has 2 empty-room recordings, one before and one after the task (...b.fif / ...d.fif). One is missing – see Known Issues.

Raw Data (sub-0X/)

anat/

T1.mgz – the individual structural MRI, defaced. One per participant.

ses-0Y/meg/

Unprocessed MEG as written by the Elekta Neuromag TRIUX, one FIF per run (as{SS}{letter}{run:02d}.fif) plus the two empty-room files. 306 channels (204 gradiometers, 102 magnetometers) at 1000 Hz. See MEG Acquisition.

ses-0Y/eyetrack/

The native EyeLink .EDF plus its three standard exports – *_events.parquet (fixations/saccades/blinks), *_messages.parquet (experiment messages and triggers) and *_samples.parquet (the 1000 Hz gaze/pupil sample stream). Named as{subject}_{session}_{index}, where the trailing index identifies the recording segment within the session: it is 0 when eye tracking ran from the session’s first block. A session where the tracker was restarted mid-session carries more than one segment (sub-04, ses-04), and one where early blocks were lost starts at a non-zero index (sub-03, ses-02) – both are documented in Known Issues.

ses-0Y/beh/

Experiment logs written by the presentation script: as_exp_data_*.parquet (with _archive_S / _archive_T companions), the transcribed caption logs (explog_transcribed_*.parquet, including a COCO-category-coded variant), and transcription_data/*.json, one JSON per transcribed caption trial. Speech audio (``.wav``) is not released – only the transcriptions. Two transcription variants ship: explog_transcribed_automated_* (raw automated speech-to-text output, unreviewed) and explog_transcribed_corrected_* (manually corrected and verified by the AVS team) – prefer the corrected variant for any analysis that needs high-quality transcriptions.

Derivatives (derivatives/pyavs/)

Produced by pyAVS; regenerable from the raw data with the package.

ses-0Y/meg/

  • ..._run-XX_raw-sss.fif – per-run Maxwell-filtered (SSS) MEG, including the two empty-room recordings (task-noise_recording-{b,d}_raw-sss.fif).

  • ..._annotations-{scene,microphone,caption}.fif – annotation-only FIFs carrying the event structure for the three recording types, small and separable from the data.

  • ..._ica.fif, ..._ica-exclusions.json, ..._ica-et-scores.parquet – the fitted ICA solution, the components marked for exclusion, and the gaze-correlation scores that identified the ocular components. ICA is shipped fitted but not applied, so you can inspect or override the exclusions; pyavs.apply_precomputed_ica() applies them. See MEG Preprocessing for the exclusion criterion and how components were selected.

ses-0Y/epochs/

Fixation- and saccade-locked sensor-space epochs, as HDF5, plus their metadata tables:

sub-01_ses-01_task-avs_fixation_scene_epochs.h5
sub-01_ses-01_task-avs_saccade_scene_epochs.h5
sub-01_ses-01_fixation_metadata.parquet
sub-01_ses-01_saccade_metadata.parquet

_scene_ means “recorded during the scene-viewing task” – not scene-onset-locked. These epochs are locked to eye movements, not to stimulus onset. No scene-onset-locked epochs are shipped; you can build them from the SSS runs and the scene annotations with pyavs.AVSComposer.

Inside each HDF5 file, one group per channel type (grad, mag), each holding an onset dataset of shape (n_epochs, n_channels, n_times) in float32, gzip-compressed and chunked one epoch per chunk so that reading a scattered subset of epochs costs roughly one chunk read per epoch. Acquisition and processing parameters are stored as file attributes (subject, session, event_type, blocks, times, rois, hz, filter). Epochs span -0.5 to 0.8 s around the event at 500 Hz (651 samples), band-pass filtered 0.2–200 Hz, with no baseline correction (AVS practice: there is no neutral pre-fixation baseline during active viewing).

The metadata table has one row per epoch, in file order, and is what you filter on. Columns cover event kinematics (duration, amplitude, peak_velocity, mean_gx / mean_gy, start_* / end_*, rms, sd), sequence position (fix_sequence, fix_sequence_from_last, sac_sequence, sac_sequence_from_last, and the *_pre / *_post neighbour columns), trial bookkeeping (subject, session, trial, block, trial_per_block, sceneID, time_in_trial, caption_task, recording), and the fixated object (object_label, object_id) derived by intersecting gaze position with the COCO-Stuff masks in stimuli/annotations/ – see Fixation Object Labeling.

Because sceneID and object_label are plain columns, cross-subject queries (“every fixation on scene X”, “every fixation on a person”) reduce to concatenating the 50 per-session metadata tables and filtering – no MEG data needs to be touched until you know which epochs you want.

ses-0Y/eyetrack/

Preprocessed gaze, four files per session: as_s{N}_el_events.parquet (fixations, saccades and blinks with scene/trial assignment), as_s{N}_el_msgs.parquet, as_s{N}_el_samples.parquet and as_s{N}_el_cleaned_samples.parquet (the sample stream, and the drift-corrected/cleaned version). See Eye Tracking.

Anatomy (derivatives/freesurfer/)

derivatives/freesurfer/ is a directly usable MNE SUBJECTS_DIR. Point subjects_dir at it and MNE will find each participant as sub-01 … sub-05:

import mne

subjects_dir = "avs-public/derivatives/freesurfer"
labels = mne.read_labels_from_annot("sub-01", parc="aparc", subjects_dir=subjects_dir)
morph = mne.compute_source_morph(src, subject_to="fsaverage", subjects_dir=subjects_dir)

Per participant it contains:

  • bem/sub-0X_oct6-src.fif – the oct6 cortical source space (8,196 sources).

  • bem/sub-0X-bem-sol.fif – single-shell (inner skull) BEM solution.

  • bem/sub-0X-fwd.fif – the precomputed forward solution.

  • mri/transforms/sub-0X-trans.fif – the MEG–MRI coregistration.

  • surf/{lh,rh}.{white,pial,inflated,sphere.reg,curv,sulc} – cortical surfaces; sphere.reg is what drives morphing to fsaverage.

  • label/ – 457 files: the FreeSurfer parcellations (aparc, a2009s, DKTatlas, BA_exvivo) as .annot/.ctab plus the individual .label files.

You supply fsaverage yourself; MNE ships one (mne.datasets.fetch_fsaverage()).

Note

Only cortex is released. Scalp and head surfaces are withheld – they reconstruct facial geometry and are therefore identifying. This is a privacy decision, not an oversight; see Known Issues for what it means in practice.

Stimuli (stimuli/)

  • images/ – the 4,080 scene JPEGs (see Stimuli).

  • annotations/coco_objects/ and annotations/cocostuff/ – per-scene segmentation masks (RLE-encoded JSON, one file per scene) used to assign a category to each fixation. pyavs.get_fixated_objects() reads these.

  • avs_scenes_all_licenses.parquet – one row per image with its licence and attribution. Check this before reproducing any scene in a figure: the images come from MS-COCO via the Natural Scenes Dataset and carry heterogeneous Flickr licences.

File Formats

Data

Format

MEG (raw, SSS, annotations, ICA)

FIF (mne)

Epochs

HDF5 (h5py; see the epochs section above for the internal layout)

Anatomy

FIF + FreeSurfer binary surface/label formats

Eye tracking (native)

EyeLink .EDF

Tabular data

Parquet – gaze events/samples, epoch metadata, behavioural logs, licence table

Scene annotations

JSON (RLE-encoded masks)

Scenes

JPEG

All tabular data ships as Parquet, not CSV. Every table in the release was converted: gaze sample streams and events, epoch metadata, behavioural and transcription logs, ICA gaze-correlation scores, and the image licence table. pandas.read_parquet reads them with the same call shape as read_csv.

The reason is the gaze sample streams: dense numeric time series at 1000 Hz are the worst case for row-oriented text and the best case for columnar storage. They account for nearly all of the ~80 GiB the conversion saved – the two eye-tracking categories dropped from 86.0 GiB as CSV to 7.0 GiB as Parquet. Beyond size, Parquet carries an embedded schema, so dtypes no longer have to be re-guessed on every read, and it supports reading one column or one row group without pulling the whole file.

Two consequences worth stating plainly:

  • Small tables can be slightly larger as Parquet than as CSV. Parquet has fixed per-file footer and schema overhead, which dominates for files of a few hundred bytes (some behavioural logs). This was accepted for format consistency across the release.

  • The behavioural logs are raw experiment output, not a derived export the way the gaze tables are of the untouched .EDF. Converting them means that tier no longer ships bit-identical to its original on-disk format. The .EDF files themselves ship untouched, so the native eye-tracking format is preserved.

Note

Still open: the release may be published with small files grouped into per-subject or per-session ZIP archives, to keep the file count manageable for the hosting repository. Large files (MEG, epochs, gaze samples) would stay individually addressable so that selective download remains possible. This affects packaging, not content. See Data Access.

What Is Not Included

Deliberate exclusions, with the reasoning:

  • Session-level concatenated/annotated MEG raws. Only per-run SSS files ship. The session-level *_raw-concatenated.fif / *_raw-annotated.fif are reconstructible from the per-run SSS files plus the small annotations-*.fif files, which do ship – at a saving of roughly 80 GB. pyavs.AVSComposer rebuilds them.

  • Microphone-locked (``mic_on``) epochs. Only fixation_scene and saccade_scene epochs ship. Other event-locked epochs can be computed from the released SSS runs and annotations.

  • Speech audio (``.wav``). Voice is identifying; the transcriptions ship instead.

  • Scalp/head surfaces (*-head*.fif, *.seghead) and non-defaced or additional MRI volumes – facial geometry.

  • Volumetric source spaces and forward solutions, FreeSurfer stats//scripts/, and intermediate reconstruction artifacts not used by any pyAVS pipeline.

  • Higher-tier derivatives – pupil dynamics, UMAP embeddings, ANN embeddings, fixation crops. These are analysis products, reproducible from what ships; several are demonstrated in Example Analyses.

  • Non-release participants – pilot and test recordings.

Modality-Specific Loading

pyAVS’s pyavs.dataloader submodule provides the loading functions for each modality (pyavs.load_meg_raw(), pyavs.load_eye_events(), pyavs.load_anatomical()), and pyavs.AVSComposer (see AVSComposer Guide) combines them into a single aligned MEG + eye-tracking workflow. See Quick Start Guide to get started.