thsolver.dataset
This module provides the built-in filelist-based dataset helper and the default binary file reader used by it.
- read_file(filename)[source]
Reads a binary sample file into a tensor of bytes.
- Parameters:
filename (str) – The file to read.
- Returns:
A 1-D tensor containing the raw file bytes.
- Return type:
- class Dataset(root, filelist, transform, read_file=<function read_file>, in_memory=False, take: int = -1)[source]
A lightweight dataset helper based on file lists.
- Parameters:
root (str) – The dataset root directory.
filelist (str) – The text file listing the samples.
transform (callable) – The callable applied to each loaded sample.
read_file (callable) – The file reader used to load raw samples.
in_memory (bool) – If True, loads all samples into memory at startup.
take (int) – Limits the number of samples used from the file list.