def preprocess_dataset(dataset, transform, num_workers=<NUM_LIT:8>):
worker_fn = partial(_worker_fn, transform=transform)<EOL>start = time.time()<EOL>pool = mp.Pool(num_workers)<EOL>dataset_transform = []<EOL>dataset_len = []<EOL>for data in pool.map(worker_fn, dataset):<EOL><INDENT>if data:<EOL><INDENT>for _data in data:<EOL><INDENT>dataset_transform.append(_data[:-<NUM_LIT:1>])<EOL>dataset_len.append(_data[-<NUM_LIT:1>])<EOL><DEDENT><DEDENT><DEDENT>dataset = SimpleDataset(dataset_transform).transform(<EOL>lambda x: (x[<NUM_LIT:0>], x[<NUM_LIT:1>], x[<NUM_LIT:2>], x[<NUM_LIT:3>], x[<NUM_LIT:4>], x[<NUM_LIT:5>]))<EOL>end = time.time()<EOL>pool.close()<EOL>print('<STR_LIT>' % (end-start))<EOL>return dataset, dataset_len<EOL>
Use multiprocessing to perform transform for dataset.

    Parameters
    ----------
    dataset: dataset-like object
        Source dataset.
    transform: callable
        Transformer function.
    num_workers: int, default 8
        The number of multiprocessing workers to use for data preprocessing.