Spark in me - Internet, data science, math, deep learning, philosophy. Страница 27

For matrix multiplication:
On FP16 inputs, all three dimensions (M, N, K) must be multiples of 8.

For convolution:
On FP16 inputs, input and output channels must be multiples of 8.

Prefer dense math operations.
For example, vanilla convolutions have much higher arithmetic intensity than depth-wise separable convolutions.

Choose mini-batch to be a multiple of 8
Choose linear layer dimensions to be a multiple of 8
Choose convolution layer channel counts to be a multiple of 8
For classification problems, pad vocabulary to be a multiple of 8
For sequence problems, pad the sequence length to be a multiple of 8

import time
import torch
import random
import string
from multiprocessing import Manager
from torch.utils.data import Dataset, DataLoader


def id_gen(size=6,
           chars=string.ascii_uppercase):
    return ''.join(random.choice(chars)
                   for _ in range(size))


class DataIter(Dataset):
    def __init__(self):
        m = Manager()
        self.data = m.dict({i: {'key': random.random(),
                                'path': id_gen(size=10)}
                            for i in range(1000000)})

    def __len__(self):
        return len(self.data)

    def __getitem__(self, idx):
        data = self.data[idx]
        return torch.tensor(data['cer']), data['path']


train_data = DataIter()

train_loader = DataLoader(train_data,
                          batch_size=60,
                          shuffle=False,
                          drop_last=False,
                          pin_memory=False,
                          num_workers=10)

tic = time.time()

for i, item in enumerate(train_loader):
    if (i + 1) % 1000 == 0:
        toc = time.time()
        print(f"Time for 1000 batches in {toc - tic} s")
        tic = time.time()