DALL-E Mini Explained with Demo Tech report: - Financed by Google Cloud and HF, essentially an advertising campaign for JAX, 8 person team - 27x smaller than the original, trained on a single TPU v3-8 for only 3 days + ~3 weeks for experiments, 400M params - 30m image-text pairs, only 2m used to fine-tune the VQGAN encoder - Could use preemptible TPU instances - Pre-trained BART Encoder - Pre-trained VQGAN encoder - Pre-trained CLIP is used to select the best generated images - (so the actual cost probably is actually ~1-2 orders of magnitude higher) - (compare with 20k GPU days stipulated by Sber) - The report is expertly written and easy to read