Обложка канала

Spark in me - Internet, data science, math, deep learning, philosophy

2440 @snakers4

Канал про интересные мне темы - интернет - статистика - наука о данных Без рекламы и буллшита.

Spark in me - Internet, data science, math, deep learning, philosophy

3 года назад
Открыть в
An interesting perspective here. What if LLMs are viewed though the lens of Microsoft willing to take some part of the search market? Trends in the dollar training cost of machine learning systems - epochai.org/blog/tr…-systems The Inference Cost Of Search Disruption – Large Language Model Cost Analysis - www.semianalysis.com/p/the-i…sruption The AI Brick Wall – A Practical Limit For Scaling Dense Transformer Models, and How GPT 4 Will Break Past It - www.semianalysis.com/p/the-a…al-limit Training Compute-Optimal Large Language Models - https://arxiv.org/pdf/2203.15556.pdf
Trends in the dollar training cost of machine learning systems

I combine training compute and GPU price-performance data to estimate the cost of compute in US dollars for the final training run of 124 machine learning systems published between 2009 and 2022, and find that the cost has grown by approximately 0.5 orders of magnitude per year.

Epoch