Deep Learning
Muon: Second Order Optimizer for Hidden Layers
·1209 words·6 mins
Muon is a second-order optimizer for deep learning models, designed to accelerate training and reduce memory usage. It leverages information about the curvature of the loss landscape to achieve faster convergence and more efficient memory utilization. By overcoming historical computational barriers and standardizing its usage, Muon brings the theoretical advantages of second-order optimization to the scale required for LLMs, potentially reshaping both practice and expectations in deep learning.
Pioneers of Machine Learning and Artificial Intelligence
·591 words·3 mins
The journey of pioneers in Machine Learning (ML) and Artificial Intelligence (AI) is a remarkable tale of innovation, collaboration, and the relentless pursuit of knowledge.
Gran Turismo's Sophy AI
·859 words·5 mins
Gran Turismo Sophy is an advanced AI racing agent developed through a collaboration between Sony AI, Polyphony Digital, and Sony Interactive Entertainment. This groundbreaking technology utilizes deep reinforcement learning to master the complexities of competitive racing in the Gran Turismo Sport simulator. Initially starting as an AI that struggled to navigate tracks, Sophy has evolved into a formidable competitor capable of challenging top human drivers by mastering racing tactics, etiquette, and vehicle control.
AlexNet Revolution
·1304 words·7 mins
In 2012, the field of artificial intelligence witnessed a seismic shift. The catalyst for this transformation was a deep learning model known as AlexNet.