MOE论文详解(4)-GLaM:Efficient Scaling of Language Models with Mixture-of-Experts

机器学习 分类

2024

10-15

MOE论文详解(4)-GLaM:Efficient Scaling of Language Models with Mixture-of-Experts

10-15

MOE论文详解(3)-Switch Transformers:Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

10-15

MOE论文详解(2)-GShard:Scaling Giant Models with Conditional Computation and Automatic Sharding

10-14

MOE论文详解(1)-OUTRAGEOUSLY LARGE NEURAL NETWORKS:THE SPARSELY-GATED MIXTURE-OF-EXPERTS LAYER

05-27

Megatron-LM源码系列(八)： Context Parallel并行

02-23

Gemma模型论文详解(附源码)

02-04

Megatron-LM源码系列(七)：Distributed-Optimizer分布式优化器实现Part2

2023

12-31

Megatron-LM源码系列(六)：Distributed-Optimizer分布式优化器实现Part1

12-23

FP16数据格式详解

12-21

Megatron-LM源码系列(五)： FP16使用