Training colossal artificial intelligence models, especially the mighty large language models or transformers, is a resource intensive endeavor. While fine tuning these pretrained models on specific tasks is incredibly powerful, updating every single weight can be a memory hungry and time consuming process. Enter Low Rank Adaptation (LoRA), a brilliant technique that makes fine tuning…

Continue Reading