Official pytorch code for "From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers" - View it on GitHub