PA1: Automatic differentiation

Question 1: Auto Diff Library

Part 1: Operators

The list of operators that you will need to implement are:

• DivOp
• DivByConstOp
• TransposeOp
• ReLUOp
• SqrtOp
• PowerOp
• MeanOp
• MatMulOp
• SoftmaxOp
• LayerNormOp