CSE234
约 283 字小于 1 分钟
PA1: Automatic differentiation
Question 1: Auto Diff Library
Part 1: Operators
The list of operators that you will need to implement are:
DivOp
DivByConstOp
TransposeOp
ReLUOp
SqrtOp
PowerOp
MeanOp
MatMulOp
SoftmaxOp
LayerNormOp
DivOp
class DivOp(Op):
"""Op to element-wise divide two nodes."""
def __call__(self, node_A: Node, node_B: Node) -> Node:
return Node(
inputs=[node_A, node_B],
op=self,
name=f"({node_A.name}/{node_B.name})",
)
def compute(self, node: Node, input_values: List[torch.Tensor]) -> torch.Tensor:
"""Return the element-wise division of input values."""
assert len(input_values) == 2
"""TODO: your code here"""
def gradient(self, node: Node, output_grad: Node) -> List[Node]:
"""Given gradient of division node, return partial adjoint to each input."""
"""TODO: your code here"""