def _masked_softmax(F, att_score, mask, dtype):
if mask is not None:<EOL><INDENT>neg = -<NUM_LIT> if np.dtype(dtype) == np.float16 else -<NUM_LIT><EOL>att_score = F.where(mask, att_score, neg * F.ones_like(att_score))<EOL>att_weights = F.softmax(att_score, axis=-<NUM_LIT:1>) * mask<EOL><DEDENT>else:<EOL><INDENT>att_weights = F.softmax(att_score, axis=-<NUM_LIT:1>)<EOL><DEDENT>return att_weights<EOL>
Ignore the masked elements when calculating the softmax

    Parameters
    ----------
    F : symbol or ndarray
    att_score : Symborl or NDArray
        Shape (batch_size, query_length, memory_length)
    mask : Symbol or NDArray or None
        Shape (batch_size, query_length, memory_length)
    Returns
    -------
    att_weights : Symborl or NDArray
        Shape (batch_size, query_length, memory_length)