def orthonormal_initializer(output_size, input_size, debug=False):
print((output_size, input_size))<EOL>if debug:<EOL><INDENT>Q = np.random.randn(input_size, output_size) / np.sqrt(output_size)<EOL>return np.transpose(Q.astype(np.float32))<EOL><DEDENT>I = np.eye(output_size)<EOL>lr = <NUM_LIT><EOL>eps = <NUM_LIT> / (output_size + input_size)<EOL>success = False<EOL>tries = <NUM_LIT:0><EOL>while not success and tries < <NUM_LIT:10>:<EOL><INDENT>Q = np.random.randn(input_size, output_size) / np.sqrt(output_size)<EOL>for i in range(<NUM_LIT:100>):<EOL><INDENT>QTQmI = Q.T.dot(Q) - I<EOL>loss = np.sum(QTQmI ** <NUM_LIT:2> / <NUM_LIT:2>)<EOL>Q2 = Q ** <NUM_LIT:2><EOL>Q -= lr * Q.dot(QTQmI) / (<EOL>np.abs(Q2 + Q2.sum(axis=<NUM_LIT:0>, keepdims=True) + Q2.sum(axis=<NUM_LIT:1>, keepdims=True) - <NUM_LIT:1>) + eps)<EOL>if np.max(Q) > <NUM_LIT> or loss > <NUM_LIT> or not np.isfinite(loss):<EOL><INDENT>tries += <NUM_LIT:1><EOL>lr /= <NUM_LIT:2><EOL>break<EOL><DEDENT><DEDENT>success = True<EOL><DEDENT>if success:<EOL><INDENT>print(('<STR_LIT>' % loss))<EOL><DEDENT>else:<EOL><INDENT>print('<STR_LIT>')<EOL>Q = np.random.randn(input_size, output_size) / np.sqrt(output_size)<EOL><DEDENT>return np.transpose(Q.astype(np.float32))<EOL>
adopted from Timothy Dozat https://github.com/tdozat/Parser/blob/master/lib/linalg.py

    Parameters
    ----------
    output_size : int
    input_size : int
    debug : bool
        Whether to skip this initializer
    Returns
    -------
    Q : np.ndarray
        The orthonormal weight matrix of input_size x output_size