Numpy: How To Get Rid Of The Minima Along Axis=1, Given The Indices - In An Efficient Way?

Given a matrix A with shape (1000000,6) I have figured out how to get the minimum rightmost value for each row and implemented it in this function: def calculate_row_minima_indices

Solution 1:

Assuming you have enough memory to hold a boolean mask the shape of your original array as well as the new array, here's one way to do it:

import numpy as np

defmain():
    np.random.seed(1) # For reproducibility
    data = generate_data((10, 6))

    indices = rightmost_min_col(data)
    new_data = pop_col(data, indices)

    print'Original data...'print data
    print'Modified data...'print new_data

defgenerate_data(shape):
    return np.random.randint(0, 10, shape)

defrightmost_min_col(data):
    nrows, ncols = data.shape[:2]
    min_indices = np.fliplr(data).argmin(axis=1)
    min_indices = (ncols - 1) - min_indices
    return min_indices

defpop_col(data, col_indices):
    nrows, ncols = data.shape[:2]
    col_indices = col_indices[:, np.newaxis]
    row_indices = np.arange(ncols)[np.newaxis, :]
    mask = col_indices != row_indices
    return data[mask].reshape((nrows, ncols-1))

if __name__ == '__main__':
    main()

This yields:

Original data...
[[5 8 9 5 0 0][1 7 6 9 2 4][5 2 4 2 4 7][7 9 1 7 0 6][9 9 7 6 9 1][0 1 8 8 3 9][8 7 3 6 5 1][9 3 4 8 1 4][0 3 9 2 0 4][9 2 7 7 9 8]]
Modified data...
[[5 8 9 5 0][7 6 9 2 4][5 2 4 4 7][7 9 1 7 6][9 9 7 6 9][1 8 8 3 9][8 7 3 6 5][9 3 4 8 4][0 3 9 2 4][9 7 7 9 8]]

One of the less readable tricks I'm using here is exploiting numpy's broadcasting during array comparisons. As a quick example, consider the following:

import numpy as np
a = np.array([[1, 2, 3]])
b = np.array([[1],[2],[3]])
print a == b

This yields:

array([[ True, False, False],
       [False,  True, False],
       [False, False,  True]], dtype=bool)

So, if we know the column index of the item we want removed, we can vectorize the operation for an array of column indices, which is what pop_col does.

Solution 2:

you can use a bool mask array to do the selection, but the memory useage is a little large.

import numpy

h = numpy.random.randint(0, 10, (20, 6))

flipped = numpy.fliplr(h) # flip the matrix to get the rightmost minimum.
flipped_indices = numpy.argmin(flipped, axis=1)
indices = 5 - flipped_indices

mask = numpy.ones(h.shape, numpy.bool)

mask[numpy.arange(h.shape[0]), indices] = False

result = h[mask].reshape(-1, 5)