Translating Matlab To Python - Speeding Up A Loop

I have been translating some code from Matlab to Python that we use to analyse data in our lab. We have two lists of time stamps and we want to use one to herald the other: for eve

Solution 1:

How to get the runtime down to 6ms

As you already have seen Python loops are extremely slow. Per default there is no jit-Compiler which speeds up loops as in Matlab. So you have following possibilities:

• Vectorize your code in Numpy, if possible.
• Use Cython to compile the function
• Use Numba to compile the function

In the following example I use Numba, because it is really simple to use in such cases.

Example

import numpy as np
import numba as nb

@nb.njit()defmain_nb(Ctrigger, Csignal, offset, gate):
    Hsignal = np.zeros(Ctrigger.shape[0])

    marker = 1for j inrange(Ctrigger.shape[0]):
        m = marker
        t_star = Ctrigger[j] + offset - gate/2
        t_sto = Ctrigger[j] + offset + gate/2while m < Csignal.shape[0]:   
            if (Csignal[m] < t_star):
                marker = m
                m = m + 1elif (Csignal[m] >= t_star and Csignal[m] <= t_sto):
                Hsignal[m] = Csignal[m]
                m = m + 1elif (Csignal[m] > t_sto):
                breakreturn Hsignal

Also note to avoid Lists if possible. Use simple arrays like you would do in Matlab.

Timings

import time

#Use simple numpy arrays if possible, not lists
Ctrigger = np.arange(0, 3000000, 3)
length_t = Ctrigger.shape[0]

Bsignal = np.arange(0, 3000000, 10)
noise = 1e-05*np.random.rand(Bsignal.shape[0])
Csignal = np.sort(np.asarray(Bsignal) + noise)

offset = 3
gate = 1

start = time.time()
Hsignal=main(Ctrigger, Csignal, offset, gate)
print("Pure Python takes:" +str(time.time()-start))
#Pure Python takes:6.049151659011841#First call takes longer (compilation overhead)#The same may be the case in matlab
start = time.time()
Hsignal=main_nb(Ctrigger, Csignal, offset, gate)
print("First Numba run takes:" +str(time.time()-start))
#First Numba run takes:0.16272664070129395

start = time.time()
Hsignal=main_nb(Ctrigger, Csignal, offset, gate)
print("All further Numba calls run takes:" +str(time.time()-start))
#All further Numba calls run takes:0.006016731262207031

Hsignal = np.unique(Hsignal)

Solution 2:

What is probably slowing down your algorithm is the use of np.append in

Hsignal = np.append(Hsignal, Csignal[m])

You should use a list, not a NumPy array:

Ctrigger = [1, 10, 11, 20, 30, 40, 50, 60]
Csignal = [4, 11, 13, 17, 25, 34, 41, 42, 50, 57, 65]

offset = 2
gate = 2

Hsignal = []
marker = 0for j inrange(len(Ctrigger)):
    m = marker
    t_start = Ctrigger[j] + offset - gate/2
    t_stop = Ctrigger[j] + offset + gate/2while m < len(Csignal):   
        if Csignal[m] < t_start:
            marker = m
            m = m + 1elif Csignal[m] <= t_stop:
            Hsignal.append(Csignal[m])
            m = m + 1else:
            break

Hsignal = sorted(set(Hsignal))

Once the list has been built, you can transform it into an array:

Hsignal = np.array(Hsignal)