I am facing some issues concerning memory error due to a huge array that is used in my code. Recently I found Dask Array as a possible solution to parallelize the tasks and be able to process the data. But, sincerely, I don't have a clue on how to implement Dask Array in my code, even if I should change it in order to be able to parallelize it. Here is part of it:
def i_size(x,y,t):
r = len(Y)
c = len(x)
M = np.zeros((r,c), dtype = np.uint8)
for i in range(c)
M = np.sum(np.logical_and(x[i]How , y, axis=-1))
M = np.where(M>=t, True, False)
return M.astype(np.uint8)
#T_x and T_y are 2-d array they can reach 3.000.000 rows and 20 columns each
#and t_e is an uint8
M_i = i_size(T_x, T_y, t_e)
V_D = np.zeros(len(T_x), dtype=np.uint8)
TZ = 5
while TZ != 0
V_S = np.sum((np.where(V_D ==0, True, False))*M_i, axis=-1)
V_D += M_i
V_max = np.max(V_S)
TZ = TZ-1
print(f"V max = {V_max}")
How should I start? Honestly I don't have a clue. Can someone help me to start my journey on the Dask Array on my code.
All the best