## Converting numpy arrays of arrays into one whole numpy array

2021-6-21 anglehua

I want to turn my array of array into just a single array. From something like :

``````array([ array([[0, 0, 0, ..., 1, 0, 0],
[0, 1, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 2, 0, 0],
...,
array([[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 8, 0, 2],
...,
[0, 0, 0, ..., 0, 0, 0],
[1, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 1, 0, 0]], dtype=uint8)], dtype=object)
``````

which has size (10,) to just the 3D numpy array which is of size (10,518, 32)

``````array([[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]]], dtype=uint8)
``````

I've tried converting everything into a list then do np.asarray and also tried defining everything as the same dtype=uint8 but I couldn't get it into the 3D form.

One way is to allocate the target array and copy the objects in as a loop.

``````import numpy as np

x = np.array([ np.array([[0, 0, 0, 1, 0, 0],
[0, 1, 0, 0, 0, 0],
[0, 0, 3, 7, 0, 0],
[0, 0, 0, 2, 0, 0]], dtype=np.uint8),
np.array([[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 4, 8, 0, 0],
[0, 0, 0, 8, 0, 2]], dtype=np.uint8),
np.array([[0, 0, 0, 0, 0, 0],
[1, 0, 0, 0, 0, 0],
[0, 0, 5, 9, 0, 0],
[0, 0, 0, 1, 0, 0]], dtype=np.uint8)], dtype=object)

print len(x)

print x[0].shape

y=np.zeros([len(x),x[0].shape[0],x[0].shape[1]],dtype=np.uint8)

print y.shape

for i in range(len(x)):
y[i,:,:] = x[i]

print y
``````

If I understand what you're asking this is the desired result:

``````3
(4L, 6L)
(3L, 4L, 6L)
[[[0 0 0 1 0 0]
[0 1 0 0 0 0]
[0 0 3 7 0 0]
[0 0 0 2 0 0]]

[[0 0 0 0 0 0]
[0 0 0 0 0 0]
[0 0 4 8 0 0]
[0 0 0 8 0 2]]

[[0 0 0 0 0 0]
[1 0 0 0 0 0]
[0 0 5 9 0 0]
[0 0 0 1 0 0]]]
``````

`np.concatenate` should do the trick:

Make an object array of arrays:

``````In [23]: arr=np.empty((4,),dtype=object)
In [24]: for i in range(4):arr[i]=np.ones((2,2),int)*i
In [25]: arr
Out[25]:
array([array([[0, 0],
[0, 0]]), array([[1, 1],
[1, 1]]),
array([[2, 2],
[2, 2]]), array([[3, 3],
[3, 3]])], dtype=object)

In [28]: np.concatenate(arr)
Out[28]:
array([[0, 0],
[0, 0],
[1, 1],
[1, 1],
[2, 2],
[2, 2],
[3, 3],
[3, 3]])
``````

Or with a reshape:

``````In [26]: np.concatenate(arr).reshape(4,2,2)
Out[26]:
array([[[0, 0],
[0, 0]],

[[1, 1],
[1, 1]],

[[2, 2],
[2, 2]],

[[3, 3],
[3, 3]]])
In [27]: _.shape
Out[27]: (4, 2, 2)
``````

`concatenate` effectively treats its input as a list of arrays. So it works regardless of whether this is an object array, a list, or 3d array.

This can't be done simply with a reshape. `arr` is an array of pointers - pointing to arrays located elsewhere in memory. To get a single 3d array, all of the pieces will have to be copied into one buffer. That's what concatenate does - it creates a large empty file, and copies each array, but it does it in compiled code.

`np.array` does not change it:

``````In [37]: np.array(arr).shape
Out[37]: (4,)
``````

but treating `arr` as a list of arrays does work (but is slower than the `concatenate` version - array analyses its inputs more).

``````In [38]: np.array([x for x in arr]).shape
Out[38]: (4, 2, 2)
``````