1.3.3. More elaborate arrays

1.3.3.1. More data types

Casting

“Bigger” type wins in mixed-type operations:

>>> np.array([1, 2, 3]) + 1.5
array([2.5,  3.5,  4.5])

Assignment never changes the type!

>>> a = np.array([1, 2, 3])
>>> a.dtype
dtype('int64')
>>> a[0] = 1.9     # <-- float is truncated to integer
>>> a
array([1, 2, 3])

Forced casts:

>>> a = np.array([1.7, 1.2, 1.6])
>>> b = a.astype(int)  # <-- truncates to integer
>>> b
array([1, 1, 1])

Rounding:

>>> a = np.array([1.2, 1.5, 1.6, 2.5, 3.5, 4.5])
>>> b = np.around(a)
>>> b                    # still floating-point
array([1.,  2.,  2.,  2.,  4.,  4.])
>>> c = np.around(a).astype(int)
>>> c
array([1, 2, 2, 2, 4, 4])

Different data type sizes

Integers (signed):

int8	8 bits
int16	16 bits
int32	32 bits (same as int on 32-bit platform)
int64	64 bits (same as int on 64-bit platform)

>>> np.array([1], dtype=int).dtype
dtype('int64')
>>> np.iinfo(np.int32).max, 2**31 - 1
(2147483647, 2147483647)

Unsigned integers:

uint8	8 bits
uint16	16 bits
uint32	32 bits
uint64	64 bits

>>> np.iinfo(np.uint32).max, 2**32 - 1
(4294967295, 4294967295)

Floating-point numbers:

float16	16 bits
float32	32 bits
float64	64 bits (same as float)
float96	96 bits, platform-dependent (same as np.longdouble)
float128	128 bits, platform-dependent (same as np.longdouble)

>>> np.finfo(np.float32).eps
np.float32(1.1920929e-07)
>>> np.finfo(np.float64).eps
np.float64(2.220446049250313e-16)

>>> np.float32(1e-8) + np.float32(1) == 1
np.True_
>>> np.float64(1e-8) + np.float64(1) == 1
np.False_

Complex floating-point numbers:

complex64	two 32-bit floats
complex128	two 64-bit floats
complex192	two 96-bit floats, platform-dependent
complex256	two 128-bit floats, platform-dependent

Smaller data types

If you don’t know you need special data types, then you probably don’t.

Comparison on using float32 instead of float64:

• Half the size in memory and on disk

• Half the memory bandwidth required (may be a bit faster in some operations)

  In [1]: a = np.zeros((int(1e6),), dtype=np.float64)
  
  In [2]: b = np.zeros((int(1e6),), dtype=np.float32)
  
  In [3]: %timeit a*a
  268 us +- 1.31 us per loop (mean +- std. dev. of 7 runs, 1,000 loops each)
  
  In [4]: %timeit b*b
  134 us +- 581 ns per loop (mean +- std. dev. of 7 runs, 10,000 loops each)
  

• But: bigger rounding errors — sometimes in surprising places (i.e., don’t use them unless you really need them)

1.3.3.2. Structured data types

sensor_code	(4-character string)
position	(float)
value	(float)

>>> samples = np.zeros((6,), dtype=[('sensor_code', 'S4'),
...                                 ('position', float), ('value', float)])
>>> samples.ndim
1
>>> samples.shape
(6,)
>>> samples.dtype.names
('sensor_code', 'position', 'value')
>>> samples[:] = [('ALFA',   1, 0.37), ('BETA', 1, 0.11), ('TAU', 1,   0.13),
...               ('ALFA', 1.5, 0.37), ('ALFA', 3, 0.11), ('TAU', 1.2, 0.13)]
>>> samples
array([(b'ALFA', 1. , 0.37), (b'BETA', 1. , 0.11), (b'TAU', 1. , 0.13),
       (b'ALFA', 1.5, 0.37), (b'ALFA', 3. , 0.11), (b'TAU', 1.2, 0.13)],
      dtype=[('sensor_code', 'S4'), ('position', '<f8'), ('value', '<f8')])

Field access works by indexing with field names:

>>> samples['sensor_code']
array([b'ALFA', b'BETA', b'TAU', b'ALFA', b'ALFA', b'TAU'], dtype='|S4')
>>> samples['value']
array([0.37,  0.11,  0.13,  0.37,  0.11,  0.13])
>>> samples[0]
np.void((b'ALFA', 1.0, 0.37), dtype=[('sensor_code', 'S4'), ('position', '<f8'), ('value', '<f8')])

>>> samples[0]['sensor_code'] = 'TAU'
>>> samples[0]
np.void((b'TAU', 1.0, 0.37), dtype=[('sensor_code', 'S4'), ('position', '<f8'), ('value', '<f8')])

Multiple fields at once:

>>> samples[['position', 'value']]
array([(1. ,  0.37), (1. ,  0.11), (1. ,  0.13), (1.5,  0.37),
       (3. ,  0.11), (1.2,  0.13)],
      dtype={'names': ['position', 'value'], 'formats': ['<f8', '<f8'], 'offsets': [4, 12], 'itemsize': 20})

Fancy indexing works, as usual:

>>> samples[samples['sensor_code'] == b'ALFA']
array([(b'ALFA', 1.5, 0.37), (b'ALFA', 3. , 0.11)],
      dtype=[('sensor_code', 'S4'), ('position', '<f8'), ('value', '<f8')])

Note

There are a bunch of other syntaxes for constructing structured arrays, see here and here.

1.3.3.3. maskedarray: dealing with (propagation of) missing data

• For floats one could use NaN’s, but masks work for all types:

  >>> x = np.ma.array([1, 2, 3, 4], mask=[0, 1, 0, 1])
  >>> x
  masked_array(data=[1, --, 3, --],
               mask=[False,  True, False,  True],
         fill_value=999999)
  
  
  >>> y = np.ma.array([1, 2, 3, 4], mask=[0, 1, 1, 1])
  >>> x + y
  masked_array(data=[2, --, --, --],
               mask=[False,  True,  True,  True],
         fill_value=999999)
  

• Masking versions of common functions:

  >>> np.ma.sqrt([1, -1, 2, -2]) 
  masked_array(data=[1.0, --, 1.41421356237... --],
               mask=[False,  True, False,  True],
         fill_value=1e+20)
  

Note

There are other useful array siblings

---

While it is off topic in a chapter on NumPy, let’s take a moment to recall good coding practice, which really do pay off in the long run:

Good practices

• Explicit variable names (no need of a comment to explain what is in the variable)

• Style: spaces after commas, around =, etc.

  A certain number of rules for writing “beautiful” code (and, more importantly, using the same conventions as everybody else!) are given in the Style Guide for Python Code and the Docstring Conventions page (to manage help strings).

• Except some rare cases, variable names and comments in English.