`import numpypy as numpy`

Alex Gaynor
RCOS – November 18, 2011

Building blocks

Source: Wikipedia

Arrays

```>>> a = numpy.array([1, 2, 3])
>>> a
array([1, 2, 3])
>>> a * 5
array([5, 10, 15])
```

Dtypes

```>>> numpy.dtype(float)
dtype('float64')
>>> numpy.dtype("i2")
dtype('int16')
>>> numpy.dtype("Q")
dtype('uint64')
```

Boxes

```>>> type(numpy.array([0])[0])
<type 'numpy.int64'>
>>> numpy.uint(260)
4
```

Ufuncs

```>>> numpy.sin(3)
0.14112000805986721
>>> numpy.sin([1, 2, 3])
array([ 0.84147098,  0.90929743,  0.14112001])
>>> numpy.sin(numpy.array([True, False, True]))
array([ 0.84130859,  0.        ,  0.84130859], dtype=float16)
```

At the start of this semester

• Mostly-working single-dimension arrays
• Many working ufuncs (didn't support all methods)
• Some built-in dtypes
• No boxes

It's all about the dtypes, stupid!

The other mess

```>>> POINT = numpy.dtype([("x", float), ("y", float)])
>>> POINT
dtype([('x', '<f8'), ('y', '<f8')])
>>> NUMBERED_POINT = numpy.dtype([("n", numpy.uint), ("p", POINT)])
>>> NUMBERED_POINT
dtype([('n', '<u8'), ('p', [('x', '<f8'), ('y', '<f8')])])
>>> numpy.dtype(complex)
dtype('complex128')
```

Horrific failures (in pursuit of complex values)

Arrays of structs

```ComplexDouble = lltype.Struct("Complex Double",
("real", lltype.Float),
("imag", lltype.Float),
)
W_Complex128Dtype = create_low_level_dtype(
num = 15, kind = COMPLEXLTR, name = "complex128",
aliases = [],
applevel_types = ["complex"],
T = ComplexDouble,
valtype = ComplexDouble,
expected_size = 16,
)
```

`0 < 1 < N < 2`

Seperation of dtype and itemtype

dtype itemtype
Exposed in Python Purely internal
Can malloc data Knows how many bytes in needs
Has a name Knows how to coerce and box values

How does this solve any problems?

```complex128 = CompositeItemType(
types = [Float64(), Float64()],
names = ["real", "imag"],
)
```

Composite item types

```def read(self, storage, width, i, offset):
boxes = []
for type in self.types:
boxes.append(type.read(storage, width, i, offset))
offset += type.get_element_size()
return CompositeBox(boxes)
```

Store a value

```def store(self, storage, width, i, offset, box):
boxes = box.subboxes
for type, box in zip(self.types, boxes):
type.store(storage, width, i, offset, box)
offset += types.get_element_size()
```

A three headed yak, to shave

Source: flickr
PyPy is a race between people shaving yaks and the universe producing more bearded yaks. So far, the universe is winning.
```@specialize.call_location()
@jit.oopspec("libffi_array_getitem(ffitype, width, addr, index, offset)")
def array_getitem(ffitype, width, addr, index, offset):
for TYPE, ffitype2 in clibffi.ffitype_map:
if ffitype is ffitype2:
assert False
```

Teach the JIT about compile-time offsets

```diff -r da2860819b0f -r 5545a3c93d51 pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py Sun Nov 13 15:23:01 2011 -0500
+++ b/pypy/jit/backend/x86/assembler.py Sun Nov 13 15:34:10 2011 -0500
@@ -1601,8 +1601,10 @@
assert isinstance(itemsize_loc, ImmedLoc)
if isinstance(index_loc, ImmedLoc):
temp_loc = imm(index_loc.value * itemsize_loc.value)
+            return AddressLoc(base_loc, index_loc, _get_scale(itemsize_loc.value), ofs_loc.value)
else:
-            # XXX should not use IMUL in most cases
+            # XXX should not use IMUL in more cases, it can use a clever LEA
```

Optimize the x86 backend so that it's as fast as the old way

```>>> numpy.int64.__bases__
(<type 'numpy.signedinteger'>, <type 'int'>)
>>> numpy.float64.__bases__
(<type 'numpy.floating'>, <type 'float'>)
```

I have exterminated the yaks. Pray they do not breed further.

Other happening things

• Multi-dimensional arrays
• f2pypy

PS: write tests

```def test_sum(self):
result = self.run("""
a = |30|
b = a + a
sum(b)
""")
assert result == 2 * sum(range(30))
self.check_loops({"getinteriorfield_raw": 2, "float_add": 2,
"int_lt": 1, "guard_true": 1, "jump": 1})
```

Status now

• n-dimensional arrays (branch)
• Most ufuncs, with advanced features like `reduce()`
• Almost all builtin dtypes, plus custom and complex ones (branch)
• Full boxes (branch)

Questions?

Thanks RCOS
http://pypy.org/
http://alexgaynor.net/
@alex_gaynor