import numpypy as numpy

Alex Gaynor
RCOS – November 18, 2011

Building blocks

Source: Wikipedia

Arrays

>>> a = numpy.array([1, 2, 3])
>>> a
array([1, 2, 3])
>>> a * 5
array([5, 10, 15])
                    

Dtypes

>>> numpy.dtype(float)
dtype('float64')
>>> numpy.dtype("i2")
dtype('int16')
>>> numpy.dtype("Q")
dtype('uint64')
                    

Boxes

>>> type(numpy.array([0])[0])
<type 'numpy.int64'>
>>> numpy.uint(260)
4
                    

Ufuncs

>>> numpy.sin(3)
0.14112000805986721
>>> numpy.sin([1, 2, 3])
array([ 0.84147098,  0.90929743,  0.14112001])
>>> numpy.sin(numpy.array([True, False, True]))
array([ 0.84130859,  0.        ,  0.84130859], dtype=float16)
                    

At the start of this semester

It's all about the dtypes, stupid!

The other mess

>>> POINT = numpy.dtype([("x", float), ("y", float)])
>>> POINT
dtype([('x', '<f8'), ('y', '<f8')])
>>> NUMBERED_POINT = numpy.dtype([("n", numpy.uint), ("p", POINT)])
>>> NUMBERED_POINT
dtype([('n', '<u8'), ('p', [('x', '<f8'), ('y', '<f8')])])
>>> numpy.dtype(complex)
dtype('complex128')
                    

Horrific failures (in pursuit of complex values)

Arrays of structs

ComplexDouble = lltype.Struct("Complex Double",
    ("real", lltype.Float),
    ("imag", lltype.Float),
)
W_Complex128Dtype = create_low_level_dtype(
    num = 15, kind = COMPLEXLTR, name = "complex128",
    aliases = [],
    applevel_types = ["complex"],
    T = ComplexDouble,
    valtype = ComplexDouble,
    expected_size = 16,
)
                    

Just make them special
(Mess too gruesome to be shown)

Software engineering nirvana

0 < 1 < N < 2

Seperation of dtype and itemtype

dtype itemtype
Exposed in Python Purely internal
Can malloc data Knows how many bytes in needs
Has a name Knows how to coerce and box values

How does this solve any problems?

complex128 = CompositeItemType(
    types = [Float64(), Float64()],
    names = ["real", "imag"],
)
                    

Composite item types

Read a value

def read(self, storage, width, i, offset):
    boxes = []
    for type in self.types:
        boxes.append(type.read(storage, width, i, offset))
        offset += type.get_element_size()
    return CompositeBox(boxes)
                    

Store a value

def store(self, storage, width, i, offset, box):
    boxes = box.subboxes
    for type, box in zip(self.types, boxes):
        type.store(storage, width, i, offset, box)
        offset += types.get_element_size()
                    

A three headed yak, to shave

Source: flickr
PyPy is a race between people shaving yaks and the universe producing more bearded yaks. So far, the universe is winning.
Dr. Antonio Cuni
@specialize.call_location()
@jit.oopspec("libffi_array_getitem(ffitype, width, addr, index, offset)")
def array_getitem(ffitype, width, addr, index, offset):
    for TYPE, ffitype2 in clibffi.ffitype_map:
        if ffitype is ffitype2:
            addr = rffi.ptradd(addr, index * width)
            addr = rffi.ptradd(addr, offset)
            return rffi.cast(rffi.CArrayPtr(TYPE), addr)[0]
    assert False
                    

Teach the JIT about compile-time offsets

diff -r da2860819b0f -r 5545a3c93d51 pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py Sun Nov 13 15:23:01 2011 -0500
+++ b/pypy/jit/backend/x86/assembler.py Sun Nov 13 15:34:10 2011 -0500
@@ -1601,8 +1601,10 @@
         assert isinstance(itemsize_loc, ImmedLoc)
         if isinstance(index_loc, ImmedLoc):
             temp_loc = imm(index_loc.value * itemsize_loc.value)
+        elif _valid_addressing_size(itemsize_loc.value):
+            return AddressLoc(base_loc, index_loc, _get_scale(itemsize_loc.value), ofs_loc.value)
         else:
-            # XXX should not use IMUL in most cases
+            # XXX should not use IMUL in more cases, it can use a clever LEA
                    

Optimize the x86 backend so that it's as fast as the old way

>>> numpy.int64.__bases__
(<type 'numpy.signedinteger'>, <type 'int'>)
>>> numpy.float64.__bases__
(<type 'numpy.floating'>, <type 'float'>)
                    

RPython multiple inheritance

I have exterminated the yaks.
Pray they do not breed further.

Other happening things

PS: write tests

def test_sum(self):
    result = self.run("""
    a = |30|
    b = a + a
    sum(b)
    """)
    assert result == 2 * sum(range(30))
    self.check_loops({"getinteriorfield_raw": 2, "float_add": 2,
                      "int_add": 1,
                      "int_lt": 1, "guard_true": 1, "jump": 1})
                    

Status now

Questions?

Thanks RCOS
http://pypy.org/
http://alexgaynor.net/
@alex_gaynor