Advertisement
Advertisement


What's the idiomatic syntax for prepending to a short python list?


Question

list.append() is the obvious choice for adding to the end of a list. Here's a reasonable explanation for the missing list.prepend(). Assuming my list is short and performance concerns are negligible, is

list.insert(0, x)

or

list[0:0] = [x]

idiomatic?

2011/12/16
1
567
12/16/2011 5:30:01 PM

Accepted Answer

The s.insert(0, x) form is the most common.

Whenever you see it though, it may be time to consider using a collections.deque instead of a list.

2011/12/16
817
12/16/2011 6:08:26 PM


What's the idiomatic syntax for prepending to a short python list?

You don't usually want to repetitively prepend to a list in Python.

If it's short, and you're not doing it a lot... then ok.

list.insert

The list.insert can be used this way.

list.insert(0, x)

But this is inefficient, because in Python, a list is an array of pointers, and Python must now take every pointer in the list and move it down by one to insert the pointer to your object in the first slot, so this is really only efficient for rather short lists, as you ask.

Here's a snippet from the CPython source where this is implemented - and as you can see, we start at the end of the array and move everything down by one for every insertion:

for (i = n; --i >= where; )
    items[i+1] = items[i];

If you want a container/list that's efficient at prepending elements, you want a linked list. Python has a doubly linked list, which can insert at the beginning and end quickly - it's called a deque.

deque.appendleft

A collections.deque has many of the methods of a list. list.sort is an exception, making deque definitively not entirely Liskov substitutable for list.

>>> set(dir(list)) - set(dir(deque))
{'sort'}

The deque also has an appendleft method (as well as popleft). The deque is a double-ended queue and a doubly-linked list - no matter the length, it always takes the same amount of time to preprend something. In big O notation, O(1) versus the O(n) time for lists. Here's the usage:

>>> import collections
>>> d = collections.deque('1234')
>>> d
deque(['1', '2', '3', '4'])
>>> d.appendleft('0')
>>> d
deque(['0', '1', '2', '3', '4'])

deque.extendleft

Also relevant is the deque's extendleft method, which iteratively prepends:

>>> from collections import deque
>>> d2 = deque('def')
>>> d2.extendleft('cba')
>>> d2
deque(['a', 'b', 'c', 'd', 'e', 'f'])

Note that each element will be prepended one at a time, thus effectively reversing their order.

Performance of list versus deque

First we setup with some iterative prepending:

import timeit
from collections import deque

def list_insert_0():
    l = []
    for i in range(20):
        l.insert(0, i)

def list_slice_insert():
    l = []
    for i in range(20):
        l[:0] = [i]      # semantically same as list.insert(0, i)

def list_add():
    l = []
    for i in range(20):
        l = [i] + l      # caveat: new list each time

def deque_appendleft():
    d = deque()
    for i in range(20):
        d.appendleft(i)  # semantically same as list.insert(0, i)

def deque_extendleft():
    d = deque()
    d.extendleft(range(20)) # semantically same as deque_appendleft above

and performance:

>>> min(timeit.repeat(list_insert_0))
2.8267281929729506
>>> min(timeit.repeat(list_slice_insert))
2.5210217320127413
>>> min(timeit.repeat(list_add))
2.0641671380144544
>>> min(timeit.repeat(deque_appendleft))
1.5863927800091915
>>> min(timeit.repeat(deque_extendleft))
0.5352169770048931

The deque is much faster. As the lists get longer, I would expect a deque to perform even better. If you can use deque's extendleft you'll probably get the best performance that way.

2019/07/23

If someone finds this question like me, here are my performance tests of proposed methods:

Python 2.7.8

In [1]: %timeit ([1]*1000000).insert(0, 0)
100 loops, best of 3: 4.62 ms per loop

In [2]: %timeit ([1]*1000000)[0:0] = [0]
100 loops, best of 3: 4.55 ms per loop

In [3]: %timeit [0] + [1]*1000000
100 loops, best of 3: 8.04 ms per loop

As you can see, insert and slice assignment are as almost twice as fast than explicit adding and are very close in results. As Raymond Hettinger noted insert is more common option and I, personally prefer this way to prepend to list.

2017/05/23

Source: https://stackoverflow.com/questions/8537916
Licensed under: CC-BY-SA with attribution
Not affiliated with: Stack Overflow
Email: [email protected]