Series: Stacks · Queues and deques · Linked lists
COL 1101 Programming with Data Structures — Lectures 14 and 15, rewritten as one guide
Linked lists
An array keeps its elements side by side, which makes indexing instant and inserting in the middle slow. A linked list makes the opposite trade: each element lives in its own node that points to the next, so you can splice anywhere you are standing in constant time, but you can only get there by walking. This page builds singly, circularly and doubly linked lists, then the positional list that lets users hold their place, and two algorithms that run on it.
Four nodes. Each holds an element and a link to the next node; the last links to nothing. The list itself only remembers where the chain starts and ends.
How to use this page
Parts 1–3 are Lecture 14; Parts 4–7 are Lecture 15 (which begins with a recap of the end of Lecture 14). Try every Check yourself before opening the answers. Dashed boxes marked Beyond the lecture are optional. Part 8 is the topic on one page. Stacks, queues and amortized cost are explained on their own pages; this one uses them freely.
Reference (pointer): a value that says where an object is in memory. A Python variable holding an object holds a reference to it. None is the reference to nothing.
Node: a small object holding one element (really, a reference to it) plus one or more references to other nodes.
Traverse (walk): follow next references from node to node. Reaching the k-th node takes k steps; there is no shortcut.
Part 1
Why not just use an array?
1.1What arrays are bad at
A Python list is a dynamic array: references stored in one contiguous block, with O(1) access to any index and O(1) amortized append. The lecture lists three drawbacks:
- Wasted space. The block's capacity is usually larger than the number of elements, by up to a constant factor, so that appends stay cheap.
- Insertions and deletions in the interior cost O(n). An array has no gaps, so inserting at index i shifts every element from i onward one place right; deleting shifts them left.
- Amortized bounds can be unacceptable in real-time systems. An append is usually O(1) but occasionally O(n) when the array resizes. A system that must respond within a deadline every time cannot accept the occasional long pause.
Part 2
Singly linked lists
2.1Structure
A singly linked list is a sequence of nodes. Each node stores a reference to its element and a reference next to the following node; the last node's next is None. The list object stores a reference to the first node, the head, and optionally a reference to the last node, the tail, and the number of nodes, size.
The lecture calls this a “collaboration of many objects”: the element objects (which the list does not own and never copies), the nodes (one per element), and the list object itself (head, tail, size).
Two consequences follow immediately:
- There is no spare capacity: a node is created for each element and discarded when it leaves, so memory is proportional to the current size at all times.
- There is no index arithmetic: to reach the 5th element you start at the head and follow
nextfour times. Access by index is O(n).
2.2Inserting at the head: O(1)
- Allocate a new node.
- Store the new element in it.
- Make the new node's
nextpoint to the old head. - Update
headto point to the new node.
head moved first, nothing would point to MSP any more and the rest of the list would be lost.In Python, steps 1–4 collapse into one line, self.head = Node(e, self.head), because the right-hand side is evaluated (using the old head) before the assignment. If the list was empty, the new node is also the tail.
2.3Inserting at the tail: O(1), if you keep a tail reference
- Allocate a new node and store the element.
- Make the new node's
nextpoint toNone(it will be last). - Make the old last node's
nextpoint to the new node. - Update
tailto point to the new node.
next before moving tail. Once tail points to MIA you have no reference to BOS left (except by walking from the head).Without a stored tail, you would have to walk from the head to find the last node: O(n). Keeping one extra reference makes it O(1). If the list was empty, the new node is also the head.
2.4Removing from the head: O(1)
- Update
headto point to the next node in the list. - Let Python's garbage collector reclaim the former first node, which nothing references any more.
If that empties the list, tail must be set to None as well; otherwise it still points to the removed node.
2.5Removing from the tail: O(n), and why
After removing the last node, tail must point to the second-to-last node, and that node's next must become None. But links only point forward: the last node does not know who points to it. The only way to find the node before the tail is to start at the head and walk until walk.next is tail. That walk is n − 2 steps.
There is no constant-time way to remove the tail of a singly linked list, even with a tail reference. A tail reference tells you where the last node is, not where the one before it is. Part 4 fixes this with backward links.
2.6Searching and deleting by value: O(n)
Search for e: walk from the head comparing each element with e; stop at the first match or at None. Worst case n comparisons.
Delete the first node containing e: search for it, but also remember the node before it (call it prev), because deleting means making prev.next skip over the found node. Three cases: the match is the head (move head instead, there is no prev); the match is the tail (also move tail to prev); otherwise just link around it.
class Empty(Exception):
pass
class SinglyLinkedList:
class _Node:
__slots__ = "element", "next" # two fields, no per-object dict
def __init__(self, element, nxt):
self.element = element
self.next = nxt
def __init__(self):
self.head = None
self.tail = None
self.size = 0
def add_head(self, e): # O(1)
self.head = self._Node(e, self.head) # new node points to old head
if self.size == 0:
self.tail = self.head # first node is also the last
self.size += 1
def add_tail(self, e): # O(1), thanks to the tail reference
newest = self._Node(e, None)
if self.size == 0:
self.head = newest
else:
self.tail.next = newest # link the old tail FIRST...
self.tail = newest # ...then move tail
self.size += 1
def remove_head(self): # O(1)
if self.size == 0:
raise Empty("list is empty")
e = self.head.element
self.head = self.head.next # old first node is now unreachable
self.size -= 1
if self.size == 0:
self.tail = None
return e
def remove_tail(self): # O(n): must find the node before tail
if self.size == 0:
raise Empty("list is empty")
if self.size == 1:
return self.remove_head()
walk = self.head
while walk.next is not self.tail:
walk = walk.next
e = self.tail.element
walk.next = None
self.tail = walk
self.size -= 1
return e
def find(self, e): # O(n)
walk = self.head
while walk is not None and walk.element != e:
walk = walk.next
return walk # the node, or None
def delete(self, e): # O(n): delete first node holding e
prev, walk = None, self.head
while walk is not None and walk.element != e:
prev, walk = walk, walk.next # remember the node BEFORE walk
if walk is None:
return False
if prev is None:
self.head = walk.next # deleting the head
else:
prev.next = walk.next # link around walk
if walk is self.tail:
self.tail = prev
self.size -= 1
return True
def __iter__(self):
walk = self.head
while walk is not None:
yield walk.element
walk = walk.next
L = SinglyLinkedList()
for code in ["MSP", "ATL", "BOS"]:
L.add_tail(code)
L.add_head("LAX"); print(list(L))
L.add_tail("MIA"); print(list(L))
print(L.remove_head(), L.remove_tail(), list(L), "tail =", L.tail.element)
print(L.find("ATL").element, L.find("SFO"), L.delete("ATL"), list(L))['LAX', 'MSP', 'ATL', 'BOS'] ['LAX', 'MSP', 'ATL', 'BOS', 'MIA'] LAX MIA ['MSP', 'ATL', 'BOS'] tail = BOS ATL None True ['MSP', 'BOS']
Singly linked list playground
Every operation reports how many nodes it had to walk past and which links it changed. Try remove_tail on a long list, then remove_head, and compare.
| Singly linked list operation | Cost | Why |
|---|---|---|
| add_head, remove_head | O(1) | head is stored |
| add_tail | O(1) | tail is stored (O(n) without it) |
| remove_tail | O(n) | must walk to the node before the tail |
| access by index, search, delete by value | O(n) | must walk from the head |
| len | O(1) | size is stored |
Check yourself: singly linked lists
- Write the two lines of
add_tailthat change links, in the right order. What goes wrong if you swap them? - A list holds 1,000 nodes. How many
nextreferences must be followed to remove the tail? - In
delete(e), why do we needprev? Why not just make the found node “disappear”? - After
remove_headempties the list, what must happen totail?
Show answers
self.tail.next = newest, thenself.tail = newest. Swapped, the first line makestailthe new node, and the second sets the new node'snextto itself: the old last node never links to it, so the new node is unreachable from the head, and walking off the end loops forever.- 998: from the head (node 1) to node 999, the one before the tail.
- A node cannot remove itself: something points to it, and that pointer must be changed to skip it. In a singly linked list the only thing pointing to a node is the previous node's
next, so we must have the previous node in hand. - It must become
None; otherwise it refers to a node that is no longer in the list, and the next add_tail would link onto that dead node.
Part 3
Stacks, queues and circular lists
3.1Linked stack and linked queue
Choose the ends so that every operation avoids remove_tail:
| ADT | Placement | Operations used | Cost |
|---|---|---|---|
| Stack | top = head | push = add_head; pop = remove_head; top = head.element | all O(1) |
| Queue | front = head, rear = tail | enqueue = add_tail; dequeue = remove_head; first = head.element | all O(1) |
Unlike the array versions, these are O(1) in the worst case, not amortized: nothing ever resizes. Space is O(n). Full code for both is on the Stacks and Queues pages.
3.2Circularly linked lists
Make the tail's next point back to the head instead of None. There is no longer a natural start or end, so instead of head and tail the list keeps one reference, usually called current. If current is treated as the “last” node, then current.next is the “first”, so both ends are reachable from one reference.
next from any node eventually returns to it. rotate() moves current one step, which makes the old first node the new last one.Circular lists fit data that is naturally cyclic: a circular bus or train route, a playlist on repeat-all, the turn order in a multiplayer game.
3.3Example: round-robin scheduling
A round-robin scheduler iterates through a collection in a circle and “services” each element in turn, for example giving each running application a slice of CPU time.
- With a queue: dequeue the next element, service it, enqueue it again. Every turn removes a node and creates a new one.
- With a circular list: service
front(), thenrotate(). One reference moves; nothing is removed or created.
class CircularList:
"""Circularly linked list, kept by one reference: current (the 'last' node)."""
class _Node:
__slots__ = "element", "next"
def __init__(self, element, nxt):
self.element, self.next = element, nxt
def __init__(self):
self.current = None
self.size = 0
def add(self, e): # insert after current, become current
newest = self._Node(e, None)
if self.current is None:
newest.next = newest # a one-node circle points at itself
else:
newest.next = self.current.next
self.current.next = newest
self.current = newest
self.size += 1
def front(self):
return self.current.next.element # the node after 'last' is the 'first'
def rotate(self): # O(1): one reference moves
self.current = self.current.next
C = CircularList()
for app in ["editor", "browser", "music"]:
C.add(app)
schedule = []
for turn in range(7):
schedule.append(C.front()) # give this app its time slice
C.rotate() # move on: no removal, no insertion
print(schedule)['editor', 'browser', 'music', 'editor', 'browser', 'music', 'editor']
Check yourself: circular lists
- A circular list holds A → B → C → (back to A) with
currentat C. What doesfront()return? Afterrotate()? - How do you know when a traversal of a circular list has visited every node once?
Show answers
front()= C.next = A. After rotate, current = A, so front() = B.- There is no
Noneto stop at. Either countsizesteps, or remember the starting node and stop when you return to it.
Part 4
Doubly linked lists
4.1Links in both directions
The singly linked list could not remove its tail quickly because a node does not know its predecessor. So give it one. In a doubly linked list every node stores three references: its element, next and prev. Now from any node you can step either way in O(1).
4.2Sentinels: the header and trailer
The list also keeps two extra nodes that hold no element: a header before the first real node and a trailer after the last. They are called sentinels (or guards), are created with the list, and are never removed. An empty list is just header ⇄ trailer.
next links; lower arrows are prev links. The header and trailer are permanent and store no element.Why spend two nodes on nothing? Because every real node is now guaranteed to have a real node on both sides. Inserting at the front is just “insert between header and its successor”; inserting into an empty list is “insert between header and trailer”. There is no special case for an empty list, the first node or the last node, and the same two short procedures below handle every insertion and every deletion. Fewer special cases means fewer bugs.
4.3Insertion: four links
To insert a new node q holding X between node p and its successor s (= p.next):
s = p.next
q = new node(X)
q.prev = p # new node's own two links
q.next = s
s.prev = q # then the neighbours' links
p.next = qOrder matters in one way only: read p.next (into s) before overwriting it. After that, the four assignments can happen in any order.
4.4Deletion: two links
To remove node p, make its neighbours point at each other:
p.prev.next = p.next
p.next.prev = p.prev
# nothing in the list refers to p any more; return p.elementBecause of the sentinels, p.prev and p.next always exist, even when p is the only real node.
Doubly linked list, one link at a time
Pick an operation and step through it. The link being set on each step is drawn heavier; dashed links are about to be replaced.
4.5Costs, and the deque
| Doubly linked list operation | Cost |
|---|---|
| insert or delete at either end | O(1) |
| insert or delete next to a node you already have a reference to | O(1) |
| find a node by index or by value | O(n) |
| space | O(n) |
That makes it the natural home for a deque: add_first and delete_first work just after the header, add_last and delete_last just before the trailer, all in O(1) worst case. The array-based deque achieves O(1) only amortized, because of occasional resizing.
class _DoublyLinkedBase:
"""Doubly linked list with header and trailer sentinels."""
class _Node:
__slots__ = "element", "prev", "next"
def __init__(self, element, prev, nxt):
self.element, self.prev, self.next = element, prev, nxt
def __init__(self):
self._header = self._Node(None, None, None) # sentinels hold no element
self._trailer = self._Node(None, None, None)
self._header.next = self._trailer
self._trailer.prev = self._header
self._size = 0
def __len__(self):
return self._size
def is_empty(self):
return self._size == 0
def _insert_between(self, e, predecessor, successor): # O(1)
newest = self._Node(e, predecessor, successor) # new node's two links
predecessor.next = newest # neighbours' two links
successor.prev = newest
self._size += 1
return newest
def _delete_node(self, node): # O(1)
predecessor, successor = node.prev, node.next
predecessor.next = successor # link around node
successor.prev = predecessor
self._size -= 1
element = node.element
node.prev = node.next = node.element = None # mark node as dead
return element
class LinkedDeque(_DoublyLinkedBase):
def first(self):
if self.is_empty(): raise IndexError("deque is empty")
return self._header.next.element
def last(self):
if self.is_empty(): raise IndexError("deque is empty")
return self._trailer.prev.element
def add_first(self, e):
self._insert_between(e, self._header, self._header.next)
def add_last(self, e):
self._insert_between(e, self._trailer.prev, self._trailer)
def delete_first(self):
if self.is_empty(): raise IndexError("deque is empty")
return self._delete_node(self._header.next)
def delete_last(self):
if self.is_empty(): raise IndexError("deque is empty")
return self._delete_node(self._trailer.prev)
D = LinkedDeque()
D.add_last(5); D.add_first(3); D.add_first(7)
print(D.first(), D.last(), len(D))
print(D.delete_last(), D.delete_first(), D.first(), len(D))7 5 3 5 7 3 1
_delete_node sets the dead node's fields to None. That helps the garbage collector, and Part 5 uses it to detect positions that have been deleted.
Check yourself: doubly linked lists
- How many link assignments does inserting into an empty doubly linked list take? Which nodes are involved?
- In the insertion code, what goes wrong if you write
p.next = qas the very first line? - Why is
delete_lastO(1) here but O(n) in a singly linked list?
Show answers
- Four, exactly as always: q.prev = header, q.next = trailer, trailer.prev = q, header.next = q. That is the point of sentinels.
- You lose the successor:
q.next = p.nextwould then makeqpoint to itself, and the old successor s is no longer reachable from p. Saves = p.nextfirst. - The last node is
trailer.prevand its predecessor istrailer.prev.prev: both one step away. A singly linked list has no prev links, so it must walk from the head.
Part 5
The positional list
5.1The problem: insert and leave anywhere
The clinic's waiting room, modified once more: a person may be inserted at any position, and anyone may leave from any position. A doubly linked list can do both in O(1), but only if the user can say where. How?
- By index? Indexes shift every time someone ahead of you arrives or leaves, so an index recorded earlier may now name a different person. And finding index i in a linked list is an O(n) walk anyway.
- By handing out nodes? Fast, but dangerous: see 5.3.
The answer is a third option, a position: a marker that stays attached to one element. Think of the cursor in a text editor. It sits between two particular characters, and when you type elsewhere it does not jump; it stays with “its” characters.
5.2The Positional List ADT
A position is a simple object supporting exactly one method:
p.element(): return the element stored at position p.
A position is unaffected by changes elsewhere in the list. The only way it becomes invalid is an explicit command to delete it.
Accessors of a positional list L:
| Method | Returns |
|---|---|
L.first(), L.last() | position of the first / last element, or None if L is empty |
L.before(p), L.after(p) | position just before / after p, or None if p is first / last |
L.is_empty(), len(L) | as usual |
iter(L) | a forward iterator over the elements |
Updates:
| Method | Effect | Returns |
|---|---|---|
L.add_first(e), L.add_last(e) | insert at the front / back | new position |
L.add_before(p, e), L.add_after(p, e) | insert just before / after p | new position |
L.replace(p, e) | put e at p | the old element |
L.delete(p) | remove the element at p; p becomes invalid | the removed element |
Slide correction. The slide says replace(p, e) returns “the position of the new element”. It returns the element formerly at p, as the slide's own example table shows (L.replace(p, 7) returns 8).
The lecture's example. Position objects are named p, q, r, s; list contents show which position holds each value.
| Operation | Returns | L |
|---|---|---|
| L.add_last(8) | p | 8p |
| L.first() | p | 8p |
| L.add_after(p, 5) | q | 8p, 5q |
| L.before(q) | p | 8p, 5q |
| L.add_before(q, 3) | r | 8p, 3r, 5q |
| r.element() | 3 | 8p, 3r, 5q |
| L.after(p) | r | 8p, 3r, 5q |
| L.before(p) | None | 8p, 3r, 5q |
| L.add_first(9) | s | 9s, 8p, 3r, 5q |
| L.delete(L.last()) | 5 | 9s, 8p, 3r |
| L.replace(p, 7) | 8 | 9s, 7p, 3r |
Notice that after add_first(9), p still refers to 8 even though 8 is now second. That is what “unaffected by changes elsewhere” means.
5.3Why not hand out node references?
- Simplicity for users. They should not have to manipulate prev/next links or know that sentinels exist.
- Robustness. A user holding a raw node could change its links and corrupt the whole list.
- Safety checks. A position records which list it belongs to, so the list can reject a position from a different list, or one that has been deleted.
- Freedom to redesign. Because users only see positions, the implementation could later switch to an array-based sequence without changing any user code.
5.4Implementation
A Position wraps a node and remembers its container. Every public method validates the position it is given (correct type, belongs to this list, not deleted) and converts it to a node; every method that returns a location converts a node back into a position, returning None instead of ever exposing a sentinel. It builds on _DoublyLinkedBase from Part 4.
class PositionalList(_DoublyLinkedBase):
class Position:
"""A marker for one element's location. Users never see nodes."""
__slots__ = "_container", "_node"
def __init__(self, container, node):
self._container, self._node = container, node
def element(self):
return self._node.element
def __eq__(self, other): # same node = same position
return type(other) is type(self) and other._node is self._node
def __ne__(self, other):
return not (self == other)
def _validate(self, p): # position -> node, with checks
if not isinstance(p, self.Position):
raise TypeError("p must be a proper Position type")
if p._container is not self:
raise ValueError("p does not belong to this container")
if p._node.next is None: # _delete_node cleared it
raise ValueError("p is no longer valid")
return p._node
def _make_position(self, node): # node -> position (never a sentinel)
if node is self._header or node is self._trailer:
return None
return self.Position(self, node)
# accessors
def first(self): return self._make_position(self._header.next)
def last(self): return self._make_position(self._trailer.prev)
def before(self, p): return self._make_position(self._validate(p).prev)
def after(self, p): return self._make_position(self._validate(p).next)
def __iter__(self):
cursor = self.first()
while cursor is not None:
yield cursor.element()
cursor = self.after(cursor)
# updates (all O(1))
def _insert_between(self, e, predecessor, successor):
return self._make_position(super()._insert_between(e, predecessor, successor))
def add_first(self, e): return self._insert_between(e, self._header, self._header.next)
def add_last(self, e): return self._insert_between(e, self._trailer.prev, self._trailer)
def add_before(self, p, e):
node = self._validate(p)
return self._insert_between(e, node.prev, node)
def add_after(self, p, e):
node = self._validate(p)
return self._insert_between(e, node, node.next)
def delete(self, p):
return self._delete_node(self._validate(p))
def replace(self, p, e):
node = self._validate(p)
old = node.element
node.element = e
return old
L = PositionalList()
p = L.add_last(8); print("add_last(8) ", list(L))
print("first() == p ", L.first() == p)
q = L.add_after(p, 5); print("add_after(p, 5) ", list(L))
print("before(q) == p ", L.before(q) == p)
r = L.add_before(q, 3); print("add_before(q, 3) ", list(L))
print("r.element() ", r.element())
print("after(p) == r ", L.after(p) == r)
print("before(p) ", L.before(p))
s = L.add_first(9); print("add_first(9) ", list(L))
print("delete(last()) ", L.delete(L.last()), list(L))
print("replace(p, 7) ", L.replace(p, 7), list(L))
try:
L.after(q) # q was deleted above
except ValueError as err:
print("after(q) ValueError:", err)add_last(8) [8] first() == p True add_after(p, 5) [8, 5] before(q) == p True add_before(q, 3) [8, 3, 5] r.element() 3 after(p) == r True before(p) None add_first(9) [9, 8, 3, 5] delete(last()) 5 [9, 8, 3] replace(p, 7) 8 [9, 7, 3] after(q) ValueError: p is no longer valid
Every operation touches only p and its immediate neighbours, so all of them are O(1). Two positions compare equal when they wrap the same node, which is why L.first() == p is True even though first() builds a fresh Position object.
A text editor on a positional list
Each character is an element of a positional list, and the cursor is a position (the character just before it, or none at the start). Typing is add_after(cursor, ch); backspace is delete(cursor). The counter compares the links touched with the elements an array would have to shift.
Check yourself: positions
- L = [4a, 6b]. Run: c = L.add_after(a, 5); L.delete(b); d = L.add_last(7); L.replace(a, 1). Give the list and the return value of replace.
- After
L.delete(b), what shouldL.after(b)do? - Why does
before(p)returnNonefor the first position instead of the header?
Show answers
- [4a, 5c, 6b] → [4a, 5c] → [4a, 5c, 7d] → [1a, 5c, 7d]. replace returns 4.
- Raise an error: b is no longer a valid position. The implementation detects this because deleting cleared the node's links.
- Sentinels are an implementation detail. Exposing them would let users read or replace a “non-element”, and would break the promise that a position always holds an element.
Part 6
Insertion sort on a positional list
6.1The idea
Insertion sort is how most people sort a hand of cards: keep the left part sorted, take the next card, and slide it left until it fits. On a positional list, three positions drive it:
marker: the rightmost position of the currently sorted portion.
pivot: the position just after the marker, holding the next element to place.
walk: moves leftward from the marker until it finds where the pivot's element belongs.
6.2The algorithm
- If the pivot's value is greater than the marker's, it is already in place: advance the marker to the pivot.
- Otherwise, set walk to the marker and move it left while the element before walk is greater than the pivot's value.
- Delete the pivot and insert its value before walk. The marker stays where it is (it is still the end of the sorted part).
- Repeat until the marker is the last position.
def insertion_sort(L):
"""Sort a PositionalList of comparable elements into nondecreasing order."""
if len(L) > 1:
marker = L.first() # end of the sorted prefix
while marker != L.last():
pivot = L.after(marker) # next element to place
value = pivot.element()
if value > marker.element(): # already in the right place
marker = pivot # the sorted prefix grows by one
else:
walk = marker # walk left to find the spot
while walk != L.first() and L.before(walk).element() > value:
walk = L.before(walk)
L.delete(pivot)
L.add_before(walk, value) # O(1) re-link, no shifting
L = PositionalList()
for x in [15, 22, 25, 29, 36, 23, 53, 11, 42]:
L.add_last(x)
insertion_sort(L)
print(list(L))[11, 15, 22, 23, 25, 29, 36, 42, 53]
Cost. Moving the pivot is O(1): delete and add_before relink nodes, with no shifting. But walking left can pass the whole sorted portion, so the worst case (a reverse-sorted list) is 1 + 2 + … + (n − 1) comparisons: O(n²). On an already sorted list, step 1 always applies and it runs in O(n).
Insertion sort, step by step
Each step is one comparison or one move. The sorted portion is underlined; M, P and W mark marker, pivot and walk.
Check yourself: insertion sort
- How many comparisons does insertion sort make on [4, 3, 2, 1]?
- An array-based insertion sort also makes O(n²) comparisons. What does the positional version save?
Show answers
- 6, counting as the code and the demo do. Each of the three pivots (3, 2, 1) is first compared with the marker: 3 comparisons. Then walk moves left comparing with its left neighbour, stopping without a comparison when it reaches the first position: 0 for the 3, 1 for the 2, 2 for the 1. Total 3 + 3 = 6. On a reversed list of length n this grows like n²/2.
- Element moves: the array version shifts every element between walk and pivot one slot right (O(n) writes per placement); the positional version relinks a single node in O(1). The walk itself is still O(n).
Part 7
The favorites list
7.1The problem
Maintain a collection of elements while counting how many times each is accessed, so you can report the most popular ones: a browser's most-visited sites, a music app's most-played songs.
| Method | What it does |
|---|---|
access(e) | access e: add 1 to its count, adding it to the list if new |
remove(e) | remove e if present |
top(k) | iterate over the k most-accessed elements |
len(), is_empty() | as usual |
7.2Design 1: keep it sorted by count
Store the elements in a positional list in non-increasing order of access count.
access(e): search for e (O(n)), add 1 to its count, then move it backward (towards the front) past any elements that now have a smaller count.remove(e): search, then delete the position.top(k): the first k entries. O(k).
The lecture's bad case: element 1 is accessed n times, then element 2 is accessed n times, …, then element n is accessed n times. When element k is being accessed, elements 1 … k − 1 are all ahead of it with count n, at least as large as its own, so it never moves forward. Every one of its n accesses searches past those k − 1 elements: about kn steps. Summing over k:
7.3Design 2: the move-to-front heuristic
Many access sequences have locality of reference: something used recently is likely to be used again soon. Exploit it: order the list by recency, and on every access move the element to the front.
access(e): search, add 1, move to the front. The move is O(1) (delete + add_first).
Same bad case: the first access to element k searches the whole list, O(n). After that, element k is at the front, so its remaining n − 1 accesses are O(1) each. Per element: O(n) + (n − 1)·O(1) = O(n). Over n elements: O(n²), a factor of n better.
A heuristic is a rule that usually helps but carries no guarantee. Move-to-front can do badly too: cycling through all n elements in the same order over and over always finds the needed element at the back, O(n) per access.
7.4The price: top(k)
With move-to-front the list is no longer sorted by count, so top(k) has to work:
- Copy all entries into a temporary list. O(n).
- Find the entry with the largest count and remove it from the temporary list. O(n). Repeat k times.
- Report the k entries removed.
Total O(kn). For a constant k, such as a top-10 list, that is O(n). For k proportional to n, such as the top 25%, it is O(n²). (Sorting the copy instead would give O(n log n), a later topic.)
class FavoritesList:
"""Elements kept in a PositionalList sorted by access count, highest first."""
class _Item:
__slots__ = "_value", "_count"
def __init__(self, e):
self._value, self._count = e, 0
def __init__(self):
self._data = PositionalList()
def _find_position(self, e): # O(n) linear search
walk = self._data.first()
while walk is not None and walk.element()._value != e:
walk = self._data.after(walk)
return walk
def _move_up(self, p): # restore the sorted order
if p != self._data.first():
cnt = p.element()._count
walk = self._data.before(p)
if cnt > walk.element()._count: # out of order: walk backwards
while (walk != self._data.first() and
cnt > self._data.before(walk).element()._count):
walk = self._data.before(walk)
self._data.add_before(walk, self._data.delete(p))
def access(self, e):
p = self._find_position(e)
if p is None:
p = self._data.add_last(self._Item(e)) # new elements start at the end
p.element()._count += 1
self._move_up(p)
def remove(self, e):
p = self._find_position(e)
if p is not None:
self._data.delete(p)
def top(self, k): # O(k): the first k are the answer
walk = self._data.first()
for _ in range(k):
item = walk.element()
yield item._value, item._count
walk = self._data.after(walk)
class FavoritesListMTF(FavoritesList):
"""Same ADT, ordered by recency: every access moves the element to the front."""
def _move_up(self, p): # O(1)
if p != self._data.first():
self._data.add_first(self._data.delete(p))
def top(self, k): # O(kn): order no longer helps
temp = PositionalList()
for item in self._data: # copy every item: O(n)
temp.add_last(item)
for _ in range(k): # k times: find and remove the max
high = temp.first()
walk = temp.after(high)
while walk is not None:
if walk.element()._count > high.element()._count:
high = walk
walk = temp.after(walk)
yield high.element()._value, high.element()._count
temp.delete(high)Measured on the bad case, counting nodes walked past:
# FavoritesList keeps items sorted by access count (highest first).
# FavoritesListMTF moves each accessed item to the front instead.
# Both count .steps = how many nodes their searches walk past.
for n in [10, 20, 40, 80]:
A, B = FavoritesList(), FavoritesListMTF()
for e in range(1, n + 1): # element 1 accessed n times, then element 2 ...
for _ in range(n):
A.access(e)
B.access(e)
print(f"n = {n:2} sorted list: {A.steps:7,} steps move-to-front: {B.steps:5,} steps")
F = FavoritesListMTF()
for site in "ACABACAAB":
F.access(site)
F.access("D")
print("list order (most recent first):", [item._value for item in F._data])
print("top(2):", list(F.top(2)))n = 10 sorted list: 450 steps move-to-front: 45 steps
n = 20 sorted list: 3,800 steps move-to-front: 190 steps
n = 40 sorted list: 31,200 steps move-to-front: 780 steps
n = 80 sorted list: 252,800 steps move-to-front: 3,160 steps
list order (most recent first): ['D', 'B', 'A', 'C']
top(2): [('A', 5), ('B', 2)]Each doubling of n multiplies the sorted list's steps by about 8 (cubic) and move-to-front's by about 4 (quadratic). The last two lines show recency order versus count order: D was accessed last so it is first in the list, yet top(2) still reports A and B by count.
Sorted versus move-to-front
Choose an access pattern and the number of distinct elements. The chart shows cumulative search-and-move steps for both designs over the whole sequence.
Check yourself: favorites
- Access sequence A, B, A, C, A, B on a move-to-front list. Give the list order after each access, and the counts at the end.
- Give an access pattern where move-to-front is no better than the sorted list.
- A music app shows a top-10 chart and has n = 100,000 songs. Is O(kn) for top(k) acceptable?
Show answers
- [A] → [B, A] → [A, B] → [C, A, B] → [A, C, B] → [B, A, C]. Counts: A 3, B 2, C 1.
- Cycling through all elements repeatedly in the same order (A, B, C, …, A, B, C, …): each access finds the element at the very back, O(n) every time.
- Yes: k = 10 is a constant, so it is O(n), about a million steps, fast. It would not be for “top 25%”.
Part 8
Linked lists on one page
8.1Arrays versus linked lists
| Feature | Array-based sequence | Link-based sequence |
|---|---|---|
| access by index | O(1) | O(n): walk from an end |
| insert / delete | O(n): shifting | O(1) given the location (e.g. a text-editor cursor) |
| memory overhead | less per element, but capacity exceeds size | extra references in every node |
| cache performance | good: contiguous memory | poor: nodes scattered across memory |
| worst case of end operations | amortized O(1) (occasional resize) | O(1) every time |
Cache, briefly: the processor keeps a small, very fast copy of recently used memory, and loads it in chunks of neighbouring bytes. Walking an array uses every byte of each chunk; walking a linked list jumps to a new, unrelated address at every node, so it waits on slow main memory far more often. Two algorithms with the same O(·) can differ several-fold in practice because of this.
Beyond the lecture: what “extra memory” means in Python
Measured on this machine, for 1,000 elements (the element objects themselves are not counted in any line):
import sys
class Node:
__slots__ = "element", "next"
def __init__(self, element, nxt):
self.element, self.next = element, nxt
class NodeNoSlots:
def __init__(self, element, nxt):
self.element, self.next = element, nxt
n = 1000
array_bytes = sys.getsizeof([None] * n)
slot_nodes = sum(sys.getsizeof(Node(None, None)) for _ in range(n))
plain = NodeNoSlots(None, None)
dict_nodes = n * (sys.getsizeof(plain) + sys.getsizeof(plain.__dict__))
print(f"list of {n} references : {array_bytes:6,} bytes")
print(f"{n} nodes with __slots__: {slot_nodes:6,} bytes")
print(f"{n} nodes without : {dict_nodes:6,} bytes")list of 1000 references : 8,056 bytes 1000 nodes with __slots__: 48,000 bytes 1000 nodes without : 344,000 bytes
A list stores one 8-byte reference per element. A node with __slots__ is a whole Python object, 48 bytes; without __slots__ each node also carries its own attribute dictionary. That is why every node class on this page declares __slots__.
8.2Summary of costs
| Structure | Ends | Middle (given a position) | Find |
|---|---|---|---|
| singly linked | add head/tail O(1); remove head O(1); remove tail O(n) | insert after O(1); delete needs predecessor | O(n) |
| circularly linked | rotate O(1); front O(1) | as singly linked | O(n) |
| doubly linked (sentinels) | all O(1) | insert / delete O(1) | O(n) |
| positional list | all O(1) | all O(1) | O(n) |
| favorites, sorted | access O(n); top(k) O(k) | bad case O(n³) total | |
| favorites, move-to-front | access O(n), O(1) for repeats; top(k) O(kn) | bad case O(n²) total | |
Ideas to carry forward
- Order of link updates matters: save any reference you are about to overwrite.
- Sentinels remove special cases. Two dummy nodes buy you code with no empty-list branches.
- O(1) insertion is “given the location”. Finding the location is a separate O(n) cost.
- Abstraction protects structure. Positions let users hold a place without touching nodes.
- How you order data changes total cost: sorted by count vs by recency differ by a factor of n on the same input.
Final mixed check
- You need a queue whose every operation must finish in bounded time. Array or linked? Why?
- Which single operation makes a singly linked list a poor choice for a deque?
- A doubly linked list has 10 elements. Inserting after the 7th element takes how many link assignments, and how many steps to reach the 7th from the header?
- Why is
replace(p, e)O(1) on a positional list butreplaceby value O(n)?
Show answers
- Linked: every operation is O(1) worst case. The array version is O(1) only amortized; a resize can take O(n).
- delete_last (remove at tail), which is O(n).
- Four link assignments. Reaching the 7th node takes 7 steps from the header (or 4 from the trailer, since you can walk backwards). The splice is O(1); the walk is not.
- With a position you already hold the node. Replacing by value first has to search for it.