The scariest bugs are not the ones that crash. They are the ones where nothing happens, forever.

This is the story of a GPU that appeared to hang, a bug that turned out to live nowhere near the GPU code, a one-hunk fix in the kernel's workqueue core, and the stress test I wrote so that a hang like this becomes a clean, bounded test failure instead of a machine that needs a hard reboot.

In plain English

Kernels hand a lot of background chores to a system called a workqueue: "do this later, on another thread." The GPU driver relies on it for housekeeping, such as signalling that a piece of work has finished.

A rarely used feature, changing which CPUs the workqueue may use, briefly pauses some queues. A bug in the code that un-pauses them forgot to write down all but one waiting chore. The forgotten chores never ran. Everything that was waiting for them waited forever, and on the affected system the only way out was a hard reboot.

The fix was a few lines. The test I wrote makes the bug show up in about 30 seconds, and fail cleanly instead of freezing the machine.

Words you'll see in this post

• Workqueue: the kernel's way to run deferred work on kernel worker threads.
• Work item: one chore queued on a workqueue.
• Worker pool / kworker: a group of kernel threads that run the chores, and one such thread.
• Bound vs unbound: bound workers are pinned to one CPU. Unbound workers may run on any allowed CPU.
• NUMA node: a group of CPUs together with their own local memory.
• cpumask: a setting that says which CPUs the unbound workqueue's workers may run on.
• CPU migration: a thread moving from one CPU to another.
• Ordered workqueue: a workqueue that runs one item at a time, in order.
• pwq (pool_workqueue): the link between a workqueue and a worker pool.
• Plugged: temporarily paused. New work is accepted but not started.
• Fence: a signal that says "the GPU has finished this piece of work." Other code waits on it.

The short version

• Symptom: the GPU driver hung because a fence never signaled. The machine could only be recovered with a hard reboot.
• Root cause: a bug in the kernel workqueue code, not the GPU driver. When a plugged ordered workqueue was unplugged, only the first queued work item was activated. The rest were never put on the list of work waiting to run, so they never ran.
• Trigger: changing the unbound workqueue cpumask while the GPU driver was queuing work.
• Fix: one extra call in unplug_oldest_pwq() that restores the second bookkeeping step. It was written by Matthew Brost, reviewed on the kernel list, and applied by the workqueue maintainer, Tejun Heo.
• Safety net: a GPU stress test that cycles the cpumask while hammering the driver's bind and unbind path, with a 5-second deadline on every fence, so a stuck fence means a quick, clean failure instead of a frozen system.

The big picture

Here is the whole story in one frame. Follow the numbers: the driver queues a chore whose only job is to signal a fence, a cpumask write plugs the queue, the unplug restores only half the bookkeeping, and that chore is never called. The CPUs are idle, the workers are free, and the fence is never signaled.

The rest of this post explains each piece, starting with the background you need.

Background 1: what is a workqueue?

Kernel code often cannot, or should not, do slow work right where it is. An interrupt handler, for example, must finish fast. So it packages the job as a work item (a function plus some data) and puts it on a workqueue. A kernel worker thread (kworker) picks it up later and runs it in a context where sleeping is allowed.

Three terms to keep in mind:

• Workqueue: the line of chores waiting to be run.
• pool_workqueue (pwq): the link between one workqueue and one pool of workers.
• max_active: how many chores from this workqueue may run at the same time.

Background 2: worker pools, CPUs and NUMA nodes

Creating a new thread for every chore would be wasteful, so the kernel keeps worker pools: groups of kworker threads that are reused and that grow and shrink with demand. There are two families.

• Bound (per-CPU) pools: every CPU has its own pools, and their workers are pinned to that one CPU. This is good for cache locality.
• Unbound pools: their workers are not tied to a CPU. The scheduler may place them on any CPU the workqueue allows. They are used for long-running or CPU-hungry work, and they are the only kind that ordered workqueues use.

Big machines also have NUMA nodes: groups of CPUs that sit next to their own local memory. Reaching local memory is fast, and reaching another node's memory is slower. Unbound pools are organised by this topology so that workers stay close to the memory they touch. Many client machines have just one node, and the same code simply uses a single one.

One detail matters for this bug. For unbound workqueues, newer kernels count the number of active chores per NUMA node. All the pwqs of a workqueue in one node share that counter, and a waiting line for pwqs that need a slot lives next to it. The data structure is called wq_node_nr_active, and the waiting line is its pending_pwqs list.

Background 3: the cpumask, and CPU migration

A cpumask is a bit mask with one bit per CPU. The file /sys/devices/virtual/workqueue/cpumask holds the set of CPUs that unbound workers are allowed to use. Written in hex:

It exists so administrators can keep kernel housekeeping off chosen CPUs, for example to protect a latency-sensitive job.

Changing the mask has real consequences:

1. The unbound pools get a new allowed-CPU set.
2. The kworker threads are re-pinned. A worker that was running on one CPU may be migrated to another CPU, or even to another NUMA node. (CPU migration just means a thread moves from one CPU to another.)
3. In broad terms, each unbound workqueue gets fresh pwqs that match the new mask, while the old ones finish their work.

This is not a hot path. Almost nobody changes the mask while a GPU workload is running, which is exactly why bugs can hide here. The reproduction script below writes four different masks in under half a second, then repeats that hundreds of times.

Background 4: ordered workqueues and plugging

An ordered workqueue is an unbound workqueue with max_active = 1: chores run one at a time, in the order they were queued.

That promise makes a cpumask change tricky. If the new pwq started running chores while the old pwq still had chores waiting, two chores could run at once, or run out of order. So when the kernel gained the ability to change the cpumask of ordered workqueues (the commit "Enable unbound cpumask update on ordered workqueues"), it did this:

• the new pwq is plugged: it accepts new chores, but holds them,
• the old pwq drains its remaining chores,
• once the old pwq is empty, the new pwq is unplugged and work resumes.

As one reviewer noted, this works because only an ordered workqueue with max_active = 1 can be plugged.

Background 5: the slot and the waiting line

Two pieces of bookkeeping decide whether a chore may run:

• The slot (nr_active): a counter, like a ticket to run. With max_active = 1 there is exactly one slot.
• The waiting line (pending_pwqs): a list of pwqs that have chores but could not get a slot. When a slot frees up, the head of the line is called.

An item that has been queued but not started is inactive. Making it runnable is activating it.

When the pwq is not plugged, queueing a chore works like this: if a slot is free, the chore takes it. If not, the pwq is added to the waiting line, so it will be called when a slot frees up.

While the pwq is plugged, queueing skips activation. That means it skips both steps: it does not take a slot, and it does not add the pwq to the waiting line.

The waiting line is the memory of the system. If a pwq is not on it, nobody will ever come back for its work.

Background 6: GPU virtual memory, bind and unbind

The stress test does not poke the workqueue directly. It drives the GPU's bind and unbind path as hard as it can, because that path leans on workqueues to finish its work. To follow the test, you need to know what bind and unbind are.

GPU virtual memory. Just like a CPU process, every GPU client gets its own virtual address space, called a VM. The GPU does not use physical addresses directly. It translates GPU virtual addresses through GPU page tables.

Buffers. Data lives in buffer objects (BOs): chunks of memory the GPU can use.

Three operations make up the life of a mapping:

Bind and unbind are asynchronous. The caller does not wait inside the call. It passes in a sync object and gets a fence back, and the fence signals when the operation has fully finished. The work is submitted on a bind exec queue, a submission channel used only for bind and unbind. A separate exec queue carries the real GPU work.

This is where the workqueue comes in. Part of the completion and cleanup work, such as finishing the TLB invalidation and releasing buffers, runs as work items on kernel workqueues. A fence only signals once that work has run. If a work item is lost, the unbind fence never signals, and whoever waits on it waits forever.

Why the test uses this loop. Each test thread repeatedly binds, executes, unbinds and waits for the unbind fence, one thread per GPU engine, while the cpumask keeps changing in the background. That produces a constant stream of completion work items on the workqueue, which is exactly the traffic that exposes the bug.

The bug: unplug restored only half the bookkeeping

With the background in place, the bug is easy to state. When the pwq is unplugged, unplug_oldest_pwq() activates the first inactive work item. That part is correct.

But suppose several items were queued while the pwq was plugged. The first one is activated. The second, third and fourth are inactive, and because queueing skipped the "add to pending_pwqs" step, the pwq is not on the list.

So nobody ever comes back for them.

Think of a restaurant. While the door is closed for cleaning, new parties are seated at tables but their names are not written on the waiting list. When the door reopens, the host calls the first party. The others sit at their tables, and the host never calls their names, because no name is on the list.

Why a GPU hangs because of this

The GPU driver queues housekeeping on workqueues: cleaning up buffers after a bind or unbind, completing a TLB invalidation, and so on. A fence only signals once that housekeeping has run.

If the work item is one of the forgotten ones:

1. the work item never runs,
2. the fence it was supposed to complete never signals,
3. anything waiting on that fence, often in an uninterruptible wait, waits forever.

From the outside it looks like a GPU hang, even though the GPU did nothing wrong. As the original report put it, the driver "appeared to hang due to a fencing signal not completing", and it was traced to work items not being scheduled. The author also noted that it had been lurking around for a while, and that the driver had been working around it for a couple of years.

Reproducing it

The fix author's reproduction used two shells:

# shell 0: hammer the GPU driver's rebind path, over and over
for i in {1..100}; do run-the-gpu-rebind-test; done

# shell 1: keep changing the unbound workqueue cpumask
for i in {1..1000}; do
    echo f    > /sys/devices/virtual/workqueue/cpumask
    echo ff   > /sys/devices/virtual/workqueue/cpumask
    echo fff  > /sys/devices/virtual/workqueue/cpumask
    echo ffff > /sys/devices/virtual/workqueue/cpumask
    sleep .1
done

Shell 1 plugs and unplugs the ordered workqueue again and again. Shell 0 keeps queueing work onto it. Sooner or later several items are queued during a plug window, and the machine freezes.

The fix

 	if (pwq->plugged) {
 		pwq->plugged = false;
-		if (pwq_activate_first_inactive(pwq, true))
+		if (pwq_activate_first_inactive(pwq, true)) {
+			pwq_activate_first_inactive(pwq, false);
+
 			kick_pool(pwq->pool);
+		}
 	}

The first call activates the first inactive item. The second call, with fill = false, puts the pwq back on pending_pwqs if more items are waiting. That restores both bookkeeping steps that queueing had skipped.

The test: make the hang cheap to catch

A fix without a test can come back. But a test for this bug has a nasty property: when it fails, it hangs the machine.

So the real design challenge was not "reproduce the bug". It was "reproduce the bug without taking the test machine down".

The subtest is threads-wq-stress-rebind-bindexecqueue, added to the driver's exec-threads IGT test. It was posted to the igt-dev list as v3 and received a Reviewed-by and later merged upstream.

The design at a glance

What it does

• Per-engine threads continuously unbind and rebind the VM, using a separate bind exec queue for each slot.
• A helper process rapidly cycles the unbound workqueue cpumask through wider and wider CPU sets (f, ff, fff, ffff) every 100 ms, just like the shell reproducer.
• The test runs for up to 30 seconds.

The trick: every fence has a deadline

Normally the test would wait for a fence with no timeout. A missing signal would then hang the test forever. Under the new WQ_STRESS flag it waits with a 5-second deadline at three checkpoints:

1. the per-exec-queue fences,
2. the bind-chain fence,
3. the unbind fence, which covers TLB invalidation, the most likely place for a stuck work item to show up.

static int64_t stress_fence_deadline(void)
{
	struct timespec ts;

	clock_gettime(CLOCK_MONOTONIC, &ts);
	return (int64_t)ts.tv_sec * NSEC_PER_SEC + ts.tv_nsec +
	       WQ_FENCE_TIMEOUT_NS;   /* 5 seconds from now */
}

If any fence misses its deadline, the first thread to notice sets a shared atomic flag, and every other thread bails out immediately instead of hammering a hung kernel for the rest of the 30 seconds.

The trick: keep the GPU work in a child process

All GPU work runs in a forked child. The parent only watches.

• The child reports its result through a pipe: one byte, pass or fail.
• The parent waits on the pipe with poll() and a 60-second timeout. If nothing arrives, that is treated as a hang too.
• The parent restores the original cpumask whatever happens, so a failing test does not leave the machine with a changed CPU mask.

The trick: do not clean up a hung GPU

This part took the most thought. When a hang is detected, the obvious cleanup is to destroy the exec queues, buffers and VM, and close the DRM file descriptor. Every one of those can block forever, because they wait for pending GPU work, and that work is exactly what is stuck. Closing the fd would sit in an uninterruptible wait for the fence.

So when a hang is detected, the child:

1. logs a critical message,
2. drops the page cache and forces a GPU reset to unblock fences that are still pending,
3. writes its result before exiting,
4. exits immediately with _exit(), without closing the fd,
5. and skips all teardown of exec queues, buffers and VM. The kernel reclaims them when the process dies.

Writing the result before exiting matters: the parent already has its answer and does not have to wait for a child that may be stuck in an uninterruptible state.

Fail loudly, and abort

If a hang is confirmed, the test aborts rather than just marking a failure. A child stuck in an uninterruptible D+ state can interfere with later tests, so continuing would only produce confusing results. Aborting makes it clear that this machine needs a reboot, instead of letting a hang pollute the rest of the run. That behaviour came out of review: v2 changed it from "fail" to "abort".

The test also skips cleanly if it is not run as root, since it needs to write the workqueue cpumask.

Debugging tips

• Watch the workqueue cpumask: cat /sys/devices/virtual/workqueue/cpumask
• See stuck workers: echo w > /proc/sysrq-trigger and read the blocked-tasks dump in dmesg
• A process in D state forever is the classic sign of a lost wake-up: check what it is waiting on with cat /proc/<pid>/stack
• If a fence never signals, look for the work item that should have signalled it, not at the GPU

Status

The kernel fix was applied by the workqueue maintainer to the fixes branch for the 7.0 cycle: the patch on patchwork. The IGT test is merged, the commit link the patch on igt-dev.

What I learned

• The symptom is not the location. A "GPU hang" was a kernel workqueue bug. When a fence never signals, ask who was supposed to signal it, and whether that code ever ran.
• Bookkeeping bugs are the quietest bugs. Nothing crashed, nothing printed. One list was missing one insertion, and the work simply was never called.
• Pause and resume paths deserve extra tests. The bug lived in the code that restores state after a temporary pause, a path that most workloads rarely exercise.
• A test that can hang the machine needs an exit plan. Deadlines on waits, work in a child process, a result written before exit, and no cleanup of something that is stuck turn a frozen system into a failed test.

Views are my own and not those of my employer. Kernel and test code shown is from public mailing-list patches.

I work on Linux graphics drivers and GPU tooling. If you have chased a "GPU hang" that turned out to live somewhere else, I would love to hear about it.

A 4K graphics benchmark that should score 4,667 came back with 2,445. Nearly half the performance was gone. The GPU was fine, the clocks were fine, and the thermals were fine. The problem was how the kernel handed out memory.

This post is about that bug, why it only showed up under heavy load, and the RFC patch I wrote to fix it.

In plain English

When a GPU needs memory, it likes big contiguous chunks. Big chunks are cheaper for the GPU to address, so it runs faster. When the system has been running for a while, RAM becomes fragmented: lots of free memory, but chopped into tiny pieces. If big chunks are not free at that moment, the driver quietly settles for tiny 4 KB pages instead. Nothing fails, and nothing logs an error. The workload just gets slower.

There is one more ingredient: the slowdown only appeared when the IOMMU was switched on. The IOMMU adds a second layer of address translation between the GPU and RAM, and tiny pages make that extra layer far more expensive.

The fix: let the driver notice when the GPU is about to ask for a lot of memory, and in that case wait for the kernel to make big chunks available instead of giving up immediately.

Words you'll see in this post

• Page: the basic unit of memory, normally 4 KB.
• Order: the size of a block of pages. Order n means 2ⁿ contiguous pages, so order 9 is 512 pages = 2 MB.
• Fragmentation: free memory that is split into small pieces, so no big block is available.
• Reclaim / compaction: the kernel's work to free up pages and shuffle them together to rebuild big blocks.
• TTM: the generic GPU memory manager in the Linux DRM subsystem.
• IOMMU: a hardware unit that sits between a device (here, the GPU) and RAM and translates the addresses the device uses into real physical addresses. It is to devices what the MMU is to CPUs.
• IOVA (I/O virtual address): the address a device uses when the IOMMU is on. It is not a physical address. The IOMMU translates it.
• TLB / IOTLB: a small cache of recent address translations. The one in the IOMMU is the IOTLB. A miss means a slow walk of the page tables in memory.
• TLB invalidation: throwing cached translations away when a mapping changes, so nobody uses a stale one.

The short version

• Symptom: a 4K GPU benchmark scored 2,445 instead of 4,667 on an integrated GPU with 16 GB of RAM.
• Condition: it only happened with the IOMMU enabled. With it off, the same workload was fine.
• Why the IOMMU matters: it adds a second translation layer (IOVA to physical). 4 KB pages mean many more IOMMU entries, more IOTLB misses and a much higher TLB invalidation rate.
• Clue: dropping the kernel caches (echo 3 > /proc/sys/vm/drop_caches) before the run restored full performance, which pointed at fragmentation.
• Cause: under a heavy allocation burst, no order-9 (2 MB) blocks were free. The GPU memory manager made an opportunistic attempt for big pages, did not wait, and fell back to 4 KB pages.
• Fix (RFC): a small memory monitor in the driver. It measures how fast GPU memory is growing, using an exponential moving average, and checks whether order-9 blocks are scarce. When both say "a burst is coming and big pages are short", it switches allocation into a high-throughput mode that is allowed to wait for reclaim.
• Why two modes: direct reclaim stalls the task that asked for memory. That is acceptable for a heavy workload and unacceptable for browsing, UI and video, so the driver pays that cost only when a burst is detected.
• Safe by default: the whole thing is behind a module parameter and is off unless enabled.

Background: why the GPU cares about page size

Every buffer the GPU touches is mapped through GPU page tables. A buffer backed by 2 MB pages needs 512 times fewer entries than the same buffer backed by 4 KB pages. That means fewer page-table walks and better TLB behaviour, which matters when a benchmark streams gigabytes of textures and render targets.

The missing piece: the IOMMU and its extra translation layer

So far I described one layer of translation: the GPU's own page tables. With the IOMMU enabled there is a second one underneath.

Without an IOMMU, the GPU is handed physical addresses. What the driver programs is what the memory bus sees. There is no extra step.

With an IOMMU, the GPU never sees physical addresses. For each buffer, the kernel's DMA layer allocates a range of IOVAs (I/O virtual addresses) and fills the IOMMU's page tables so that each IOVA maps to a physical page. The GPU is given the IOVAs. Every system-memory access it makes then goes through two lookups:

The IOMMU is there for good reasons: it isolates devices from memory they shouldn't touch, and it is a building block for virtualization. But it is not free, and the cost depends heavily on page size:

So low-order pages hurt twice:

1. More TLB misses. The same buffer needs 512 times as many translations, far beyond what the IOTLB can cache. Constant misses mean constant slow table walks, and the GPU stalls waiting for them.
2. A higher TLB invalidation rate. Whenever buffers are freed, evicted or remapped, the IOMMU mappings change and the cached translations must be invalidated. With 4 KB pages there are many more mappings to change, so invalidations are far more frequent, and each one costs time on the same path the GPU is trying to use.

That is why the bug only appeared with the IOMMU on. With it off, 4 KB pages still cost something in the GPU's own page tables, but there is no second layer to multiply the damage. With it on, the fallback from 2 MB to 4 KB turned a mild inefficiency into a halving of the benchmark score.

So when the GPU memory manager can get 2 MB blocks, performance is good. When it can't, it still works, which is exactly what makes this bug hard to spot.

How a GPU buffer gets its memory

The stack looks like this:

Step 3 is where the story turns.

The bug: an optimistic allocator that never waits

The big-order attempts in TTM are deliberately opportunistic. They are not allowed to stall, because blocking the whole system for a nice-to-have 2 MB block would be a bad trade in general. So the allocator asks the buddy system for a 2 MB block, and if none is free right now, it does not wait for reclaim or compaction. It just falls back to a smaller order.

On a freshly booted machine this is invisible: big blocks are everywhere. On a machine that has been busy, with 16 GB of RAM mostly spoken for and a heavy 4K workload allocating GPU memory in a burst, the picture is different:

I could see this in /proc/buddyinfo: the high-order columns, order 9 and above, were nearly empty while the low-order columns held plenty of free memory.

That is external fragmentation: the memory exists, it is just in the wrong shape.

How the issue was narrowed down

• The regression correlated with allocation behaviour, not with GPU frequency or power.

• The key clue: it disappeared when the IOMMU was disabled. That ruled out the GPU itself and pointed at the extra translation layer, and from there at page size.

• The decisive experiment: drop the caches. Before a run, flush the kernel's reclaimable memory from the command line:

  sync
  echo 3 > /proc/sys/vm/drop_caches    # as root
  

  sync writes dirty data to storage first, so the pages can be dropped safely. Writing 3 frees the clean page cache plus dentries and inodes (1 is page cache only, 2 is dentries and inodes only). All that memory goes back to the free lists, and neighbouring free pages merge back into big blocks. After that, the benchmark ran at full speed. The same run without the drop was slow. That is exactly what you'd expect if the problem is shape, not amount: the memory was already there, it just wasn't in large contiguous pieces until the caches were released.

• Comparing /proc/buddyinfo before and during the run showed high-order free blocks collapsing under load.

• The place where TTM's page-allocation flags are chosen is exactly where a driver-level hook can influence the behaviour, and changing the flags there recovered the lost performance. That pointed straight at "the allocator gave up too early".

Why dropping caches is a diagnosis, not a fix

Dropping caches proved the theory, but it can't be the solution. It throws away useful cached files, so everything gets slower for a while as they are re-read. It needs root, and nobody can run it by hand before every game or benchmark. The driver has to get the same effect by itself, at the right moment: ask the kernel to reclaim memory when, and only when, a burst of GPU allocations is arriving.

The idea: wait only when it's worth waiting

The obvious fix is "always allow the allocator to wait for big pages". That is a bad idea, for the reasons below.

Why two modes? Because most of the time, nobody needs to wait

If waiting for big pages fixes the benchmark, why not wait all the time? Because direct reclaim is a tax paid by whoever is asking for memory. When an allocation can't be satisfied, the kernel makes that very task stop and clean house: write back pages, evict caches, compact memory. Until it finishes, the task is stuck.

For a benchmark that streams gigabytes, that stall is a rounding error next to the speed-up. For everyday use it is the opposite: a stall of even a few milliseconds is a visible hitch.

The trade-off flips completely depending on the workload:

• Small, steady allocations: the cost of waiting is paid immediately and repeatedly, and the benefit is tiny. Fast 4 KB fallback wins.
• A big burst: the cost of waiting is paid once, and the benefit lasts as long as the workload does. Waiting wins by a mile.

A single fixed policy gets one of these wrong, whichever way you set it. "Never wait" costs us half the benchmark score. "Always wait" makes a quiet browsing session feel sluggish. So the driver needs two modes, and a way to decide, right now, which situation it is in:

• Latency mode (default): no direct reclaim. Allocation stays instant. This is what you're in 99% of the time while browsing, scrolling and watching videos.
• High-throughput mode: direct reclaim is allowed, so the allocator can wait for big blocks. This is used only during a heavy GPU workload.

The question is how to detect "a burst is happening" without being fooled by tiny spikes.

The memory monitor

I added a small periodic worker, mem_monitor, in the driver. It does two jobs.

1. Measure how fast GPU memory is growing

The driver already keeps a counter of the total GPU memory pages in use. It is the same counter that feeds the gpu_mem_total tracepoint. Every sample interval, the monitor reads it and computes:

1. Delta (growth): how much the memory grew since the last sample. If it shrank or stayed flat, the delta is clamped to 0.

   delta(t) = max(0, mem(t) - mem(t-1))

2. Exponential moving average (EMA): blend the new growth with history.

   EMA(t) = alpha * delta(t) + (1 - alpha) * EMA(t-1)

   The patch uses a fixed alpha = 0.8: 80% weight on the latest growth, 20% on everything before it.

An alpha of 0.8 makes the monitor a low-pass filter that reacts quickly. A real allocation burst pushes the average up within a sample or two, while a single tiny spike is smoothed away.

If the EMA crosses a threshold (default 250 MB per interval), the driver switches from latency mode to high-throughput mode.

2. Don't flap

Switching modes back and forth would be worse than either mode. So the switch back to latency mode requires several consecutive below-threshold samples (the default is 5). That gives a built-in minimum dwell time of debounce_count * poll_ms, with no second timer or second alpha to tune.

3. Check whether big pages are actually scarce

A growth burst alone isn't enough. If order-9 blocks are plentiful, the allocator will find them without waiting. The patch also gates the wake-ups on order-9 external fragmentation, using the free-block counts that /proc/buddyinfo exposes. So the expensive behaviour is only used when both conditions hold: GPU memory is growing fast, and big blocks are short.

4. Be good citizens

• Power: the worker goes to sleep after a period with no significant activity in latency mode (default 10 s) and is re-armed when activity returns.
• Suspend and resume: the worker is paused and resumed across system and runtime power-management transitions.
• Opt-in: everything is behind module parameters and off by default.

How the pieces connect

1. The monitor samples the GPU page counter and checks order-9 availability.
2. It publishes a mode: latency or high-throughput.
3. A small set of vendor hooks on the TTM page-allocation path reads that mode.
4. In high-throughput mode the hooks let TTM's allocation attempts wait for reclaim and compaction instead of bailing out. In latency mode nothing changes.

The first patch adds the monitor, the second registers the hooks, the third adds the fragmentation gate, and the fourth adds the safe enable toggle. The RFC covers all four.

Observing it

• Watch the mode flip with a tracepoint: cat /sys/kernel/debug/tracing/events/<driver>/<mode_switch_event>.
• Watch fragmentation live: watch -n1 cat /proc/buddyinfo and look at the orders 9 and above.
• Compare gpu_mem_total against the mode switches to see the monitor reacting to a burst.

What I learned

• "It works" can hide a performance bug. A fallback path that never fails never raises an alarm. It just quietly costs you half your frame rate.
• Every translation layer multiplies the cost of small pages. The IOMMU is worth having, but it turns "slightly less efficient" into "twice as slow" when pages are tiny. Test with the IOMMU both on and off.
• Fragmentation is invisible in free. Free memory can be high while useful free memory is low. /proc/buddyinfo tells the real story.
• Don't fix a latency trade-off globally. "Always wait" would have helped one benchmark and hurt every interactive workload. The monitor makes the trade-off only when the workload justifies it.
• Smooth, then threshold, then debounce. An EMA removes noise, a threshold makes the decision, and the debounce stops it flapping. Three small ideas make a stable control loop.
• Ship risky behaviour off by default. An RFC with a kill switch is much easier to review than one without.

Status

This is an RFC: the patches are posted for discussion and feedback, not merged. You can read the thread here: RFC: Dynamic GPU Memory Monitor. Thoughts, objections and better ideas are welcome.

I work on Linux graphics, display drivers and GPU tools. If you have fought a fragmentation bug of your own, I'd love to hear how you tackled it.

Some kernel bugs hide for a long time because the code path that triggers them is simply never exercised by the usual software stack. This is the story of one of those bugs in the display code used by the GPU driver, and the GPU test I wrote so it can't come back.

In plain English

Your screen is built from layers (called planes). Before showing a new frame, the display driver has to wait until the GPU has finished drawing every layer. A tiny type mistake, a big number squeezed into a smaller box, made the driver think the waiting had failed, so it stopped early and showed unfinished frames. The result was flicker and visual glitches. The fix was one line, and I wrote the test that makes sure it can't come back.

Words you'll see in this post

• Fence: a signal that says "the GPU has finished this piece of work."
• Plane: one layer of the screen, such as the background, a video or a pop-up.
• Atomic commit: updating all the layers together in one step, all or nothing.
• Explicit fencing: the app hands its fences to the kernel itself (as Android does), instead of the kernel tracking them behind the scenes.

The short version

• Trigger: Android's compositor passes explicit fences to the kernel, including already-finished placeholder ("stub") fences on some planes, next to real pending ones.
• Bug: in a function in Display Driver Code <gpu>_atomic_commit_fence_wait(), dma_fence_wait_timeout() returns a 64-bit long, but the result was stored in a 32-bit int. For an already-signaled fence that value is MAX_SCHEDULE_TIMEOUT (0x7FFFFFFFFFFFFFFF). Only the lower 32 bits survived, 0xFFFFFFFF, which is -1, and the loop read that as an error.
• Effect: the driver skipped waiting on the remaining fences, so a GPU-drawn layer was shown before it was ready: flicker and visual artifacts.
• Fix: a one-line type change (int to long).
• Safety net: a GPU test I wrote that reproduces the exact mix of signaled and pending fences, so the bug can't return unnoticed.

Background: what is a fence?

A fence is a synchronization primitive, exposed to userspace as a file descriptor. It has three states:

In the display (KMS) world there are two fence properties:

• IN_FENCE_FD (per plane): "do not display this buffer until this fence signals."
• OUT_FENCE_PTR (per CRTC): "give me a fence that signals when this update has reached the screen."

Implicit vs explicit fencing

• Implicit fencing: the kernel attaches fences to buffers automatically and userspace never sees them. The kernel must track them all.
• Explicit fencing: userspace owns the fences. It passes them around as file descriptors, can poll() on them without blocking a thread, and the kernel only waits when told to. This suits event-driven compositors.

Android's compositor works this way. Each frame has an acquire fence (the GPU is done drawing), and the display returns a present/release fence (the buffer can be reused).

The Android frame lifecycle in one picture

The app draws a frame and passes the buffer, with an acquire fence, down to the display driver through SurfaceFlinger and the Hardware Composer. The kernel waits for those fences, flips the display, then signals an out-fence that travels back up as the release fence, telling the app the buffer is safe to reuse. The kernel’s wait is where this bug lived: it must wait on every plane’s fence.

Following one commit through the kernel

When userspace calls drmModeAtomicCommit() with an IN_FENCE_FD, the DRM core turns the file descriptor into a dma_fence and stores it in the plane state. The driver's <gpu>_atomic_commit_fence_wait() then walks every plane in the commit and waits on its fence before the hardware is programmed. Only after all of them signal does it flip the buffers.

The bug

The wait loop looked like this (simplified):

int ret, i;

for_each_new_plane_in_state(&<gpu>_state->base, plane, new_plane_state, i) {
	if (new_plane_state->fence) {
		ret = dma_fence_wait_timeout(new_plane_state->fence, false, timeout);
		/* ... ret < 0 is treated as an error, the remaining waits are skipped ... */
	}
}

dma_fence_wait_timeout() returns a long: the remaining timeout, or a negative error. For a fence that is already signaled, it returns immediately with the timeout value, here MAX_SCHEDULE_TIMEOUT (0x7fffffffffffffff).

The driver stored that in an int, so only the lower 32 bits survived: 0xFFFFFFFF, which is -1. The code read this as an error and skipped the wait on the remaining fences.

So the failing recipe is narrow: one plane with an already-signaled (or stub) fence, plus another plane whose fence is still pending. The Android compositor produces exactly that, with stub fences on overlay planes and a real GPU fence on the composited layer. With only pending fences, or only signaled ones, nothing went wrong, which is why it hid for so long. The symptom was the GPU-composited layer being scanned out too early: visual artifacts.

The fix

The fix is one line of type, drm kernel patch :

-	int ret, i;
+	long ret;
+	int i;

It restores the full 64-bit return value, so an already-signaled fence is no longer mistaken for an error and the loop keeps waiting on the rest.

The GPU test

GPU Tools is the Linux kernel graphics test suite (IGT). I wanted a test that:

1. needs no GPU rendering (so it is deterministic and runs anywhere), and
2. controls fence timing exactly.

The trick is sw_sync, a kernel software fence facility. You create a timeline (a monotonically increasing counter), create fences at a sequence number, and signal them yourself by advancing the timeline.

A timeline starting at 0 behaves like this: a fence at point 1 is unsignaled while the timeline is 0, and becomes signaled the moment the timeline reaches 1.

What the test does

The subtest is multiplane-atomic-fence-wait in tests/kms_explicit_fence.c. It needs a CRTC with at least two overlay planes and skips otherwise.

1. Set up a modeset with three planes, each showing a solid color: primary = blue, overlay 1 = red, overlay 2 = green. No fences yet.
2. Capture a baseline CRC ("before").
3. For each of the three planes, create a sw_sync timeline and a fence at sequence point 1. Then:
   • primary and overlay 1: advance the timeline, so the fence is already signaled,
   • overlay 2: leave the fence unsignaled, the long-running GPU job,
   • attach each fence to its plane as IN_FENCE_FD.
4. Swap the overlay framebuffers (red and green) so a scanout update would change the CRC.
5. Request an OUT_FENCE_PTR on the CRTC and commit atomically, non-blocking. The commit must succeed immediately.
6. Verify nothing has happened yet:
   • overlay 2's fence is still unsignaled,
   • the out-fence is not signaled,
   • after sleeping 100 ms, the out-fence is still not signaled and the CRC still equals the baseline.
7. Signal overlay 2's fence.
8. Wait for the out-fence (2 s timeout), then check the CRC differs from the baseline: the update happened, and only after the last fence.
9. Close the fences and timelines.

A driver with this bug sees the two already-signaled fences, misreads the first return value as an error, and never waits for overlay 2. The display updates early, the out-fence signals, and the test fails at step 6 instead of letting the artifacts reach a real screen.

I have submitted the test upstream. It went through seven revisions on the dev mailing list and its merged now, with changes such as CRC checks instead of visual inspection, a vendor-neutral setup and a scalable overlay-plane check.

You can read the patch and the review thread here: Add Explicit Fencing Tests

Debugging tips

• List plane properties: cat /sys/kernel/debug/dri/0/i915_display_info | grep -A 20 "plane"

• Enable fence tracing:

  echo 1 > /sys/kernel/debug/tracing/events/dma_fence/enable
  echo 1 > /sys/kernel/debug/tracing/events/sync_timeline/enable
  cat /sys/kernel/debug/tracing/trace_pipe
  

• On Android: dumpsys SurfaceFlinger --hwclayers

What I learned

• A fix is not done until a test pins it down. Without a test, the same regression can return unnoticed.
• Software fences (sw_sync) turn a timing-dependent GPU problem into a deterministic one.
• Different userspace stacks exercise different kernel paths. A bug can be invisible on one platform and obvious on another.
• Mind return types. A long stored in an int looked harmless for years, until a "very large but valid" value turned into -1.

Questions or feedback? I'd love to hear from you. I work on Linux graphics and GPU drivers.