Linux Dynticks Being Extended For Performance Wins

Dynticks, the Dynamic Tick Timer for allowing the Linux kernel to skip ticks while idling and resume to running at full HZ when encountering load, is in the process of being extended. Developers are working on making Dynticks work even under select workloads in order to enhance the performance of CPU-intensive tasks.

Dynticks up to this point has been a power savings win for Linux systems that are frequently idling since the tick timer goes off much less frequently. However, Dynticks can also be a performance win if the kernel can tick less while experiencing high CPU load. Developers have been working towards this feat and some of the bits are likely to be merged into the Linux 3.8 kernel.

There's an outstanding pull request for full Dynticks support for printk, the Linux kernel print function, in Linux 3.8. The pull request can be found below and describes what's being done for making Dynticks more relevant to CPU-intense workloads.

Frederic Weisbecker wrote:

We are currently working on extending the dynticks mode to broader contexts than just idle. Under some conditions on a busy CPU, the tick can be avoided (no need of preemption for one task running, no need of RCU state machine maintainance in userspace, etc...).

The most popular application of this is the implementation of CPU isolation. On HPC workloads, where people run one task per-CPU in order to maximize the CPU performances, the kernel sets itself too much on the way with these often unnecessary interrupts.

The result is a performance loss due to stolen CPU time and cache trashing of the userspace workset.

Now CPU isolation is the most famous user. I expect more. For example we should be able to avoid the tick when we run in guest mode. And more generally this may be a win for most CPU-bound workloads.

So in order to implement this full dynticks mode, we need to find alternatives to handle the many maintainance operations performed periodically and turn them to more one-shot event driven solutions.

printk() is part of the problem. It must be safely callable from most places and for that purpose it performs an asynchronous wake up of the readers by probing on the tick for pending messages and readers through printk_tick().

Of course if we use printk while the tick is stopped, the pending readers may not be woken up for a while. So a solution to make printk() working even if the CPU is in dynticks mode
is to use the irq_work subsystem. This subsystem is typically able to fire self-IPIs. So when printk() is called, it now enqueues an irq_work that does the asynchronous wakeup:

* If the tick is stopped, it raises a self-IPI
* If the tick is running periodically then don't fire a self-IPI but wait for the next tick to handle that instead (irq work probes on the timer tick). This avoids self-IPIs storm in case of frequent printk() in short periods of time.

I know this is a sensitive area. We want printk() to stay minimal and not rely too much on other subsystems that add complications and that may use printk themselves. That's why we chose irq_work because:

- It's pretty small and self-contained
- It's lockless
- It handles most recursivity cases (if it uses printk() itself from the IPI path, this won't fire another IPI)

But because it's sensitive, I'm proposing it as an RFC pull request.

A win for most CPU-bound workloads? Yes please. Linus has yet to respond whether he will honor this merge request for the Linux 3.8 release.