vtable, vptr & dynamic dispatch overhead
Scenario
You are writing a High-Frequency Trading (HFT) matching engine. You use a clean OOP design with a base `Order` class and virtual `execute()` methods. Profiling reveals your application is running 20% slower than expected.
Why does using the `virtual` keyword silently tank performance in ultra-low-latency systems?
Why it exists
While dynamic polymorphism (OOP) is a powerful design pattern for decoupling code, C++ is designed for zero-overhead abstractions. C++ exposes the exact costs of the vtable so systems engineers can choose when to pay the price for polymorphism, and when to stick to static dispatch (like compile-time templates) for maximum performance.
Mental model
Standard function calls are like a direct flight—the CPU knows exactly where to go. Virtual function calls are like a connecting flight with a layover. You first have to fly to a lookup table (vtable) to figure out where your final destination is. That extra stop takes time.
Dynamic dispatch in C++ is implemented using a Virtual Table (vtable) and a hidden Virtual Pointer (vptr) in every object. This introduces memory overhead (the vptr), an extra pointer dereference at runtime, and inhibits compiler optimizations like inlining, causing cache misses and branch mispredictions.
Explanation
**1. Memory Overhead (vptr):**
When a class contains even one `virtual` function, the compiler silently inserts a hidden pointer (the `vptr`) into every instance of that class. This pointer points to the class's `vtable`. If you create an array of 1,000,000 objects, that's an extra 8 Megabytes of RAM consumed just by hidden pointers.
**2. The Double Dereference (vtable):**
The `vtable` is a static array of function pointers created once per class. When you call `obj->virtual_func()`, the CPU must:
1. Dereference `obj` to find the `vptr`.
2. Dereference the `vptr` to find the `vtable`.
3. Look up the correct function pointer in the `vtable`.
4. Jump to that function's address.
This indirection takes more CPU cycles than a direct `CALL` instruction.
**3. Cache Misses & Branch Prediction:**
The biggest hidden cost is not the dereference itself, but memory latency. The `vtable` might not be in the L1 CPU cache. Fetching it from main memory causes a massive pipeline stall. Furthermore, because the destination address is unknown until runtime, the CPU's branch predictor often fails, requiring the CPU to flush its pipeline.
**4. Loss of Inlining:**
Compilers speed up code by 'inlining' small functions (copy-pasting the function body directly into the caller to avoid function call overhead). Because the compiler doesn't know which virtual function will be called until runtime, it usually *cannot* inline virtual functions. **Devirtualization** is the exception: if the compiler can statically prove the exact type at compile time (e.g., if you call the virtual function on a concrete, stack-allocated object rather than a base pointer), it bypasses the vtable and inlines the call.
Code examples
The Hidden Costs of Virtual Functions
struct NonVirtual {
void doWork() {} // Direct call, can be inlined
};
struct Virtual {
virtual void doWork() {} // Requires vptr and vtable
};
int main() {
NonVirtual n;
Virtual v;
// sizeof(n) is 1 byte (empty structs are 1 byte in C++)
// sizeof(v) is 8 bytes (because it contains a hidden 64-bit vptr)
}Adding a virtual function silently increases the size of every object. In tight data-oriented designs (like ECS in game engines or order books in HFT), this ruins cache density.
Key points
- Double memory lookup: Virtual calls require dereferencing the vptr, then looking up the function in the vtable.
- Cache and pipeline killer: Virtual calls prevent inlining, cause instruction cache misses, and hinder branch prediction.
Common mistakes
- Making everything virtual 'just in case': Developers coming from Java or C# (where methods are virtually dispatched by default) often apply the `virtual` keyword to every C++ method out of habit. In C++, you only pay for what you use. Making functions virtual by default introduces widespread, unnecessary performance penalties.
Recall questions
- What two internal mechanisms does a C++ compiler use to implement dynamic dispatch?
- Why do virtual functions prevent compiler inlining?
- How does adding a virtual function affect the `sizeof()` an object?
Questions & answers
A game engine loop iterates over 10,000 `GameObject` pointers, calling a virtual `update()` method on each. Profiling shows intense CPU stalling. Why?
Because the objects are scattered in memory, iterating over pointers causes continuous L1 cache misses. Each virtual call then requires fetching a `vtable` (another potential cache miss) and executing an indirect branch, which breaks CPU pipeline prediction.
Approach: Understand the interaction between virtual dispatch and CPU caches/branch prediction.
An interviewer asks: 'Can a C++ compiler ever inline a virtual function call?'
Yes, but only through 'Devirtualization'. If the compiler can statically prove the exact type of the object at compile time (e.g., if you call the virtual function on a concrete stack-allocated object rather than a base pointer/reference), it bypasses the vtable and can inline the call.
Approach: Recognize edge cases where the compiler can optimize away dynamic dispatch.
Continue learning
Previous: Inheritance & virtual functions / vtable