Understanding the Stack: Functions and Calling Conventions
In the previous tutorials, we reversed if/else statements and loops in GDB. We saw local variables sitting at [ebp-0x4], [ebp-0x8], etc. We saw call instructions invoke printf and puts. We saw leave and ret at the end of main.
But we never really explained why. Why does EBP matter? Why are local variables at negative offsets from EBP? What exactly happens when a call instruction executes? Where do function arguments go? What does ret actually do?
In this post, we’re going to answer all of those questions. We’ll trace the complete lifecycle of a function call — from the caller pushing arguments, through the call instruction, the prologue, the function body, the epilogue, and the return. We’ll watch the stack change at every step in GDB, and by the end, you’ll understand exactly how the stack works and why buffer overflows are so devastating.
The Stack — A Quick Refresher
The stack is a region of memory that grows downward — from high addresses to low addresses. Every time you push something, the stack pointer (ESP) decreases. Every time you pop, it increases.
High Address (e.g., 0xBFFFFFFF)
┌────────────────────────┐
│ │
│ Stack grows downward │
│ ↓ │
│ │
│ ┌──────────────┐ │
│ │ Latest push │ ◄── ESP points here
│ └──────────────┘ │
│ │
Low Address (e.g., 0xBFFFF000)
The stack has two primary uses:
- Saving state — Return addresses, saved registers, local variables
- Passing data — Function arguments (in 32-bit x86 cdecl convention)
The Calling Convention — cdecl
On 32-bit Linux, the standard calling convention is cdecl (C declaration). The rules are:
| Rule | Description |
|---|---|
| Arguments | Pushed onto the stack right to left |
| Return value | Placed in EAX |
| Caller saves | EAX, ECX, EDX (caller-saved / volatile) |
| Callee saves | EBX, ESI, EDI, EBP (callee-saved / non-volatile) |
| Stack cleanup | The caller cleans up the arguments after the call returns |
| Stack alignment | The stack should be 16-byte aligned before call on modern systems |
Why are arguments pushed right to left? So that the first argument ends up closest to the top of the stack (lowest address). This makes it easy for functions like printf to find their first argument (the format string) at a known offset, regardless of how many additional arguments follow.
Example: A Function Call, Step by Step
Let’s work with a simple program that calls a function:
user@protostar:~$ ./func1
Result: 8
Disassembly
(gdb) disass main
Dump of assembler code for function main:
0x08048404 <+0>: push ebp
0x08048405 <+1>: mov ebp,esp
0x08048407 <+3>: sub esp,0x18
0x0804840a <+6>: mov DWORD PTR [esp+0x4],0x5
0x08048412 <+14>: mov DWORD PTR [esp],0x3
0x08048419 <+21>: call 0x80483c4 <add>
0x0804841e <+26>: mov DWORD PTR [ebp-0x4],eax
0x08048421 <+29>: mov eax,DWORD PTR [ebp-0x4]
0x08048424 <+32>: mov DWORD PTR [esp+0x4],eax
0x08048428 <+36>: mov DWORD PTR [esp],0x80484e0
0x0804842f <+43>: call 0x80482f8 <printf@plt>
0x08048434 <+48>: mov eax,0x0
0x08048439 <+53>: leave
0x0804843a <+54>: ret
End of assembler dump.
(gdb) disass add
Dump of assembler code for function add:
0x080483c4 <+0>: push ebp
0x080483c5 <+1>: mov ebp,esp
0x080483c7 <+3>: sub esp,0x10
0x080483ca <+6>: mov eax,DWORD PTR [ebp+0x8]
0x080483cd <+9>: mov edx,DWORD PTR [ebp+0xc]
0x080483d0 <+12>: lea eax,[edx+eax*1]
0x080483d3 <+15>: mov DWORD PTR [ebp-0x4],eax
0x080483d6 <+18>: mov eax,DWORD PTR [ebp-0x4]
0x080483d9 <+21>: leave
0x080483da <+22>: ret
End of assembler dump.
Now let’s trace the entire call, watching the stack at every stage.
Stage 1: Before the Call — Pushing Arguments
0x0804840a <+6>: mov DWORD PTR [esp+0x4],0x5
0x08048412 <+14>: mov DWORD PTR [esp],0x3
0x08048419 <+21>: call 0x80483c4 <add>
Instead of using push instructions, the compiler placed arguments directly at stack offsets. This is an optimization — the sub esp, 0x18 in the prologue already reserved enough space, so the compiler writes to the pre-allocated slots. The effect is the same as push 5 then push 3.
Arguments are placed right to left: first 5 (second argument, at [esp+0x4]), then 3 (first argument, at [esp]).
Let’s set a breakpoint right before the call and examine the stack:
(gdb) b *0x08048419
Breakpoint 1 at 0x8048419
(gdb) run
Breakpoint 1, 0x08048419 in main ()
(gdb) x/4xw $esp
0xbffff7d0: 0x00000003 0x00000005 0xbffff898 0xb7fd7ff4
arg1=3 arg2=5
The stack has our arguments ready: 3 at the top, 5 right after. The call instruction is about to execute.
The Stack Before CALL
┌──────────────────────────┐
│ ...main's stack frame...│
│ │
│ 0x00000005 (arg2) │ ESP+4
│ 0x00000003 (arg1) │ ◄── ESP
└──────────────────────────┘
Stage 2: The CALL Instruction
0x08048419 <+21>: call 0x80483c4 <add>
The call instruction does exactly two things:
- Pushes the return address onto the stack — the address of the instruction after the
call(which is0x0804841e) - Jumps to the target function (
0x080483c4)
This is equivalent to:
push 0x0804841e ; push return address
jmp 0x080483c4 ; jump to add()
Let’s step into the call and check:
(gdb) si
0x080483c4 in add ()
(gdb) x/8xw $esp
0xbffff7cc: 0x0804841e 0x00000003 0x00000005 0xbffff898
return addr arg1=3 arg2=5
There it is — 0x0804841e (the return address) was pushed onto the stack by call.
The Stack After CALL
┌──────────────────────────┐
│ ...main's stack frame...│
│ │
│ 0x00000005 (arg2) │ ESP+C
│ 0x00000003 (arg1) │ ESP+8
│ 0x0804841e (ret addr) │ ◄── ESP (pushed by CALL)
└──────────────────────────┘
This return address is what makes ret work later — and it’s what buffer overflow exploits overwrite.
Stage 3: The Function Prologue
0x080483c4 <+0>: push ebp
0x080483c5 <+1>: mov ebp,esp
0x080483c7 <+3>: sub esp,0x10
We’ve seen this in every function, but now let’s understand why each instruction exists.
push ebp — Save the caller’s base pointer. main was using EBP to access its own local variables. If add overwrites EBP, main won’t be able to find its variables when add returns. So we save it.
(gdb) ni
0x080483c5 in add ()
(gdb) x/8xw $esp
0xbffff7c8: 0xbffff7e8 0x0804841e 0x00000003 0x00000005
saved EBP return addr arg1=3 arg2=5
0xbffff7e8 is main’s EBP value — now safely saved on the stack.
mov ebp, esp — Set up the new base pointer for add(). From now on, EBP is a fixed reference point. No matter how ESP changes (due to local allocations or pushes), EBP stays constant.
(gdb) ni
0x080483c7 in add ()
(gdb) i r ebp esp
ebp 0xbffff7c8 0xbffff7c8
esp 0xbffff7c8 0xbffff7c8
EBP and ESP now point to the same place — the top of the new stack frame.
sub esp, 0x10 — Allocate 16 bytes for local variables. ESP moves down, creating space.
(gdb) ni
0x080483ca in add ()
(gdb) i r ebp esp
ebp 0xbffff7c8 0xbffff7c8
esp 0xbffff7b8 0xbffff7b8
The Stack After the Prologue
┌──────────────────────────┐
│ ...main's stack frame...│
│ │
│ 0x00000005 (arg2) │ EBP+0xC
│ 0x00000003 (arg1) │ EBP+0x8
│ 0x0804841e (ret addr) │ EBP+0x4
│ 0xbffff7e8 (saved EBP) │ ◄── EBP points here
├──────────────────────────┤
│ [ebp-0x4] local var │
│ [ebp-0x8] │
│ [ebp-0xc] │
│ [ebp-0x10] │ ◄── ESP
└──────────────────────────┘
This is the key diagram. Notice the pattern:
- Arguments are at positive offsets from EBP:
[ebp+0x8],[ebp+0xc], etc. - Local variables are at negative offsets from EBP:
[ebp-0x4],[ebp-0x8], etc. - Saved EBP is at
[ebp+0x0] - Return address is at
[ebp+0x4]
This is why EBP matters — it gives us a stable reference point. Arguments are above it (pushed by the caller before the call), local variables are below it (allocated by the callee after the prologue).
Stage 4: Accessing Arguments
0x080483ca <+6>: mov eax,DWORD PTR [ebp+0x8]
0x080483cd <+9>: mov edx,DWORD PTR [ebp+0xc]
Now we see why arguments are at positive offsets. [ebp+0x8] is the first argument (3), and [ebp+0xc] is the second argument (5). The +0x8 skip accounts for the saved EBP (4 bytes at [ebp+0x0]) and the return address (4 bytes at [ebp+0x4]).
(gdb) x/d $ebp+0x8
0xbffff7d0: 3
(gdb) x/d $ebp+0xc
0xbffff7d4: 5
The arguments are right where we expected them.
Stage 5: The Function Body
0x080483d0 <+12>: lea eax,[edx+eax*1]
0x080483d3 <+15>: mov DWORD PTR [ebp-0x4],eax
0x080483d6 <+18>: mov eax,DWORD PTR [ebp-0x4]
This adds the two arguments (3 + 5 = 8), stores the result in the local variable at [ebp-0x4], then loads it back into EAX.
Why store to a local and immediately load it back? The original C code likely has int result = a + b; return result;. The compiler stored the result in result (on the stack), then loaded it into EAX to return it. With optimizations, this would collapse into a single lea eax, [edx+eax].
The return value is in EAX — that’s the cdecl convention. The caller will read EAX to get the result.
Stage 6: The Function Epilogue
0x080483d9 <+21>: leave
0x080483da <+22>: ret
leave is shorthand for two instructions:
mov esp, ebp ; Deallocate local variables (ESP = EBP, discarding everything below)
pop ebp ; Restore the caller's EBP from the stack
After leave:
(gdb) ni
0x080483da in add ()
(gdb) i r ebp esp
ebp 0xbffff7e8 0xbffff7e8 ← main's EBP restored!
esp 0xbffff7cc 0xbffff7cc
(gdb) x/4xw $esp
0xbffff7cc: 0x0804841e 0x00000003 0x00000005 0xbffff898
return addr arg1 arg2
EBP is restored to main’s value. ESP points to the return address — exactly where it needs to be for ret.
ret pops the return address from the stack and jumps to it:
pop eip ; EIP = 0x0804841e (the instruction after the call in main)
(gdb) ni
0x0804841e in main ()
(gdb) i r eax
eax 0x8 8
We’re back in main, and EAX holds 8 — the return value from add(3, 5).
Stage 7: Back in the Caller
0x0804841e <+26>: mov DWORD PTR [ebp-0x4],eax
main stores the return value (EAX = 8) into its own local variable at [ebp-0x4]. This is int result = add(3, 5);.
The rest of main passes this value to printf:
(gdb) x/s 0x80484e0
0x80484e0: "Result: %d\n"
The Reconstructed Code
#include <stdio.h>
int add(int a, int b) {
int result = a + b;
return result;
}
int main() {
int result = add(3, 5);
printf("Result: %d\n", result);
return 0;
}
Nested Function Calls
What happens when functions call other functions? Each call creates a new stack frame, and they chain together like links in a chain.
Consider: main calls foo, and foo calls bar.
┌──────────────────────────┐
│ main's stack frame │
│ ┌────────────────────┐ │
│ │ local variables │ │
│ │ saved EBP ─────────┼──┼──► (previous frame)
│ │ return addr │ │
│ └────────────────────┘ │
│ foo's arguments │
├──────────────────────────┤
│ foo's stack frame │
│ ┌────────────────────┐ │
│ │ local variables │ │
│ │ saved EBP ─────────┼──┼──► main's EBP
│ │ return addr ───────┼──┼──► back to main
│ └────────────────────┘ │
│ bar's arguments │
├──────────────────────────┤
│ bar's stack frame │
│ ┌────────────────────┐ │
│ │ local variables │ │
│ │ saved EBP ─────────┼──┼──► foo's EBP
│ │ return addr ───────┼──┼──► back to foo
│ └────────────────────┘ │ ◄── ESP
└──────────────────────────┘
Each saved EBP points to the previous frame’s EBP, forming a linked list of stack frames. This is how GDB’s backtrace command works — it follows this chain to show you the call stack:
(gdb) bt
#0 bar () at prog.c:4
#1 0x08048420 in foo () at prog.c:9
#2 0x08048450 in main () at prog.c:14
GDB starts at the current EBP, reads the saved EBP to find the previous frame, reads its saved EBP for the one before that, and so on until it reaches the bottom of the stack. Each return address along the way tells GDB which function the frame belongs to.
The EBP Chain in GDB
You can walk the frame chain manually:
(gdb) i r ebp
ebp 0xbffff798 0xbffff798 ← bar's frame
(gdb) x/xw $ebp
0xbffff798: 0xbffff7b8 ← foo's EBP (saved by bar's prologue)
(gdb) x/xw 0xbffff7b8
0xbffff7b8: 0xbffff7e8 ← main's EBP (saved by foo's prologue)
(gdb) x/xw 0xbffff7e8
0xbffff7e8: 0x00000000 ← bottom of the chain (no previous frame)
And the return addresses are always at saved_ebp + 4:
(gdb) x/xw 0xbffff798+4
0xbffff79c: 0x08048420 ← return to foo
(gdb) x/xw 0xbffff7b8+4
0xbffff7bc: 0x08048450 ← return to main
Why Arguments Are Pushed Right to Left
Consider printf("x=%d y=%d", 10, 20). The arguments are pushed in reverse order:
push 20 ; third argument (rightmost)
push 10 ; second argument
push fmt_str ; first argument (leftmost) — pushed last, so it's on top
After pushing, the stack looks like:
ESP ──► fmt_str (first argument — always at [esp])
10 (second argument — at [esp+4])
20 (third argument — at [esp+8])
The first argument is always at the same position relative to ESP, regardless of how many arguments follow. This is critical for variadic functions like printf — the function reads the format string first, then uses it to determine how many more arguments to read from the stack.
If arguments were pushed left to right, the format string would be buried under a variable number of arguments, and printf wouldn’t know where to find it.
Why This Matters for Exploitation
Now that you understand the stack layout, the classic buffer overflow attack should make perfect sense.
Look at the stack inside a function:
┌──────────────────────────┐
│ function arguments │
│ RETURN ADDRESS │ ◄── TARGET
│ saved EBP │
│ ──────────────────── │
│ char buffer[64] │ ◄── overflow starts here
│ │
│ │ ◄── ESP
└──────────────────────────┘
If the function has a local char buffer[64] and uses an unsafe function like gets(buffer) or strcpy(buffer, input), an attacker can write more than 64 bytes. The extra bytes overflow past the buffer, past the saved EBP, and overwrite the return address.
When the function executes ret, it pops the attacker-controlled return address into EIP — and the CPU jumps wherever the attacker wants. If that address points to shellcode the attacker injected into the buffer, it’s game over.
Normal stack: After overflow:
┌──────────────┐ ┌──────────────────┐
│ ret: 0x0804XX │ (legit) │ ret: 0xBFFFF7D0 │ (attacker's address!)
│ saved EBP │ │ AAAA (overwritten)│
│ buffer[64] │ │ AAAAAAAAAAAAAAA..│
│ │ │ [shellcode here] │ ◄── 0xBFFFF7D0
└──────────────┘ └──────────────────┘
This is why stack canaries, ASLR, and DEP exist — to make this exact attack harder. But understanding the attack starts with understanding the stack.
Key Takeaways
Here’s a summary of everything we covered:
The function call lifecycle:
1. Caller pushes arguments (right to left)
2. CALL pushes return address, jumps to function
3. Callee prologue: push ebp / mov ebp,esp / sub esp,N
4. Function body executes (args at [ebp+8], [ebp+C], locals at [ebp-4], [ebp-8])
5. Return value goes in EAX
6. Callee epilogue: leave (mov esp,ebp + pop ebp) / ret
7. Execution returns to caller at the saved return address
The stack frame layout:
| Offset from EBP | Contents |
|---|---|
[ebp+0xC] |
Second argument |
[ebp+0x8] |
First argument |
[ebp+0x4] |
Return address (pushed by call) |
[ebp+0x0] |
Saved EBP (pushed by prologue) |
[ebp-0x4] |
First local variable |
[ebp-0x8] |
Second local variable |
| … | … |
Essential GDB commands for stack analysis:
x/20xw $esp — Examine 20 words at the stack pointer
x/xw $ebp — See the saved EBP (previous frame)
x/xw $ebp+4 — See the return address
x/xw $ebp+8 — See the first argument
bt — Full backtrace (walk the EBP chain)
info frame — Detailed info about the current frame
Final Thoughts
The stack is the backbone of program execution. Every function call, every local variable, every return — it all lives on the stack. And once you understand how it works at the assembly level, a huge number of things click into place:
- Why buffer overflows overwrite return addresses
- Why stack canaries are placed between local variables and the return address
- Why ASLR randomizes the stack location
- Why
leaveandretappear at the end of every function - Why GDB’s
btcommand can show you the call chain
This knowledge is fundamental to exploit development, reverse engineering, and understanding how computers actually run your code. Everything else builds on top of this.
Happy reversing!
