__cdecl

The __cdecl calling convention is the default calling convention in C/C++. In this calling convention:

• The arguments are pushed from Right to Left (so that the first argument is nearest to the top-of-stack).
• Caller cleans the stack.
• Creates larger executables than __stdcall, because it requires each function call to include stack cleanup code.

As the caller cleans the stack in this convention, we can provide variable arguments to the functions with the __cdecl calling convention.

Compiling the above program in Debug mode using assembly code, we get:

main()::result = cdeclAdd(1, 2);

Note: If you use Release Mode, you might not see the variables passing onto the stack like this as the compiler will produce optimized code.

The function with calling convention __cdecl is cdeclAdd() which is called on line 42 in the main function.

cdeclAdd()

Explanation: When we look into the assembly code of cdeclAdd(), the last statement is: ret 0. This means that the callee will do nothing to the stack pointer and the control returns to the main().

If we look at the assembly code in main(), after the function call, there is another statement: add esp, 8. When the control is returned to the main, it will add in stack pointer(esp) 8 bytes. Adding in memory means that we are popping from the stack or clearing up the stack. If we subtract from esp, this means we are pushing to the stack. The stack grows in reverse order with respect to memory.

Note: Here we are incrementing 8 bytes in esp because we are popping 2 integer variables passed. Since each integer variable in the 32-bit arch is 4 bytes, we are clearing 8-byte memory for both variables.

This means that the caller (i.e. main) will clear the stack and not the callee (i.e. cdeclAdd).

In C/C++ programming, a calling convention is a set of rules that specify how a function will be called. You might have seen keywords like __cdecl or __stdcall when you get linking errors. For example:

error LNK2019: unresolved external symbol "void __cdecl A(void)" (?A@@YAXXZ) referenced in function _main

Here, we see __cdecl being used. It is one of the calling conventions in C/C++. You can also see code like this in 3rd party libraries.

For example:

extern __m128i __cdecl _mm256_mask_cvtepi32_epi16(__m128i, __mmask8, __m256i);

What are caller and callee?

When we call a function, a stack frame is allocated for that function, arguments are passed to the function, and after the function does its work, the allocated stack frame is deallocated and control is passed to the calling function.

The function that calls the subroutine is called the caller. The function that gets called(i.e. subroutine) by the caller is called the callee.

In the above code, main() is the caller and func() is the callee.