Stack Variable Layout & Pointer Danger

Last evening, I was trying to think of a good interview question that tests pointer arithmetic. As I'm a compiler developer, I'm familiar with the various ways pointers are abused. Often, the compiler works hard to meet the developer's expectations even if it means not conforming to the standard. Consider this C program:

#include <stdio.h>

void dangerous()
{
   int cnt;
   int stack_0 = 0, stack_1 = 1, stack_2 = 2, stack_3 = 3;
   int *ptr = &stack_0;

   for (cnt = 0; cnt < 4; cnt++)
   {
      printf(" %i", (*ptr));
      ptr++;
   }
}

void safe()
{
   int cnt;
   int stack[4] = { 0, 1, 2, 3 };
   int *ptr = stack;

   for (cnt = 0; cnt < 4; cnt++)
   {
      printf(" %i", (*ptr));
      ptr++;
   }
}

int main()
{
   dangerous();
   printf("\n");
   safe();
   return 0;
}

Now, with the Microsoft Visual C compiler version 16.10.30716.3, this program will have different behavior with different optimization levels:

cl.exe layout.c
Microsoft (R) C/C++ Optimizing Compiler Version 16.10.30716.03 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

layout.c
Microsoft (R) Incremental Linker Version 10.10.30716.03
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:layout.exe
layout.obj

layout.exe
 0 1 2 3
 0 1 2 3

cl.exe /O2 layout.c
Microsoft (R) C/C++ Optimizing Compiler Version 16.10.30716.03 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

layout.c
Microsoft (R) Incremental Linker Version 10.10.30716.03
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:layout.exe
layout.obj

layout.exe
 0 0 0 0
 0 1 2 3

The interview question is to explain: What's going on here?

The assembly of the 'dangerous' and 'safe' functions under /Od (that is, debug mode, no compiler optimizations) is nearly identical. Each function has 0, 1, 2, 3 laid out neatly on its stack frame in sequential order. Because of this, the programs behave identically. But what happens with compiler optimizations enabled (/O2 compilation)? Well, if we turn on warnings, we get a hint:

layout.c(7) : warning C4189: 'stack_2' : local variable is initialized but not referenced
layout.c(7) : warning C4189: 'stack_3' : local variable is initialized but not referenced
layout.c(7) : warning C4189: 'stack_1' : local variable is initialized but not referenced

From the perspective of the compiler, the 'stack_1', 'stack_2', and 'stack_3' initializations are dead code. Because there exist no references, they can be thrown away. And that's exactly what happens. Beside the first '0' printed by the optimized 'dangerous' function, the other values are garbage. Depending on what values are on the stack, they could be anything.