Igor’s Tip of the Week #148: Fixing “call analysis failed”

This error is not very common but may appear in some situations.

804D5DD: call analysis failed

Such errors happen when there is a function call in the code, but the decompiler fails to convert it to a high-level function call, e.g.:

  1. the target function’s prototype is missing or invalid;
  2. the decompiler failed to figure out the function arguments: how many of them, or how exactly they’re being passed to the callee;
  3. the usage of the stack by the call does not make sense.

Let’s look at some examples where it happens and how to fix it.

Wrong function info

The first action on seeing the error should be to inspect the address mentioned and the surrounding code. For example, here’s the snippet around the address in the first screenshot:

.text:0804D5CD                 push    [ebp+var_10]
.text:0804D5D0                 push    offset sub_804D6E8
.text:0804D5D5                 push    [ebp+var_28]
.text:0804D5D8                 push    offset sub_804CF24 ; oset
.text:0804D5DD                 call    sub_8058FF0
.text:0804D5E2                 mov     edx, [ebp+var_14]
.text:0804D5E5                 or      dword ptr [edx+28h], 10h
.text:0804D5E9                 mov     eax, [ebp+var_18]
.text:0804D5EC                 add     esp, 10h
.text:0804D5EF                 test    eax, eax
.text:0804D5F1                 jz      loc_804D1D3
.text:0804D5F7                 sub     esp, 0Ch
.text:0804D5FA                 push    [ebp+var_18]
.text:0804D5FD                 call    sub_8055A0C

At the first glance, there doesn’t seem to be anything unusual: four arguments are pushed on the stack before calling the function sub_8058FF0. However, if we go inside the function and try to decompile it, we get another error:

8058FF0: function frame is wrong

Also, the header of the function looks strange:

.text:08058FF0 ; =============== S U B R O U T I N E =======================================
.text:08058FF0 ; Attributes: bp-based frame
.text:08058FF0 ; int __cdecl sub_8058FF0(sigset_t oset)
.text:08058FF0 sub_8058FF0     proc near               ; CODE XREF: sub_804CF6C+671↑p
.text:08058FF0                                         ; sub_804F798+126↑p ...
.text:08058FF0 var_48          = dword ptr -48h
.text:08058FF0 oset            = sigset_t ptr -38h

I.e. the function was detected not to take four arguments, but one structure by value. While this can indeed happen in some cases, the argument is in a wrong location: the local variables area (note the negative offset). 

Fixing the function itself is a topic for another post, but a quick fix for the original issue would be to delete the current prototype and let the decompiler fall back to guessing the arguments. For this, put the cursor on the function name or its first line, then press Y (edit type), Del, Enter. This will clear the wrong prototype and decompilation should succeed, showing the four arguments we’ve seen in the disassembly:

snippet of pseudocode with the call:
sub_8058FF0(sub_804CF24, v23, sub_804D6E8, a1);

Sometimes the decompiler’s guessing of the prototype still fails, so try to specify one based on the actual arguments being passed to the call (look at the assembly around the call). In some cases this may require the __usercall calling convention.

Indirect calls

Instead of the direct function address, indirect calls use a register or a memory location which holds the destination address to perform the call. For example, on x86 it may look like one of the following:

call eax 
call dword ptr [edx+14h] 
call [ebp+arg_0] 
call g_myfuncptr

In rare cases, the decompiler may fail to detect the actual arguments being passed to the call, especially if optimizer interleaves arguments passed to different calls. In that case, you can give it a hint by adding a cross-reference to the actual function being called (if you know it), or a function of the matching type, for example using the Set callee address feature. You should also check that the stack pointer is properly balanced before and after each call for stack-using calling conventions.


See also:

Igor’s tip of the week #27: Fixing the stack pointer

Decompiler Manual: Failures and troubleshooting

Article Link: Igor’s Tip of the Week #148: Fixing “call analysis failed” – Hex Rays