audit: More cross compile fixes

There was an issue that was not fixed by the last set of cross compile
patches where architecture specific CFLAGS were being used when building
for the host and not the target. This patch fixes that issue.

Fixes:
  http://autobuild.buildroot.net/results/3e85c2253f6bd4cfe6ac1dde947eb6d5afc78cfe/
  http://autobuild.buildroot.net/results/d0f0f7e7462d68331d4a2f87b1df05cc9a6fecfd/
  http://autobuild.buildroot.net/results/114de2422c56071141284fb2eb8044ffa48e77f4/
  http://autobuild.buildroot.net/results/0862cf008e7e4b52c46c40257adeda954afee394/



Signed-off-by: Clayton Shotwell <clayton.shotwell@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>