[Firmware] Building cross-compiler.sh natively on arm.

[Rob Landley]

So I'm testing the uClibc snapshot, and the most recent version builds for all 
the targets.  The next step is to run a smoke test script that fires up 
system-image for a target, does a "wget 
http://landley.net/hg/firmware/archive/tip.tar.gz", extracts it, 
runs ./download.sh, and then does "BUILD_STATIC=1 ./cross-compiler.sh" for 
the arch.

So I'm trying to do a native build of the cross compiler for the target, if 
that makes sense.  (Hey, easy smoke test of the build infrastructure.)

This works fine on i686, and goes "boing" on armv4l.  Sigh.  Ok, dig in.

Building binutils 2.17 on armv4l is breaking in libiberty/regex.c, saying that 
memcmp() and such are implicitly defined and then dying on improper use of 
them.  This is because "strings.h" and such are not being #included.  At 
first I was blaming the uClibc headers, but the problem is actually that 
those headers are only included if STDC_HEADERS is #defined, and it isn't.

  # grep STDC_HEADERS libiberty/config.h
  /* #undef STDC_HEADERS */

Yes, ./configure is checking whether or not it has standard header files, and 
is getting conflicting answers in different directories.  Sigh.

Now the _top_ level ./configure seems to be producing reasonable values.  It's 
the subdirectories that "make configure-host" populates that are screwing up.

At first I thought it might be a parallelism issue.  I'm doing "make -j $CPUS 
configure-host", and I tried taking that out.  No change.  However, the 
single processor logs are much easier to read, and they're saying:

  checking for ANSI C header files... yes

Meaning this test is _succeeding_, and config.h is winding up with the wrong 
value anyway.  Hmmm...  Looking at config.h with vi, it hasn't got _anything_ 
set.  Something's up.

The way configure works in the subdirectories is the main configure script 
runs 8 gazillion tests, then generates a second script called "config.status" 
which generates the half-dozen output files used by the actual build, 
including config.h.  If you cd libiberty and run ./config.status it 
rebuilds 'em all, and you can also "rm config.h; ./config.status config.h" to 
beat slightly more granular behavior out of the thing.  (Don't ask me why 
running it without arguments rebuilds them all, but running it on just one 
does dependency checking and won't rebuild the file if it exists.  Isn't that 
backwards?  Oh well, it's autoconf.  It is NOT SANE.)

After drilling through layers of horrible, horrible shell script, it turns out 
the core of what config.status _does_ is run a huge sed invocation 
against "binutils/libiberty/config.in" to produce config.h (and similar for 
the other files, but this is the one that I'm trying to debug).  The source 
file's in the binutils source directory.  (This package is one of those 
FSF "you must build out of tree" things so you have to look 
at ../../binutils/libiberty to see the files it's working on.  However, you 
must _be_ in build/libiberty to run the script.  If you just run 
libiberty/config.status from the top level build directory it'll complain it 
can't find its source files.  Sigh.)

The config.in file contains #undef lines for every possible config entry, and 
the sed invocation searches for each enabled one and changes it to a #define, 
and comments out the rest.  Hideously roundabout way to do it?  But of 
course!  This is <strike>sparta</strike> autoconf.

The easy way to figure out what it's feeding to sed is to replace sed with 
something that captures it.  So, I create the subdirectory "squish" and put a 
file "sed" in there:

  #!/bin/bash

  echo "sed $@" >> /home/walrus
  exec /tools/bin/sed "$@"

#chmod +x squish/sed
#rm config.h
#PATH=`pwd`/squish:$PATH ./config.status config.h

And then /home/walrus contains:

  sed h;s/./-/g;s/^.../## /;s/...$/ ##/;p;x;p;x
  sed 1q
  sed s,[^:]*:,,
  sed s,:.*,,
  sed 
s/[  ]*$// /home/firmware-763712cce21f/build/temp-armv4l/binutils/libiberty/
  sed -f ./confstatAK4AMo/undefs.sed ./confstatAK4AMo/in

That last line looks interesting, so let's tell the script to copy those files 
by adding this to squish/sed:

  if [ $# -eq 3 ]
  then
    cp $2 /home/two
    cp $3 /home/three
  fi

And now:
#cd /home
#sed -f two three | grep STDC
/* #undef STDC_HEADERS */

So we know it's reproducing the problem.  (Incidentally, "three" is identical 
to binutils/libiberty/config.in.  Why it had to copy it into a temp directory 
when it's used read only, I have no clue.)

So the next question is, will running that on x86 _not_ reproduce the problem?  
Or was the creation of "two" screwed up by an earlier step?  Looking 
at "two", it contains:

/^[      ]*#[    ]*undef/!b
t clr
: clr
s,^\([   ]*\)#\([        ]*\)undef\([    ][      ]*\)PACKAGE_NAME$,
\1#\2define\3PACKAGE_NAME "",;t
s,^[     ]*#[    ]*undef[        ][      ]*[a-zA-Z_][a-zA-Z_0-9]*,/* & */,

And if you'll notice, there are no interersting symbol names in there.  
Nothing to recognize STDC_HEADERS and switch it on instead of off.

At this point, I need to compare what's happening on a working system with 
what's happening on the broken system.  It's probably a busybox sed thing 
that's only happening on arm for some reason (Endianness?  Alignment?  
Dunno.)  But that's just a guess.  More debugging to come...

Rob

P.S.  The above took something like 4 hours to work out, and I am now _tired_.  
Wheee...