The system provided Perl and Ruby libraries adds -arch flags since they were built as universal binaries. This causes Vim to build with these flags which is undesirable (the default should be to only build for the native architecture). The solution is to strip any -arch flags from Perl and Ruby. A universal binary can be built by using the --with-archs configure flag.
The supporting files for CTGradient and RBSplitView causes several warnings when compiling for 64 bit. Since nobody uses MacVim plugins I have decided to disable them instead of having to support these files (although these warnings are easy enough to fix for anybody interested).