Permalink
Browse files

AVRO-551. C: Compiles on Win32 via VC++ 2008.

Patch submitted by Vivek Nadkarni.  Includes a README file describing the
changes that were necessary, and what we'll need to do to maintain compatibility
on Windows moving forward.


git-svn-id: https://svn.apache.org/repos/asf/avro/trunk@1331812 13f79535-47bb-0310-9956-ffa450edef68
  • Loading branch information...
1 parent 64664a6 commit 52ea2d88416fbf964cfd6bf1c6c959596ed1063e @dcreager dcreager committed Apr 28, 2012
Showing with 2,372 additions and 951 deletions.
  1. +3 −0 CHANGES.txt
  2. +59 −0 LICENSE.txt
  3. +57 −37 lang/c/CMakeLists.txt
  4. +154 −0 lang/c/README.maintaining_win32.txt
  5. +8 −1 lang/c/examples/CMakeLists.txt
  6. +1 −3 lang/c/examples/quickstop.c
  7. +3 −3 lang/c/jansson/src/dump.c
  8. +4 −4 lang/c/jansson/src/hashtable.c
  9. +7 −0 lang/c/jansson/src/hashtable.h
  10. +11 −0 lang/c/jansson/src/jansson_private.h
  11. +6 −2 lang/c/jansson/src/load.c
  12. +1 −1 lang/c/jansson/src/memory.c
  13. +2 −2 lang/c/jansson/src/strbuffer.c
  14. +7 −0 lang/c/jansson/src/strbuffer.h
  15. +8 −1 lang/c/jansson/src/utf.h
  16. +11 −11 lang/c/jansson/src/value.c
  17. +17 −1 lang/c/src/CMakeLists.txt
  18. +4 −3 lang/c/src/allocation.c
  19. +3 −3 lang/c/src/array.c
  20. +1 −0 lang/c/src/avro.h
  21. +8 −0 lang/c/src/avro/allocation.h
  22. +1 −1 lang/c/src/avro/consumer.h
  23. +9 −1 lang/c/src/avro/data.h
  24. +7 −0 lang/c/src/avro/errors.h
  25. +1 −1 lang/c/src/avro/generic.h
  26. +1 −1 lang/c/src/avro/io.h
  27. +1 −1 lang/c/src/avro/legacy.h
  28. +315 −0 lang/c/src/avro/msinttypes.h
  29. +247 −0 lang/c/src/avro/msstdint.h
  30. +45 −0 lang/c/src/avro/platform.h
  31. +43 −8 lang/c/src/avro/refcount.h
  32. +1 −1 lang/c/src/avro/schema.h
  33. +1 −1 lang/c/src/avro/value.h
  34. +34 −1 lang/c/src/avro_private.h
  35. +11 −12 lang/c/src/avropipe.c
  36. +9 −0 lang/c/src/codec.h
  37. +1 −1 lang/c/src/consume-binary.c
  38. +30 −17 lang/c/src/datafile.c
  39. +25 −25 lang/c/src/datum.c
  40. +1 −1 lang/c/src/datum.h
  41. +2 −2 lang/c/src/datum_equal.c
  42. +1 −1 lang/c/src/datum_size.c
  43. +1 −1 lang/c/src/datum_validate.c
  44. +34 −22 lang/c/src/datum_value.c
  45. +2 −3 lang/c/src/dump.c
  46. +8 −2 lang/c/src/dump.h
  47. +9 −2 lang/c/src/encoding.h
  48. +47 −44 lang/c/src/encoding_binary.c
  49. +134 −134 lang/c/src/generic.c
  50. +13 −13 lang/c/src/io.c
  51. +13 −13 lang/c/src/map.c
  52. +10 −10 lang/c/src/memoize.c
  53. +171 −172 lang/c/src/resolved-reader.c
  54. +182 −169 lang/c/src/resolved-writer.c
  55. +38 −38 lang/c/src/resolver.c
  56. +24 −16 lang/c/src/schema.c
  57. +1 −1 lang/c/src/schema.h
  58. +29 −63 lang/c/src/st.c
  59. +24 −10 lang/c/src/st.h
  60. +16 −16 lang/c/src/string.c
  61. +1 −1 lang/c/src/value-hash.c
  62. +5 −5 lang/c/src/value-json.c
  63. +4 −4 lang/c/src/value-read.c
  64. +2 −2 lang/c/src/value-sizeof.c
  65. +2 −2 lang/c/src/value-write.c
  66. +7 −7 lang/c/src/wrapped-buffer.c
  67. +7 −1 lang/c/tests/CMakeLists.txt
  68. +0 −1 lang/c/tests/generate_interop_data.c
  69. +372 −0 lang/c/tests/msdirent.h
  70. +5 −6 lang/c/tests/test_avro_1034.c
  71. +4 −6 lang/c/tests/test_avro_984.c
  72. +3 −4 lang/c/tests/test_avro_data.c
  73. +10 −1 lang/c/tests/test_avro_schema.c
  74. +15 −14 lang/c/tests/test_avro_values.c
  75. +18 −18 lang/c/tests/test_data_structures.c
  76. +0 −1 lang/c/tests/test_interop_data.c
  77. +0 −2 lang/c/tests/test_refcount.c
View
@@ -22,6 +22,9 @@ Avro 1.7.0 (unreleased)
AVRO-1068. Avro Java does not build on recent Eclipse versions with m2eclipse (thiru)
+ AVRO-551. C: Now compiles on Win32, using Visual Studio C++ 2008.
+ (Vivek Nadkarni via dcreager)
+
BUG FIXES
AVRO-1045. Java: Fix a bug in GenericData#deepCopy() of ByteBuffer values.
View
@@ -247,3 +247,62 @@ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+----------------------------------------------------------------------
+License for msinttypes used in the C implementation:
+Source from:
+http://code.google.com/p/msinttypes/downloads/detail?name=msinttypes-r26.zip
+
+Copyright (c) 2006-2008 Alexander Chemeris
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+ 1. Redistributions of source code must retain the above copyright notice,
+ this list of conditions and the following disclaimer.
+
+ 2. Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in the
+ documentation and/or other materials provided with the distribution.
+
+ 3. The name of the author may be used to endorse or promote products
+ derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED
+WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
+EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
+OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+----------------------------------------------------------------------
+License for Dirent API for Microsoft Visual Studio used in the C implementation:
+Source from:
+http://www.softagalleria.net/download/dirent/dirent-1.11.zip
+
+Copyright (C) 2006 Toni Ronkko
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+``Software''), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be included
+in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS
+OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL TONI RONKKO BE LIABLE FOR ANY CLAIM, DAMAGES OR
+OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.
+
+----------------------------------------------------------------------
View
@@ -27,47 +27,56 @@ cmake_policy(SET CMP0003 NEW)
#-----------------------------------------------------------------------
# Retrieve the current version number
-execute_process(COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/version.sh project
- WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
- RESULT_VARIABLE AVRO_VERSION_RESULT
- OUTPUT_VARIABLE AVRO_VERSION
- OUTPUT_STRIP_TRAILING_WHITESPACE)
-if(AVRO_VERSION_RESULT)
- message(FATAL_ERROR "Cannot determine Avro version number")
-endif(AVRO_VERSION_RESULT)
-
-execute_process(COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/version.sh libtool
- WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
- RESULT_VARIABLE LIBAVRO_VERSION_RESULT
- OUTPUT_VARIABLE LIBAVRO_VERSION
- OUTPUT_STRIP_TRAILING_WHITESPACE)
-if(LIBAVRO_VERSION_RESULT)
- message(FATAL_ERROR "Cannot determine libavro version number")
-endif(LIBAVRO_VERSION_RESULT)
+if (UNIX)
+ execute_process(COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/version.sh project
+ WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
+ RESULT_VARIABLE AVRO_VERSION_RESULT
+ OUTPUT_VARIABLE AVRO_VERSION
+ OUTPUT_STRIP_TRAILING_WHITESPACE)
+ if(AVRO_VERSION_RESULT)
+ message(FATAL_ERROR "Cannot determine Avro version number")
+ endif(AVRO_VERSION_RESULT)
+
+ execute_process(COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/version.sh libtool
+ WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
+ RESULT_VARIABLE LIBAVRO_VERSION_RESULT
+ OUTPUT_VARIABLE LIBAVRO_VERSION
+ OUTPUT_STRIP_TRAILING_WHITESPACE)
+ if(LIBAVRO_VERSION_RESULT)
+ message(FATAL_ERROR "Cannot determine libavro version number")
+ endif(LIBAVRO_VERSION_RESULT)
+else(UNIX)
+ # Hard code for win32 -- need to figure out how to port version.sh for
+ # Windows.
+ set(LIBAVRO_VERSION "22:0:0")
+endif(UNIX)
+
#-----------------------------------------------------------------------
# Extract major.minor.patch from version number
-string(REGEX REPLACE "([0-9]+)\\..*"
- "\\1"
- AVRO_MAJOR_VERSION
- ${AVRO_VERSION}
-)
-string(REGEX REPLACE ".*\\.([0-9]+)\\..*"
- "\\1"
- AVRO_MINOR_VERSION
- ${AVRO_VERSION}
-)
-string(REGEX REPLACE ".*\\..*\\.([0-9]+).*"
- "\\1"
- AVRO_PATCH_VERSION
- ${AVRO_VERSION}
-)
-string(REGEX REPLACE ".*\\..*\\.[0-9]+(.*)"
- "\\1"
- AVRO_VERSION_EXTENSION
- ${AVRO_VERSION}
-)
+if (UNIX)
+ string(REGEX REPLACE "([0-9]+)\\..*"
+ "\\1"
+ AVRO_MAJOR_VERSION
+ ${AVRO_VERSION}
+ )
+ string(REGEX REPLACE ".*\\.([0-9]+)\\..*"
+ "\\1"
+ AVRO_MINOR_VERSION
+ ${AVRO_VERSION}
+ )
+ string(REGEX REPLACE ".*\\..*\\.([0-9]+).*"
+ "\\1"
+ AVRO_PATCH_VERSION
+ ${AVRO_VERSION}
+ )
+ string(REGEX REPLACE ".*\\..*\\.[0-9]+(.*)"
+ "\\1"
+ AVRO_VERSION_EXTENSION
+ ${AVRO_VERSION}
+ )
+endif(UNIX)
#-----------------------------------------------------------------------
# Source package support
@@ -89,6 +98,17 @@ if(CMAKE_COMPILER_IS_GNUCC)
add_definitions(-W -Wall)
endif(CMAKE_COMPILER_IS_GNUCC)
+if (WIN32)
+ # Compile win32 in C++ to allow declarations after statements
+ add_definitions(/TP)
+endif(WIN32)
+
+# Uncomment to allow missing fields in the resolved-writer
+# add_definitions(-DAVRO_ALLOW_MISSING_FIELDS_IN_RESOLVED_WRITER)
+
+# Uncomment to allow non-atomic increment/decrement of reference count
+# add_definitions(-DAVRO_ALLOW_NON_ATOMIC_REFCOUNT)
+
include_directories(${AvroC_SOURCE_DIR}/src)
include_directories(${AvroC_SOURCE_DIR}/jansson/src)
@@ -0,0 +1,154 @@
+Win32 C++ builds of Avro-C
+*****************************
+
+April 2, 2012
+
+These instructions describe some of the changes required to allow
+Avro-C to compile under the Microsoft Visual C++ 2008 compiler, as
+well as some limitations of the Windows build.
+
+Status of the Windows Build:
+****************************
+
+The Windows build of Avro-C compiles under Microsoft Visual C++
+2008. You can use C-Make to create the solution file (AvroC.sln) for
+the build. The solution file contains projects for the build as well
+as projects for the tests. All the tests are run and pass when the
+project RUN_TESTS is built from within MS Visual C++ 2008.
+
+Limitations of Windows Build:
+******************************
+
+1. The Windows build of Avro-C is compiled using Microsoft's C++
+ compiler and not Microsoft's C compiler. This is done using the /TP
+ flag in the MSVC++ compiler. This flag is automatically set using
+ CMakeLists.txt
+
+ The reason to compile Win32 under C++ instead of C is that there
+ are lots of places where variables are declared after statements,
+ the Microsoft C compiler does not support declarations after
+ statements. It is possible, that if all the declarations after
+ statements were removed, that Avro-C would compile under
+ Microsoft's C compiler also. I have not tried this.
+
+2. The shared library, i.e. DLL, for avro has not been built. There
+ are instructions on how to build DLLs using CMake at
+ http://www.cmake.org/Wiki/BuildingWinDLL
+
+3. Currently avropipe.c and avromod.c do not compile under Windows. In
+ order for them to compile we would have to either port getopt() to
+ Windows, or remove their dependency on getopt().
+
+4. Windows cannot run version.sh to generate the version
+ number. Currently, LIBAVRO_VERSION is hard coded to "22:0:0" for
+ the Windows build, in the top level CMakeLists.txt.
+
+5. Found two bugs related to improper return values under error
+ conditions in Avro. These bugs were marked with #warnings in the
+ code.
+
+
+Instructions for Maintenance
+*****************************
+
+1. Instructions to check name mangling in Visual C++:
+
+ In a DOS prompt go to "C:\Program Files(x86)\Microsoft Visual Studio 9.0\VC\"
+ Run the program vcvarsall.bat . This will set up environment variables.
+
+ Now go to the avro_c\build_win32\src\Debug\ directory.
+ Run the command
+
+ dumpbin /ALL avro.lib > tmp.txt
+
+ View tmp.txt in your favorite editor. This will allow you to see
+ which names are mangled and which names are not mangled.
+
+ Every header file should start with
+
+ #ifndef HEADER_FILE_NAME_H
+ #define HEADER_FILE_NAME_H
+ #ifdef __cplusplus
+ extern "C" {
+ #define CLOSE_EXTERN }
+ #else
+ #define CLOSE_EXTERN
+ #endif
+
+ and end with
+
+ CLOSE_EXTERN
+ #endif /* HEADER_FILE_NAME_H */
+
+ This will ensure that all exported (public) functions are mangled
+ using C name mangling instead of C++ name mangling.
+
+2. All file I/O operations should have "b" for binary in the fopen
+ statements. Otherwise Windows will replace LF with CRLF in binary
+ data.
+
+3. Windows does not allow writing to a file with exclusive access
+ using the mode "wbx". Use the non-exclusive mode "wb" instead.
+
+4. If the hashtable from st.c is used, the functions in the struct
+ st_hash_type should be cast to HASH_FUNCTION_CAST.
+
+5. Do not use "%zu" to print size_t. Use '"%" PRIsz' without the
+ single quotes, instead.
+
+6. MS Visual C++ 2008 does not properly expand variadic preprocessor
+ macros by default. It is possible to "trick" MSVC++ to properly
+ expand variadic preprocessor macros by using an extra (dummy)
+ preprocessor macro, whose only purpose is to properly expand its
+ input. This method is described here:
+
+ http://stackoverflow.com/questions/2575864/the-problem-about-different-treatment-to-va-args-when-using-vs-2008-and-gcc
+ See the solution described by monkeyman.
+
+ This method is used in the macro expand_args(...) in test_avro_values.c.
+
+7. Web site describing how to debug macro expansion in Visual C++:
+ http://fneep.fiffa.net/?p=66
+
+8. Sometimes it is necessary to declare a struct at the top of a file
+ and define it at the bottom of a file. An example is
+ AVRO_DATUM_VALUE_CLASS in src/datum_value.c. A C++ compiler will
+ complain that the struct is defined twice. To avoid this, declare
+ the struct with the modifier "extern" at the top of the file, and
+ then define it at the bottom of the file. Note that it cannot be
+ defined as "static" because Win32 does not like an extern struct
+ mapping to a static struct.
+
+9. Use __FUNCTION__ instead of __func__ for generating the function
+ name.
+
+10. All variables need to be explicitly cast when calling functions
+ with differing function signatures
+
+11. All pointers need to be explicitly cast when assigning to other
+ pointers of a different type.
+
+12. Do not perform pointer arithmetic on void * pointers. Cast the
+ pointers to char * before performing pointer arithmetic.
+
+13. The executable names of any examples and tests need to be set
+ explicitly to include the "Debug/" directory in the path, and the
+ ".exe" ending. See the CMakeLists.txt in the examples and the
+ tests directory to see how this is done.
+
+14. Do not include the headers inttypes.h or unistd.h or
+ stdint.h. Instead include avro/platform.h in your C files.
+
+15. Do not include dirent.h in your tests. When _WIN32 is defined
+ include msdirent.h. See example in test_avro_schema.c.
+
+16. If _WIN32 is defined, define snprintf() to_snprintf(), which MS
+ Visual C++ recognizes. See jansson_private.h for an example.
+
+17. MSVC++ does not recognize strtoll(). Define it to _strtoi64()
+ instead. See jansson/src/load.c for an example.
+
+18. Old-style C function declarations are not allowed in C++. See the
+ changes in st.c and st.h -- which were converted to new-style
+ function declarations.
+
@@ -20,6 +20,13 @@
add_executable(quickstop quickstop.c)
target_link_libraries(quickstop avro-static)
+if (WIN32)
+ set(exec_name ${CMAKE_CURRENT_BINARY_DIR}/Debug/quickstop.exe)
+else (WIN32)
+ set(exec_name ${CMAKE_CURRENT_BINARY_DIR}/quickstop)
+endif (WIN32)
+
add_test(quickstop
${CMAKE_COMMAND} -E chdir ${AvroC_SOURCE_DIR}/examples
- ${CMAKE_CURRENT_BINARY_DIR}/quickstop)
+ ${exec_name}
+)
@@ -16,10 +16,8 @@
*/
#include <avro.h>
-#include <inttypes.h>
#include <stdio.h>
#include <stdlib.h>
-#include <unistd.h>
#ifdef DEFLATE_CODEC
#define QUICKSTOP_CODEC "deflate"
@@ -144,7 +142,7 @@ int main(void)
init_schema();
/* Delete the database if it exists */
- unlink(dbname);
+ remove(dbname);
/* Create a new database */
rval = avro_file_writer_create_with_codec
(dbname, person_schema, &db, QUICKSTOP_CODEC, 0);
Oops, something went wrong.

0 comments on commit 52ea2d8

Please sign in to comment.