Skip to content

#268: update json-c from 0.12 to 0.13.1 #269

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 23, 2018
Merged

#268: update json-c from 0.12 to 0.13.1 #269

merged 2 commits into from
Aug 23, 2018

Conversation

patrikjuvonen
Copy link
Contributor

@patrikjuvonen patrikjuvonen commented Jul 27, 2018

GitHub issue:
Resolves #268

Summary:

  • Contains performance improvements and bug fixes
  • Not many changes required to our code - the following changes have been made:
    1. Removal in our code: is_error() -> a simple NULL check is the way to go now
    2. Change in our code: "Use size_t for array length and size." -> we need to change any affected parts to use unsigned ints instead of signed ints to avoid compiler warnings. In our case this only affects a couple for loops int -> uint.
    3. Change in json-c code (double precision): json-c has changed it a few times, by default it is at %.17g but we've had it at %.16g so it has to be edited (due to backwards compatibility reasons) in the source code at json_object.c:808 (issue: https://bugs.mtasa.com/view.php?id=8853) (commit: 4cc6fda)
    4. Change in json-c code (pretty mode improvements): See @qaisjp's commit here: e5b1c3f
  • Couldn't find any other commits related to json-c that we should take into account: https://github.com/multitheftauto/mtasa-blue/commits/master/vendor/json-c
  • It is good to update while we select and upgrade out of json-c to some other json library. The upgrade is unlikely to happen before 1.6, so it doesn't hurt to update now for whatever it's worth.
  • Changelog: https://github.com/json-c/json-c/blob/master/ChangeLog
  • Releases: 0.12.1, 0.13, 0.13.1

Performance test:

function doToJSONAndFromJSONTestAndPushDurationToResults(amount, times, results)
	if (not results) then
		amount = amount or 25e4
		times = times or 20
		results = {}

		print(("="):rep(62))
		print("Running toJSON + fromJSON test [" .. amount .. "] x [" .. times .. "] times.")
		print(("="):rep(62))
	end

	local startTickCount = getTickCount()
	for i = 1, amount do
		local json = toJSON({
			255,
			255,
			255,
			{
				id = 128,
				name = "test name goes here",
				available = false
			},
			nil,
			192.000000001,
			127.000000001,
			false,
			true,
			"Some text over here",
			{ 1, 2, true }
		})
		local tb = fromJSON(json)
	end
	local endTickCount = getTickCount() - startTickCount

	results[#results + 1] = endTickCount
	print("Test: [" .. #results .. "/" .. times .. "]; Duration: [" .. endTickCount .. "ms]")

	if (#results == times) then
		local totalDuration = 0

		for _, duration in pairs(results) do
			totalDuration = totalDuration + duration
		end

		print(("="):rep(62))
		print("Total duration: [" .. totalDuration .. "ms]")
		print("Average duration: [" .. totalDuration / #results .. "ms]")
		print("Tests completed: [" .. #results .. "/" .. times .. "]")
		print(("="):rep(62))
	else
		testTimer = setTimer(doToJSONAndFromJSONTestAndPushDurationToResults, 50, 1, amount, times, results)
	end
end
setTimer(doToJSONAndFromJSONTestAndPushDurationToResults, 500, 1)

Results (250,000 toJSON + fromJSON executions, 20 times):

  • json-c 0.12 (MTA:SA r13816 client-side): 194195ms
  • json-c 0.12 (MTA:SA r13816 server-side): 246032ms
  • json-c 0.13.1 (MTA:SA 8c3cddd client-side): 179426ms (7.61% improvement)
  • json-c 0.13.1 (MTA:SA 8c3cddd server-side): 210169ms (14.58% improvement)

Copy of changelog:

Build changes:
--------------

0.13.1 (up to commit 0f814e5, 2018/03/04)
=========================================

* Bump the major version of the .so library generated up to 4.0 to avoid 
  conflicts because some downstream packagers of json-c had already done
  their own bump to ".so.3" for a much older 0.12 release.
* Add const size_t json_c_object_sizeof()
* Avoid invalid free (and thus a segfault) when ref_count gets < 0
* PR#394: fix handling of custom double formats that include a ".0"
* Avoid uninitialized variable warnings in json_object_object_foreach
* Issue #396: fix build for certain uClibc based systems.
* Add a top level fuzz directory for fuzzers run by OSS-Fuzz


0.13 (up to commit 5dae561, 2017/11/29)
=================================

This release, being three and a half years after the 0.12 branch (f84d9c),
   has quite a number of changes included.  The following is a sampling of
   the most significant ones.

Since the 0.12 release, 250 issues and pull requests have been closed.
See issues_closed_for_0.13.md for a complete list.


Deprecated and removed features:
--------------------------------
* All internal use of bits.h has been eliminated.  The file will be removed.
	Do not use: hexdigit(), error_ptr(), error_descrition() and it_error() 
* lh_abort() is deprecated.  It will be removed.

Behavior changes:
-----------------
* Tighten the number parsing algorithm to raise errors instead of truncating
     the results.  For example 12.3.4 or 2015-01-15, which now return null.
	 See commit 99d8fc

* Use size_t for array length and size.  Platforms where sizeof(size_t) != sizeof(int) may not be backwards compatible
	See commits 45c56b, 92e9a5 and others.

* Check for failue when allocating memory, returning NULL and errno=ENOMEM.
	 See commit 2149a04.

* Change json_object_object_add() return type from void to int, and will return -1 on failures, instead of exiting. (Note: this is not an ABI change)

New features:
-------------
* We're aiming to follow RFC 7159 now.

* Add a couple of additional option to json_object_to_json_string_ext:
	JSON_C_TO_STRING_PRETTY_TAB
	JSON_C_TO_STRING_NOSLASHESCAPE

* Add a json_object_object_add_ex() function to allow for performance
	improvements when certain constraints are known to be true.

* Make serialization format of doubles configurable, in two different ways:
	Call json_object_set_serializer with json_object_double_to_json_string and a custom
	 format on each double object, or
	Call json_c_set_serialization_double_format() to set a global or thread-wide format.

* Add utility function for comparing json_objects - json_object_equal()

* Add a way to copy entire object trees: json_object_deep_copy()
* Add json_object_set_<type> function to modify the value of existing json_object's
 without the need to recreate them.  Also add a json_object_int_inc function to
 adjust an int's value.
* Add support for JSON pointer, RFC 6901.  See json_pointer.h
* Add a json_util_get_last_err() function to retrieve the string describing the
 cause of errors, instead of printing to stderr.
* Add perllike hash function for strings, and json_global_set_string_hash() 8f8d03d
* Add a json_c_visit() function to provide a way to iterate over a tree of json-c objects.

Notable bug fixes and other improvements:
-----------------------------------------
* Make reference increment and decrement atomic to allow passing json objects between threads.
* Fix json_object_object_foreach to avoid uninitialized variable warnings.
* Improve performance by removing unneeded data items from hashtable code and reducing duplicate hash computation.
* Improve performance by storing small strings inside json_object
* Improve performance of json_object_to_json_string by removing variadic printf. commit 9ff0f49
* Issue #371: fix parsing of "-Infinity", and avoid needlessly copying the input when doing so.
* Fix stack buffer overflow in json_object_double_to_json_string_format() - commit 2c2deb87
* Fix various potential null ptr deref and int32 overflows
* Issue #332: fix a long-standing bug in array_list_put_idx() where it would attempt to free previously free'd entries due to not checking the current array length.
* Issue #195: use uselocale() instead of setlocale() in json_tokener to behave better in threaded environments.
* Issue #275: fix out of bounds read when handling unicode surrogate pairs.
* Ensure doubles that happen to be a whole number are emitted with ".0" - commit ca7a19
* PR#331: for Visual Studio, use a snprintf/vsnprintf wrapper that ensures the string is terminated.
* Fix double to int cast overflow in json_object_get_int64.
* Clamp double to int32 when narrowing in json_object_get_int.
* Use strtoll() to parse ints - instead of sscanf
* Miscellaneous smaller changes, including removing unused variables, fixing warning
 about uninitialized variables adding const qualifiers, reformatting code, etc...

Build changes:
--------------
* Add Appveyor and Travis build support
* Switch to using CMake when building on Windows with Visual Studio.
	A dynamic .dll is generated instead of a .lib
	config.h is now generated, config.h.win32 should no longer be manually copied
* Add support for MacOS through CMake too.
* Enable silent build by default
* Link against libm when needed
* Add support for building with AddressSanitizer
* Add support for building with Clang
* Add a --enable-threading configure option, and only use the (slower) __sync_add_and_fetch()/__sync_sub_and_fetch() function when it is specified.

List of new functions added:
----------------------------
### json_object.h
* array_list_bsearch()
* array_list_del_idx()
* json_object_to_json_string_length()
* json_object_get_userdata()
* json_object_set_userdata()
* json_object_object_add_ex()
* json_object_array_bsearch()
* json_object_array_del_idx()
* json_object_set_boolean()
* json_object_set_int()
* json_object_int_inc()
* json_object_set_int64()
* json_c_set_serialization_double_format()
* json_object_double_to_json_string()
* json_object_set_double()
* json_object_set_string()
* json_object_set_string_len()
* json_object_equal()
* json_object_deep_copy()

### json_pointer.h
* json_pointer_get()
* json_pointer_getf()
* json_pointer_set()
* json_pointer_setf()

### json_util.h
* json_object_from_fd()
* json_object_to_fd()
* json_util_get_last_err()

### json_visit.h
* json_c_visit()

### linkhash.h
* json_global_set_string_hash()
* lh_table_resize()

### printbuf.h
* printbuf_strappend()


0.12.1
======

  * Minimal changes to address compile issues.

@Pirulax
Copy link
Contributor

Pirulax commented Jul 27, 2018

Are there any new functions at Lua side?
Or still just the good ol' to/fromJSON?

@botder botder added the enhancement New feature or request label Jul 27, 2018
@patrikjuvonen
Copy link
Contributor Author

No new functionality. It is just a vendor update.

@Dutchman101 Dutchman101 requested a review from ccw808 July 27, 2018 23:18
@qaisjp qaisjp added this to the 1.5.6 milestone Jul 28, 2018
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

@qaisjp qaisjp self-assigned this Aug 13, 2018
Copy link
Contributor

@qaisjp qaisjp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍Thank you!

  • Checked this tree against 0.13.1 and it's pretty much identical except for our changes.
  • Checked the current master against 0.12 and there are some other changes here and there but it should be fine.

@qaisjp qaisjp merged commit 424560d into multitheftauto:master Aug 23, 2018
@patrikjuvonen patrikjuvonen deleted the json-c-0.13.1 branch August 29, 2018 11:45
@patrikjuvonen patrikjuvonen added the upstream Related to vendor library label Aug 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request upstream Related to vendor library
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update json-c from 0.12 to 0.13.1
4 participants