Alternative to Malloc for embedded device

[mtangoo]

Hi,
Thanks for the useful library, it is great!
I have faced one problem with embedded device (NEW8110 EFT POS to be explicit) whereby using malloc() ends up with Data abort exception that have some code I cannot understand. Now since I want to use CJSON in this device, is there a way out? Can I use CJSON without invoking malloc at all?

Google plus searching issues here have not helped. The SDK documentation mentions nothing and the support have expired which means am on my own!

Device: NEW8110 EFT POS, ARM9
Compiler: arm-linux-uclib-gcc 4.1.2

Let me know if any more information is needed

TIA,
Stefano

[FSMaxB]

It really depends on what you mean by that.

You can:

• Print Json from a given set of cJSON struct data without using the heap with cJSON_PrintPreallocated
• Globally swap out the allocator so that your own allocator gets used.

When it comes to parsing, you don't get around using some allocator. The same goes for building JSON that you want to print later using cJSON_Create... methods. Although you can in theory create all of the structs manually and store them somewhere statically, but I wouldn't recommend it.

As mentioned above, you can replace the global allocator completely, if this helps in your case. Use cJSON_InitHooks for that.

[mtangoo]

Hi @FSMaxB,
Thank you for explanations. However am lost as I'm not yet well versed in internals of CJSON. Let me explain what I want to do.

I get JSON from API call from which I use cJSON_Create and get the element and print results.

Now since I cannot use malloc/free in this particular device, how can I create my own? My JSON string is always less than 5k which means I can do allocation statically. But never have used init hooks am at loss.

TIA,
Stefano

[mtangoo]

here is a sample code:

cJSON *root = NULL;
const cJSON *message = NULL;
const cJSON *success = NULL;   

root = cJSON_Parse(api_json_string);
if(root == NULL)
{
    show_error(cJSON_GetErrorPtr()); 
}

success = cJSON_GetObjectItemCaseSensitive(root, "success");
message = cJSON_GetObjectItemCaseSensitive(root, "message");
if(cJSON_IsBool(success) && cJSON_IsTrue(success))
{
    show_success(message->valuestring);
}
cJSON_Delete(root); 

[mtangoo]

I have found that this device uses bare-metal approach and as such malloc is not available. Do you have advice on how I can use library for that? Is it outside use-case for this library?

[mtangoo]

So more tests, reveals that I can use alloca(). Would simple replacement work?
Am testing that right now

[FSMaxB]

I would not recommend using alloca. It only works under one condition: If you never ever return anything that you get from cJSON without copying it first.
If you don't follow this rule, you will most certainly seriously corrupt your data. (This is because alloca allocates memory on the current stack frame, and that gets "destroyed" once you return from the current function, where "destroyed" means that it is still there, but will be overwritten by consecutive function calls, thereby corrupting your data).
Also with alloca you might get a stackoverflow. That is also something you really want to avoid.

Since you are using cJSON_Parse you have to do allocations of some kind, you can't get around it. You could in theory write your own global pool allocator with a statically allocated pool that you reset every time you are done using cJSON. Or allocate a large pool statically and use a real malloc/free implementation on that.

If I researched that correctly, your system has rather large memory and performance (16MB RAM, 180MHz) compared to some other systems I have seen people use cJSON on, this should definitely be enough if your JSON is not incredibly large. Probably you just have to turn on the Heap somehow, so you can use malloc and free, or go the route I mentioned earlier: Statically allocate a large chunk of memory and put your own heap in there, there should be several allocator libraries out there that you can use for that.

There are several malloc implementations in uClibc that you seem to be using btw.

[mtangoo]

Hi,
Thanks for the comment. It is really useful.

If you never ever return anything that you get from cJSON without copying it first.

Can you explain this comment? I want to avoid it, but then use it as a last resort if I fail to find any solution (since alloca works right now but malloc crashes in simple call. Not tested with CJSON yet). Do you mean I should copy element->valuestring instea of return element->valuestring?

You could in theory write your own global pool allocator with a statically allocated pool that you reset every time you are done using cJSON. Or allocate a large pool statically and use a real malloc/free implementation on that.

Currently I cannot use malloc. Using it even for the simplest code for sake of testing crashes. Being mainly a C++ programmer I have hard time to create statically allocated pool (especially managing global variables effectively)

If I researched that correctly, your system has rather large memory and performance (16MB RAM, 180MHz) compared to some other systems I have seen people use cJSON on, this should definitely be enough if your JSON is not incredibly large.

You are right, memory is not an issue at all. I wonder why would malloc be a garbage in such a system with little memory constraint.

Probably you just have to turn on the Heap somehow, so you can use malloc and free, or go the route I mentioned earlier: Statically allocate a large chunk of memory and put your own heap in there

I have wasted good time trying both to no avail. I have also searched for libraries only to be confused. I will try the uclib's other malloc implementations and see if any helps.

Thank you again!

[FSMaxB]

Nevermind, alloca won't work now that I think about it. Because the functions in cJSON itself do return, so the memory will already be gone once it returns. And it will write all over itself.

[FSMaxB]

You don't need the cJSON_IsBool btw., cJSON_IsTrue already handles that for you.

There is no way around either fixing malloc and free or making your own.

What you need to use a custom allocator in cJSON is to fill the cJSON_Hooks struct with function pointers to an allocator and deallocator function that have the same signature as malloc and free:

cJSON/cJSON.h

Lines 75 to 79 in 0e0c463

	typedef struct cJSON_Hooks
	{
	void *(*malloc_fn)(size_t sz);
	void (*free_fn)(void *ptr);
	} cJSON_Hooks;

Then you pass a pointer to that struct to cJSON_InitHooks and cJSON will use your custom allocator functions from then on.

cJSON/cJSON.h

Lines 135 to 136 in 0e0c463

	/* Supply malloc, realloc and free functions to cJSON */
	CJSON_PUBLIC(void) cJSON_InitHooks(cJSON_Hooks* hooks);

[mtangoo]

Ok Thanks for the help.
Let me find a way to enable somehow the malloc function.
I will post my findings!

[mtangoo]

After fighting for two days, I think it is a lost battle. Malloc there does not work at all. Since I could not get Malloc to work and am not competent enough to write my own pool, am forced to find another library that is malloc free. It will be another waste of time but have no choice, sadly!

I thank you for help and would keep using it in the other machines.

[mtangoo]

Though am moving to another library, I suggest you think about non-malloc version of cJSON_Parse if you find time or anything that can help where malloc is aither unavailable (rare case) or where it is discouraged (very common)

Thank you!

[FSMaxB]

It is simply impossible to implement cJSON_Parse without doing dynamic allocations.

I can't do any more here than provide a way to exchange the allocator that cJSON uses with your own implementation. (that will also improve in the future when you will be able to pass userdata around so you can create pool allocators with different memory pools).

I doubt that it is at all possible to write a JSON library that can parse arbitrary JSON into a different data structure without requiring some kind of dynamic allocations at all. (Although it should be possible to write a parser that takes something like a JSON pointer or other kind of path and returns one single value given that, but cJSON just isn't that kind of library).

I wish you good luck and please report back if you found a solution.

[iC00kScripts]

Hi @mtangoo

I'll like information on how you were able to parse json on the NEW8110 terminal so I may do the same. I'll appreciate libraries, sample code snippets and every other resources I can get.

@FSMaxB I followed the thread above and managed to dig up two working implementations of malloc and free. When I called cJSON_InitHooks for each before parsing, not exceptions were thrown; but at the point of calling cJSON_Parse the Data Abort Exception was thrown.

[mtangoo]

@arashifunsho are you dealing with NEW8110 too? I ended up using Tiny JSON. It is a non-malloc JSON that I have been using before diving into CJSon

[iC00kScripts]

Yes, I stumbled on Tiny JSON few hours after posting that. Worked perfectly.....