Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing with cpp_ reference examples #106

Closed
mingodad opened this issue Feb 20, 2024 · 39 comments
Closed

Testing with cpp_ reference examples #106

mingodad opened this issue Feb 20, 2024 · 39 comments

Comments

@mingodad
Copy link

Using a Lua script shown bellow we can test cake against the examples in https://en.cppreference.com/w/c that can be downloaded for offline view here https://en.cppreference.com/w/Cppreference:Archives .

local base_dir = "/home/mingo/.local/share/Zeal/Zeal/docsets/C.docset/Contents/Resources/Documents/en.cppreference.com/w/c/";

local cCodeFName = "/tmp/cCode.c";
--local cxx_cmd = "../build/src/frontend/cxx -fsyntax-only -toolchain linux " .. cppCodeFName;
--local cxx_cmd = "/usr/bin/time ../build-release/src/frontend/cxx -fsyntax-only -toolchain linux " .. cppCodeFName;
--local cxx_cmd = "clang-17-env /usr/bin/time clang++ -std=c++23 -fsyntax-only " .. cppCodeFName;
--local cxx_cmd = "clang-17-env /usr/bin/time clang++ -fsyntax-only " .. cppCodeFName;
--local cxx_cmd = "gcc-13-env /usr/bin/time g++ -std=c++23 -fsyntax-only " .. cppCodeFName;
--local cxx_cmd = "gcc-13-env /usr/bin/time g++ -fsyntax-only " .. cppCodeFName;
local cxx_cmd = "/usr/bin/time timeout 3s src/cake -Isrc/include " .. cCodeFName;

--<div dir="ltr" class="mw-geshi t-example-code" style="text-align: left;"><div class="c source-c">
local code_re = "<div .-class=\"[^\"t]*t%-example%-[^\"]+\".->.-<div class=\"c source%-c\">(.-)</pre></div>";

local find_cmd = "find " .. base_dir .. " -name '*.html'";
local start_t = os.time();

local html_list = {};
for fname in io.popen(find_cmd, "r"):lines("l") do
	table.insert(html_list, fname);
end
table.sort(html_list);

local count_with_code = 0;
local fname_with_code_failed = {};
for idx, fname in ipairs(html_list) do
	print(fname);

	local html = io.open(fname, "r"):read("a");
	for code in html:gmatch(code_re) do
		code = code:gsub("%b<>", "");
		code = code:gsub("&nbsp;", " ");
		code = code:gsub("&lt;", "<");
		code = code:gsub("&gt;", ">");
		code = code:gsub("&amp;", "&");

		--print("//====Start");
		--print(code);
		--print("//====End");

		local fd = io.open(cCodeFName, "w");
		fd:write(code);
		fd:close();
		count_with_code = count_with_code + 1;
		print("==CHECK==")
		print(cxx_cmd);
		io.stdout:flush();
		local rc, rc_type, rc_code = os.execute(cxx_cmd);
		if rc_code == 0 then
			print("==rc:OK==")
		else
			table.insert(fname_with_code_failed, idx);
			print("==rc:FAILED==")
			print("//====Start");
			print(code);
			print("//====End");
		end
	end

end
local end_t = os.time();
print("elapsed time: ", os.difftime(end_t, start_t));
print("Total files scanned     :", #html_list);
print("Total files with code   :", count_with_code);
print("Total files failed code :", #fname_with_code_failed);
for idx, fname_idx in ipairs(fname_with_code_failed) do
	print(html_list[fname_idx]);
end
@mingodad
Copy link
Author

Also trying to test with sqlite3.c we get this:

/usr/bin/time ./cake -Iinclude -DSQLITE_THREADSAFE=0 sqlite3-orig/sqlite3.c

Cake 0.7.4


sqlite3-orig/sqlite3.c
sqlite3-orig/sqlite3.c:36582:29: note: you can use "adjacent" "strings"
 36582 |      __asm__ __volatile__ (
       |                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:38167:49: error: file sys/types.h not found
 38167 |#include <sys/types.h>   /* amalgamator: keep */
       |                                                ^
sqlite3-orig/sqlite3.c:38167:49: note: dir = 'include'
 38167 |#include <sys/types.h>   /* amalgamator: keep */
       |                                                ^

 1 files in 0.41 seconds
 0 errors 0 warnings 0 notes 
0.32user 0.09system 0:00.42elapsed 99%CPU (0avgtext+0avgdata 212044maxresident)k
0inputs+0outputs (0major+52649minor)pagefaults 0swaps

>/usr/bin/time gcc -fsyntax-only -DSQLITE_THREADSAFE=0 sqlite3-orig/sqlite3.c
0.38user 0.04system 0:00.42elapsed 99%CPU (0avgtext+0avgdata 78836maxresident)k
0inputs+0outputs (0major+19596minor)pagefaults 0swaps

@thradams
Copy link
Owner

The error here is because the include dir (gcc and system headers) are not configured

     error: file sys/types.h not found

Create a cakeconfig.h file at same path of cake executable.

//Edit this file adding inside pragma dir "" .. the result of
// echo | gcc -E -Wp,-v -

#pragma dir "add dir here..."
#pragma dir "add dir here..."
...

Next time you run cake

/cake -Iinclude -DSQLITE_THREADSAFE=0 sqlite3-orig/sqlite3.c

Then the includes configured inside cakeconfig.h will be used.
Otherwise each time you need add -I ...

(The same on windows)

See "Include directories"
http://thradams.com/cake/manual.html

On windows.
https://youtu.be/gO6DfsU7qj4

I have created this repository with real source code to be checked
https://github.com/thradams/cake-tests
but It has been a while I don't check.

@mingodad
Copy link
Author

Using valgrind and cakeconfig.h:

valgrind ./cake -Iinclude sqlite3-orig/sqlite3.c
==27122== Memcheck, a memory error detector
==27122== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==27122== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==27122== Command: ./cake -Iinclude sqlite3-orig/sqlite3.c
==27122== 
==27122== Invalid read of size 4
==27122==    at 0x111ECC: pp_tokens_opt (tokenizer.c:2681)
==27122==    by 0x1141B6: control_line (tokenizer.c:3248)
==27122==    by 0x1166DB: group_part (tokenizer.c:4339)
==27122==    by 0x10FA31: group_opt (tokenizer.c:1818)
==27122==    by 0x116798: preprocessor (tokenizer.c:4364)
==27122==    by 0x116A6A: include_config_header (tokenizer.c:4443)
==27122==    by 0x13E5D7: compile_one_file (parser.c:6663)
==27122==    by 0x13F3E7: compile (parser.c:6974)
==27122==    by 0x14CA17: main (main.c:58)
==27122==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==27122== 
==27122== 
==27122== Process terminating with default action of signal 11 (SIGSEGV)
==27122==  Access not within mapped region at address 0x0
==27122==    at 0x111ECC: pp_tokens_opt (tokenizer.c:2681)
==27122==    by 0x1141B6: control_line (tokenizer.c:3248)
==27122==    by 0x1166DB: group_part (tokenizer.c:4339)
==27122==    by 0x10FA31: group_opt (tokenizer.c:1818)
==27122==    by 0x116798: preprocessor (tokenizer.c:4364)
==27122==    by 0x116A6A: include_config_header (tokenizer.c:4443)
==27122==    by 0x13E5D7: compile_one_file (parser.c:6663)
==27122==    by 0x13F3E7: compile (parser.c:6974)
==27122==    by 0x14CA17: main (main.c:58)
==27122==  If you believe this happened as a result of a stack
==27122==  overflow in your program's main thread (unlikely but
==27122==  possible), you can try to increase the size of the
==27122==  main thread stack using the --main-stacksize= flag.
==27122==  The main thread stack size used in this run was 8388608.
==27122== 
==27122== HEAP SUMMARY:
==27122==     in use at exit: 77,033 bytes in 973 blocks
==27122==   total heap usage: 3,041 allocs, 2,068 frees, 157,981 bytes allocated
==27122== 
==27122== LEAK SUMMARY:
==27122==    definitely lost: 0 bytes in 0 blocks
==27122==    indirectly lost: 0 bytes in 0 blocks
==27122==      possibly lost: 0 bytes in 0 blocks
==27122==    still reachable: 77,033 bytes in 973 blocks
==27122==         suppressed: 0 bytes in 0 blocks
==27122== Rerun with --leak-check=full to see details of leaked memory
==27122== 
==27122== For lists of detected and suppressed errors, rerun with: -s
==27122== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)

Then applying this fix:

------------------------------- src/tokenizer.c -------------------------------
index 38a4e81..8eb69c3 100644
@@ -2678,7 +2678,7 @@ struct token_list replacement_list(struct preprocessor_ctx* ctx, struct macro* m
 struct token_list pp_tokens_opt(struct preprocessor_ctx* ctx, struct token_list* input_list, int level)
 {
 	struct token_list r = { 0 };
-	while (input_list->head->type != TK_NEWLINE)
+	while (input_list->head && input_list->head->type != TK_NEWLINE)
 	{
 		prematch_level(&r, input_list, level);
 	}

Then:

valgrind ./cake -Iinclude sqlite3-orig/sqlite3.c
==27365== Memcheck, a memory error detector
==27365== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==27365== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==27365== Command: ./cake -Iinclude sqlite3-orig/sqlite3.c
==27365== 
Cake 0.7.4


sqlite3-orig/sqlite3.c
sqlite3-orig/sqlite3.c:36582:29: note: you can use "adjacent" "strings"
 36582 |      __asm__ __volatile__ (
       |                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:46561:58: warning: unnecessary line-slicing
 46561 |WINBASEAPI HANDLE WINAPI CreateFileMappingFromApp(HANDLE,
       |                                                         ^~~~~~~~~~~
sqlite3-orig/sqlite3.c:46567:75: warning: unnecessary line-slicing
 46567 |WINBASEAPI HANDLE WINAPI CreateFileMappingA(HANDLE, LPSECURITY_ATTRIBUTES,
       |                                                                          ^~~~~~~~~~~
sqlite3-orig/sqlite3.c:46572:75: warning: unnecessary line-slicing
 46572 |WINBASEAPI HANDLE WINAPI CreateFileMappingW(HANDLE, LPSECURITY_ATTRIBUTES,
       |                                                                          ^~~~~~~~~~~
sqlite3-orig/sqlite3.c:129301:6: warning: unnecessary line-slicing
 129301 |    }
        |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:24249:22: error: unexpected
 24249 |    *va_arg(ap,int*) = val;
       |                     ^
sqlite3-orig/sqlite3.c:24249:22: error: expected TK_SEMICOLON
 24249 |    *va_arg(ap,int*) = val;
       |                     ^
sqlite3-orig/sqlite3.c:28818:3: error: not found 'mutexIsInit'
 28818 |  assert( GLOBAL(int, mutexIsInit) ) assert(mutexIsInit);
       |          ^~~~~~~~~~~
sqlite3-orig/sqlite3.c:28818:3: error: expected TK_RIGHT_PARENTHESIS
 28818 |  assert( GLOBAL(int, mutexIsInit) ) assert(mutexIsInit);
       |          ^~~~~~~~~~~
sqlite3-orig/sqlite3.c:28818:3: error: expected TK_SEMICOLON
 28818 |  assert( GLOBAL(int, mutexIsInit) ) assert(mutexIsInit);
       |                     ^
sqlite3-orig/sqlite3.c:29331:3: error: member 'nRef' not found in struct 'sqlite3_mutex'
 29331 |  assert( p->nRef==0 ) assert(p->nRef==0);
       |             ^~~~
sqlite3-orig/sqlite3.c:29358:3: error: member 'id' not found in struct 'sqlite3_mutex'
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |             ^~
sqlite3-orig/sqlite3.c:29358:3: error: not found 'pthreadMutexNotheld'
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                     ^~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29358:3: error: right type is not scalar for or expression
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                     ^~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29358:3: error: expected TK_RIGHT_PARENTHESIS
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                     ^~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29358:3: error: expected TK_SEMICOLON
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                         ^
sqlite3-orig/sqlite3.c:29358:3: error: expected TK_SEMICOLON
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                           ^
sqlite3-orig/sqlite3.c:29358:3: error: unexpected
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                            ^
sqlite3-orig/sqlite3.c:29358:3: error: expected TK_SEMICOLON
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                            ^
sqlite3-orig/sqlite3.c:29401:3: error: member 'id' not found in struct 'sqlite3_mutex'
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |             ^~
sqlite3-orig/sqlite3.c:29401:3: error: not found 'pthreadMutexNotheld'
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                     ^~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29401:3: error: right type is not scalar for or expression
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                     ^~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29401:3: error: expected TK_RIGHT_PARENTHESIS
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                     ^~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29401:3: error: expected TK_SEMICOLON
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                         ^
sqlite3-orig/sqlite3.c:29401:3: error: expected TK_SEMICOLON
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                           ^
sqlite3-orig/sqlite3.c:29401:3: error: unexpected
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                            ^
sqlite3-orig/sqlite3.c:29401:3: error: expected TK_SEMICOLON
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                            ^
sqlite3-orig/sqlite3.c:29457:3: error: not found 'pthreadMutexHeld'
 29457 |  assert( pthreadMutexHeld(p) ) assert(pthreadMutexHeld(p));
       |          ^~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29457:3: error: expected TK_RIGHT_PARENTHESIS
 29457 |  assert( pthreadMutexHeld(p) ) assert(pthreadMutexHeld(p));
       |          ^~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29457:3: error: expected TK_SEMICOLON
 29457 |  assert( pthreadMutexHeld(p) ) assert(pthreadMutexHeld(p));
       |                          ^
sqlite3-orig/sqlite3.c:29457:3: error: expected TK_SEMICOLON
 29457 |  assert( pthreadMutexHeld(p) ) assert(pthreadMutexHeld(p));
       |                            ^
sqlite3-orig/sqlite3.c:29457:3: error: unexpected
 29457 |  assert( pthreadMutexHeld(p) ) assert(pthreadMutexHeld(p));
       |                             ^
sqlite3-orig/sqlite3.c:29457:3: error: expected TK_SEMICOLON
 29457 |  assert( pthreadMutexHeld(p==27365== Invalid read of size 4
==27365==    at 0x126259: type_is_void (type.c:611)
==27365==    by 0x13C1E3: unlabeled_statement (parser.c:5558)
==27365==    by 0x13CC6C: block_item (parser.c:5894)
==27365==    by 0x13C906: block_item_list (parser.c:5788)
==27365==    by 0x13C6C7: compound_statement (parser.c:5702)
==27365==    by 0x13BE52: primary_block (parser.c:5446)
==27365==    by 0x13C15F: unlabeled_statement (parser.c:5545)
==27365==    by 0x13BDDA: statement (parser.c:5434)
==27365==    by 0x13BF86: secondary_block (parser.c:5480)
==27365==    by 0x13D096: selection_statement (parser.c:6031)
==27365==    by 0x13BE7A: primary_block (parser.c:5450)
==27365==    by 0x13C15F: unlabeled_statement (parser.c:5545)
==27365==  Address 0x23e28578 is 8 bytes inside a block of size 232 free'd
==27365==    at 0x4C35A6F: free (vg_replace_malloc.c:974)
==27365==    by 0x12295A: expression_delete (expressions.c:3565)
==27365==    by 0x122906: expression_delete (expressions.c:3560)
==27365==    by 0x1225B3: assignment_expression (expressions.c:3508)
==27365==    by 0x122990: expression (expressions.c:3577)
==27365==    by 0x13D9FE: expression_statement (parser.c:6314)
==27365==    by 0x13C1A1: unlabeled_statement (parser.c:5553)
==27365==    by 0x13CC6C: block_item (parser.c:5894)
==27365==    by 0x13C906: block_item_list (parser.c:5788)
==27365==    by 0x13C6C7: compound_statement (parser.c:5702)
==27365==    by 0x13BE52: primary_block (parser.c:5446)
==27365==    by 0x13C15F: unlabeled_statement (parser.c:5545)
==27365==  Block was alloc'd at
==27365==    at 0x4C37F23: calloc (vg_replace_malloc.c:1554)
==27365==    by 0x11C0B4: primary_expression (expressions.c:1165)
==27365==    by 0x11E2F4: postfix_expression (expressions.c:1943)
==27365==    by 0x11F906: unary_expression (expressions.c:2374)
==27365==    by 0x11FE3A: cast_expression (expressions.c:2498)
==27365==    by 0x11FEE0: multiplicative_expression (expressions.c:2533)
==27365==    by 0x120418: additive_expression (expressions.c:2664)
==27365==    by 0x120F4B: shift_expression (expressions.c:2875)
==27365==    by 0x12117F: relational_expression (expressions.c:2946)
==27365==    by 0x121685: equality_expression (expressions.c:3091)
==27365==    by 0x12198E: and_expression (expressions.c:3164)
==27365==    by 0x121B81: exclusive_or_expression (expressions.c:3218)
==27365== 
...
==27365== 
==27365== HEAP SUMMARY:
==27365==     in use at exit: 167,251,640 bytes in 4,385,178 blocks
==27365==   total heap usage: 4,755,509 allocs, 370,331 frees, 183,171,918 bytes allocated
==27365== 
==27365== LEAK SUMMARY:
==27365==    definitely lost: 0 bytes in 0 blocks
==27365==    indirectly lost: 0 bytes in 0 blocks
==27365==      possibly lost: 0 bytes in 0 blocks
==27365==    still reachable: 167,251,640 bytes in 4,385,178 blocks
==27365==         suppressed: 0 bytes in 0 blocks
==27365== Rerun with --leak-check=full to see details of leaked memory
==27365== 
==27365== For lists of detected and suppressed errors, rerun with: -s
==27365== ERROR SUMMARY: 179 errors from 19 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)

@thradams
Copy link
Owner

(
Note:
I never compiled sqlite on linux.. just on windows.
So it is a good opportunity to fix some bugs.
)

@mingodad
Copy link
Author

Looking through the usage (see bellow) of struct expression::type it seems that it should be struct type owner * type; instead of struct type type; :

  • p_expression_node->type = type_dup(&p_expression_node->generic_selection->p_view_selected_expression->type);

Because here

cake/src/type.c

Line 1794 in d503e53

struct type r = *l.head;
you are making a raw copy (memcpy) .

thradams added a commit that referenced this issue Feb 20, 2024
@mingodad
Copy link
Author

Attached is the output of running test-c-reference.lua , it includes the extracted code for the ones that fail.

Total files scanned     :	611
Total files with code   :	432
Total files failed code :	129

test-c-reference.lua.log.zip

@thradams
Copy link
Owner

thradams commented Feb 23, 2024 via email

@mingodad
Copy link
Author

The zip file is the output of running the Lua script, you can see the errors and the source code for the failures direct there (no need to to run the script yourself).

@mingodad
Copy link
Author

Here is a simplified code that cake can't handle but gcc can:

#include <stdio.h>
int main(void)
{
    printf("isgreater(2.0,1.0)      = %d\n",  ( ({int x=2; int y=3; x > y; })));

    return 0;
}

@mingodad
Copy link
Author

In the zip file search for ==rc:FAILED== to go straight to a failure.

@mingodad
Copy link
Author

This mingodad@9294ae7 temporary fix allow more files to be parsed.

@mingodad
Copy link
Author

I found this two minimal examples that show a bug in cake when handling wchar_t:
This one parses fine:

#include <stdio.h>
#include <wchar.h>

int main(void)
{
    wchar_t *wstr =L"御休みなさい";
    int rc = fputws(wstr, stdout);
    return rc;
}

This one that do not use an intermediate variable fail:

#include <stdio.h>
#include <wchar.h>

int main(void)
{
    int rc = fputws(L"御休みなさい", stdout);
    return rc;
}

Output:

src/cake dad/test-wchar1.c
Cake 0.7.4

dad/test-wchar1.c

 1 files in 0.02 seconds
 0 errors 0 warnings 0 notes

src/cake dad/test-wchar2.c
Cake 0.7.4

dad/test-wchar2.c
type mismatch:
unsigned *
const int * __ws => wchar_t
dad/test-wchar2.c:6:21: error:  incompatible types at argument 1 [-W]
 6 |    int rc = fputws(L"御休みなさい", stdout stdout);
   |                    ^~~~~~~~~~~~~~~~~~~~~

 1 files in 0.03 seconds
 1 errors 0 warnings 0 notes 

@mingodad
Copy link
Author

With this fix mingodad@603abc4 I could get cake to parse wchar_t literals type and parse several files:
Before (lua "test-c-reference.lua"):

Total files scanned     :	611
Total files with code   :	432
Total files failed code :	129

After (lua "test-c-reference.lua"):

Total files scanned     :	611
Total files with code   :	432
Total files failed code :	107

@mingodad
Copy link
Author

With this sample:

enum { SIZE = 5 };

int main(void)
{
    const double a[SIZE] = {1.0, 2.0, 3.0, 4.0, 5.0};
    unsigned int sz1 = sizeof a;
    unsigned int sz2 =  sizeof *a;
}

And with this fix mingodad@a4bcf34 we get:

src/cake  dad/test-type-pointer.c
Cake 0.7.4


/home/mingo/dev/c/A_programming-languages/cake-dad2/dad/test-type-pointer.c
/home/mingo/dev/c/A_programming-languages/cake-dad2/dad/test-type-pointer.c:7:32: error: indirection requires pointer operand [-W]
 7 |    unsigned int sz2 =  sizeof *a;
   |                               ^

 1 files in 0.00 seconds
 1 errors 0 warnings 0 notes 

Before it was:

/usr/bin/time timeout 3s src/cake /tmp/cCode.c
cake: type.c:1573: type_remove_pointer: Assertion `type_is_pointer(p_type)' failed.
timeout: the monitored command dumped core
Command terminated by signal 6

@mingodad
Copy link
Author

With this fix mingodad@5e61573 :
Sample:

#include <complex.h>
#include <stdio.h>

int main(void)
{
    double imaginary z = 3*I;
    z = 1 / z;
    printf("1/(3.0i) = %+.1fi\n", cimag(z));
}

Output:

src/cake  dad/test-double-free.c
>src/cake  dad/test-double-free.c
Cake 0.7.4


/home/mingo/dev/c/A_programming-languages/cake-dad2/dad/test-double-free.c
/home/mingo/dev/c/A_programming-languages/cake-dad2/dad/test-double-free.c:6:22: error: expected TK_SEMICOLON [-W]
 6 |    double imaginary z = 3*I( 1.0iF);
   |                     ^
...
 1 files in 0.02 seconds
 14 errors 0 warnings 0 notes 

Before:

/usr/bin/time timeout 3s src/cake /tmp/cCode.c
free(): double free detected in tcache 2
timeout: the monitored command dumped core
Command terminated by signal 6

@mingodad
Copy link
Author

With this temporary fix mingodad@0a59d95 :
Sample:

#include <math.h>
#include <stdio.h>

// Possible implementation of the tgmath.h macro cbrt
#define cbrt(X) _Generic((X), \
              long double: cbrtl, \
                  xdefault: cbrt,  \
                    float: cbrtf  \
              )(X)

int main(void)
{
    double x = 8.0;
    const float y = 3.375;
    printf("cbrt(8.0) = %f\n", cbrt(x)); // selects the default cbrt
    printf("cbrtf(3.375) = %f\n", cbrt(y)); // converts const float to float,
                                            // then selects cbrtf
}

Output:

src/cake  dad/test-generic.c
>src/cake  dad/test-generic.c
Cake 0.7.4


/home/mingo/dev/c/A_programming-languages/cake-dad2/dad/test-generic.c
/home/mingo/dev/c/A_programming-languages/cake-dad2/dad/test-generic.c:5:9: note: use SCREAMING_CASE for macros
 5 |#define cbrt(X) _Generic((X), \
              long double: cbrtl, \
                  xdefault: cbrt,  \
                    float: cbrtf  \
              )(X)
   |        ^~~~
/home/mingo/dev/c/A_programming-languages/cake-dad2/dad/test-generic.c:15:32: error: unexpected [-W]
 15 |    printf("cbrt(8.0) = %f\n", cbrt(x) _Generic((x), long double: cbrtl, xdefault: cbrt, float: cbrtf )(x)); // selects the default cbrt
    |                                                                 ^~~~~~~~
...
 1 files in 0.03 seconds
 24 errors 0 warnings 0 notes 

Before:

/usr/bin/time timeout 3s src/cake /tmp/cCode.c
timeout: the monitored command dumped core
Command terminated by signal 11

This pattern in expressions.c need to be revised because several fixes seem to be by returning dangle pointers:

            {
                expression_delete(new_expression);
                throw;
            }

@mingodad
Copy link
Author

This mingodad@aa07dbd fix the UTF8 range assert that was failing with the sample bellow:

#include <errno.h>
#include <locale.h>
#include <stdio.h>
#include <stdlib.h>
#include <wchar.h>

int main(void)
{
    setlocale(LC_ALL, "en_US.utf8");

    errno = 0;
    if (fputwc(L'🍌', stdout) == WEOF)
    {
        if (errno == EILSEQ)
            puts("Encoding error in fputwc.");
        else
            puts("I/O error in fputwc.");
        return EXIT_FAILURE;
    }
}

@thradams
Copy link
Owner

thradams commented Feb 24, 2024 via email

@mingodad
Copy link
Author

Thank you for reply !
Yes you're right !
I've updated it now.

@mingodad
Copy link
Author

This fix mingodad@b3c7c11 memory use after free detected by valgrind when parsing this sample:

#include <stdio.h>
#include <complex.h>
#include <math.h>

int main(void)
{
    double complex z1 = 0.0 + INFINITY * _Imaginary_I;
    printf("z1 = %.1f%+.1fi\n", creal(z1), cimag(z1));

    double complex z2 = 0.0 + INFINITY * _Complex_I;
    printf("z2 = %.1f%+.1fi\n", creal(z2), cimag(z2));
}

Before the fix:

src/cake /tmp/cCode.c
free(): double free detected in tcache 2
timeout: the monitored command dumped core
Command terminated by signal 6

After the fix:

src/cake /tmp/cCode.c
Cake 0.7.4


/tmp/cCode.c
/tmp/cCode.c:7:42: error: not found '_Imaginary_I'
 7 |    double complex _Complex z1 = 0.0 + INFINITY 1e10000f * _Imaginary_I;
   |                                                           ^~~~~~~~~~~~
/tmp/cCode.c:7:42: error: right * is not arithmetic
 7 |    double complex _Complex z1 = 0.0 + INFINITY 1e10000f * _Imaginary_I;
   |                                                           ^~~~~~~~~~~~
/tmp/cCode.c:7:42: error: invalid type multiplicative expression
 7 |    double complex _Complex z1 = 0.0 + INFINITY 1e10000f * _Imaginary_I;
   |                                                           ^~~~~~~~~~~~
/tmp/cCode.c:7:42: error: expected TK_SEMICOLON
 7 |    double complex _Complex z1 = 0.0 + INFINITY 1e10000f * _Imaginary_I;
   |                                                           ^~~~~~~~~~~~

thradams added a commit that referenced this issue Feb 25, 2024
@thradams
Copy link
Owner

thradams commented Feb 25, 2024

I did a checkin with a pattern that I think it is better

with fails like malloc inside try blocks I want to return null to avoid ignoring half
AST not begin reported. (unless we add a report error for out-of-memory.)
then the AST could be half correct, because it reports an error.

    catch
    {
        if (p_expression_node)
        {
            expression_node_delete(p_expression_node);
            p_expression_node = NULL;
        }
    }

double free is fixed just puting

     new_expression->left = p_expression_node;
     p_expression_node = NULL; /*MOVED*/

after moving it

thradams added a commit that referenced this issue Feb 25, 2024
@thradams
Copy link
Owner

thradams commented Feb 25, 2024 via email

@thradams
Copy link
Owner

thradams commented Feb 25, 2024 via email

@mingodad
Copy link
Author

Here it's a zip with all examples extracted.
en-cpp-reference-c.zip

@thradams
Copy link
Owner

thradams commented Feb 25, 2024 via email

thradams added a commit that referenced this issue Feb 25, 2024
@thradams
Copy link
Owner

ccode424.c is failing..but nullptr_t is a typedef.

void puts(const char*);

#define DETECT_NULL_POINTER_CONSTANT(e) \
    _Generic(e,                         \
        void* : puts("void*"),          \
        nullptr_t : puts("nullptr_t"),  \
        default : puts("other")       \
    )

int main()
{
    DETECT_NULL_POINTER_CONSTANT(((void*)0));
    DETECT_NULL_POINTER_CONSTANT(0);
    DETECT_NULL_POINTER_CONSTANT(nullptr);
}

I think cake is correct.

thradams added a commit that referenced this issue Feb 25, 2024
out of bound is a warning now
@mingodad
Copy link
Author

Why do you have src/file.c and tests/en-cpp-reference-c/out checked in the repository ?
Aren't those files temporary or generated files ?

@thradams
Copy link
Owner

The reason I kept tests/en-cpp-reference-c/out is to compare the code generation ouput.
Lets say regression in code generation.
file.c is not generated but it is the file I use as "temp" and generally it has the last sample I worked.

@thradams
Copy link
Owner

Most of errors now are from imaginary samples that where not implemented.

@thradams
Copy link
Owner

sqlite is compiling.
one problem was assert as keyword.
the other was __builtinoffset

assert assert can be fixed. it was used for instance

void flow_visit_ctx_destroy(struct flow_visit_ctx* obj_owner p)
{
    assert(p->tail_block == NULL);
}

it works at same time as runtime and static information.
the problem is that when debug is not defined we cannot parse the innert part we need to use it as empty macro.
the detection of GCC assert ((void)0) is missing;;

@thradams
Copy link
Owner

thradams commented Feb 25, 2024

ouput
.\cake sqlite.c
on linux

I added cakeconfig.h at src

/home/tra/cake/src/sqlite3.c
/home/tra/cake/src/sqlite3.c:46447:58: warning: unnecessary line-slicing
 46447 |WINBASEAPI HANDLE WINAPI CreateFileMappingFromApp(HANDLE,
       |                                                         ^~~~~~~~~~~
/home/tra/cake/src/sqlite3.c:46453:75: warning: unnecessary line-slicing
 46453 |WINBASEAPI HANDLE WINAPI CreateFileMappingA(HANDLE, LPSECURITY_ATTRIBUTES,
       |                                                                          ^~~~~~~~~~~
/home/tra/cake/src/sqlite3.c:46458:75: warning: unnecessary line-slicing
 46458 |WINBASEAPI HANDLE WINAPI CreateFileMappingW(HANDLE, LPSECURITY_ATTRIBUTES,
       |                                                                          ^~~~~~~~~~~
/home/tra/cake/src/sqlite3.c:129074:6: warning: unnecessary line-slicing
 129074 |    }
        |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/tra/cake/src/sqlite3.c:40309:23: warning:  passing null as array [-Wnonnull]
 40309 |    utimes(zLockFile, NULL ((void *)0));
       |                      ^
/home/tra/cake/src/sqlite3.c:107205:55: warning: index 1 is past the end of the array [-Wout-of-bounds]
 107205 |            pExpr->iTable = exprProbability(pList->a[1].pExpr);
        |                                                      ^
/home/tra/cake/src/sqlite3.c:107475:61: warning: index 1 is past the end of the array [-Wout-of-bounds]
 107475 |          nRight = sqlite3ExprVectorSize(pExpr->x.pList->a[1].pExpr);
        |                                                            ^
/home/tra/cake/src/sqlite3.c:112595:60: warning: index 1 is past the end of the array [-Wout-of-bounds]
 112595 |         sqlite3ExprCompare(0,pFarg->a[0].pExpr, pFarg->a[1].pExpr,-1),
        |                                                           ^
/home/tra/cake/src/sqlite3.c:112604:69: warning: index 1 is past the end of the array [-Wout-of-bounds]
 112604 |         sqlite3ExprImpliesExpr(pParse,pFarg->a[0].pExpr, pFarg->a[1].pExpr,-1),
        |                                                                    ^
/home/tra/cake/src/sqlite3.c:112613:23: warning: index 1 is past the end of the array [-Wout-of-bounds]
 112613 |      pA1 = pFarg->a[1].pExpr;
        |                      ^
/home/tra/cake/src/sqlite3.c:113225:71: warning: index 1 is past the end of the array [-Wout-of-bounds]
 113225 |        pDef = sqlite3VtabOverloadFunction(db, pDef, nFarg, pFarg->a[1].pExpr);
        |                                                                      ^
/home/tra/cake/src/sqlite3.c:113831:43: warning: index 1 is past the end of the array [-Wout-of-bounds]
 113831 |    compRight.pRight = pExpr->x.pList->a[1].pExpr;
        |                                          ^
/home/tra/cake/src/sqlite3.c:114428:48: warning: index 1 is past the end of the array [-Wout-of-bounds]
 114428 |       || exprImpliesNotNull(pParse, pList->a[1].pExpr, pNN, iTab, 1)
        |                                               ^
/home/tra/cake/src/sqlite3.c:114618:55: warning: index 1 is past the end of the array [-Wout-of-bounds]
 114618 |                                   pExpr->x.pList->a[1].pExpr);
        |                                                      ^
/home/tra/cake/src/sqlite3.c:125624:22: warning: index 1 is past the end of the array [-Wout-of-bounds]
 125624 |      memcpy(&p1->a[1], p2->a, p2->nSrc*sizeof(SrcItem));
        |                     ^
/home/tra/cake/src/sqlite3.c:125626:50: warning: index 1 is past the end of the array [-Wout-of-bounds]
 125626 |      p1->a[0].fg.jointype |= (JT_LTORJ0x40 & p1->a[1].fg.jointype);
        |                                                     ^
/home/tra/cake/src/sqlite3.c:130152:40: warning: index 2 is past the end of the array [-Wout-of-bounds]
 130152 |    Expr *pEscape = pExpr->x.pList->a[2].pExpr;
        |                                       ^
/home/tra/cake/src/sqlite3.c:148965:22: warning: index 1 is past the end of the array [-Wout-of-bounds]
 148965 |    if( pTabList->a[1].fg.jointype & JT_CROSS 0x02 ) return 1;         /* (1b) */
        |                     ^
/home/tra/cake/src/sqlite3.c:149271:27: warning: index 1 is past the end of the array [-Wout-of-bounds]
 149271 |         || (pTabList->a[1].fg.jointype&(JT_OUTER0x20|JT_CROSS0x02))!=0)
        |                          ^
/home/tra/cake/src/sqlite3.c:159094:21: warning: index 1 is past the end of the array [-Wout-of-bounds]
 159094 |  pLeft = pList->a[1].pExpr;
        |                    ^
/home/tra/cake/src/sqlite3.c:159281:22: warning: index 1 is past the end of the array [-Wout-of-bounds]
 159281 |    pCol = pList->a[1].pExpr;
        |                     ^
/home/tra/cake/src/sqlite3.c:159322:32: warning: index 1 is past the end of the array [-Wout-of-bounds]
 159322 |          *ppRight = pList->a[1].pExpr;
        |                               ^
/home/tra/cake/src/sqlite3.c:160233:32: warning: index 1 is past the end of the array [-Wout-of-bounds]
 160233 |    pLeft = pExpr->x.pList->a[1].pExpr;
        |                               ^

@mingodad
Copy link
Author

Great work !
About memory usage cake is using around 5x more than gcc-9.4 and 4x more than clang-17:

/usr/bin/time myvalgrind src/cake  sqlite3-orig/sqlite3.c
...
1 files in 0.94 seconds
 2415 errors 2 warnings 3 notes 
Command exited with non-zero status 1
0.77user 0.18system 0:00.96elapsed 99%CPU (0avgtext+0avgdata 382088maxresident)k
0inputs+0outputs (0major+95108minor)pagefaults 0swaps

/usr/bin/time gcc -fsyntax-only  sqlite3-orig/sqlite3.c
0.38user 0.05system 0:00.44elapsed 99%CPU (0avgtext+0avgdata 79596maxresident)k
0inputs+0outputs (0major+19793minor)pagefaults 0swaps

/usr/bin/time clang-17-env clang -fsyntax-only  sqlite3-orig/sqlite3.c
0.73user 0.08system 0:01.05elapsed 76%CPU (0avgtext+0avgdata 100064maxresident)k
151992inputs+0outputs (545major+9762minor)pagefaults 0swaps

@thradams
Copy link
Owner

Not sure, but probably it is because comments, spaces, inactive preprocessor blocks are preserved to be able to generate the "c backend" .
But I think if cake is used as static analyzer only it could discard a lot of tokens. or when used with the option -no-ouput.
It also preserve tokens inside includes because initially I was sure if I could use comments
This also needs to be reviewed and discard tokens from included files.also with a option.

@mingodad
Copy link
Author

Also for comparison https://github.com/rui314/chibicc uses more memory than cake when compiling sqlite3.c:

chibicc -c sqlite3.c 
2.59user 0.37system 0:02.97elapsed 99%CPU (0avgtext+0avgdata 525964maxresident)k
0inputs+6336outputs (0major+201446minor)pagefaults 0swaps

And tinycc uses almost nothing compared to the others:

/usr/bin/time tcc -c sqlite3.c  
0.09user 0.01system 0:00.11elapsed 93%CPU (0avgtext+0avgdata 16748maxresident)k
840inputs+3696outputs (2major+3726minor)pagefaults 0swaps

@thradams
Copy link
Owner

thradams commented Feb 25, 2024 via email

@thradams
Copy link
Owner

A lot of samples using imaginary.
Some errors like missing strdup is caused because cake uses windows headers.

@thradams
Copy link
Owner

nullptr_t is suppose to be a typedef and it is used as builtin type on samples

@thradams
Copy link
Owner

alignas error
missing GCC asm. it is not the same of standar asm
ccode358 char is typedef
strdup not present on windows headers

@thradams
Copy link
Owner

individual errors will be created. asm, align errors already exist.

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants