Development/json types #1433

msieben · 2023-10-20T09:40:39Z

No description provided.

…f possible. In case the JSONRPC message reeived is malformatted, Thunder should try to return a JSONRPC error on malformatted JSON message in stead of: 1) Not responding at all. 2) Send an Web Error status even if the ID was sxtracted from the message. These fixes are related to: METROL-802/METROL-803

…ing.

Starting or ending of an opaque structure should *only* contain white spaces. Anything else should be considered something to be parsed..

…earlier. In the mean time also prevented quoted areas from being accounted in the cope counting and in case we are deserializing in opaque areas drop the white space for further processing and storage.

…well.. Checking the validity of the JSONRPC messages coming in over the websockets. If incorrect, rturn an error if possible.

…heck the parameters for parsing errors.

…pedControlCharacters

Empty (opaque) strings should be quoted if enabled.

See issues: - METROL-779 - METROL 780 -METROL-781 - METROL-782 - METROL-795 - METROL-796 - METROL-797

… set

Also remove unused code

Also disable failing test files.

…st cases. - See issue: METROL-798 - Rename 'test_jsonnumbertype.cpp' to 'test_jsontypes.cpp'

…rayType<>'s Deserialize

A convenience wrapper to define 'ArrayType<>' of 'ArrayType<>' up to depth N for type T

- Add EnumType<> tests - Use 'offset' as starting position - Add Container use case

pwielders

Would like to avoid allocation using new. So far it was never needed hence my quesion what are we fixing that requires this ?

pwielders · 2023-10-24T07:59:48Z

Source/core/JSON.h

+
+// FIXME: Currently fallback to underlying String instead of actual type
+                                        switch(depth) {
+                                        case 1  : element = new ArrayType<String>;


What issue is resolved by these allocations. Allocations definitely on these low levels of functionality, might cause fragmentation of memory. So far no issues have been reported that require this allocation so would like to understand the need of this construction. What is the use-case that gives erroneous output without this allocation construction?

As discussed with @MFransen69 the code path is there not to resolve any reported issue although it shows fields in config.json have no corresponding IElement in the contained list. For debugging purposes and to simplify unit testing for now it has been quite helpful. The use of Add(..) prior Deserialize(..) and ignoring unmatched fields is preferred. A proposed (recommended) approach is to remove the code path from a release delivery and save it elsewhere for use with the aforementioned purposes.

rdkcmf-jenkins · 2023-11-16T15:05:16Z

Blackduck scan failure details

Summary: 0 violations, 0 files pending approval, 1 file pending identification.

Protex Server Path: /home/blackduck/github/Thunder/1433/rdkcentral/Thunder
Commit: 071c318

Report detail: gist

rdkcmf-jenkins · 2023-11-16T16:21:13Z

Blackduck scan failure details

Summary: 0 violations, 0 files pending approval, 1 file pending identification.

Protex Server Path: /home/blackduck/github/Thunder/1433/rdkcentral/Thunder
Commit: 071c318

Report detail: gist

rdkcmf-jenkins · 2023-11-16T16:24:38Z

WARNING: A Blackduck scan failure has been waived

A prior failure has been upvoted

Upvote reason: not a problem
Commit: 071c318

Incorporate new insights, especially, JSON string and opaque string

With surrogate pair encoding and decoding

… unicode

…9f2c657b3d4ea3' from master.

msieben · 2024-06-04T08:46:12Z

Closing the PR but keeping the related branch.

pwielders and others added 30 commits July 7, 2023 22:33

[CONFORMANCE] Use the dedicated JSONRPC error code for incorrect pars…

3de0fa9

…ing.

[JSONRPC] Also rubbish after an opaque string should be considered info.

39f83e6

Starting or ending of an opaque structure should *only* contain white spaces. Anything else should be considered something to be parsed..

[JSON] Fixes in the JSON parsing to detect issues (added bogus data) …

a9c1ba6

…earlier. In the mean time also prevented quoted areas from being accounted in the cope counting and in case we are deserializing in opaque areas drop the white space for further processing and storage.

[WEBSOCKET] Messages coming in over websockets could be corrupted as …

bd15e04

…well.. Checking the validity of the JSONRPC messages coming in over the websockets. If incorrect, rturn an error if possible.

Merge branch 'master' into development/NonHappyDayJSONRPC

00df30a

[JSONRPC] Add to each and every JSONRPC register the possibility to c…

521ebb8

…heck the parameters for parsing errors.

Merge branch 'master' into development/NonHappyDayJSONRPC

608670a

[Core/JSON] : Produce an error on non-escaped control characters.

6518f2a

[FIXES] After testing several issues fixed.

43fae4b

[JSON] Final fix... Finally :-)

a1a6dd5

[CLOSE] Report on JSONRPC Notification on HTTP.

2976049

Merge branch 'development/NonHappyDayJSONRPC' into development/Unesca…

cb7c4b0

…pedControlCharacters

[Core/JSON]: METROL-784.

5ed9d2a

Empty (opaque) strings should be quoted if enabled.

Merge branch 'master' into development/METROL-784

b9313f1

Merge branch 'master' into development/METROL-784

93948b3

Merge branch 'master' into development/METROL-784

7b6c753

[Core] : Improve NumberType<> JSON parsing.

5b1b5f5

See issues: - METROL-779 - METROL 780 -METROL-781 - METROL-782 - METROL-795 - METROL-796 - METROL-797

Merge branch 'master' into development/JSONNumberType

59eebb2

[Core] : Amend '5b1b5f53efd7cc41408beee05e90ead57ca9bfe8'

779d191

[Core] : Amend '779d1917295cf5b59cf5011f05b839b878e49a77'

49a03b7

[Core]: Apply SFINEA on SIGNED and UNSIGNED NumberType<>::Convert(..)

9611cc2

[Core] : Distinghuish SIGNED and UNSIGNED in AddDigitToValue()

10066ce

[Tests/unit/core] : Introduce new JSON NumberType<> test set

c29b58d

[Tests/unit/core] : Enable additional tests in JSON NumberType<> test…

a0480fd

… set

Merge branch 'master' into development/JSONNumberType

57cf0aa

Merge branch 'master' into development/JSONNumberType

1e1c641

[Tests/unit/core] : Add tests to JSON NumberType<> test set

5c07913

Also remove unused code

[Tests/unit/core] : Enable 'test_jsonnumbertype.cpp' in build.

b048a1a

Also disable failing test files.

[Core / Test/unit/core] : Improve FloatType<> JSON parsing and add te…

8c7cccc

…st cases. - See issue: METROL-798 - Rename 'test_jsonnumbertype.cpp' to 'test_jsontypes.cpp'

msieben added 6 commits September 14, 2023 13:17

[Core / Tests/unit/core] : Fix (nested) scopes for ArrayType<>

dde023e

[Core / Tests/unit/core] : Use more expressive variables names for Ar…

2f4b6b1

…rayType<>'s Deserialize

[Core] : Introduce 'NestedArrayType<>'

8cdc41f

A convenience wrapper to define 'ArrayType<>' of 'ArrayType<>' up to depth N for type T

[Core / Tests/unit/core] : improve JSON Container parsing

af4511f

[Core / Tests/unit/core] : Improve JSON parsing and testing

8f660b3

- Add EnumType<> tests - Use 'offset' as starting position - Add Container use case

Merge branch 'master' into development/JSONTypes

1dd7de2

msieben requested review from pwielders and MFransen69 October 20, 2023 09:40

pwielders requested changes Oct 24, 2023

View reviewed changes

MFransen69 and others added 3 commits December 8, 2023 14:50

[Controller] Bring back IController and fix COMRPCStarter (#1436)

7d0085f

[Core / Tests/unit/core] : Improve JSON parsing and testing

7fff524

Incorporate new insights, especially, JSON string and opaque string

[Core / Tests/unit/core] : Amend '071c318002a74a0aa0e7ce2d05fee4b0d64804

789068e

With surrogate pair encoding and decoding

msieben force-pushed the development/JSONTypes branch from 071c318 to 789068e Compare December 8, 2023 15:00

msieben added 5 commits December 13, 2023 18:29

[Core / Tests/unit/core] : Improve surrogate pair code point parsing.

10c3b27

[Core / Tests/unit/core ] : Add UTF-8 compression / decompression for…

c3e5570

… unicode

[Core] : Unify 'unicode' scope variables.

73a1f8d

[ Core / Tests/unit/core/ ] : Cherry pick '7ca5f7a73858483229ac2bad4a…

f1b0f6d

…9f2c657b3d4ea3' from master.

Merge branch 'master' into development/JSONTypes

3a3f864

msieben closed this Jun 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Development/json types #1433

Development/json types #1433

msieben commented Oct 20, 2023

pwielders left a comment

pwielders Oct 24, 2023

msieben Oct 27, 2023

rdkcmf-jenkins commented Nov 16, 2023

rdkcmf-jenkins commented Nov 16, 2023

rdkcmf-jenkins commented Nov 16, 2023

msieben commented Jun 4, 2024

Development/json types #1433

Development/json types #1433

Conversation

msieben commented Oct 20, 2023

pwielders left a comment

Choose a reason for hiding this comment

pwielders Oct 24, 2023

Choose a reason for hiding this comment

msieben Oct 27, 2023

Choose a reason for hiding this comment

rdkcmf-jenkins commented Nov 16, 2023

Blackduck scan failure details

rdkcmf-jenkins commented Nov 16, 2023

Blackduck scan failure details

rdkcmf-jenkins commented Nov 16, 2023

WARNING: A Blackduck scan failure has been waived

msieben commented Jun 4, 2024