System emulator updates

[m-1-k-3]

• What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

Feature

• What is the current behavior? (You can also link to an open issue here)

Sticking on the outdated kernel v4.1.17 which was used by FirmAE with some patches provided from FirmAE and firmadyne.

• What is the new behavior (if this is a feature change)? If possible add a screenshot.

We include new kernels (v4.14.336 v4.1.52) and new compiled binaries for L10
As we have seen issues with out x86 kernel we switch automatically to kernel v4.1.17
As we are currently experiencing issues with the new kernel we will include the old 4.1.17 as well as 4.14.336 4.1.52 in the latest base image. The 4.1.17 will stay as the default kernel till we solved the issues
New netcat listener can be enabled in system emulation via kernel parameter (this can be used in cases were the local shell does not work)
Version fixes for binary detection - see also #1138
Switch binwalk installation to new fork here https://github.com/OSPG/binwalk
Introduce priority in identified or guessed L10 network config
Add multiple new backup network configurations with debug mode - results in longer l10 runs if debug mode is on
Rewrite variables from FIRMAE to EMBA
Add gdb and gdbserver to the firmware binaries environment
Update busybox to v1.36.1 in firmware binaries environment
Included multiple patches from here to our nvram emulation environment - see here
Update docker base image to kali 2024-2 - closes #1187
Installer is now able to handle available exernal directory

• Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

Absolute! After years of sticking on the outdated kernel v4.1.17 we started migrating to a much newer kernel v4.14.336 v4.1.52 from 01/2024 2018 - see also here.

Note: the default kernel is for now the 4.1.17, not the new 4.14.336. To use the new 4.14.336 you need to adjust the setting in the default settings helper file. We have seen massive performance issues with the new kernel!

Update: As we were faced with massive issues with newer kernel we decided to move forward with teh v4.1.52 (from 2018) which is a little update of two years compared to the the original v4.1.17 (from 2016). With this update landed we can spend more time in further testing and evaluation.

[m-1-k-3]

Outdated - Update will be available soon!
we are now in a state to do some basic public tests. There is a new docker base image available and it should be possible to start the installation (from this branch) with the following commands:

└─$ git pull https://github.com/m-1-k-3/emba.git --branch system_emulation_updates
└─$ sudo ./installer.sh -d -c embeddedanalyzer/emba:1.4.1a

Afterwards, the installation should download the EMBA image version 1.4.1a (which is currently under test but not verified)

If this works as expected you should be able to perform EMBA firmware scans with the new docker image.
Please check scans with the following scan profiles:

scan-profiles/default-scan.emba
scan-profiles/default-scan-emulation.emba

The emulation profile should now use the new binaries and the new kernel:

Be warned: The emulation results are not that good as we have seen them with the old kernel. We are currently looking into the details.

Feedback is highly appreciate

[m-1-k-3]

We are now in a state to do some basic public tests. There is a new docker base image available and it should be possible to start the installation (from this branch) with the following commands:

└─$ git pull https://github.com/m-1-k-3/emba.git --branch system_emulation_updates
└─$ sudo ./installer.sh -d

Afterwards, the installation should download the EMBA image version 1.4.1b (which is currently under test but not verified)

└─$ sudo docker tag embeddedanalyzer/emba:1.4.1b embeddedanalyzer/emba:latest                                                          
└─$ sudo docker images                             
REPOSITORY               TAG       IMAGE ID       CREATED          SIZE
embeddedanalyzer/emba    1.4.1b    a677cf89098e   15 minutes ago   13.5GB
embeddedanalyzer/emba    latest    a677cf89098e   15 minutes ago   13.5GB

If this works as expected you should be able to perform EMBA firmware scans with the new docker image.
Please check scans with the following scan profiles:

scan-profiles/default-scan.emba
scan-profiles/default-scan-emulation.emba

The emulation profile should now use the new binaries and the new kernel:

Feedback is highly appreciate

[m-1-k-3]

New base image available (1.4.1c) and enforced from the dependency checker!
PR ready for review

@BenediktMKuehne @HoxhaEndri @torabi12 please give it a try

After landing this we will make it stable and hunt for bugs in the next week and then move on to version 1.4.1 :)