Update Driver to existing Binary of specified GUID #12

YuweiChen1110 · 2023-04-28T08:12:57Z

Functions Introduction Functions:
1. Parse the input FV file into a origin binary tree. Currently, we just support FV level replacement operation, so we define two different types of tree nodes - FV/ FFS.
2. Given a new FFS file , parse the new FFS file into a new binary tree and get the new FFS Node from the new binary tree.
3. Find target FFS node from the origin binary tree with new FFS's driver guid.
4. Replace the target FFS node with New FFS node in the binary tree.
5. Encapsulate binary with replaced FFS.
  Design thoughts:
  A binary tree will be created for binary file to include all the info.
  The whole binary file will be seen as a Root Node of the tree. When parse the binary, the FV and FFS part will be seen as the FVNode/FFSNode which is the Child of the ROOT/FVNode.
  With the tree, both the structure and data of ROOT/FV/FFS... can be saved which can be used for function extensions.
Use method Method 1:
Step 1: Copy the origin "bios_image.bin" & target "driver_image.ffs" (The file name can not be changed) into the XmlCli repo root folder.
Step 2: Open windows console with XmlCli repo root folder, then run "py -3 src\xmlcli\common\bios_fw_parser.py".
Then the expected output binary file "replaced_image.bin" will be generated into the root folder.
Method 2:
Step 1: Copy the origin "bios_image.bin" & target "driver_image.ffs"( with your own file name )into the XmlCli repo root folder.
Step 2: Open windows console with XmlCli repo root folder, then run "py -3 src\xmlcli\common\bios_fw_parser.py -r $(input_binary_file) $(target_driver_ffs).ffs $(output_binary_file)".
Then the expected output binary file $(output_binary_file) will be generated into the root folder.
Known Issue 3.1 Only support replace the first find driver ffs. 3.2 Only support replace the same guid driver ffs file. 3.3 The entry extension in xmlcli_registry_listener.py does not been complemented.
Extension direction 4.1 All the Known Issues can be enhanced with more details requirements descriptions. 4.2 The binary tree can be extended to save all level's FV and FFS and Section Node through binary parsing. 4.3 The existed binary Tree can be used to extend more functions like "Add a new driver / Delete a driver...".

1. Functions Introduction Functions: 1. Parse the input FV file into a origin binary tree. Currently, we just support FV level replacement operation, so we define two different types of tree nodes - FV/ FFS. 2. Given a new FFS file , parse the new FFS file into a new binary tree and get the new FFS Node from the new binary tree. 3. Find target FFS node from the origin binary tree with new FFS's driver guid. 4. Replace the target FFS node with New FFS node in the binary tree. 5. Encapsulate binary with replaced FFS. Design thoughts: A binary tree will be created for binary file to include all the info. The whole binary file will be seen as a Root Node of the tree. When parse the binary, the FV and FFS part will be seen as the FVNode/FFSNode which is the Child of the ROOT/FVNode. With the tree, both the structure and data of ROOT/FV/FFS... can be saved which can be used for function extensions. 2. Use method Method 1: Step 1: Copy the origin "bios_image.bin" & target "driver_image.ffs" (The file name can not be changed) into the XmlCli repo root folder. Step 2: Open windows console with XmlCli repo root folder, then run "py -3 src\xmlcli\common\bios_fw_parser.py". Then the expected output binary file "replaced_image.bin" will be generated into the root folder. Method 2: Step 1: Copy the origin "bios_image.bin" & target "driver_image.ffs"( with your own file name )into the XmlCli repo root folder. Step 2: Open windows console with XmlCli repo root folder, then run "py -3 src\xmlcli\common\bios_fw_parser.py -r $(input_binary_file) $(target_driver_ffs).ffs $(output_binary_file)". Then the expected output binary file $(output_binary_file) will be generated into the root folder. 3. Known Issue 3.1 Only support replace the first find driver ffs. 3.2 Only support replace the same guid driver ffs file. 3.3 The entry extension in xmlcli_registry_listener.py does not been complemented. 4. Extension direction 4.1 All the Known Issues can be enhanced with more details requirements descriptions. 4.2 The binary tree can be extended to save all level's FV and FFS and Section Node through binary parsing. 4.3 The existed binary Tree can be used to extend more functions like "Add a new driver / Delete a driver...". Signed-off-by: Yuwei Chen <yuwei.chen@intel.com> Signed-off-by: Yuting Yang <yuting2.yang@intel.com>

YuweiChen1110 · 2023-04-28T08:46:05Z

Update the Descriptions:

Design Ideas:
A multiway tree will be created for input bios image file to include all the info.
The whole bios image will be seen as the root node of the tree. When parse the image, the FV and FFS part in it will be seen as the node (FVNode/FFSNode) which is the child of the root/FVNode.
With the tree, both the structure and data of different level layout ROOT/FV/FFS... can be saved which can be used for function extensions.
Feature Introductions
1. Parse the input bios image binary file into an origin multiway tree.
PS: Currently, only root FFS level replacement operation is supported, so only two different types of tree nodes are defined FV/ FFS, the multiway tree only have FV/FFS Node.
2. Parse new driver FFS file. Given a new FFS file, the file can be parsed into a multiway tree with defined FFS Node which have the FFS info got from binary file.
3. Find FFS node. Go through the whole bios image tree to find the target FFS (which will be replaced by our new driver) with specific guid (get from new driver file).
4. Replace the target FFS node with new driver FFS node in the multiway tree.
5. Encapsulate multiway tree which has the replaced FFS into the output bios image.
Code Function Descriptions
1. uefi_parser.parse_binary(): extend the parser to parse both bios image and driver ffs file:
We add the multiway tree establishment process into this function, and the input file of this function is extended to both FV and FFS file. All FV/FFS tree nodes with both multiway data and position info can be collected by this function.
2. uefi_parser.find_ffs_node()
This function is used to find target FFS Node in the multiway tree.
3. uefi_parser.ReplaceFfs()
This function is used to replace target FFS Node with given new FFS node.
4. uefi_parser.Encapsulate_binary()
This function is used to encapsulate multiway tree data into the byte buffer.
5. uefi_parser.dump_binary()
This function is used to dump the multiway tree data info into a file.
Use Methods
Method 1:
Step 1: Copy the origin "bios_image.bin" & target "driver_image.ffs" (The file name can't be changed) into the XmlCli repo root folder.
Step 2: Open windows console with XmlCli repo root folder, then run "py -3 src\xmlcli\common\bios_fw_parser.py".
Then the expected output bios image file "replaced_image.bin" which has the replaced driver ffs will be generated into the root folder.
Method 2:
Step 1: Copy the origin "bios_image.bin" & target "driver_image.ffs" (with your own file name) into the XmlCli repo root folder.
Step 2: Open windows console with XmlCli repo root folder, then run "py -3 src\xmlcli\common\bios_fw_parser.py -r $(input_bios_image) $(target_driver_ffs).ffs $(output_bios_image)".
Then the expected output bios image file $(output_bios_image) will be generated into the root folder.
Known Issues
1. Only support replace the first find driver ffs.
2. Only support replace the same guid driver ffs file.
3. The entry extension in xmlcli_registry_listener.py does not been complemented.
Extension directions
1. All the Known Issues can be enhanced with more details requirements descriptions.
2. The multiway tree can be extended to save all level's FV and FFS and Section Node through binary parsing.
3. The existed multiway tree can be used to extend more functions like "Add a new driver / Delete a driver...".
4. More features that you want ....

gahan9 · 2023-04-30T03:10:27Z

src/xmlcli/common/bios_fw_parser.py

 import json
 import shutil
 from collections import namedtuple

+sys.path.append(".")


would this method be working when installed module as Python package on system?

src/xmlcli/common/tree_and_node.py

+# -*- coding: utf-8 -*-
+
+# Built-in imports
+from collections import namedtuple


tests/UefiParserTest.py

+                                    )
+
+    # parse bios image into a binary_tree
+    bios_output_dict = uefi_parser.parse_binary()


tests/UefiParserTest.py

+    bios_output_dict = uefi_parser.parse_binary()
+
+    # parse driver ffs image into a binary tree node
+    ffs_output_dict = newffs_parser.parse_binary()


src/xmlcli/common/bios_fw_parser.py

+                                  )
+
+  # parse bios image into a binary_tree
+  bios_output_dict = uefi_parser.parse_binary()


src/xmlcli/common/bios_fw_parser.py

+  bios_output_dict = uefi_parser.parse_binary()
+
+  # parse driver ffs image into a binary tree node
+  ffs_output_dict = newffs_parser.parse_binary()


gahan9 · 2023-04-30T03:11:53Z

src/xmlcli/common/bios_fw_parser.py

    log.result(self.stored_guids)
    return self.output

-  def parse_firmware_volume(self, buffer, buffer_pointer, end_point, nesting_level=0, is_compressed=False, **kwargs):
+  def parse_firmware_volume(self, buffer, buffer_pointer, end_point, par_tree_node=None, nesting_level=0, is_compressed=False, is_root_level=False, **kwargs):


better to add new argument as suffix of function to maintain API function argument order

gahan9 · 2023-04-30T03:12:58Z

src/xmlcli/common/bios_fw_parser.py

@@ -282,22 +297,30 @@ def parse_firmware_volume(self, buffer, buffer_pointer, end_point, nesting_level
    firmware_volume_header = fv.read_from(buffer)  # read header structure from buffer
    log.debug("FV parsed...")

+    if (firmware_volume_header.HeaderLength - 56) // 8 != 1:


what is logic with numbers 56 and 8 ?

gahan9 · 2023-04-30T03:15:43Z

src/xmlcli/common/utils.py

@@ -19,8 +19,8 @@
 from collections import OrderedDict

 # Custom imports
-from .logger import log
-from .configurations import XMLCLI_CONFIG, ENCODING, XMLCLI_DIR, OUT_DIR, PY3, STATUS_CODE_RECORD_FILE
+from logger import log


would it work when installed as python site package module?

gahan9 · 2023-04-30T03:17:12Z

src/xmlcli/common/tree_and_node.py

@@ -0,0 +1,131 @@
+# -*- coding: utf-8 -*-


This file breaks PEP-8 naming convention guideline

gahan9

We need to ensure below before merging changes:

Follows naming convention as per PEP:8 guideline
Should work when installed as pip module
Can it be convenient if it can have modular API calls for operation of delete followed by update? so that same code flow be used as 2 features.
In case need of command line implementation, entry point https://github.com/intel/xml-cli/blob/main/setup.cfg#L35 can be used

gahan9 · 2023-05-16T03:15:52Z

This Updates Driver with GUID from another binary to fulfil usecase where:

User has label01.bin which is working BIOS
User tried label_different.bin which can be newer or older but eventually it does not have working driver which is expected by user.
1. Eventually This would likely to go with failure.
2. To aid regression without modifying code, this can replace driver from working firmware

TODO:

Test changes with working setups.

YuweiChen1110 requested a review from gahan9 as a code owner April 28, 2023 08:12

gahan9 reviewed Apr 30, 2023

View reviewed changes

github-advanced-security bot found potential problems Apr 30, 2023

View reviewed changes

gahan9 reviewed Apr 30, 2023

View reviewed changes

src/xmlcli/common/tree_and_node.py

@@ -0,0 +1,131 @@

# -*- coding: utf-8 -*-

Copy link

Member

gahan9 Apr 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file breaks PEP-8 naming convention guideline

gahan9 reviewed Apr 30, 2023

View reviewed changes

Merge branch 'main' into XmlCli_Submit0

409d636

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Driver to existing Binary of specified GUID #12

Update Driver to existing Binary of specified GUID #12

YuweiChen1110 commented Apr 28, 2023

YuweiChen1110 commented Apr 28, 2023

gahan9 Apr 30, 2023

gahan9 Apr 30, 2023

gahan9 Apr 30, 2023

gahan9 Apr 30, 2023

gahan9 Apr 30, 2023

gahan9 left a comment

gahan9 commented May 16, 2023

Update Driver to existing Binary of specified GUID #12

Are you sure you want to change the base?

Update Driver to existing Binary of specified GUID #12

Conversation

YuweiChen1110 commented Apr 28, 2023

YuweiChen1110 commented Apr 28, 2023

gahan9 Apr 30, 2023

Choose a reason for hiding this comment

gahan9 Apr 30, 2023

Choose a reason for hiding this comment

gahan9 Apr 30, 2023

Choose a reason for hiding this comment

gahan9 Apr 30, 2023

Choose a reason for hiding this comment

gahan9 Apr 30, 2023

Choose a reason for hiding this comment

gahan9 left a comment

Choose a reason for hiding this comment

gahan9 commented May 16, 2023