Split commands from shell script in dockerfile

[ForgetMe17]

Description
As the first step to parse Dockerfile RUN instruction commands, we need to split commands from a dockerfile RUN commands. A full shell script contains these elements,

• variable assignment : eg. dpkgArch="$(dpkg --print-architecture)";
• command line: eg. wget -O go.tgz "$url";
• branch: eg. case .... esac;, if ... fi;
• function: eg. foo(){.....}
  In this task, we need to parse a shell script into above parts and store them, possible in a List. For branches, we may need another task to parse its branches into variable assignment and command line.

To Do
Given a full shell script from RUN instruction of dockerfile, return a data structure that stores the parsing result. Most modifications will be made in general.py.

• We can add a function called parse_shell_script() in general.py as a main function. It receives a string of RUN instruction, which is a shell script. It returns a command dict containing all the possible command in the given shell script.
• Once the above command dict is given, we can modify the parse_command() in general.py to parse the command dict to find possible installed software. This could be implemented by the following task issues.

Something need to be discussed:

• Data structure can be List.

1. Maybe we can define a class for the parsed elements. It has a property shows what is this element, like variable assignment or command line etc. We can implement further parse like parsing the branch as the class function.

2. Or maybe we can use dict as the elements, it has a key called statement whose value can be variable assignment or command line etc. And we add new functions in general.py.

• How to handle branch.
  I am thinking adding a extra key pr property for the command line indicating if it is in a branch.
  At first we can take the default branch as the command for parse. In the further plan, we can add more analyze on this like considering varibles, or if this branch only infects the version we install but not the software name we install.
• Should we consider function?

Background
On looking at the type of parsing needed for full shell scripts embedded in the run command, we may need to develop a shell script parser to catch all places where software could have been installed.

Super Issues
#521

[nishakm]

An example of RUN commands in a Dockerfile that is difficult to parse:
https://github.com/docker-library/python/blob/06d7952f07eae237666f2fd0ccf6189a5d8ee83c/3.9-rc/buster/Dockerfile

[ForgetMe17]

Here is a shell script containing loop, and the current output is
buster-slim.txt.
In line 63, the for loop is treated as a command, so we need to consider adding another element loop to the parsing procedure. I have file an issue #706 to fix this.

[ForgetMe17]

Fixed in PR #717. Closing.