Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add apt/brew package info for binwalk and {mk,un}squashfs #418

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

ANogin
Copy link
Contributor

@ANogin ANogin commented Feb 10, 2024

One sentence summary of this PR (This should go in the CHANGELOG!)
Add apt/brew package info for binwalk and mksqashfs/unsquashfs

Link to Related Issue(s)
N/A

Please describe the changes in your request.
Added apt/brew package info for binwalk and mksqashfs/unsquashfs - not sure why it was not there already?

Anyone you think should look at this, specifically?
Not sure

@ANogin ANogin changed the title Add apt/brew package info for binwalk Add apt/brew package info for binwalk and {mk,un}squashfs Feb 10, 2024
@rbs-jacob
Copy link
Member

Binwalk apt/brew package info not included because it is assumed binwalk is installed from source, not from either of those package managers.

binwalk @ git+https://github.com/ReFirmLabs/binwalk.git@v2.3.4

If I recall correctly, the apt version of binwalk in the base Docker images we use is insanely out of date, and caused some problems at some point.

But also, in general, having as many of the dependencies as possible installed via pip (rather than using system package managers) is preferable in my opinion. It means that a user who does pip install ofrak gets many of the dependencies, even if they don't do further installation steps using their system package manager. It also means OFRAK is more portable to systems with package managers for which we do not have explicit support.

@ANogin
Copy link
Contributor Author

ANogin commented Feb 12, 2024

@rbs-jacob makes sense. Unfortunately binwalk in PyPI is even more out of date, so we end up having to have the ofrak/ofrak_core/requirements-non-pypi.txt (since PyPI disallows a dependency on a non-PyPI package :( ).

Does apt/brew make sense to include for {mk,un}squashfs? If yes, I will mutate this PR to just that (or should I do a fresh one instead)?; if no, should just close this PR.

@rbs-jacob
Copy link
Member

Tough to say about {mk,un}squashfs. We do build a specific version in Docker, I believe because apt ones were too out-of-date to have support for a particular command-line flag we need. But I don't know if that's necessarily a good case for not having the brew/apt package names listed in OFRAK.

# Install the correct version of squashfs-tools. We specifically need the
# "-no-exit" argument, which is only available in version 4.5+
RUN cd /tmp && \
git clone https://github.com/plougher/squashfs-tools.git && \
cd squashfs-tools/squashfs-tools && \
git checkout 4.5.1 && \
sed -i 's/^#\(XZ\|LZO\|LZ4\|ZSTD\)_SUPPORT/\1_SUPPORT/g' Makefile && \
make -j && \
make install && \
cd /tmp && \
rm -r squashfs-tools

I defer to @whyitfor and/or @EdwardLarson on this one.

@ANogin
Copy link
Contributor Author

ANogin commented Feb 12, 2024

Actually, brew does install a recent binwalk (2.3.3) and squashfstools v4.6.1 (newer than what we do), so probably should be included.

For apt:

  • Ubuntu 20.04 installs binwalk 2.2.0+dfsg1 and squashfs-tools 4.4
  • Ubuntu 22.04 installs binwalk 2.3.3+dfsg1 and squashfs-tools 4.5

so perhaps good enough at least in some cases?

P.S. Broader longer-term question - should ofrak deps be capable of checking and then reporting to the user when a dependency is present, but is too old to be usable? At least for some of these, where the OS is likely to come with ones that are too old?

@rbs-jacob
Copy link
Member

In the case of unsquashfs, the ofrak deps command will check that a new enough version is used.

class _UnsquashfsV45Tool(ComponentExternalTool):
def __init__(self):
super().__init__("unsquashfs", "https://github.com/plougher/squashfs-tools", "")
async def is_tool_installed(self) -> bool:
try:
cmd = ["unsquashfs", "-help"]
proc = await asyncio.create_subprocess_exec(
*cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.DEVNULL,
)
stdout, stderr = await proc.communicate()
except FileNotFoundError:
return False
if 0 != proc.returncode:
return False
if b"-no-exit" not in stdout:
# Version 4.5+ has the required -no-exit option
return False
return True

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants