Over the years, I've found myself using Virtual Machines for running reverse engineering and firmware analysis tools. Unfortunately, I also keep thinking "A 32GB disk will last for a while." Each time, I have to search and find the tools and dependencies I need to get started again using - in particular - binwalk. Binwalk can be tricky to install in its most useful state, as it uses unpacking and disk image tools provided by the operating system instead of bundling in dozens of projects you might not need.
Driven by a bad habit of distro hopping as well, I've found that the ReFirm Labs-provided dependency installer script is selective about which operating systems it will run on successfully. Partly for my own convenience and partly to advocate for a more varied use of Linux distributions (monocultures make for easier targets), I have collected bits of the dependency installers, located fixes, and stripped out extra details to create copy-paste (or scripting) friendly blocks to quickly get binwalk installed.
There were a few issues trying to create a single install script, most of which are interesting to consider when writing generic tooling to be used on Linux in general. The “Python Versioning” problem dives into technical errata, while the “Source Control” problem is more cultural/social.
Python Versioning
The first speed bump turned out to be how differently these distributions handle Python. The provided dependency installer script wants to run `python` and the official Python standards documentation for what version of Python is started when running `python` is from 2011, long before the Python 2.x series was discontinued and before widespread Python 3.x acceptance. Ubuntu and Debian will ask if you meant to type `python3` or install the `python-is-python3` package. Fedora will run version 3. OpenSUSE will run version 2, and `python3` is the correct way to run version 3.
Ubuntu, Debian and OpenSUSE needed an extra installation step or two here. For Ubuntu and Debian, it is as easy as installing python-is-python3 while gathering dependencies. However, OpenSUSE was a bit more of an issue. My solution is to symlink python3 to the user’s ~/bin/python in order to handle calling python in additional scripts, and explicitly call python3 for any command requiring sudo during installation. This should minimize its overall effect on the system while creating a functional binwalk environment. I am now even more certain that I will be explicit in which version of Python to use for any tooling I write in the future.
Source Control
To keep this brief, I will acknowledge that without pinning an exact commit, tag, or release while pulling in external code, one can get newer versions of these tools without having to update one’s own scripts, as the external changes are pulled in each time. However, the external changes are pulled in each time. The two biggest changes:
- Indentation changes in one codebase caused compiling on Ubuntu to throw a warning, which was treated as an error, breaking the build. Solution: find a forked repository with the fix (pull requests to the original project have been waiting for months)
- A legacy “master” branch was moved to a different “main” branch of another user account, and git couldn’t figure out both changes at once. Solution: install the pip version of this tool instead.
I am sure there are plenty of other blog posts out there with opinions on how to “solve” issues similar to these.
Ubuntu 22.04
#packages from distro repo
sudo apt install git locales build-essential qtbase5-dev mtd-utils gzip bzip2 tar arj lhasa p7zip p7zip-full cabextract cramfsswap squashfs-tools zlib1g-dev liblzma-dev liblzo2-dev sleuthkit default-jdk lzop srecord cpio pip python-is-python3
#packages from pip
sudo pip install setuptools matplotlib capstone pycryptodome gnupg tk ubi_reader
#sasquatch from source - repository with committed patch fixing a build warning treated as an error
git clone --quiet --depth 1 --branch "master" https://github.com/threadexio/sasquatch
(cd sasquatch && sudo ./build.sh)
sudo rm -rf sasquatch
#yaffshiv
git clone --quiet --depth 1 --branch "master" https://github.com/devttys0/yaffshiv
(cd yaffshiv && sudo python setup.py install)
sudo rm -rf yaffshiv
#jefferson
git clone --quiet --depth 1 --branch "master" https://github.com/sviehb/jefferson
(cd jefferson && sudo python -mpip install -r requirements.txt && sudo python setup.py install)
sudo rm -rf jefferson
OR
Fedora
#packages from distro repo
sudo yum install git gcc gcc-c++ make openssl-devel qtwebkit-devel qt-devel gzip bzip2 tar arj p7zip p7zip-plugins cabextract squashfs-tools zlib zlib-devel lzo lzo-devel xz xz-libs xz-devel xz-lzma-compat lzip perl-Compress-Raw-Lzma lzop srecord pip patch python3-devel
#packages from pip
sudo pip install setuptools matplotlib capstone pycryptodome gnupg tk ubi_reader
#sasquatch from source - repository with committed patch fixing a build warning treated as an error
git clone --quiet --depth 1 --branch "master" https://github.com/threadexio/sasquatch
(cd sasquatch && sudo ./build.sh)
sudo rm -rf sasquatch
#yaffshiv
git clone --quiet --depth 1 --branch "master" https://github.com/devttys0/yaffshiv
(cd yaffshiv && sudo python setup.py install)
sudo rm -rf yaffshiv
#jefferson
git clone --quiet --depth 1 --branch "master" https://github.com/sviehb/jefferson
(cd jefferson && sudo python -mpip install -r requirements.txt && sudo python setup.py install)
sudo rm -rf jefferson
OR
OpenSUSE Tumbleweed
#packages from distro repo
sudo zypper install git gcc gcc-c++ make openssl-devel gzip bzip2 tar arj p7zip cabextract zlib zlib-devel lzo lzo-devel xz xz-devel lzip perl-Compress-Raw-Lzma lzop srecord patch python3-devel
ln -s /usr/bin/python3 ~/bin/python
#packages from pip
sudo pip install setuptools matplotlib capstone pycryptodome gnupg tk ubi_reader
#Sasquatch from source - repository with committed patch fixing a build warning treated as an error
git clone --quiet --depth 1 --branch "master" https://github.com/threadexio/sasquatch
(cd sasquatch && sudo ./build.sh)
sudo rm -rf sasquatch
#yaffshiv, python3 explicitly
git clone --quiet --depth 1 --branch "master" https://github.com/devttys0/yaffshiv
(cd yaffshiv && sudo python3 setup.py install)
sudo rm -rf yaffshiv
#jefferson, python3 explicitly
git clone --quiet --depth 1 --branch "master" https://github.com/sviehb/jefferson
(cd jefferson && sudo python3 -mpip install -r requirements.txt && sudo python3 setup.py install)
sudo rm -rf jefferson
OR
Debian 12
After making sure you're able to use sudo or are already root,
sudo apt install python-is-python3 python3-pip
Then, the binwalk-packaged deps.sh script functions as intended without needing extra care and feeding
OR
Arch Linux
# please don't take the time to install Arch from scratch each time your VM runs out of disk space just to install binwalk.
#However, if anything goes wrong trying to install packages in other distros, the Arch wiki is an amazing resource
