Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
Initial release
The project code was hard-forked and moved to Kaspersky Lab repository.
- Loading branch information
Vitaly Kamluk
committed
Feb 6, 2018
1 parent
96eb824
commit 8ab4be8
Showing
90 changed files
with
4,422 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
[submodule "resources/kernel/writeblocker"] | ||
path = resources/kernel/writeblocker | ||
url = git://github.com/vitaly-kamluk/Linux-write-blocker | ||
[submodule "resources/apt-fast"] | ||
path = resources/apt-fast | ||
url = git://github.com/vitaly-kamluk/apt-fast |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,85 @@ | ||
Project Bitscout | ||
Author: Vitaly Kamluk // bitscout[at]kaspersky.com | ||
|
||
This file briefly describes the structure of the project for developers. | ||
|
||
Base system | ||
Current base system is Ubuntu 16.04 LTS. | ||
|
||
Virtualization | ||
Bitscout relies on LXC 1.0 for virtualization. One option was to use full VM, but | ||
given that we are runing from live media, it had to be something lightweight | ||
and optimal. LXC perfectly works here, because kernel becomes shared between the | ||
container and the host. Also, kernel mechanisms for IPC, such as file locking | ||
and pipes can be used for IPC between the host and the guest. We used | ||
unprivileged container that runs as user. However, we need to provide rootfs to | ||
such container and as far as we used the same source from squashfs, we had to | ||
remap uid/gid for some otherwise inaccessible files. So far the script that | ||
starts the container does this job and remaps uid/gid to subuid/subgid of the | ||
target user for the whole root filesystem. This lets us avoid data duplication | ||
in memory. | ||
|
||
Optimisation | ||
We aimed to make the system optimal in size. You may find extensive usage of | ||
overlayfs, compressed memory block device (zram) and more tuning to save space. | ||
Both host and guest system run from the same source image, but each of the | ||
system can be independently extended or modified. | ||
|
||
Extension | ||
The system is designed to be extended in memory. Both host and guest users can | ||
install additional packages in RAM. | ||
|
||
Customisation | ||
The best way to customize your own build of Bitscout is to create additional | ||
script in scripts directory. Unless you suggest a bugfix, please stick to | ||
extension of the code, rather than patching existing pieces. And you are very | ||
welcome to submit you suggested fixes and improvements. We know there is a lot | ||
to be done to improve security, stability and usability of the project. | ||
When you make a new task to modify rootfs or other things, please write in the | ||
way that doesn't apply changes twice if the same script is executed again. | ||
Current scripts follow this idea, this allows to make tiny changes and rerun the | ||
subscripts without risk to break the final system. I.e. you can make some change to | ||
the way how container is created (./scripts/chroot_create_container.sh) and then | ||
manually run ISO rebuild (./scripts/image_build.sh). That shall generate you the | ||
new ISO file with your changes applied without the need to run all other scripts. | ||
|
||
Structure and description of the files: | ||
automake.sh - this is core script that runs others. Feel free to run it whole or | ||
part of it (i.e. to avoid recurring base system download). | ||
|
||
scripts/ - this is directory with scripts to do main tasks. Some are prefixed | ||
according to current stage of execution: | ||
welcome.sh - the initial global configuration script and welcome message. | ||
chroot_* - scripts to install, configure and change root filesystem | ||
image_* - scripts to pack rootfs and generate ISO file | ||
export_generate.sh - prepare configs for other systems (VPN server, expert) | ||
initrd_* - these can pack/unpack initrd image. Used for initrd patches. | ||
chroot_enter.sh/chroot_leave.sh - these scripts can be used if you would like | ||
to run shell in chrooted environment. For the sake of automation it | ||
relies on screen tool that runs as root. To use this, run the first | ||
script and then "sudo screen -r" to join the screen session. To | ||
leave it simply run scripts/chroot_leave.sh from a separate shell. | ||
And, yes, if you know a better way to do that, please suggest! | ||
functions - this file is included by others, provides most common settings and | ||
global utility functions, i.e. running commands in chrooted screen | ||
session. | ||
|
||
|
||
Running scripts | ||
Currently all scripts have to be called from the root directory of the project, i.e. | ||
$ ./scripts/image_build.sh | ||
|
||
How to contribute | ||
We'd love to have you contribute! Our goal is to have a builder that can build | ||
the smallest Live OS useful for forensics. It would be nice to further reduce | ||
the size of the ISO file to te best possible minimum. If you are into it, look | ||
into image_prebuild_cleanup.sh. Perhaps you could bring it to a new level! | ||
One thing to consider: we need to keep essential features, which include: | ||
1. Disk controllers' and RAID modules | ||
2. Network devices | ||
3. LXC support | ||
4. Filesystems support | ||
|
||
If you want to try your ideas, make sure you read these articles first: | ||
https://wiki.debian.org/ReduceDebian | ||
https://wiki.ubuntu.com/ReducingDiskFootprint |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,205 @@ | ||
Project Bitscout | ||
Author: Vitaly Kamluk // bitscout[at]kaspersky.com | ||
|
||
Bitscout is customizable live OS constructor tool written purely in bash. It's | ||
main purpose is to help you quickly create own remote forensics bootable disk | ||
image. | ||
|
||
This project was created by security researchers for security researchers and | ||
incident handlers. Do not expect user-friendly interface and if you are not | ||
familiar with Linux commandline, it's wise idea to learn that first. This | ||
constructor can be customised to include your tools, however one of the core | ||
ideas was to remotely assist Law Enforcement investigations as well as incident | ||
responders, which is why Bitscout by default includes a number of forensic | ||
packages and settings. | ||
|
||
Basic usage: | ||
|
||
1. Build new ISO file: | ||
$ ./automake.sh | ||
|
||
After running this command, you may need to answer some questions such as | ||
location of your VPN server, type of build, etc. | ||
|
||
Hint: if you are not running Linux and want to try Bitscout, you can start | ||
Ubuntu from LiveCD. It works perfectly fine, but we suggest you to have at least | ||
2Gb+ RAM in this case. To download Ubuntu LiveCD, please go here: | ||
https://www.ubuntu.com/download/desktop | ||
|
||
Bitscout build demo from Ubuntu LiveCD (older version of Bitscout): | ||
https://www.youtube.com/watch?v=knA0NS9tWsY | ||
|
||
Note: the automake.sh script runs some commands as root, such as mounting local | ||
cache directories and creating new root filesystem permissions. However, all | ||
changes shall affect only current directory and subdirectories, unless your | ||
system is missing some essential packages to build the ISO (in this | ||
case they will be installed). | ||
|
||
2. Test new ISO file: | ||
$ ./autotest.sh | ||
|
||
This command shall run tests against freshly build ISO file. It verifies | ||
components' presence on the ISO and attempts to boot the ISO file using qemu to | ||
verify that all essential services are running. | ||
|
||
|
||
To better understand what Bitscout is, we suggest you read further description | ||
of users' roles in the process and the FAQ below. | ||
|
||
User roles: | ||
Bitscout relies on at least three components in the process of remote forensics: | ||
1. The owner | ||
The owner is a user who has physical access to the target system and owns it. | ||
The owner's role is to download, verify and burn Live ISO image file to a | ||
removable storage (CD-Rom or USB). After that the target system must be | ||
started from this bootable media. In case of LAN DHCP network configuration | ||
everything shall work automatically. In case of other setup, the owner has | ||
to configure network access using simple management interface that is | ||
brought up on physical console once Bitscout is loaded. | ||
|
||
2. The expert | ||
The expert is a remote user who connects to the target system over SSH using | ||
VPN link via the expert's server. Bitscout attempts to find existing VPN | ||
configuration and SSH keys in ./config directory. If it doesn't exist it | ||
will get the default config and generate new VPN certificates and SSH keys | ||
during the build. | ||
|
||
Hint: let Bitscout generate new keys for you and populate ./config | ||
directory, which you can customise later and rebuild the ISO by running | ||
automake.sh again. | ||
|
||
3. Expert's server | ||
The expert's server shall be accessible from the network of target system. | ||
It shall run an instance of OpenVPN server, and recommended to have an IRC | ||
chat server for communication. Suggested server configuration files can be | ||
found in the ./exports directory after successfull build of ISO file. | ||
|
||
Bitscout Features: | ||
1. Transparency | ||
a. You build your own live disk instead of using someone else's. The build | ||
process is rather straightforward and detailed. One of the core principles of | ||
Bitscout is to not use proprietary binary executables during build process. | ||
Project Bitscout is a plaintext OS constructor. | ||
b. You may choose what packages you put on Bitscout ISO. This lets you | ||
decide which binaries you trust. | ||
b.The owner can monitor what is going on in expert's container live or via | ||
recorded session, which can be replayed. This is useful for training or | ||
understanding of forensic process in the court. | ||
|
||
2. Forensics | ||
a. Bitscout is designed to not modify hard drive data or other | ||
storage media attached to the system. This is essential for forensic | ||
analysis. | ||
b. Bitscout contains most popular tools to acquire and analyze storage drives. | ||
c. The owner of the system controls which disk devices are accessible to the | ||
expert in read-only (or read-write) mode. | ||
d. Even running as root the expert cannot modify or reset access to the | ||
provided storage devices, which prevents potential data loss from the source | ||
disk. This is achieved via layers of virtualization. | ||
|
||
3. Customisation | ||
a. The set of tools available on Bitscout can be customized by editing | ||
respective scripts before running the build. You can add standard packages | ||
or your own tools. Make it available to expert, system owner or both. | ||
b. Both system owner and expert can install additional software packages on | ||
already running (booted) system. All changes will be done indepently | ||
(expert cannot change owner's environment). All installed software will exist | ||
only in RAM and will be gone when system is restarted. | ||
c. If certain operations require more memory or large disk which is not | ||
available on the system, the owner may attach writable external storage device | ||
(such as fast USB flash memory) to be used for storage or swap by the expert. | ||
|
||
4. Compact | ||
a. Bitscout project is designed to be minimal yet universal tool to access | ||
remote systems. It contains minimal set of packages, libraries and | ||
tools to start the system and provide most common forensic tools to the expert | ||
immediately. Certain optimizations yet to be added to reduce size even | ||
further. All suggestions and contributions are welcome! | ||
b. The system uses no graphical interface on purpose. This reduces disk image | ||
size and RAM consumption. | ||
c. The expert's runs inside unprivileged LXC container, which saves from | ||
overhead of full virtualization. The container relies on the same kernel as | ||
the host system, but doesn't allow kernel module manipulation. | ||
d. The container root filesystems is overlayed from the live CD rootfs. | ||
This enables to reuse the system binaries and configuration and avoid data | ||
duplication. Yet, mapped with copy-on-write access it provides almost unlimited | ||
modification of the whole OS. The real limit is just the size of available | ||
memory and swap. | ||
|
||
As a matter of fact fully running OS with a child OS inside the container used | ||
less than 200Mb of RAM in some of our tests in the past. | ||
|
||
|
||
F.A.Q: | ||
|
||
Q: Why was the system created? | ||
A: There is a lot of commercial and rather expensive forensic software suites | ||
out there. We tried several most popular of them and always bumped into | ||
functionality limitations and lack of transparency in the processes. While some | ||
suites provide scriptability, they lack remote analysis features that do not | ||
modify the evidence disk. Most of forensic tools are not designed for remote | ||
analysis, lacked flexibility and cost a fortune. | ||
We found that there was a niche for a new tool which is | ||
a. trusted, transparent and opensource (you build your own OS!) | ||
b. customisable (you put your own tools!) | ||
c. stable and reliable | ||
d. optimal and compact | ||
e. fast and optimal (with low RAM usage) | ||
f. free of charge | ||
g. runs on old and new hardware | ||
|
||
Q: How was the project developed? | ||
A: The project was initially developed as a hobby project. The first variant | ||
relied on full trust to the remote user, who was provided with root access to | ||
the live system. Soon we realized that the remote system owner is willing to | ||
track the progress, communicate with the expert and be able to approve access | ||
to storage media. To increase trust level between system's owner and the remote | ||
expert we decided to isolate the expert within virtualized container. This | ||
assured the owner of the system that the source disk information | ||
will never be tampered (unless it is permitted by the owner in case of | ||
system remediation request). | ||
|
||
Q: Does the author provide VPN server with this project? | ||
A: No, you have to use own server. All you need is an OpenVPN instance. It's | ||
free and opensource, it runs on all platforms. | ||
For more information, see https://openvpn.net | ||
|
||
Q: Will the product be supported? | ||
A: It will be supported as long as there is need for such tool. We will migrate | ||
to newer LTS versions of Ubuntu as it is released. This is important to upgrade | ||
forensic tools. However, you can always update already running live system from a | ||
newer repository and install more recent versions of certain packages. | ||
|
||
Q: Do I need to re-run ./automake.sh every time I change anything, i.e. put my | ||
own VPN certificate or SSH key? | ||
A: No. automake.sh script is just an easy do-everything script to build new ISO | ||
file from scratch in one run. Feel free to copy and modify it. Comment out | ||
stages that you don't want to pass again from top to bottom and run it. Make | ||
sure you run the last stage image_build.sh to rebuild the ISO file. If you | ||
didn't modify the rootfs in chroot directory, you can also use | ||
scripts/image_build-nosquashfs-rebuild.sh to save even more time. | ||
|
||
Q: Is this the best forensic product to save the world? | ||
A: It is not and was never meant to be so. It serves it's task though and did | ||
help us in the past in some compicated circumstances and under time pressure. | ||
If it works for you we will be happy to hear your story. If not, perhaps you | ||
could suggest a clever patch? | ||
|
||
Q: Is this project used for Kaspersky Lab business? | ||
A: This project was created independently of Kaspersky Lab product line and | ||
outside of scope of company's business operation. The developed tool is not | ||
limited to particular users and might be useful to researchers, high-tech crime | ||
units of Law Enforcement and educational institutions. | ||
|
||
|
||
Credits: | ||
Kaspersky Lab | ||
INTERPOL Global Complex For Innovation (IGCI) | ||
IGCI Digital Forensics Lab | ||
|
||
Thanks to | ||
Linux kernel developers | ||
Canonical Ltd | ||
All open-source software developers | ||
LXC developers | ||
All those awesome authors of Linux forensics tools |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
#!/bin/bash | ||
# This file is part of Bitscout remote digital forensics project. | ||
# Copyright Kaspersky Lab. Contact: bitscout[at]kaspersky.com | ||
# Bitscout is free software: you can redistribute it and/or modify it under the | ||
# terms of the GNU General Public License as published by the Free Software | ||
# Foundation, either version 2 of the License, or (at your option) any later | ||
# version. | ||
# Bitscout is distributed in the hope that it will be useful, but WITHOUT ANY | ||
# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR | ||
# A PARTICULAR PURPOSE. See the GNU General Public License for more details. | ||
# You should have received a copy of the GNU General Public License along with | ||
# Bitscout. If not, see <http://www.gnu.org/licenses/>. | ||
|
||
#welcome and initial settings: | ||
( | ||
scripts/welcome.sh && | ||
scripts/submodules_fetch.sh && | ||
|
||
#prepare rootfs directory: | ||
scripts/chroot_download.sh && | ||
scripts/chroot_postdownload_setup.sh && | ||
scripts/chroot_install_base.sh && | ||
scripts/chroot_install_kernel.sh && | ||
scripts/chroot_install_forensics.sh && | ||
scripts/chroot_install_remoteaccess.sh && | ||
|
||
#configure target system: | ||
scripts/chroot_create_user.sh && | ||
scripts/chroot_create_container.sh && | ||
scripts/chroot_configure.sh && | ||
scripts/chroot_configure_openvpn.sh && | ||
scripts/chroot_configure_ssh.sh && | ||
scripts/chroot_configure_irc.sh && | ||
|
||
#customize with own tools: | ||
scripts/chroot_add_managementtool.sh && | ||
|
||
#prepare ISO files: | ||
scripts/image_prepare.sh && | ||
|
||
#apply initrd/casper fixes: | ||
scripts/initrd_unpack.sh && | ||
scripts/casper_findlivefs_fix.sh && | ||
scripts/casper_integritycheck_fix.sh && | ||
scripts/casper_writeblocker.sh && | ||
scripts/initrd_pack.sh && | ||
|
||
#reduce size of rootfs and build ISO: | ||
scripts/image_prebuild_cleanup.sh && | ||
scripts/image_build.sh && | ||
|
||
#prepare exportable configs/certs/keys: | ||
scripts/export_generate.sh | ||
) 2>&1 | tee ./automake.log | ||
|
Oops, something went wrong.