Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alpha File System and Upgraded FTP #382

Merged
merged 26 commits into from Aug 9, 2018

Conversation

@xandfury
Copy link
Collaborator

commented Jul 8, 2018

File System

Conpot now has a new bright and shiny file system! It is designed to have "safe to use" os.* wrappers that could be used by protocols. We cannot allow os.chmod like commands that may allow attackers to make arbitrary system calls. At the same time - protocols such as FTP need chmod like methods. Same goes for os.stat etc. For this reason, we needed a file system that can operate on a layer above the actual file system and still provide the flexibility/robustness.

The Conpot's file system solves this problem by proxying the actual files kept at a controlled location.

+---------------+                      +----------------------+
|               |                      |                      |
|               | <----------------+   |  Actual FileSystem   |
|  Conpot VFS   |       Proxy          |         at           |
|               | +---------------->   |  '/tmp/__conpot__*/' |
|               |                      |                      |
+---------------+                      +----------------------+

Consequently, we would keep a cache (a dictionary where we would store all file related data - (information regarding access, permissions, owners, stat etc.). Note that no matter what, we won't change the actual permissions of the file system.

Let us see a demo shall we?

This is what my ls -la look like:

total 8
drwxrwxr-x 2 abhinav abhinav 4096 Jul  9 01:20 .
drwxrwxr-x 4 abhinav abhinav 4096 Jul  9 01:17 ..
-rw-rw-r-- 1 abhinav abhinav    0 Jul  9 01:20 hacked.png

Notice the permissions and the user/group. (There is bug with mtime - please ignore it for now, it shall be fixed in the next PR - I forgot to add it to recent commits)

>>> import conpot.core as conpot_core
>>> conpot_core.initialize_vfs('.', data_fs_path='../data_fs')
>>> vfs = conpot_core.get_vfs()
>>> vfs.listdir('.')
['hacked.png']
>>> [print(i) for i in vfs.format_list('', vfs.listdir('.'))]
rwxrwxrwx   1 root     root            0 Jul 08 19:53 hacked.png

As you can see, the permissions have changed and so have the user/groups(By default the uid:gid is 0:0 and permissions is 777 - this is configurable).
This is not all. Check this out!

>>> vfs.register_user('daniel', 2000)
>>> vfs.create_group('daniel', 3000)
>>> vfs.chown('/', uid=2000, gid=3000, recursive=True)
>>> vfs.chmod('/', 0o755, recursive=True)
>>> [print(i) for i in vfs.format_list('', vfs.listdir('.'))]
rwxr-xr-x   1 daniel   daniel          0 Jul 08 19:53 hacked.png

There is no change with the uid:gid:perms of the actual 'hacked.png' file though.

Another big advantage of this approach is : VFS is independent of the physical storage media it is located in. We are currently keeping the contents in '/tmp'. But in future if we want to replace this with somewhat better storage media(or location), we can simply detach the VFS - replace it with new storage media URL and it'll fit right in.

FTP

After discussion with Daniel, I have kept for the command channel and the data channel full duplex. Since it said so in the RFC. IOLoop is based on Gevent; Following commands are supported:

Command Channel related:

  • USER
  • PASS
  • HELP
  • NOOP
  • QUIT
  • SITE HELP
  • SYST
  • TYPE
  • ALLO
  • MODE
  • SIZE
  • PWD
  • MKD
  • RMD
  • CWD
  • CDUP
  • MDTM
  • DELE
  • SITE CHMOD
  • RNFR
  • RNTO
  • STAT

Data Channel related

  • PASV
  • PORT
  • LIST
  • NLIST
  • RETR
  • REIN
  • ABOR
  • STOR
  • APPE
  • REST
  • STRU
  • STOU

Note that commands like STOR would create a copy in the user specified data_fs with the format like:

@xandfury xandfury requested a review from creolis Jul 8, 2018

@coveralls

This comment has been minimized.

Copy link

commented Jul 8, 2018

Pull Request Test Coverage Report for Build 1065

  • 1641 of 1991 (82.42%) changed or added relevant lines in 17 files are covered.
  • 103 unchanged lines in 6 files lost coverage.
  • Overall coverage increased (+9.1%) to 71.122%

Changes Missing Coverage Covered Lines Changed/Added Lines %
conpot/core/init.py 10 11 90.91%
conpot/protocols/modbus/modbus_server.py 1 4 25.0%
conpot/protocols/ipmi/fakesession.py 14 18 77.78%
conpot/emulators/proxy.py 13 18 72.22%
conpot/protocols/ftp/ftp_server.py 108 113 95.58%
conpot/core/fs_utils.py 123 136 90.44%
conpot/protocols/ipmi/ipmi_server.py 51 67 76.12%
conpot/core/virtual_fs.py 24 42 57.14%
conpot/protocols/tftp/tftp_server.py 72 90 80.0%
conpot/protocols/tftp/tftp_handler.py 123 161 76.4%
Files with Coverage Reduction New Missed Lines %
conpot/protocols/modbus/modbus_server.py 2 68.14%
conpot/protocols/enip/enip_server.py 2 65.69%
conpot/protocols/http/web_server.py 3 85.71%
conpot/protocols/guardian_ast/guardian_ast_server.py 3 81.19%
conpot/protocols/IEC104/IEC104_server.py 21 47.31%
conpot/protocols/http/command_responder.py 72 55.49%
Totals Coverage Status
Change from base Build 1047: 9.1%
Covered Lines: 5332
Relevant Lines: 7497

💛 - Coveralls
xandfury added 6 commits Jul 10, 2018
TFTP and Unit Tests
- Add TFTP Protocol support
- Add VFS unit tests and TFTP unit tests
FTP updates - remaining tasks:
- more tests/coverage
- socket timeout and connection metrics
- template completion with xsd etc.
- docstrings/comments

@xandfury xandfury force-pushed the xandfury:ftp-tftp branch from a4f0007 to f8b50e5 Jul 25, 2018

@xandfury xandfury force-pushed the xandfury:ftp-tftp branch from f8b50e5 to 8ee0227 Jul 25, 2018

@xandfury xandfury added this to the 0.6.0 milestone Aug 2, 2018

xandfury added 11 commits Aug 3, 2018
fixing minor bugs + tests
- Add test for set time
- comment out drop privs
- add docstring to sanitize_file
run conpot without root
* change ports to be <1024 in default template
* ignore drop_privs
fix minor bugs
allow mac_addr change with root

@creolis creolis merged commit 7f3502b into mushorg:py3 Aug 9, 2018

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage increased (+9.1%) to 71.122%
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.