Skip to content

Commit

Permalink
Merge 3.6.9 (#174)
Browse files Browse the repository at this point in the history
* HOTFIX: EFR01 Enterprise feature request (MobSF#1908)

* Replace Warning with Medium and added Hotspot
* Add file analysis to hotspot
* Enterprise Feature Request Flag
* EFR01 changes
* version bump

* update quark & frida (MobSF#1903)

Co-authored-by: Ajin Abraham <ajin25@gmail.com>

* Update tldextract from 3.1.2 to 3.2.0 (MobSF#1910)

* upgrade apktool to 2.6.1 (MobSF#1915)

* Hotfix: Update slack link

* Hotfix: update slack link

* Hotfix: Slack link

* Hotfix:Slack link

* Hotfix:Slack link

* Introduce jadx decompilation timeout with env var (MobSF#1916)

* Introduce jadx decompilation timeout with env var
- exception for timeout
- replace subprocess.call for run


Co-authored-by: Ajin Abraham <ajin25@gmail.com>

* Update ip2location from 8.6.4 to 8.7.2 (MobSF#1926)

Co-authored-by: Ajin Abraham <ajin25@gmail.com>

* Scheduled weekly dependency update for week 13 (MobSF#1931)

* Update quark-engine from 22.2.1 to 22.3.1

* update lief

Co-authored-by: Ajin Abraham <ajin25@gmail.com>

* update apkid (MobSF#1939)

* Fix dynamic report_json api bug (MobSF#1934)

Co-authored-by: Ajin Abraham <ajin25@gmail.com>

* Hotfix: LIEF

* Update README.md (MobSF#1951)

* update jadx to 1.3.4 (MobSF#1941)

* update jadx to 1.3.4
* update lief
* update jadx and requirements

* Scheduled weekly dependency update for week 22 (MobSF#1972)

* Update ip2location from 8.7.3 to 8.7.4

* Update quark-engine from 22.4.1 to 22.5.1

* Update frida from 15.1.17 to 15.1.23

* Update tldextract from 3.2.1 to 3.3.0

* Check for updates via GitHub releases (MobSF#1957)

* Check the GitHub releases page for latest version number

* Update utils.py

Only log distro if not empty (or spaces)

Co-authored-by: Ajin Abraham <ajin25@gmail.com>

* Update cert_analysis.py (MobSF#1948)

* Update cert_analysis.py

Flag on MD5 hash algorithm in signer certificate

* Update cert_analysis.py

Co-authored-by: Ajin Abraham <ajin25@gmail.com>

* HOTFIX: Update Readme with Rewards Banner

* Update frida from 15.1.23 to 15.1.24 (MobSF#1975)

Co-authored-by: Ajin Abraham <ajin25@gmail.com>

* HOTFIX: openSSL link and readme update

* Hotfix: Broken slack channel link fix

* Hotfix: Windows setup script

* Feature Parity Allow iOS IPA download (MobSF#1977)

* Allow iOS IPA download

* Code QA

* Add the checking of the parent element of the permission-related elements to manifest analysis (MobSF#1905)

* Add the checking of the parent element of the permission-related elements to manifest analysis

Co-authored-by: Ajin Abraham <ajin25@gmail.com>

* Remove RELRO (MobSF#1978)

* Revert "Add the checking of the parent element of the permission-related elements to manifest analysis (MobSF#1905)" (MobSF#1984)

HOTFIX: Revert MobSF#1905

* Scheduled weekly dependency update for week 26 (MobSF#1986)

* Update ip2location from 8.7.4 to 8.8.0

* Update frida from 15.1.24 to 15.1.27

* Update quark-engine from 22.5.1 to 22.6.1 (MobSF#1989)

* Scheduled weekly dependency update for week 28 (MobSF#1993)

* Update frida from 15.1.27 to 15.1.28

* Update tldextract from 3.3.0 to 3.3.1

* HOTFIX: libsast, iOS Rule, M1 Mac support

* Hotfix MobSF#1999

* Update frida from 15.1.28 to 15.2.2 (MobSF#2002)

* Update README.md (MobSF#2020)

add Badge App

* Fix bug MobSF#1917 where checking for stripped debugging symbols produces false positives in iOS. (MobSF#2023)

Co-authored-by: Toor <toor@DES-macOS-pentest.local>
Co-authored-by: Ajin Abraham <ajin25@gmail.com>

* Update ip2location from 8.8.0 to 8.8.1 (MobSF#2035)

Co-authored-by: Ajin Abraham <ajin25@gmail.com>

* update apkid to 2.1.4 (MobSF#2037)

* Adding tarfile member sanitization to extractall() (MobSF#2039)

Co-authored-by: TrellixVulnTeam <kasimir.schulz@trellix.com>
Co-authored-by: Ajin Abraham <ajin25@gmail.com>

* fix res directory not exist (MobSF#2042)

Fix the problem that the res resource folder does not exist, the solution is to copy from the apktool_out directory

* [EFR-02]Enterprise Feature Request - False Positive Triaging (MobSF#2000)

* Suppression logic

* Android code analysis suppression

* Fixes MobSF#1981

* iOS source support bundle id extraction

* iOS Source Code - Suppression support

* Remove check in CFBundleURLName

* iOS Binary code analysis suppression support

* Add Code QL

* Suppression support for Manifest analysis

* Fixes MobSF#2014

* REST API + Docs

* Address review comments

* update suppression wordings

* Fixes MobSF#2043

* Icon analysis code QA

* Unit Test for False Positive Triaging

* Adding numeric_owner as a keyword argument (MobSF#2050)

numeric_owner needs to be a keyword argument.

* Scheduled weekly dependency update for week 41 (MobSF#2046)

* Update quark-engine from 22.6.1 to 22.9.1

* Update frida from 15.2.2 to 16.0.1

* Update tldextract from 3.3.1 to 3.4.0

* Update openstep-parser from 1.5.3 to 1.5.4

Co-authored-by: Ajin Abraham <ajin25@gmail.com>

* HOTFIX: revert frida to 15.X

* HOTFIX: UI changes and warning on mobsf.live (MobSF#2051)

* UI changes and warning on mobsf.live

* Update home.html

* HOTFIX: Split certificate analysis out, suppression list fixes (MobSF#2052)

* Hotfix: ui on donate page

* Hotfix: Homescreen Navbar

* Hotfix: UI icon

* hotfix for quyark rules location (MobSF#2053)

* HOTFIX: jadx update to 1.4.5  (MobSF#2064)

* jadx update to 1.4.5
* MobSF version bump
* Fixes CVE-2022-42889 in third party dependency

* Installation script error: Solving spelling error (MobSF#2067)

changed "installtion" to "installation"

* Android APK support extracting icon SVG from XML (MobSF#2060)

* Added support for SVG icon extraction
* Add jar binaries
* code refactoring
* Update settings.py

* HOTFIX: Setup improvement (MobSF#2078)

* Improve setup scripts.
* Python support to 3.8 - 3.10
* Delete MobSF data directory on running setup.
* Bump applicable dependencies.

* Apktool 2.7.0 update (MobSF#2082)

* Update apktool to version 2.7.0

* HOTFIX: Icon should be a file

* version bump

* New Android Manifest Rule: App support vulnerable android versions (MobSF#2114)

* add a new rule: dangerous os version

* qa

* lint checks

* run lint test on one os

* Support for filenames containing & (MobSF#2129)

Co-authored-by: none <none@none.com>

* HOTFIX: Fix docker build (MobSF#2135)

* Fix Scorecard Severity Distribution chart data (MobSF#2140)

* HOTIX: Update Dockerfile to install jq (MobSF#2149)

* Update Dockerfile

* Update tox.ini

* [HOTFIX] Add support for environment variable for MobSF config (MobSF#2150)

* add support for environment variable config
* Fixes MobSF#2109
* update lief

* HOTFIX: Fixes MobSF#2144

* HOTFIX: Android min SDK check on janus vulnerability detection (MobSF#2159)

* Android min SDK  check on janus check

* Update README.md

* [Enterprise Feature Request EFR02] Support summary of severity in each section. (MobSF#2160)

* Summary for Android and iOS SCA

* [EFR05] Enterprise Feature Request: AAR and JAR support (MobSF#2163)

* AAR and JAR support

* Enable binary analysis for aar/jar

* Scheduled weekly dependency update for week 24 (MobSF#2187)

* Update ip2location from 8.9.0 to 8.10.0

* Update quark-engine from 22.10.1 to 23.5.1

* Update LIEF from to 0.13.1

* Update tldextract from 3.4.0 to 3.4.4

* Update requirements.txt

---------

Co-authored-by: Ajin Abraham <ajin25@gmail.com>

* Update requirements.txt

0.13.1 not available.

* HOTFIX: update lief

* Revert Hotfix

* HOTFIX: Feature updates and Bug Fixes (MobSF#2197)

* OFAC, jquery bump, tox fix
* AAR handle multiple application tags

* HOTFIX: MobSF Android Dynamic Analysis Docker Support (MobSF#2214)

* MobSF Android Docker Support

* Pin pip version

* Update mobsf-test.yml

* updated requirements.txt to most recent django backend version and returned the data object internal to the class method scan_apk in mobsf/MobSF/views/scanning.py instead of the class's self.data.

* had to remove all returns of self.data from the scanning methods in mobsf/MobSF/views/scanning.py and just return the data object local to the method.

* Bug and lint fixes

* Lint fixes, JAR/AAR fix

* Lint fix

* Spell check update

* Attempt at fixing template error

---------

Co-authored-by: Ajin Abraham <ajin25@gmail.com>
Co-authored-by: superpoussin22 <vincent.nadal@orange.fr>
Co-authored-by: pyup.io bot <github-bot@pyup.io>
Co-authored-by: Matej Soroka <hi@matejsoroka.com>
Co-authored-by: N1neSun <917549681@qq.com>
Co-authored-by: Ajin.Abraham <ajin.abraham@chime.com>
Co-authored-by: Dapo Adedire <adedireadedapo19@gmail.com>
Co-authored-by: Atarii <atarii@users.noreply.github.com>
Co-authored-by: Han0nly <byxiaohanzhang@foxmail.com>
Co-authored-by: rustaska <11994805+rustaska@users.noreply.github.com>
Co-authored-by: Toor <toor@DES-macOS-pentest.local>
Co-authored-by: TrellixVulnTeam <112716341+TrellixVulnTeam@users.noreply.github.com>
Co-authored-by: TrellixVulnTeam <kasimir.schulz@trellix.com>
Co-authored-by: ohyeah521 <ohyeah521@gmail.com>
Co-authored-by: th3-d4v1d-c0de <116191845+th3-d4v1d-c0de@users.noreply.github.com>
Co-authored-by: evmxattr <evmxattr@users.noreply.github.com>
Co-authored-by: none <none@none.com>
Co-authored-by: antoinbo <87284775+antoinbo@users.noreply.github.com>
Co-authored-by: brice-syslogic <65510350+brice-syslogic@users.noreply.github.com>
  • Loading branch information
20 people committed Sep 7, 2023
1 parent 84b1b4d commit a7a31bb
Show file tree
Hide file tree
Showing 158 changed files with 3,591 additions and 1,154 deletions.
8 changes: 7 additions & 1 deletion .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
name: "CodeQL"

on:
workflow_dispatch:
push:
branches: [ "master" ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ "master" ]
schedule:
- cron: '17 16 * * 0'

jobs:
analyze:
Expand Down
6 changes: 5 additions & 1 deletion .github/workflows/mobsf-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,13 @@ jobs:
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.python-version }}
- name: Setup Pip
run: |
python -m pip install pip==22.3.1
- name: Lint
if: startsWith(matrix.os, 'ubuntu')
run: |
python -m pip install --upgrade pip tox
python -m pip install --upgrade tox
tox -e lint
- name: Install Ubuntu Dependencies
if: startsWith(matrix.os, 'ubuntu')
Expand Down
7 changes: 4 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ RUN apt update -y && apt install -y --no-install-recommends \
wget \
curl \
git \
jq \
android-tools-adb

# Set locales
Expand Down Expand Up @@ -79,9 +80,9 @@ WORKDIR /home/mobsf/Mobile-Security-Framework-MobSF
# Copy source code
COPY . .

# Set adb binary path and apktool directory
RUN sed -i "s#ADB_BINARY = ''#ADB_BINARY = '/usr/bin/adb'#" mobsf/MobSF/settings.py && \
mkdir -p /home/mobsf/.local/share/apktool/framework
# Set adb binary path and create apktool framework directory
ENV MOBSF_ADB_BINARY=/usr/bin/adb
RUN mkdir -p /home/mobsf/.local/share/apktool/framework

# Enable Postgres support by default
ARG POSTGRES=True
Expand Down
18 changes: 8 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ Made with ![Love](https://cloud.githubusercontent.com/assets/4301109/16754758/82

[![MobSF tests](https://github.com/MobSF/Mobile-Security-Framework-MobSF/workflows/MobSF%20tests/badge.svg?branch=master)](https://github.com/MobSF/Mobile-Security-Framework-MobSF/actions)
[![Requirements Status](https://pyup.io/repos/github/MobSF/Mobile-Security-Framework-MobSF/shield.svg)](https://pyup.io/repos/github/MobSF/Mobile-Security-Framework-MobSF/)
[![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/MobSF/Mobile-Security-Framework-MobSF.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/MobSF/Mobile-Security-Framework-MobSF/context:python)
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=MobSF_Mobile-Security-Framework-MobSF&metric=alert_status)](https://sonarcloud.io/dashboard?id=MobSF_Mobile-Security-Framework-MobSF)
![GitHub closed issues](https://img.shields.io/github/issues-closed/MobSF/Mobile-Security-Framework-MobSF)
[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/6392/badge)](https://bestpractices.coreinfrastructure.org/projects/6392)
Expand All @@ -37,6 +36,14 @@ If you liked MobSF and find it useful, please consider donating.
*It's easy to build open source, try maintaining a project once. Long live open source!*

## Documentation

Quick setup

```
docker pull opensecurity/mobile-security-framework-mobsf:latest
docker run -it --rm -p 8000:8000 opensecurity/mobile-security-framework-mobsf:latest
```

[![See MobSF Documentation](https://user-images.githubusercontent.com/4301109/70686099-3855f780-1c79-11ea-8141-899e39459da2.png)](https://mobsf.github.io/docs)
[![See MobSF Documentation in Chinese](https://user-images.githubusercontent.com/4301109/117404947-b09d0880-aebf-11eb-9db8-3d7360f47914.png)](https://mobsf.github.io/docs/#/zh-cn/)
[![See MobSF Documentation in Japanese](https://user-images.githubusercontent.com/4301109/148662149-7ee671b4-66a2-4232-9522-276b8e88d924.png)](https://mobsf.github.io/docs/#/ja-jp/)
Expand Down Expand Up @@ -69,15 +76,6 @@ If you liked MobSF and find it useful, please consider donating.
* For Project updates and announcements, follow [@ajinabraham](https://twitter.com/ajinabraham) or [@OpenSecurity_IN](https://twitter.com/OpenSecurity_IN).
* Github Issues are only for tracking bugs and feature requests. Do not post support or help queries there. We have a slack channel for that.

### Launching MobSF Rewards

Contributed to MobSF? Here is a big thank you from our community to you. Claim your badge, a soulbound NFT and showcase them with pride. Let us inspire more folks !

![MobSF Badges](https://aviyel.com/assets/uploads/rewards/share/project/7/512/share.png)

[Claim Now!](https://aviyel.com/projects/7/mobile-security-framework/rewards)


### Static Analysis - Android

![mobsf_android_static_analysis](https://user-images.githubusercontent.com/4301109/95506503-f9b6c980-097d-11eb-803a-f88321e1feb7.gif)
Expand Down
10 changes: 5 additions & 5 deletions mobsf/DynamicAnalyzer/tools/webproxy.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

from django.conf import settings

from mobsf.MobSF.utils import is_file_exists, upstream_proxy
from mobsf.MobSF.utils import upstream_proxy

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -61,17 +61,17 @@ def create_ca():
stdout=None,
stderr=None,
close_fds=True)
time.sleep(2)
time.sleep(3)


def get_ca_file():
"""Get CA Dir."""
from mitmproxy import ctx
ca_dir = Path(ctx.mitmproxy.options.CONF_DIR).expanduser()
ca_file = os.path.join(str(ca_dir), 'mitmproxy-ca-cert.pem')
if not is_file_exists(ca_file):
ca_file = ca_dir / 'mitmproxy-ca-cert.pem'
if not ca_file.exists():
create_ca()
return ca_file
return ca_file.as_posix()


def get_traffic(package):
Expand Down
2 changes: 1 addition & 1 deletion mobsf/DynamicAnalyzer/views/android/analysis.py
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ def safe_extract(tar, path='.',
member_path = os.path.join(path, member.name)
if not is_within_directory(path, member_path):
raise Exception('Attempted Path Traversal in Tar File')
tar.extractall(path, members, numeric_owner)
tar.extractall(path, members, numeric_owner=numeric_owner)

safe_extract(tar, untar_dir, members=safe_paths(tar))
except FileExistsError:
Expand Down
1 change: 1 addition & 0 deletions mobsf/DynamicAnalyzer/views/android/dynamic_analyzer.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ def dynamic_analysis(request, api=False):
try:
if identifier:
env = Environment(identifier)
env.connect()
device_packages = env.get_device_packages()
pkg_file = Path(settings.DWD_DIR) / 'packages.json'
with pkg_file.open('w', encoding='utf-8') as target:
Expand Down
20 changes: 15 additions & 5 deletions mobsf/DynamicAnalyzer/views/android/environment.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from frida import __version__ as frida_version

from mobsf.DynamicAnalyzer.tools.webproxy import (
create_ca,
get_ca_file,
get_http_tools_url,
start_proxy,
Expand All @@ -35,7 +36,7 @@
from mobsf.StaticAnalyzer.models import StaticAnalyzerAndroid

logger = logging.getLogger(__name__)
ANDROID_API_SUPPORTED = 29
ANDROID_API_SUPPORTED = 30


class Environment:
Expand All @@ -51,8 +52,9 @@ def __init__(self, identifier=None):

def wait(self, sec):
"""Wait in Seconds."""
logger.info('Waiting for %s seconds...', str(sec))
time.sleep(sec)
if sec > 0:
logger.info('Waiting for %s seconds...', str(sec))
time.sleep(sec)

def check_connect_error(self, output):
"""Check if connect failed."""
Expand All @@ -61,12 +63,19 @@ def check_connect_error(self, output):
return False
return True

def run_subprocess_verify_output(self, cmd):
def run_subprocess_verify_output(self, cmd, wait=2):
"""Run subprocess and verify execution."""
out = subprocess.check_output(cmd) # lgtm [py/command-line-injection]
self.wait(2) # adb shell is allowed
self.wait(wait) # adb shell is allowed
return self.check_connect_error(out)

def connect(self):
"""ADB Connect."""
logger.info('Connecting to Android %s', self.identifier)
self.run_subprocess_verify_output([get_adb(),
'connect',
self.identifier], 0)

def connect_n_mount(self):
"""Test ADB Connection."""
self.adb_command(['kill-server'])
Expand Down Expand Up @@ -564,6 +573,7 @@ def mobsfy_init(self):

def mobsf_agents_setup(self, agent):
"""Setup MobSF agents."""
create_ca()
# Install MITM RootCA
self.install_mobsf_ca('install')
# Install MobSF Agents
Expand Down
23 changes: 23 additions & 0 deletions mobsf/MalwareAnalyzer/views/MalwareDomainCheck.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,19 @@ def update_maltrail_db(self):
def gelocation(self):
"""Perform Geolocation."""
try:
ofac_list = {
'cuba', 'iran', 'north korea',
'russia', 'syria', 'balkans',
'belarus', 'myanmar', 'congo',
'ethiopia', 'hong kong', 'iraq',
'lebanon', 'libya', 'sudan',
'venezuela', 'yemen', 'zimbabwe',
'crimea', 'donetsk', 'luhansk',
'afghanistan', 'china', 'ivory coast',
'cyprus', 'eritrea', 'haiti',
'liberia', 'somalia', 'sri lanka',
'vietnam', 'south sudan',
}
self.IP2Loc.open(self.iplocbin)
for domain in self.domainlist:
# Tag Good Domains
Expand All @@ -94,6 +107,16 @@ def gelocation(self):
if ip:
rec = self.IP2Loc.get_all(ip)
self.result[domain]['geolocation'] = rec.__dict__
country = rec.__dict__.get('country_long')
region = rec.__dict__.get('region')
city = rec.__dict__.get('city')
self.result[domain]['ofac'] = False
if country and country.lower() in ofac_list:
self.result[domain]['ofac'] = True
elif region and region.lower() in ofac_list:
self.result[domain]['ofac'] = True
elif city and city.lower() in ofac_list:
self.result[domain]['ofac'] = True
else:
self.result[domain]['geolocation'] = None
except Exception:
Expand Down
2 changes: 1 addition & 1 deletion mobsf/MalwareAnalyzer/views/quark.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ def quark_analysis(app_dir, apk_file):
enable_print()

# default rules path
rules_dir = Path(f'{config.HOME_DIR}quark-rules')
rules_dir = Path(f'{config.HOME_DIR}quark-rules/rules')
report = Report()

# Analyze apk
Expand Down
2 changes: 1 addition & 1 deletion mobsf/MobSF/init.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

logger = logging.getLogger(__name__)

VERSION = '2023.06'
VERSION = '2023.08'
BANNER = """
__ __ _ ____ _____ _____ __
| \/ | ___ | |__/ ___|| ___|_ _|___ / / /_
Expand Down
Loading

0 comments on commit a7a31bb

Please sign in to comment.