2 ‐ The SecureRequests Class

The SecureRequests Class

This class provides methods to make HTTP requests with enhanced security features, including SSL/TLS configuration, certificate management, and custom headers and cookies powered with enums.

Parameters

requests: Module reference to requests
os: Module reference to os
datetime: Module reference to datetime.datetime
useEVar: Boolean flag to determine if the configuration should be loaded from environment variables. Default is False.
envVars: Dictionary mapping configuration attributes to their corresponding environment variable names.
useTLS: Boolean indicating whether to use TLS. Default is True.
unsafe: Boolean indicating if unsafe operations are allowed. Default is False.
certificateNeedFetch: Boolean indicating if a certificate needs to be fetched. Default is True.
certificateURL: String URL from which the certificate should be fetched. Default is False.
certificatePath: String path to the certificate file. Default is cacert.pem.
logToFile: Boolean indicating if logs should be written to a file. Default is True.
logLevel: Logging level. Default is logging.INFO.
logPath: Specifies the log path.
logExtensive: If to include extra data such as cookies and headers in the log.
silent: Boolean indicating if logging should be silent. Default is False.
suppressWarnings: Boolean indicating if warnings should be suppressed. Default is False.

Attributes

session: The requests.Session object used for making HTTP requests.
headers: A dictionary containting the header data to include in the requests.
cookies : A dictionary containting the cookies to include in the requests.

Constructor

    def __init__(
            self,
            requests: requests = requests,
            os: os = os,
            datetime: datetime = datetime,
            useEnv: bool = False,
            customEnvVars: Optional[Dict[str, str]] = None,
            headers: Optional[Dict[str, str]] = None,
            useTLS: Optional[bool] = None,
            unsafe: Optional[bool] = None,
            certificateNeedFetch: Optional[bool] = None,
            certificateURL: Optional[str] = None,
            certificatePath: Optional[str] = None,
            logToFile: Optional[bool] = None,
            logLevel: int = logging.INFO,
            logPath: str = None,
            logExtensive: bool = None,
            silent: Optional[bool] = None,
            suppressWarnings: Optional[bool] = None,
            session: requests.Session = None) -> None:

Initialization with Arguments

secureRequest = SecureRequests(useTLS=True, unsafe=False, 
        fetchCert=True, logToFile=True, silent=False)

Initialization with Config

    config.setUseTLS(False)
    config.setUnsafe(True)
    config.setFetchCert(False)
    config.setLogToFile(True)

    # Print current configuration for verification
    # print(f"useTLS: {config.getUseTLS()}")
    # print(f"unsafe: {config.getUnsafe()}")
    # print(f"fetchCert: {config.getFetchCert()}")
    # print(f"logToFile: {config.getLogToFile()}")

    # Initialize 
    secureRequest = SecureRequests(config=config)

Methods

`makeRequest`

@handleResponse
    def makeRequest(
        self,
        url: str,
        method: str = "GET",
        payload: Optional[Dict[str, Any]] = None,
        headers: Optional[Dict[str, str]] = None,
    ) -> requests.Response:
        """
        Makes an HTTP request with the specified parameters.

        Parameters
        ----------
        url : str
            The URL for the request.
        method : str, optional
            The HTTP method to use (e.g., 'GET', 'POST'). Defaults to 'GET'.
        payload : dict, optional
            The payload to send with the request. Defaults to None.
        headers : dict, optional
            Custom headers to include with the request. Defaults to None.

        Returns
        -------
        requests.Response
            The HTTP response.

        Raises
        ------
        ValueError
            If an unsupported HTTP method is provided.
            Any StatusCode related exception from secureRequests.decorator

        """
        srMethods = {
            "GET": self.session.get,
            "POST": self.session.post,
            "PUT": self.session.put,
            "DELETE": self.session.delete,
            "PATCH": self.session.patch,
        }

        if method not in srMethods:
            raise ValueError(f"Unsupported HTTP method: {method}")

        headers = headers or self.headers

        response = srMethods[method](
            url, json=payload, headers=headers, verify=self.verify
        )
        self._logRequest(method, url, response=response, json=payload, headers=headers)
        return response

`_logRequest`

def _logRequest(self, method: str, url: str, response: requests.Response, **kwargs: Any) -> None:
        """
        Logs the details of the HTTP request if logging is enabled.

        Parameters
        ----------
        method : str
            The HTTP method used for the request (e.g., 'GET', 'POST').
        url : str
            The URL of the request.
        response : requests.Response
            The HTTP response object.
        kwargs : Any
            Additional keyword arguments passed to the request, such as headers or parameters.

        Returns
        -------
        None
            This method does not return any value.

        Example
        -------
        >>> self._logRequest('GET', 'https://example.com/api/data', response, headers={'Authorization': 'Bearer token'}, params={'key': 'value'})
        INFO:root:GET request to https://example.com/api/data with headers {'Authorization': 'Bearer token'} and params {'headers': {'Authorization': 'Bearer token'}, 'params': {'key': 'value'}} with Status Code 200
        """
        if hasattr(self, "logger"):
            timestamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
            safetyStatus = "[UNSAFE]" if self.unsafe else "[SAFE]"
            tlsStatus = "[TLS]" if self.useTLS else "[NO TLS]"
            logExtra = f' with headers {kwargs.get("headers")} and params {kwargs}' if self.logExtensive else ''
            logDefaults = f' with Status Code {response.status_code} - {response.reason}'
            logBase = f'[{timestamp}]{safetyStatus}{tlsStatus} {method} request to {url}{logExtra}{logDefaults}'
            self.logger.info(logBase)

Header Management

The secureRequests.header* functions provide functionality for managing HTTP headers used in web requests. This includes generating default headers, setting specific header values, and updating or removing headers. This module is particularly useful for ensuring that HTTP requests adhere to certain standards and practices, such as setting appropriate user agent strings and content types.

`headerGenerate`

    @staticmethod
    def headerGenerate(customHeaders: Optional[Dict[str, Any]] = None) -> Dict[str, str]:
        """
        Generates headers for the session with optional custom values.

        Parameters
        ----------
        customHeaders : Optional[Dict[str, Any]], optional
            A dictionary of headers to include or override the default headers. By default, None.

        Returns
        -------
        Dict[str, str]
            A dictionary of the generated headers, including both default headers and any custom headers provided.

        Default Header Generation
        -------------------------
        >>> chromeMajors = list(range(110, 126))
        >>> SecCHUAPlatform = ["Windows", "Macintosh", "X11"]
        >>> platforms = [
        ...     "Windows NT 10.0; Win64; x64",
        ...     "Windows NT 6.1; Win64; x64",
        ...     "Macintosh; Intel Mac OS X 10_15_7",
        ...     "Macintosh; Intel Mac OS X 11_2_3",
        ...     "X11; Linux x86_64",
        ...     "X11; Ubuntu; Linux x86_64",
        ... ]

        >>> randPlatform = random.choice(platforms)
        >>> randSecCHUAPlatform = random.choice(SecCHUAPlatform)
        >>> randChromeMajors = random.choice(chromeMajors)

        >>> if not customHeaders:
        >>>     customHeaders = {}

        Default Header
        --------------
        >>> defaultHeader = {
        ...     HeaderKeys.ACCEPT.value: "application/x-www-form-urlencoded",
        ...     HeaderKeys.CONTENT_TYPE.value: "application/x-www-form-urlencoded",
        ...     HeaderKeys.SEC_CH_UA.value: f'"Google Chrome";v="{randChromeMajors}", "Chromium";v="{randChromeMajors}", "Not.A/Brand";v="24"',
        ...     HeaderKeys.SEC_CH_UA_MOBILE.value: "?0",
        ...     HeaderKeys.SEC_CH_UA_PLATFORM.value: f'"{randSecCHUAPlatform}"',
        ...     HeaderKeys.SEC_FETCH_DEST.value: "empty",
        ...     HeaderKeys.SEC_FETCH_MODE.value: "cors",
        ...     HeaderKeys.SEC_FETCH_SITE.value: "same-site",
        ...     HeaderKeys.USER_AGENT.value: f"Mozilla/5.0 ({randPlatform}) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/{randChromeMajors}.0.0.0 Safari/537.36"
        ... }

        Example
        -------
        >>> from secureRequests import HeaderKeys
        >>> from secureRequests import SecureRequests
        >>> customHeaders = {HeaderKeys.CONTENT_TYPE.value: "application/json", HeaderKeys.AUTHORIZATION.value: "Bearer Token"}
        >>> # Directly genereate them on creation
        >>> sr = SecureRequests(headers=customHeaders)
        >>> # Regenerate a needed one works aswell
        >>> headers = secureRequests.headerGenerate(customHeaders=customHeaders)
        >>> print(sr.headers)
        >>> {
        ... 'Accept': 'application/x-www-form-urlencoded', 
        ... 'Authorization': 'Bearer Token',
        ... 'Content-Type': 'application/json', 
        ... 'Sec-CH-UA': '"Google Chrome";v="120", "Chromium";v="120", "Not.A/Brand";v="24"', 
        ... 'Sec-CH-UA-Mobile': '?0', 
        ... 'Sec-CH-UA-Platform': '"Windows"', 
        ... 'Sec-Fetch-Dest': 'empty', 
        ... 'Sec-Fetch-Mode': 'cors', 
        ... 'Sec-Fetch-Site': 'same-site', 
        ... 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'}
        """
        chromeMajors = list(range(110, 126))
        SecCHUAPlatform = ["Windows", "Macintosh", "X11"]
        platforms = [
            "Windows NT 10.0; Win64; x64",
            "Windows NT 6.1; Win64; x64",
            "Macintosh; Intel Mac OS X 10_15_7",
            "Macintosh; Intel Mac OS X 11_2_3",
            "X11; Linux x86_64",
            "X11; Ubuntu; Linux x86_64",
        ]

        randPlatform = random.choice(platforms)
        randSecCHUAPlatform = random.choice(SecCHUAPlatform)
        randChromeMajors = random.choice(chromeMajors)

        if not customHeaders:
            customHeaders = {}

        defaultHeader = {
            HeaderKeys.ACCEPT.value: "application/x-www-form-urlencoded",
            HeaderKeys.CONTENT_TYPE.value: "application/x-www-form-urlencoded",
            HeaderKeys.SEC_CH_UA.value: f'"Google Chrome";v="{randChromeMajors}", "Chromium";v="{randChromeMajors}", "Not.A/Brand";v="24"',
            HeaderKeys.SEC_CH_UA_MOBILE.value: "?0",
            HeaderKeys.SEC_CH_UA_PLATFORM.value: f'"{randSecCHUAPlatform}"',
            HeaderKeys.SEC_FETCH_DEST.value: "empty",
            HeaderKeys.SEC_FETCH_MODE.value: "cors",
            HeaderKeys.SEC_FETCH_SITE.value: "same-site",
            HeaderKeys.USER_AGENT.value: f"Mozilla/5.0 ({randPlatform}) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/{randChromeMajors}.0.0.0 Safari/537.36"
        }
        headers = {key: customHeaders.get(key, defaultValue) for key, defaultValue in defaultHeader.items()}

        # Add any custom headers that are not in the default set
        for key, value in customHeaders.items():
            if key not in headers:
                headers[key] = value

        return headers

`headerSetKey`

    def headerSetKey(self, key: HeaderKeys, value: str) -> None:
        """
        Sets a specific header key to a given value.

        Parameters
        ----------
        key : HeaderKeys
            The header key to set.
        value : str
            The value to set for the header key.

        Returns
        -------
        None

        Example
        -------
        >>> headers = {"Content-Type": "application/json"}
        >>> MyClass.headerSetKey(HeaderKeys.AUTHORIZATION, "Bearer token123")
        >>> print(headers)
        {'Content-Type': 'application/json', 'Authorization': 'Bearer token123'}
        """
        self.headers[key.value] = value

`headerRemoveKey`

    def headerRemoveKey(self, key: HeaderKeys) -> None:
        """
        Removes a specific header key.

        Parameters
        ----------
        key : HeaderKeys
            The header key to remove.

        Returns
        -------
        None

        Example
        -------
        >>> headers = {"Content-Type": "application/json", "Authorization": "Bearer token123"}
        >>> MyClass.headerRemoveKey(HeaderKeys.AUTHORIZATION)
        >>> print(headers)
        {'Content-Type': 'application/json'}
        """
        if key.value in self.headers:
            del self.headers[key.value]

`headerUpdateMultiple`

    def headerUpdateMultiple(self, newHeader: Dict[HeaderKeys, str]) -> None:
        """
        Updates specific headers based on the provided dictionary.

        Parameters
        ----------
        newHeader : Dict[HeaderKeys, str]
            The new headers to update, where keys are of type HeaderKeys.

        Returns
        -------
        None

        Example
        -------
        >>> headers = {"Content-Type": "application/json"}
        >>> MyClass.headerUpdateMultiple({HeaderKeys.AUTHORIZATION: "Bearer token123", HeaderKeys.ACCEPT: "application/xml"})
        >>> print(headers)
        {'Content-Type': 'application/json', 'Authorization': 'Bearer token123', 'Accept': 'application/xml'}
        """
        for key, value in newHeader.items():
            self.headers[key.value] = value

`headerRemoveMultiple`

    def headerRemoveMultiple(self, keys: List[HeaderKeys]) -> None:
        """
        Removes multiple header keys at once.

        Parameters
        ----------
        keys : List[HeaderKeys]
            The list of header keys to remove.

        Returns
        -------
        None

        Example
        -------
        >>> headers = {"Content-Type": "application/json", "Authorization": "Bearer token123", "Accept": "application/xml"}
        >>> MyClass.headerRemoveMultiple([HeaderKeys.AUTHORIZATION, HeaderKeys.ACCEPT])
        >>> print(headers)
        {'Content-Type': 'application/json'}
        """
        for key in keys:
            if key.value in self.headers:
                del self.headers[key.value]

Cookie Management

The secureRequests.cookie* functions provide functionality for managing HTTP cookies used in web requests.

`_serializeCookieInfo`

    def _serializeCookieInfo(self, cookieInfo: Dict[CookieAttributeKeys, Union[str, bool, int, datetime]]) -> str:
        """
        Serializes the cookie attributes into a string.

        Parameters:
        ------------
        cookieInfo: Dict[CookieAttributeKeys, Union[str, bool, int, datetime]]
            A dictionary containing the attributes of the cookie.

        Returns:
        ------------
        str
            The serialized string of cookie attributes.
        """
        return '|'.join(f'{key.value}={value}' for key, value in cookieInfo.items())

`_deserializeCookieInfo`

    def _deserializeCookieInfo(self, cookieInfoStr: str) -> Dict[CookieAttributeKeys, Union[str, bool, int, datetime]]:
        """
        Deserializes the string back into a dictionary of cookie attributes.

        Parameters:
        ------------
        cookieInfoStr: str
            The serialized string of cookie attributes.

        Returns:
        ------------
        Dict[CookieAttributeKeys, Union[str, bool, int, datetime]]
            The deserialized dictionary of cookie attributes.
        """
        items = cookieInfoStr.split('|')
        cookieInfo = {}
        for item in items:
            key, value = item.split('=')
            cookieInfo[CookieAttributeKeys(key)] = value
        return cookieInfo

`cookieUpdate`

    def cookieUpdate(self, key: CookieKeys, cookieInfo: Dict[CookieAttributeKeys, Union[str, bool, int, datetime]]) -> None:
        """
        Sets or updates a single cookie with specified attributes.

        Parameters:
        ------------
        key: CookieKeys
            The key identifying the cookie to be set or updated.
        cookieInfo: Dict[CookieAttributeKeys, Union[str, bool, int, datetime]]
            A dictionary containing the attributes of the cookie.

        Example:
        ------------
        >>> cookie_attributes = {
        ...     CookieAttributeKeys.DOMAIN: 'example.com',
        ...     CookieAttributeKeys.PATH: '/',
        ...     CookieAttributeKeys.EXPIRES: datetime.now() + timedelta(days=7),
        ...     CookieAttributeKeys.SECURE: True,
        ...     CookieAttributeKeys.HTTP_ONLY: True,
        ...     CookieAttributeKeys.SAME_SITE: 'Strict',
        ...     CookieAttributeKeys.MAX_AGE: 604800  # 7 days in seconds
        ... }

        # Set the cookie
        secureRequests.cookieUpdate(CookieKeys.SESSION_ID, cookieAttributes)
        """
        cookieValue = self._serializeCookieInfo(cookieInfo)
        self.session.cookies.set(key.value, cookieValue)

`cookieGet`

    def cookieGet(self, key: CookieKeys) -> Optional[Dict[CookieAttributeKeys, Union[str, bool, int, datetime]]]:
        """
        Retrieves the attributes of a single cookie.

        Parameters:
        ------------
        key: CookieKeys
            The key identifying the cookie to be retrieved.

        Returns:
        ------------
        Optional[Dict[CookieAttributeKeys, Union[str, bool, int, datetime]]]
            A dictionary containing the attributes of the cookie if it exists, otherwise None.
        """
        cookieValue = self.session.cookies.get(key.value)
        if cookieValue:
            return self._deserializeCookieInfo(cookieValue)
        return None

`cookieRemove`

    def cookieRemove(self, key: CookieKeys) -> None:
        """
        Removes a single cookie.

        Parameters:
        ------------
        key: CookieKeys
            The key identifying the cookie to be removed.
        """
        self.session.cookies.pop(key.value, None)

`cookieUpdateMultiple`

    def cookieUpdateMultiple(self, cookies: Dict[CookieKeys, Dict[CookieAttributeKeys, Union[str, bool, int, datetime]]]) -> None:
        """
        Adds or updates multiple cookies at once.

        Parameters:
        ------------
        cookies: Dict[CookieKeys, Dict[CookieAttributeKeys, Union[str, bool, int, datetime]]]
            A dictionary of cookies with their attributes to be set or updated.

        Example:
        ------------
        >>> cookies = {
        ...     CookieKeys.AUTH_TOKEN: {
        ...         CookieAttributeKeys.DOMAIN: 'example.com',
        ...         CookieAttributeKeys.PATH: '/',
        ...         CookieAttributeKeys.EXPIRES: datetime.now() + timedelta(days=30),
        ...         CookieAttributeKeys.SECURE: True,
        ...         CookieAttributeKeys.HTTP_ONLY: True,
        ...         CookieAttributeKeys.SAME_SITE: 'Lax'
        ...     },
        ...     CookieKeys.USER_PREFERENCES: {
        ...         CookieAttributeKeys.DOMAIN: 'example.com',
        ...         CookieAttributeKeys.PATH: '/',
        ...         CookieAttributeKeys.EXPIRES: datetime.now() + timedelta(days=7),
        ...         CookieAttributeKeys.SAME_SITE: 'None',
        ...         CookieAttributeKeys.SECURE: True
        ...     }
        ... }

        # Set or update multiple cookies
        secureRequests.cookieUpdateMultiple(cookies)
        """
        for key, cookieInfo in cookies.items():
            self.cookieUpdate(key, cookieInfo)

`CookieGetAll`

    def cookieGetAll(self) -> Dict[CookieKeys, Dict[CookieAttributeKeys, Union[str, bool, int, datetime]]]:
        """
        Retrieves all cookies with their attributes.

        Returns:
        ------------
        Dict[CookieKeys, Dict[CookieAttributeKeys, Union[str, bool, int, datetime]]]
            A dictionary of all cookies with their attributes.
        """
        allCookies = {}
        for cookie in self.session.cookies:
            key = CookieKeys(cookie.name)
            cookieInfo = self._deserializeCookieInfo(cookie.value)
            allCookies[key] = cookieInfo
        return allCookies

Full Example

from datetime import datetime, timedelta

from secureRequests import SecureRequests
from secureRequests import HeaderKeys, CookieKeys, CookieAttributeKeys


# Creating secureRequests empty to demonstrate a third way of configuring it
sr = SecureRequests(headers={HeaderKeys.CONTENT_TYPE.value: "application/json"})
sr.headerSetKey(HeaderKeys.AUTHORIZATION, "Bearer token123")
sr.cookieUpdate(CookieKeys.SESSION_ID, {
    CookieAttributeKeys.PATH: '/',
    CookieAttributeKeys.EXPIRES: datetime.now() + timedelta(days=7),
    CookieAttributeKeys.SECURE: True
})
response = sr.makeRequest("https://httpbin.org/get")
print(response.status_code)
print(sr.headers)

print(sr.cookieGetAll())

Navigation Menu

Repository navigation

2 ‐ The SecureRequests Class

The SecureRequests Class

Parameters

Attributes

Constructor

Initialization with Arguments

Initialization with Config

Methods

`makeRequest`

`_logRequest`

Header Management

`headerGenerate`

`headerSetKey`

`headerRemoveKey`

`headerUpdateMultiple`

`headerRemoveMultiple`

Cookie Management

`_serializeCookieInfo`

`_deserializeCookieInfo`

`cookieUpdate`

`cookieGet`

`cookieRemove`

`cookieUpdateMultiple`

`CookieGetAll`

Full Example

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally