:py~awslimitchecker.checker.AwsLimitChecker
provides the full and only public interface to this project; it's used by the awslimitchecker
command line script (entry point to :py~awslimitchecker.runner
) and should be the only portion directly used by external code.
Each AWS Service is represented by a subclass of the :py~awslimitchecker.services.base._AwsService
abstract base class; these Service Classes are responsible for knowing which limits exist for the service they represent, what the default values for these limits are, querying current limits from the service's API (if supported), and how to check the current usage via the AWS API (boto3
). When the Service Classes are instantiated, they build a dict of all of their limits, correlating a string key (the "limit name") with an :py~awslimitchecker.limit.AwsLimit
object. The Service Class constructors must not make any network connections; connections are created lazily as needed and stored as a class attribute. This allows us to inspect the services, limits and default limit values without ever connecting to AWS (this is also used to generate the Supported Limits <limits>
documentation automatically).
All calls to boto3 client ("low-level") methods that return a dict response that can include 'NextToken' or another pagination marker, should be called through :py~awslimitchecker.utils.paginate_dict
with the appropriate parameters.
When :py~awslimitchecker.checker.AwsLimitChecker
is instantiated, it imports :py~awslimitchecker.services
which in turn creates instances of all awslimitchecker.services.*
classes and adds them to a dict mapping the string Service Name to the Service Class instance. These instances are used for all interaction with the services.
So, once an instance of :py~awslimitchecker.checker.AwsLimitChecker
is created, we should have instant access to the services and limits without any connection to AWS. This is utilized by the --list-services
and --list-defaults
options for the command line client <cli_usage>
.
Attention
Trusted Advisor support in awslimitchecker is deprecated outside of the China and GovCloud regions, and now defaults to disabled/skipped in standard AWS, as the information available from TA can now be retrieved faster and more accurately via other means. See changelog.10_0_0
for further information.
When :py~awslimitchecker.checker.AwsLimitChecker
is initialized, it also initializes an instance of :py~awslimitchecker.trustedadvisor.TrustedAdvisor
. In :py~.AwsLimitChecker.get_limits
, :py~.AwsLimitChecker.find_usage
and :py~.AwsLimitChecker.check_thresholds
, when called with use_ta == True
(the default), :py~.TrustedAdvisor.update_limits
is called on the TrustedAdvisor instance.
:py~.TrustedAdvisor.update_limits
polls Trusted Advisor data from the Support API via :py~.TrustedAdvisor._poll
; this will retrieve the limits for all "flaggedResources" items in the Service Limits
Trusted Advisor check result for the current AWS account. It then calls :py~.TrustedAdvisor._update_services
, passing in the Trusted Advisor check results and the dict of :py~._AwsService
objects it was called with (from :py~.AwsLimitChecker
).
:py~.TrustedAdvisor._update_services
iterates over the Services in the Trusted Advisor check result and attempts to find a matching :py~._AwsService
(by string service name) in the dict passed in from :py~.AwsLimitChecker
. If a match is found, it iterates over all limits for that service in the TA result and attempts to call the Service
's :py~._AwsService._set_ta_limit
method. If a matching Service is not found, or if _set_ta_limit
raises a ValueError (matching Limit not found for that Service), an error is logged.
When :py~awslimitchecker.checker.AwsLimitChecker
initializes :py~awslimitchecker.trustedadvisor.TrustedAdvisor
, it passes in the self.services
dictionary of all services and limits. At initialization time, :py~awslimitchecker.trustedadvisor.TrustedAdvisor
iterates all services and limits, and builds a new dictionary mapping the limit objects by the return values of their :py~awslimitchecker.limit.AwsLimit.ta_service_name
and :py~awslimitchecker.limit.AwsLimit.ta_limit_name
properties. This allows limits to override the Trusted Advisor service and limit name that their data comes from. In the default case, their service and limit names will be used as they are set in the awslimitchecker code, and limits which have matching Trusted Advisor data will be automatically populated.
In the :py~awslimitchecker.trustedadvisor.TrustedAdvisor
class's :py~.TrustedAdvisor._poll
method, :py~.TrustedAdvisor._get_refreshed_check_result
is used to retrieve the check result data from Trusted Advisor. This method also implements the check refresh logic. See the comments in the source code for the specific logic. There are three methods of refreshing checks (refresh modes), which are controlled by the ta_refresh_mode
parameter to :py~awslimitchecker.trustedadvisor.TrustedAdvisor
:
- If
ta_refresh_mode
is the string "wait", the check will be refreshed and awslimitchecker will poll for the refresh result every 30 seconds, waiting for the refresh to complete (or untilta_refresh_timeout
seconds have elapsed). This is exposed via the CLI as the--ta-refresh-wait
option. - If
ta_refresh_mode
is an integer, it will operate like the "wait" mode above, but only if the current result data for the check is more thanta_refresh_mode
seconds old. This is exposed via the CLI as the--ta-refresh-older
option. - If
ta_refresh_mode
is the string "trigger", the check will be refreshed and the program will continue on immediately, without waiting for the refresh to complete; this will almost certainly result in stale check results in the current run. However, this may be useful if you desire to keepawslimitchecker
runs short, and run it on a regular schedule (i.e. if you runawslimitchecker
every 6 hours, and are OK with Trusted Advisor check data being 6 hours old). This is exposed via the CLI as the--ta-refresh-trigger
option.
Additionally, :py~awslimitchecker.trustedadvisor.TrustedAdvisor
has a ta_refresh_timeout
parameter. If this is set to a non-None
value (an integer), refreshes of the check will time out after that number of seconds. If a timeout occurs, a message will be logged at error level, but the program will continue running (most likely using the old result data). This parameter is exposed via the CLI as the --ta-refresh-timeout
option.
Important: It may take 30 to 60 minutes for the Service Limits check to refresh on large accounts. Please be aware of this when enabling the refresh options.
Using the check refresh options will require the trustedadvisor:RefreshCheck
IAM permission.
For use via Python, these same parameters (ta_refresh_mode
and ta_refresh_timeout
) are exposed as parameters on the :py~awslimitchecker.checker.AwsLimitChecker
constructor.
Unless use of Serivce Quotas is disabled with the --skip-quotas
command line option or by passing skip_quotas=False
to the :py~awslimitchecker.checker.AwsLimitChecker
constructor, awslimitchecker will retrieve all relevant data from the Service Quotas service. In the :py~.AwsLimitChecker
constructor (so long as skip_quotas
is True), an instance of the :py~.ServiceQuotasClient
class is constructed, passing in our boto3 connection keyword arguments for the current region. This client class instance is then passed to the constructor of every Service class (:py~._AwsService
subclass) when the class is created, via the quotas_client
argument. Each :py~._AwsService
class stores this as the _quotas_client
instance variable.
As the :py~.AwsLimitChecker
class iterates over all (configured) services in its :py~.AwsLimitChecker.get_limits
, :py~.AwsLimitChecker.find_usage
, and :py~.AwsLimitChecker.check_thresholds
methods, it will call the service class's :py~._AwsService._update_service_quotas
method after calling :py~.TrustedAdvisor.update_limits
and the service class's _update_limits_from_api()
method (if present), and before the actual operation of getting limits, finding usage, or checking thresholds.
The :py._AwsService._update_service_quotas
method will iterate through all limits (:py~.AwsLimit
) for the service and call the :py~.ServiceQuotasClient.get_quota_value
method for each. Assuming it returns a non-None
result, that result will be passed to the limit's :py~.AwsLimit._set_quotas_limit
method for later use in :py~.AwsLimit.get_limit
.
When retrieving values from Service Quotas, the ServiceCode
is taken from the :py._AwsService.quotas_service_code
attribute on the Service class. If that is set to None
, Service Quotas will not be consulted for that service. The ServiceCode
can also be overridden on a per-limit basis via the quotas_service_code
argument to the :py~.AwsLimit
constructor. The QuotaName
used by each limit defaults to the limit name itself (:py.AwsLimit
instance variable name
) but can be overridden with the quota_name
argument to the :py~.AwsLimit
constructor.
Note that quota names are stored and compared in lower case.
Some services provide API calls to retrieve at least some of the current limits, such as the DescribeAccountAttributes
API calls for RDS and EC2. Services that support such calls should make them in a _update_limits_from_api()
method, which will be automatically called from :py~.awslimitchecker.checker.AwsLimitChecker.get_limits
. The _update_limits_from_api()
method should make the API call, and then update all relevant limits via the :py~.AwsLimit
class's :py~.AwsLimit._set_api_limit
method.
The value used for a limit is the first match in the following list:
- Limit Override (set at runtime)
- API Limit
- Service Quotas
- Trusted Advisor
- Hard-coded default
For more information on overriding thresholds, see Python Usage / Setting a Threshold Override <python_usage.threshold_overrides>
as well as the documentation for :py.AwsLimitChecker.check_thresholds
and :py.AwsLimitChecker.set_threshold_override
.