Skip to content

Commit

Permalink
feat: Migrated to common foremast-utils (from legacy gogo-utils)
Browse files Browse the repository at this point in the history
- This will keep the project inline with the new open source standards set
- Attributions remain, yet referring to contributions to project as Foremast Org (as its been 3-4 years now)
- Migrated to local assets, still waiting and cleaning up gogo-utils to foremast-utils in PyPI.

At this time we will not be deleting the old packages, but in time they may no longer work. Foremast-Utils will
continue to be maintained and in new versions of foremast be required (updated docs and build packages).
  • Loading branch information
Joel Vasallo authored and linjmeyer committed Oct 28, 2021
1 parent 4d4a49e commit d369205
Show file tree
Hide file tree
Showing 24 changed files with 41 additions and 41 deletions.
4 changes: 2 additions & 2 deletions CONTRIBUTING.rst
Expand Up @@ -146,5 +146,5 @@ Try to keep history as linear as possible using a `rebase` merge strategy.
#. Delete merged Branch

.. _Issue: https://github.com/gogoit/foremast/issues
.. _Pull Request: https://github.com/gogoit/foremast/pulls
.. _Issue: https://github.com/foremast/foremast/issues
.. _Pull Request: https://github.com/foremast/foremast/pulls
2 changes: 1 addition & 1 deletion README.rst
Expand Up @@ -19,7 +19,7 @@ Foremast is a Spinnaker pipeline and infrastructure configuration and
templating tool. Just create a couple JSON configuration files and then
manually creating Spinnaker pipelines becomes a thing of the past.

.. image:: https://s3.amazonaws.com/gogo-oss-logos/Foremast/Foremast+Logo-text-300.png
.. image:: _docs/_static/Foremast+Logo-text-300.png
:align: center


Expand Down
Binary file added _docs/_static/Foremast+Logo-text-300.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
4 changes: 2 additions & 2 deletions _docs/advance_usages.rst
Expand Up @@ -51,7 +51,7 @@ Both methods will generate the same outcome. An application created in Spinnaker
Method 1
********

This is our recommended method and how we internally at Gogo run Foremast. You need to first set the environment variables from above.
This is the recommended method on how to run Foremast. You need to first set the environment variables from above.

With the environment variables defined, you can simply run the command ``foremast-pipeline`` from the command line. This will create the Application in Spinnaker as well as generate a base pipeline.

Expand All @@ -70,4 +70,4 @@ Next Steps
----------
Take a look at the :doc:`infra_assumptions` docs for details on the necessary Jenkins jobs.

This is only the tip of what Foremast can do. It also has functionality for creating scaling policies, setting up AWS infrastructure (elbs, security groups, iam policies, s3 buckets), sending slack notifications, and destroying old infrastructure. Take a look at our internal workflow docs for more detail on how Foremast is used at Gogo.
Since Foremast is based on Jinja templates, it can be extended to do anything you need. For example, you can add functionality for creating scaling policies, setting up AWS infrastructure (elbs, security groups, iam policies, s3 buckets), sending slack notifications, and destroying old infrastructure. Take a look at our internal workflow docs for more detail on how Foremast is used at various organizations!
4 changes: 2 additions & 2 deletions _docs/configuration_files/foremast_config.rst
Expand Up @@ -389,7 +389,7 @@ Section handling the naming convention of applications, elb, iam, s3 buckets and
other services.

The most common sections are shown. The complete list of sections and defaults
are defined by the underlying library gogo-utils_.
are defined by the underlying library foremast-utils_.

Any of the possible variables below can be used as the value.

Expand Down Expand Up @@ -463,7 +463,7 @@ keyed by task name. This section only applies to AWS environments.
| *Default*: 120
| *Required*: No
.. _gogo-utils: https://github.com/gogoair/gogo-utils#formats
.. _foremast-utils: https://github.com/foremast/foremast-utils#formats

.. _gcp-section:

Expand Down
4 changes: 2 additions & 2 deletions _docs/getting_started.rst
Expand Up @@ -45,9 +45,9 @@ Method 2 - Using git

.. code-block:: sh
$ git clone https://github.com/gogoit/foremast.git
$ git clone https://github.com/foremast/foremast.git
$ cd foremast
$ pip3 install -U .
$ pip install -U .
Configuration Files
*******************
Expand Down
12 changes: 6 additions & 6 deletions _docs/pipeline_examples.rst
Expand Up @@ -14,9 +14,9 @@ transition.
Default Pipeline Flow
---------------------

The below flow can repeat for as many environments as defined in the configs. At
Gogo, most applications go through these stages 3 times as we deploy to dev,
stage, and production.
The below flow can repeat for as many environments as defined in the configs.
In general, most applications repeat a common set of stages 2-3 times. Typically,
the same way to deploy to dev (if used), stage, and production.

1. Configuration

Expand Down Expand Up @@ -77,11 +77,11 @@ templates.
Example Workflow
-----------------

At Gogo we have a detailed workflow for using Foremast internally. Feel free to
copy our workflow or use it as inspiration for your own. You can view all of our
Most Foremast users have a detailed workflow for using Foremast internally. Feel free to
copy this example workflow or use it as inspiration for your own. You can view all of our
internal templates on the `foremast-templates repo`_.

.. image:: _static/gogo-pipeline.png
.. image:: _static/foremast-pipeline.png

#. The :ref:`application_json` and :ref:`pipeline_json` are bundled directly
with the application code
Expand Down
2 changes: 1 addition & 1 deletion _docs/requirements.txt
@@ -1,7 +1,7 @@
awscli
boto3
deepmerge
gogo-utils
foremast-utils
jinja2
pluginbase
python-gitlab
Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
@@ -1,7 +1,7 @@
awscli
boto3
deepmerge
gogo-utils
foremast-utils
jinja2
pluginbase
python-gitlab
Expand Down
6 changes: 3 additions & 3 deletions src/foremast/app/__main__.py
Expand Up @@ -20,7 +20,7 @@
import argparse
import logging

import gogoutils
import foremastutils

from ..args import add_app, add_debug, add_properties, add_provider
from ..consts import APP_FORMATS, LOGGING_FORMAT
Expand Down Expand Up @@ -48,8 +48,8 @@ def main():
logging.getLogger(__package__.split('.')[0]).setLevel(args.debug)

if args.git and args.git != 'None':
parsed = gogoutils.Parser(args.git).parse_url()
generated = gogoutils.Generator(*parsed, formats=APP_FORMATS)
parsed = foremastutils.Parser(args.git).parse_url()
generated = foremastutils.Generator(*parsed, formats=APP_FORMATS)
project = generated.project
repo = generated.repo
else:
Expand Down
4 changes: 2 additions & 2 deletions src/foremast/configs/__main__.py
Expand Up @@ -20,7 +20,7 @@
import argparse
import logging

import gogoutils
import foremastutils

from ..args import add_debug
from ..consts import APP_FORMATS, LOGGING_FORMAT
Expand All @@ -45,7 +45,7 @@ def main():
LOG.setLevel(args.debug)
logging.getLogger(__package__.split('.')[0]).setLevel(args.debug)

generated = gogoutils.Generator(*gogoutils.Parser(args.git_short).parse_url(), formats=APP_FORMATS)
generated = foremastutils.Generator(*foremastutils.Parser(args.git_short).parse_url(), formats=APP_FORMATS)
git_short = generated.gitlab()['main']

if args.runway_dir:
Expand Down
4 changes: 2 additions & 2 deletions src/foremast/configs/outputs.py
Expand Up @@ -18,7 +18,7 @@
import logging
from pprint import pformat

import gogoutils
import foremastutils

from ..consts import APP_FORMATS
from ..utils import DeepChainMap, get_template
Expand Down Expand Up @@ -79,7 +79,7 @@ def write_variables(app_configs=None, out_file='', git_short=''):
Returns:
dict: Configuration equivalent to the JSON output.
"""
generated = gogoutils.Generator(*gogoutils.Parser(git_short).parse_url(), formats=APP_FORMATS)
generated = foremastutils.Generator(*foremastutils.Parser(git_short).parse_url(), formats=APP_FORMATS)

json_configs = {}
for env, configs in app_configs.items():
Expand Down
2 changes: 1 addition & 1 deletion src/foremast/consts.py
Expand Up @@ -78,7 +78,7 @@ def validate_key_values(config_handle, section, key, default=None):
def extract_formats(config_handle):
"""Get application formats.
See :class:`gogoutils.Formats` for available options.
See :class:`foremastutils.Formats` for available options.
Args:
config_handle (configparser.ConfigParser): Instance of configurations.
Expand Down
6 changes: 3 additions & 3 deletions src/foremast/pipeline/construct_pipeline_block.py
Expand Up @@ -115,7 +115,7 @@ def construct_pipeline_block(env='',
Args:
env (str): Deploy environment name, e.g. dev, stage, prod.
generated (gogoutils.Generator): Gogo Application name generator.
generated (foremastutils.Generator): Foremast Application name generator.
kwargs (dict): Extra variables to pass to Pipeline Templates.
previous_env (str): The previous deploy environment to use as
Trigger.
Expand Down Expand Up @@ -181,7 +181,7 @@ def ec2_pipeline_setup(
"""Handles ec2 pipeline data setup
Args:
generated (gogoutils.Generator): Generated naming formats.
generated (foremastutils.Generator): Generated naming formats.
project (str): Group name of application
settings (dict): Environment settings from configurations.
env (str): Deploy environment name, e.g. dev, stage, prod.
Expand Down Expand Up @@ -239,7 +239,7 @@ def ec2_pipeline_setup(
grace_period = hc_grace_period + app_grace_period

# TODO: Migrate the naming logic to an external library to make it easier
# to update in the future. Gogo-Utils looks like a good candidate
# to update in the future. foremast-utils looks like a good candidate
ssh_keypair = data['asg'].get('ssh_keypair', None)
if not ssh_keypair:
ssh_keypair = '{0}_{1}_default'.format(env, region)
Expand Down
Expand Up @@ -37,7 +37,7 @@ def construct_pipeline_block_cloudfunction(env='',
Args:
env (str): Deploy environment name, e.g. dev, stage, prod.
generated (gogoutils.Generator): Gogo Application name generator.
generated (foremastutils.Generator): Foremast Application name generator.
previous_env (str): The previous deploy environment to use as
Trigger.
region (str): GCP Region to deploy to.
Expand Down
Expand Up @@ -37,7 +37,7 @@ def construct_datapipeline(env='',
Args:
env (str): Deploy environment name, e.g. dev, stage, prod.
generated (gogoutils.Generator): Gogo Application name generator.
generated (foremastutils.Generator): Foremast Application name generator.
previous_env (str): The previous deploy environment to use as
Trigger.
region (str): AWS Region to deploy to.
Expand Down
2 changes: 1 addition & 1 deletion src/foremast/pipeline/construct_pipeline_block_lambda.py
Expand Up @@ -39,7 +39,7 @@ def construct_pipeline_block_lambda(env='',
Args:
env (str): Deploy environment name, e.g. dev, stage, prod.
generated (gogoutils.Generator): Gogo Application name generator.
generated (foremastutils.Generator): Foremast Application name generator.
previous_env (str): The previous deploy environment to use as
Trigger.
region (str): AWS Region to deploy to.
Expand Down
2 changes: 1 addition & 1 deletion src/foremast/pipeline/construct_pipeline_block_s3.py
Expand Up @@ -37,7 +37,7 @@ def construct_pipeline_block_s3(env='',
Args:
env (str): Deploy environment name, e.g. dev, stage, prod.
generated (gogoutils.Generator): Gogo Application name generator.
generated (foremastutils.Generator): Foremast Application name generator.
previous_env (str): The previous deploy environment to use as
Trigger.
region (str): AWS Region to deploy to.
Expand Down
Expand Up @@ -37,7 +37,7 @@ def construct_stepfunction(env='',
Args:
env (str): Deploy environment name, e.g. dev, stage, prod.
generated (gogoutils.Generator): Application name generator.
generated (foremastutils.Generator): Application name generator.
previous_env (str): The previous deploy environment to use as
Trigger.
region (str): AWS Region to deploy to.
Expand Down
6 changes: 3 additions & 3 deletions src/foremast/runner.py
Expand Up @@ -30,7 +30,7 @@
import logging
import os

import gogoutils
import foremastutils

from . import (autoscaling_policy, awslambda, configs, consts, datapipeline, dns, elb, iam, pipeline, s3,
scheduled_actions, securitygroup, slacknotify, stepfunction, utils)
Expand Down Expand Up @@ -65,8 +65,8 @@ def __init__(self):
self.provider = os.getenv("PROVIDER", "aws")

self.git_project = "{}/{}".format(self.group, self.repo)
parsed = gogoutils.Parser(self.git_project)
generated = gogoutils.Generator(*parsed.parse_url(), formats=consts.APP_FORMATS)
parsed = foremastutils.Parser(self.git_project)
generated = foremastutils.Generator(*parsed.parse_url(), formats=consts.APP_FORMATS)

self.app = generated.app_name()
self.trigger_job = generated.jenkins()['name']
Expand Down
Expand Up @@ -10,6 +10,6 @@
{% if data.app.approval_timeout %}
"stageTimeoutMs": {{ data.app.approval_timeout }},
{% endif %}
"instructions": "By approving this S3 content promotion to <strong>{{ data.app.environment }} LATEST</strong>, I certify that:\n<ul>\n<li>I am authorized by Gogo to perform this change</li>\n<li> To the best of my knowledge and ability, this version of the software is fit for purpose</li>\n<li>I did not develop any aspects of the code or configuration that will be promoted with this approval</li>\n</ul>",
"instructions": "By approving this S3 content promotion to <strong>{{ data.app.environment }} LATEST</strong>, I certify that:\n<ul>\n<li>I am authorized to perform this change</li>\n<li> To the best of my knowledge and ability, this version of the software is fit for purpose</li>\n<li>I did not develop any aspects of the code or configuration that will be promoted with this approval</li>\n</ul>",
"comments": "application: ${ trigger.properties.APP_NAME }"
}
4 changes: 2 additions & 2 deletions src/foremast/utils/apps.py
Expand Up @@ -16,7 +16,7 @@
"""Application related utilities."""
import logging

import gogoutils
import foremastutils

from ..consts import APP_FORMATS
from ..exceptions import SpinnakerAppNotFound
Expand Down Expand Up @@ -67,7 +67,7 @@ def get_details(app='groupproject', env='dev', region='us-east-1'):
LOG.debug('App details: %s', app_details)
group = app_details['attributes'].get('repoProjectKey')
project = app_details['attributes'].get('repoSlug')
generated = gogoutils.Generator(group, project, env=env, region=region, formats=APP_FORMATS)
generated = foremastutils.Generator(group, project, env=env, region=region, formats=APP_FORMATS)

LOG.debug('Application details: %s', generated)
return generated
4 changes: 2 additions & 2 deletions src/foremast/utils/encoding.py
Expand Up @@ -32,7 +32,7 @@ def generate_encoded_user_data(
Args:
env (str): Deployment environment, e.g. dev, stage.
region (str): AWS Region, e.g. us-east-1.
generated (gogoutils.Generator): Generated naming formats.
generated (foremastutils.Generator): Generated naming formats.
group_name (str): Application group nane, e.g. core.
pipeline_type (str): Type of Foremast Pipeline to configure.
Expand All @@ -49,7 +49,7 @@ def generate_encoded_user_data(
export CLOUD_STACK=forrest
export EC2_REGION=us-east-1
export CLOUD_DOMAIN=dev.example.com
printenv | grep 'CLOUD\|EC2' | awk '$0="export "$0'>> /etc/gogo/cloud_env
printenv | grep 'CLOUD\|EC2' | awk '$0="export "$0'>> /etc/foremast/cloud_env
"""
# We need to handle the case of prodp and prods for different URL generation
Expand Down

0 comments on commit d369205

Please sign in to comment.