Sourcery Starbot ⭐ refactored mikeasilva/cgr-work #1

sourcery-ai-bot · 2020-06-18T20:35:00Z

Thanks for starring sourcery-ai/sourcery ✨ 🌟 ✨

Here's your pull request refactoring your most popular Python repo.

If you want Sourcery to refactor all your Python repos and incoming pull requests install our bot.

Review changes via command line

To manually merge these changes, make sure you're on the master branch, then run:

git fetch https://github.com/sourcery-ai-bot/cgr-work master
git merge --ff-only FETCH_HEAD
git reset HEAD^

sourcery-ai-bot · 2020-06-18T20:35:01Z

general-python-scripts/03-Fill in Missing Income and Assets.py

-holes = list()
+holes = []


Lines 96-96 refactored with the following changes:

Replace list() with []

sourcery-ai-bot · 2020-06-18T20:35:01Z

general-python-scripts/2005 5 Year ACS Download.py

-    if n%2 == 0 or (n+1)%2 == 0:
-        return True
-    return False
+    return n%2 == 0 or (n+1)%2 == 0


Function validMetaData refactored with the following changes:

Simplify conditional into return statement

sourcery-ai-bot · 2020-06-18T20:35:02Z

general-python-scripts/2013 5 Year ACS Download.py

-    if n%2 == 0 or (n+1)%2 == 0:
-        return True
-    return False
+    return n%2 == 0 or (n+1)%2 == 0


Function validMetaData refactored with the following changes:

Simplify conditional into return statement

sourcery-ai-bot · 2020-06-18T20:35:02Z

general-python-scripts/2014 5 Year ACS Download.py

-    if n%2 == 0 or (n+1)%2 == 0:
-        return True
-    return False
+    return n%2 == 0 or (n+1)%2 == 0


Function validMetaData refactored with the following changes:

Simplify conditional into return statement

sourcery-ai-bot · 2020-06-18T20:35:02Z

general-python-scripts/ACS Processing.py

-file_names = list()
+file_names = []


Lines 41-91 refactored with the following changes:

Replace assignment with augmented assignment

Replace list() with []

Simplify logical expression

sourcery-ai-bot · 2020-06-18T20:35:06Z

general-python-scripts/Erie Peer Finder.py

-    return sum([i*j for i,j in zip(a, b)])/(math.sqrt(sum([i*i for i in a]))* math.sqrt(sum([i*i for i in b])))
+    return sum(i * j for i, j in zip(a, b)) / (
+        math.sqrt(sum(i * i for i in a)) * math.sqrt(sum(i * i for i in b))
+    )


Function cosine_similarity refactored with the following changes:

Replace unneeded comprehension with generator

sourcery-ai-bot · 2020-06-18T20:35:06Z

general-python-scripts/Extract Response Times.py

@@ -5,6 +5,7 @@
 @author: Michael
 """



Lines 25-74 refactored with the following changes:

Use any() instead of for loop

Simplify inverted any() and all() calls

sourcery-ai-bot · 2020-06-18T20:35:06Z

general-python-scripts/FBI UCR.py

-    temp_dict = dict()
+    temp_dict = {}

    # Open the file
    f = open(path, 'r')

    for line in f:
        # Loop through the items
-        line_number = line_number + 1
+        line_number += 1
        temp_dict = dict()
-        for i in range(0,len(head)):
+        for i in range(len(head)):


Function fbi_ucr refactored with the following changes:

Replace assignment with augmented assignment

Replace range(0, x) with range(x)

Replace dict() with {}

sourcery-ai-bot · 2020-06-18T20:35:06Z

general-python-scripts/FBI UCR.py

-        for i, l in enumerate(f):
-            pass
+        pass


Function file_len refactored with the following changes:

Hoist statements out of for/while loops

sourcery-ai-bot · 2020-06-18T20:35:07Z

general-python-scripts/Feeding America Map Scrapper.py

+
 import requests
 import pandas as pd
 from sqlalchemy import create_engine

 # This will hold the data scrapped from the API
-scrapped_data = list()
+scrapped_data = []

 for year in range(2013, 2016):
    url = "http://map.feedingamerica.org/api/v1/map/county."+str(year)+".overall.null.null.null?sideload[]=county&sideload[]=national&&sideload[]=states"
    response = requests.get(url)
    j = response.json()
-    states = dict()
+    states = {}


Lines 13-34 refactored with the following changes:

Replace list() with []

Replace dict() with {}

sourcery-ai-bot · 2020-06-18T20:35:09Z

general-python-scripts/Google Distance to AC.py

@@ -4,6 +4,7 @@

 @author: Michael
 """
+


Lines 15-36 refactored with the following changes:

Replace if statement with if expression

Replace list() with []

sourcery-ai-bot · 2020-06-18T20:35:09Z

general-python-scripts/LEHD Scrapper.py

-    data = dict()
+    data = {}


Function parse_lehd_html refactored with the following changes:

Replace dict() with {}

sourcery-ai-bot · 2020-06-18T20:35:10Z

general-python-scripts/LEHD Scrapper.py

-    
+
 sql = 'SELECT CGR_GEO_ID, NAME, LEHD_URL FROM CGR_GeographyIndex WHERE CI_GEO=1 AND LEHD_URL IS NOT NULL'

 query = conn.execute(sql)
 browser = webdriver.Chrome()

-data = list()
+data = []


Lines 42-68 refactored with the following changes:

Replace list() with []

sourcery-ai-bot · 2020-06-18T20:35:10Z

general-python-scripts/Living Wage Web Scrapper.py

@@ -5,6 +5,7 @@
 @author: Michael Silva
 """



Lines 18-61 refactored with the following changes:

Replace list() with []

sourcery-ai-bot · 2020-06-18T20:35:10Z

general-python-scripts/Local Arts Index Cultural Attendance.py

+
 from bs4 import BeautifulSoup
 import requests as re
 import pandas as pd

 base_url = 'http://www.artsindexusa.org/where-i-live?c4='
 data_url = 'http://www.artsindexusa.org/fetchCounty.php?selectedCounty='
-data = list()
+data = []


Lines 13-43 refactored with the following changes:

Replace assignment with augmented assignment

Replace list() with []

Replace dict() with {}

sourcery-ai-bot · 2020-06-18T20:35:13Z

general-python-scripts/QCEW Update.py

-    # the enb extension.   
-    file_name_list = list()
+    # the enb extension.
+    file_name_list = []


Function get_file_name_list refactored with the following changes:

Replace list() with []

sourcery-ai-bot · 2020-06-18T20:35:13Z

general-python-scripts/ROC Docs Real Estate Scrapper.py

@@ -5,6 +5,7 @@
 @author: Michael Silva
 """



Lines 35-78 refactored with the following changes:

Replace if statement with if expression

Replace list() with []

sourcery-ai-bot · 2020-06-18T20:35:13Z

general-python-scripts/Scrape Opportunity Index.py

+
 import requests
 import pandas as pd

-opportunity_index_data = list()
+opportunity_index_data = []

 # Loop through the list of fips codes
 for line in open('fips.csv', 'r', encoding='latin-1'):
-    csv_row = line.split(',') 
+    csv_row = line.split(',')
    # Pull the name
    pos = line.find('"')
-    if pos > 0:
-        name = line[pos:].strip().replace('"','')
-    else:
-        name = csv_row[2].strip()
+    name = line[pos:].strip().replace('"','') if pos > 0 else csv_row[2].strip()


Lines 10-35 refactored with the following changes:

Replace if statement with if expression

Replace list() with []

sourcery-ai-bot · 2020-06-18T20:35:14Z

general-python-scripts/Total nonprofit arts revenue per capita.py

+
 from bs4 import BeautifulSoup
 import requests as re
 import pandas as pd

 base_url = 'http://www.artsindexusa.org/where-i-live?c4='
 data_url = 'http://www.artsindexusa.org/fetchCounty.php?selectedCounty='
-data = list()
+data = []


Lines 13-43 refactored with the following changes:

Replace assignment with augmented assignment

Replace list() with []

Replace dict() with {}

sourcery-ai-bot · 2020-06-18T20:35:14Z

general-python-scripts/Unbanked and Underbanked Scrapper.py

@@ -5,6 +5,7 @@
 @author: Michael Silva
 """



Lines 19-19 refactored with the following changes:

Replace list() with []

sourcery-ai-bot · 2020-06-18T20:35:16Z

general-python-scripts/Update BLS_QCEW_Data.py

-    file_name_list = list()
+    file_name_list = []


Function get_file_name_list refactored with the following changes:

Replace list() with []

sourcery-ai-bot · 2020-06-18T20:35:17Z

general-python-scripts/Vera Incarceration Rates Scrapper.py

+
 import requests
 from sqlalchemy import create_engine
 import pandas as pd

 # This will hold the data
-data = list()
+data = []


Lines 13-71 refactored with the following changes:

Replace list() with []

Replace dict() with {}

sourcery-ai-bot · 2020-06-18T20:35:17Z

general-python-scripts/Youth Employment Statistics - Wrangle psam_p06.csv.py

+    "RAC1P",
+    "SCH",
+    "SCHL",
+    "PWGTP",
+    "PWGTP1",
+    "PWGTP2",
+    "PWGTP3",
+    "PWGTP4",
+    "PWGTP5",
+    "PWGTP6",
+    "PWGTP7",
+    "PWGTP8",
+    "PWGTP9",
+    "PWGTP10",
+    "PWGTP11",
+    "PWGTP12",
+    "PWGTP13",
+    "PWGTP14",
+    "PWGTP15",
+    "PWGTP16",
+    "PWGTP17",
+    "PWGTP18",
+    "PWGTP19",
+    "PWGTP20",
+    "PWGTP21",
+    "PWGTP22",
+    "PWGTP23",
+    "PWGTP24",
+    "PWGTP25",
+    "PWGTP26",
+    "PWGTP27",
+    "PWGTP28",
+    "PWGTP29",
+    "PWGTP30",
+    "PWGTP31",
+    "PWGTP32",
+    "PWGTP33",
+    "PWGTP34",
+    "PWGTP35",
+    "PWGTP36",
+    "PWGTP37",
+    "PWGTP38",
+    "PWGTP39",
+    "PWGTP40",
+    "PWGTP41",
+    "PWGTP42",
+    "PWGTP43",
+    "PWGTP44",
+    "PWGTP45",
+    "PWGTP46",
+    "PWGTP47",
+    "PWGTP48",
+    "PWGTP49",
+    "PWGTP50",
+    "PWGTP51",
+    "PWGTP52",
+    "PWGTP53",
+    "PWGTP54",
+    "PWGTP55",
+    "PWGTP56",
+    "PWGTP57",
+    "PWGTP58",
+    "PWGTP59",
+    "PWGTP60",
+    "PWGTP61",
+    "PWGTP62",
+    "PWGTP63",
+    "PWGTP64",
+    "PWGTP65",
+    "PWGTP66",
+    "PWGTP67",
+    "PWGTP68",
+    "PWGTP69",
+    "PWGTP70",
+    "PWGTP71",
+    "PWGTP72",
+    "PWGTP73",
+    "PWGTP74",
+    "PWGTP75",
+    "PWGTP76",
+    "PWGTP77",
+    "PWGTP78",
+    "PWGTP79",
+    "PWGTP80",
+]

+for chunk in pd.read_csv("psam_p06.csv", low_memory = False, chunksize = chunksize):
    chunk = chunk[p_vars]
    chunk = chunk[chunk["PUMA"].isin(keep_me)]
    data.append(chunk)
-    
+
 df = pd.concat(data)
 df.to_csv("psam_p06_small.csv", index=False)

 with zipfile.ZipFile("csv_pca_small.zip", "w") as z:
   z.write("psam_p06_small.csv")
   z.write("ACS2013_2017_PUMS_README.pdf")
-   
+


Lines 36-139 refactored with the following changes:

Hoist statements out of for/while loops

sourcery-ai-bot · 2020-06-18T20:35:18Z

general-python-scripts/Youth Employment Statistics.py

-        if val == 1:
-            return False
-        else:
-            return True
+    return val != 1


Function recode_hisp refactored with the following changes:

Simplify conditional into return statement

sourcery-ai-bot · 2020-06-18T20:35:18Z

general-python-scripts/Youth Employment Statistics.py

-    elif val == 3 or val == 4 or val == 5:
+    elif val in [3, 4, 5]:


Function recode_rac1p refactored with the following changes:

Replace multiple comparisons of same variable with in operator

sourcery-ai-bot · 2020-06-18T20:35:21Z

general-python-scripts/presto.py

-    est = list()
-    moe = list()
+    est = []
+    moe = []


Function aggregate_acs_data refactored with the following changes:

Replace list() with []

sourcery-ai-bot · 2020-06-18T20:35:21Z

general-python-scripts/presto.py

-    inflator = base_cpi / cpi
-    return inflator
+    return base_cpi / cpi


Function get_cpi_inflator refactored with the following changes:

Inline variable that is only used once

sourcery-ai-bot · 2020-06-18T20:35:21Z

general-python-scripts/presto.py

-    aggregate_data = list()
-    for a in aggregate_me:
-        if a[1] not in geos:
-            aggregate_data.append({'geo_id': a[0], 'aggregate_id': a[1]})
+    aggregate_data = [
+        {'geo_id': a[0], 'aggregate_id': a[1]}
+        for a in aggregate_me
+        if a[1] not in geos
+    ]
+
    aggregate_data = pd.DataFrame(aggregate_data)
-    
+
    # Merge it with our data (inner join)
    df = pd.merge(df, aggregate_data, on='geo_id')
-    
+
    # Set some common values
    df['NAME'] = df['geo_id'] = df['aggregate_id']
-    
+
    # Drop the extra column
    df = df.drop(['aggregate_id'], axis=1)   
-    
+
    # Determine the columns
-    cols_to_keep = list(df.columns.values) 
+    cols_to_keep = list(df.columns.values)
    cols_to_aggregate = [e for e in cols_to_keep if e not in ('NAME', 'geo_id')]
-    
+
    # Deal with the margin of error
    if 'count_moe' in cols_to_aggregate:
        # Square the Margin of Error
        df['count_moe'] = df['count_moe'] ** 2
-    
+
    # Aggregate (Sum)
    df = df.groupby(['NAME', 'geo_id'])[cols_to_aggregate].sum().reset_index()
-    
+
    # Deal with margin of error
    if 'count_moe' in cols_to_aggregate:
        # Square root the sum of squares
        df['count_moe'] = df['count_moe'] ** .5    
-    
+


Function get_sum_aggregates refactored with the following changes:

Replace list() with []

Convert for loop into list comprehension

sourcery-ai-bot · 2020-06-18T20:35:21Z

general-python-scripts/presto.py

-    aggregate_data = list()
-    for a in aggregate_me:
-        aggregate_data.append({'geo_id': a[0], 'aggregate_id': a[1]})
+    aggregate_data = [{'geo_id': a[0], 'aggregate_id': a[1]} for a in aggregate_me]
    aggregate_data = pd.DataFrame(aggregate_data)
-    
+
    # Merge it with our data (inner join)
    df = pd.merge(df, aggregate_data, on='geo_id')
-    
+


Function get_weighted_average_aggregate refactored with the following changes:

Replace list() with []

Convert for loop into list comprehension

sourcery-ai-bot · 2020-06-18T20:35:22Z

general-python-scripts/Responder API/data/Make Model JSON.py

-model = dict()
+model = {}


Lines 12-12 refactored with the following changes:

Replace dict() with {}

Refactored by Sourcery

fcef011

sourcery-ai-bot commented Jun 18, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sourcery Starbot ⭐ refactored mikeasilva/cgr-work #1

Sourcery Starbot ⭐ refactored mikeasilva/cgr-work #1

sourcery-ai-bot commented Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

sourcery-ai-bot Jun 18, 2020

		elif val == 3 or val == 4 or val == 5:
		elif val in [3, 4, 5]:

		holes = list()
		holes = []

		file_names = list()
		file_names = []

		model = dict()
		model = {}

Sourcery Starbot ⭐ refactored mikeasilva/cgr-work #1

Are you sure you want to change the base?

Sourcery Starbot ⭐ refactored mikeasilva/cgr-work #1

Conversation

sourcery-ai-bot commented Jun 18, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment