04_Phrase_matching_and_Vocabulary.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "NLP - 4. Phrase matching and Vocabulary",
      "provenance": [],
      "authorship_tag": "ABX9TyMzsDDoAjAqfv/o06JAGwXx",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Justabhi96/NLP/blob/master/04_Phrase_matching_and_Vocabulary.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kJmzWNuWzivH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import spacy\n",
        "nlp = spacy.load(\"en_core_web_sm\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eb7DG0i_zynV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from spacy.matcher import Matcher"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HzP0DgMr0dVf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "matcher = Matcher(nlp.vocab)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nQaIF20E0iGU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#SolarPower\n",
        "pattern1 = [{\"LOWER\": \"solarpower\"}]\n",
        "#Solar-power\n",
        "pattern2 = [{\"LOWER\": \"solar\"}, {\"IS_PUNCT\": True}, {\"LOWER\": \"power\"}]\n",
        "#Solar power\n",
        "pattern3 = [{\"LOWER\": \"solar\"}, {\"LOWER\": \"power\"}]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ed_SKqXX1flR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "matcher.add(\"SolarPower\", None, pattern1, pattern2, pattern3)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HAFYi4W32Jq3",
        "colab_type": "code",
        "outputId": "e33b3914-910d-439f-f025-448b8c4e769f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 69
        }
      },
      "source": [
        "doc = nlp(\"Solar power industry continues to grow as solarpower grows. Solar-Power is great.\")\n",
        "found_matches = matcher(doc)\n",
        "found_matches"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[(8656102463236116519, 0, 2),\n",
              " (8656102463236116519, 7, 8),\n",
              " (8656102463236116519, 10, 13)]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zFCcqCP62a1h",
        "colab_type": "code",
        "outputId": "04cc7747-9dfc-4f67-f741-7524d1d2a535",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 69
        }
      },
      "source": [
        "for match_id, start, end in found_matches:\n",
        "  print(doc[start:end])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Solar power\n",
            "solarpower\n",
            "Solar-Power\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AlIxyfAI233O",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "matcher.remove(\"SolarPower\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c7paKrKM2643",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "pattern1 = [{\"LOWER\": \"solarpower\"}]\n",
        "# Solar--power or solar**power\n",
        "pattern2 = [{\"LOWER\": \"solar\"}, {\"IS_PUNCT\": True, \"OP\":\"*\"}, {\"LOWER\": \"power\"}]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "h4mZkcGs3WO8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "matcher.add(\"SolarPower\", None, pattern1, pattern2)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rKgFvYNW3fAA",
        "colab_type": "code",
        "outputId": "a14c8f6a-c7fd-4d4a-ff1a-1d2d069c9e28",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "doc2 = nlp(\"Solar--Power is great solarpower\")\n",
        "found_matches = matcher(doc2)\n",
        "for match_id, start, end in found_matches:\n",
        "  print(doc2[start:end])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Solar--Power\n",
            "solarpower\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m0_w4geHXf7O",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from spacy.matcher import PhraseMatcher\n",
        "matcher = PhraseMatcher(nlp.vocab)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j0r14BVAp2EL",
        "colab_type": "code",
        "outputId": "f971d687-2936-462c-f41b-13855c7c44d3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "%%writefile reagonomics.txt\n",
        "REAGANOMICS\n",
        "https://en.wikipedia.org/wiki/Reaganomics\n",
        "\n",
        "Reaganomics (a portmanteau of [Ronald] Reagan and economics attributed to Paul Harvey)[1] refers to the economic policies promoted by U.S. President Ronald Reagan during the 1980s. These policies are commonly associated with supply-side economics, referred to as trickle-down economics or voodoo economics by political opponents, and free-market economics by political advocates.\n",
        "\n",
        "The four pillars of Reagan's economic policy were to reduce the growth of government spending, reduce the federal income tax and capital gains tax, reduce government regulation, and tighten the money supply in order to reduce inflation.[2]\n",
        "\n",
        "The results of Reaganomics are still debated. Supporters point to the end of stagflation, stronger GDP growth, and an entrepreneur revolution in the decades that followed.[3][4] Critics point to the widening income gap, an atmosphere of greed, and the national debt tripling in eight years which ultimately reversed the post-World War II trend of a shrinking national debt as percentage of GDP.[5][6]\n",
        "\n",
        "HISTORICAL CONTEXT\n",
        "\n",
        "Prior to the Reagan administration, the United States economy experienced a decade of high unemployment and persistently high inflation (known as stagflation). Attacks on Keynesian economic orthodoxy as well as empirical economic models such as the Phillips Curve grew. Political pressure favored stimulus resulting in an expansion of the money supply. President Richard Nixon's wage and price controls were phased out.[7] The federal oil reserves were created to ease any future short term shocks. President Jimmy Carter had begun phasing out price controls on petroleum while he created the Department of Energy. Much of the credit for the resolution of the stagflation is given to two causes: a three-year contraction of the money supply by the Federal Reserve Board under Paul Volcker, initiated in the last year of Carter's presidency, and long-term easing of supply and pricing in oil during the 1980s oil glut.[citation needed]\n",
        "\n",
        "In stating that his intention was to lower taxes, Reagan's approach was a departure from his immediate predecessors. Reagan enacted lower marginal tax rates as well as simplified income tax codes and continued deregulation. During Reagan's eight year presidency, the annual deficits averaged 4.0% of GDP, compared to a 2.2% average during the preceding eight years.[8] The real (inflation adjusted) average rate of growth in federal spending fell from 4% under Jimmy Carter to 2.5% under Ronald Reagan.[9][10] GDP per employed person increased at an average 1.5% rate during the Reagan administration, compared to an average 0.6% during the preceding eight years.[11] Private sector productivity growth, measured as real output per hour of all persons, increased at an average rate of 1.9% during Reagan's eight years, compared to an average 1.3% during the preceding eight years.[12] Federal net outlays as a percent of GDP averaged 21.4% under Reagan, compared to 19.1% during the preceding eight years.[13]\n",
        "\n",
        "During the Nixon and Ford Administrations, before Reagan's election, a combined supply and demand side policy was considered unconventional by the moderate wing of the Republican Party. While running against Reagan for the Presidential nomination in 1980, George H. W. Bush had derided Reaganomics as \"voodoo economics\".[14] Similarly, in 1976, Gerald Ford had severely criticized Reagan's proposal to turn back a large part of the Federal budget to the states.\n",
        "\n",
        "JUSTIFICATIONS\n",
        "\n",
        "In his 1980 campaign speeches, Reagan presented his economic proposals as a return to the free enterprise principles, free market economy that had been in favor before the Great Depression and FDR's New Deal policies. At the same time he attracted a following from the supply-side economics movement, which formed in opposition to Keynesian demand-stimulus economics. This movement produced some of the strongest supporters for Reagan's policies during his term in office.\n",
        "\n",
        "The contention of the proponents, that the tax rate cuts would more than cover any increases in federal debt, was influenced by a theoretical taxation model based on the elasticity of tax rates, known as the Laffer curve. Arthur Laffer's model predicts that excessive tax rates actually reduce potential tax revenues, by lowering the incentive to produce; the model also predicts that insufficient tax rates (rates below the optimum level for a given economy) lead directly to a reduction in tax revenues.\n",
        "\n",
        "POLICIES\n",
        "\n",
        "Reagan lifted remaining domestic petroleum price and allocation controls on January 28, 1981,[15] and lowered the oil windfall profits tax in August 1981. He ended the oil windfall profits tax in 1988.[16] During the first year of Reagan's presidency, federal income tax rates were lowered significantly with the signing of the Economic Recovery Tax Act of 1981,[17] which lowered the top marginal tax bracket from 70% to 50% and the lowest bracket from 14% to 11%. This act slashed estate taxes and trimmed taxes paid by business corporations by $150 billion over a five-year period. In 1982 Reagan agreed to a rollback of corporate tax cuts and a smaller rollback of individual income tax cuts. The 1982 tax increase undid a third of the initial tax cut. In 1983 Reagan instituted a payroll tax increase on Social Security and Medicare hospital insurance.[18] In 1984 another bill was introduced that closed tax loopholes. According to tax historian Joseph Thorndike, the bills of 1982 and 1984 \"constituted the biggest tax increase ever enacted during peacetime\".[19]\n",
        "\n",
        "With the Tax Reform Act of 1986, Reagan and Congress sought to simplify the tax system by eliminating many deductions, reducing the highest marginal rates, and reducing the number of tax brackets.[20][21][22][23] In 1983, Democrats Bill Bradley and Dick Gephardt had offered a proposal; in 1984 Reagan had the Treasury Department produce its own plan. The 1986 act aimed to be revenue-neutral: while it reduced the top marginal rate, it also cleaned up the tax base by removing certain tax write-offs, preferences, and exceptions, thus raising the effective tax on activities previously specially favored by the code. Ultimately, the combination of the decrease in deductions and decrease in rates raised revenue equal to about 4% of existing tax revenue.[24]\n",
        "\n",
        "Federal revenue share of GDP fell from 19.6% in fiscal 1981 to 17.3% in 1984, before rising back to 18.4% by fiscal year 1989. Personal income tax revenues fell during this period relative to GDP, while payroll tax revenues rose relative to GDP.[25] Reagan's 1981 cut in the top regular tax rate on unearned income reduced the maximum capital gains rate to only 20%  its lowest level since the Hoover administration.[26] The 1986 act set tax rates on capital gains at the same level as the rates on ordinary income like salaries and wages, with both topping out at 28%.[27]\n",
        "\n",
        "Reagan significantly increased public expenditures, primarily the Department of Defense, which rose (in constant 2000 dollars) from $267.1 billion in 1980 (4.9% of GDP and 22.7% of public expenditure) to $393.1 billion in 1988 (5.8% of GDP and 27.3% of public expenditure); most of those years military spending was about 6% of GDP, exceeding this number in 4 different years. All these numbers had not been seen since the end of U.S. involvement in the Vietnam War in 1973.[28] In 1981, Reagan significantly reduced the maximum tax rate, which affected the highest income earners, and lowered the top marginal tax rate from 70% to 50%; in 1986 he further reduced the rate to 28%.[29] The federal deficit under Reagan peaked at 6% of GDP in 1983, falling to 3.2% of GDP in 1987[30] and to 3.1% of GDP in his final budget.[31] The inflation-adjusted rate of growth in federal spending fell from 4% under Jimmy Carter to 2.5% under Ronald Reagan. This was the slowest rate of growth in inflation adjusted spending since Eisenhower. However, federal deficit as percent of GDP was up throughout the Reagan presidency from 2.7% at the end of (and throughout) the Carter administration.[9][31][32] As a short-run strategy to reduce inflation and lower nominal interest rates, the U.S. borrowed both domestically and abroad to cover the Federal budget deficits, raising the national debt from $997 billion to $2.85 trillion.[33] This led to the U.S. moving from the world's largest international creditor to the world's largest debtor nation.[5] Reagan described the new debt as the \"greatest disappointment\" of his presidency.[34]\n",
        "\n",
        "According to William A. Niskanen, one of the architects of Reaganomics, \"Reagan delivered on each of his four major policy objectives, although not to the extent that he and his supporters had hoped\", and notes that the most substantial change was in the tax code, where the top marginal individual income tax rate fell from 70.1% to 28.4%, and there was a \"major reversal in the tax treatment of business income\", with effect of \"reducing the tax bias among types of investment but increasing the average effective tax rate on new investment\". Roger Porter, another architect of the program, acknowledges that the program was weakened by the many hands that changed the President's calculus, such as Congress.[2][35] President Reagan raised taxes eleven times over the course of his presidency, but the overall tax burden went down during his presidency.[36][37] According to Paul Krugman, \"Over all, the 1982 tax increase undid about a third of the 1981 cut; as a share of GDP, the increase was substantially larger than Mr. Clinton's 1993 tax increase.\"[18] According to historian and domestic policy adviser Bruce Bartlett, Reagan's tax increases over the course of his presidency took back half of the 1981 tax cut. Though since the Reagan tax reductions, top marginal tax rates have remained lower than at any point in US history since 1931, when the top marginal rate was raised from 25% to 63%.[38]\n",
        "\n",
        "RESULTS\n",
        "\n",
        "Overview\n",
        "\n",
        "Spending during the years Reagan budgeted (FY 198289) averaged 21.6% GDP, roughly tied with President Obama for the highest among any recent President. Each faced a severe recession early in their administration. In addition, the public debt rose from 26% GDP in 1980 to 41% GDP by 1988. In dollar terms, the public debt rose from $712 billion in 1980 to $2.052 trillion in 1988, a roughly three-fold increase.[25]:143 The unemployment rate rose from 7% in 1980 to 11% in 1982, then declined to 5% in 1988. The inflation rate declined from 10% in 1980 to 4% in 1988.[2]\n",
        "\n",
        "Some economists have stated that Reagan's policies were an important part of bringing about the third longest peacetime economic expansion in U.S. history.[39][40] During the Reagan administration, real GDP growth averaged 3.5%, compared to 2.9% during the preceding eight years.[41] The annual average unemployment rate declined by 1.7 percentage points, from 7.2% in 1980 to 5.5% in 1988, after it had increased by 1.6 percentage points over the preceding eight years.[42][43] Nonfarm employment increased by 16.1 million during Reagan's presidency, compared to 15.4 million during the preceding eight years,[44] while manufacturing employment declined by 582,000 after rising 363,000 during the preceding eight years.[45] Reagan's administration is the only one not to have raised the minimum wage.[46] The inflation rate, 13.5% in 1980, fell to 4.1% in 1988, due to the Federal Reserve increasing interest rates (prime rate peaking at 20.5% in August 1981[47]).[48] The latter contributed to a recession from July 1981 to November 1982 during which unemployment rose to 9.7% and GDP fell by 1.9%. Additionally, income growth slowed for middle- and lower-class (2.4% to 1.8%) and rose for the upper-class (2.2% to 4.83%).[49]\n",
        "\n",
        "The misery index, defined as the inflation rate added to the unemployment rate, shrank from 19.33 when he began his administration to 9.72 when he left, the greatest improvement record for a President since Harry S. Truman left office.[50] In terms of American households, the percentage of total households making less than $10,000 a year (in real 2007 dollars) shrank from 8.8% in 1980 to 8.3% in 1988 while the percentage of households making over $75,000 went from 20.2% to 25.7% during that period, both signs of progress.[51]\n",
        "\n",
        "Employment and wages\n",
        "\n",
        "The job growth (measured for non-farm payrolls) under the Reagan administration averaged 168,000 per month, versus 216,000 for Carter, 55,000 for H.W. Bush, and 239,000 for Clinton. Measuring the number of jobs created per month is limited for longer time periods as the population grows. To address this, we can measure annual job growth percentages, comparing the beginning and ending number of jobs during their time in office to determine an annual growth rate. Jobs grew by 2.0% annually under Reagan, versus 3.1% under Carter, 0.6% under H.W. Bush, and 2.4% under Clinton.[52]\n",
        "\n",
        "The unemployment rate averaged 7.5% under Reagan, compared to an average 6.6% during the preceding eight years. Declining steadily after December 1982, the rate was 5.4% the month Reagan left office.[53]\n",
        "\n",
        "The average real hourly wage for production and nonsupervisory workers continued the decline that had begun in 1973, albeit at a slower rate, and remained below the pre-Reagan level in every Reagan year.[54]\n",
        "\n",
        "The labor force participation rate increased by 2.6 percentage points during Reagan's eight years, compared to 3.9 percentage points during the preceding eight years.[55]\n",
        "\n",
        "Growth rates\n",
        "\n",
        "Following the 1981 recession, the unemployment rate had averaged slightly higher (6.75% vs. 6.35%), productivity growth lower (1.38% vs. 1.92%), and private investment as a percentage of GDP slightly less (16.08% vs. 16.86%).[citation needed] In the 1980's, industrial productivity growth in the United States matched that of its trading partners after trailing them in the 1970's. By 1990, manufacturing's share of GNP exceeded the post-World War II low hit in 1982 and matched \"the level of output achieved in the 1960's when American factories hummed at a feverish clip\".[56]\n",
        "\n",
        "GDP growth\n",
        "\n",
        "Real GDP grew over one-third during Reagans presidency, an over $2 trillion increase. The compound annual growth rate of GDP was 3.6% during Reagan's eight years, compared to 2.7% during the preceding eight years.[57] Real GDP per capita grew 2.6% under Reagan, compared to 1.9% average growth during the preceding eight years.[58]\n",
        "\n",
        "Income and wealth\n",
        "In nominal terms, median household income grew at a compound annual growth rate (CAGR) of 5.5% during the Reagan presidency, compared to 8.5% during the preceding five years (pre-1975 data are unavailable).[59] Real median family income grew by $4,492 during the Reagan period, compared to a $1,270 increase during the preceding eight years.[60] After declining from 1974 through 1980, real mean personal income rose $4,708 by 1988.[61] Nominal household net worth increased by a CAGR of 8.4%, compared to 9.3% during the preceding eight years.[62]\n",
        "\n",
        "Poverty level\n",
        "\n",
        "The percentage of the total population below the poverty level increased from 13.0% in 1980 to 15.2% in 1983, then declined back to 13.0% in 1988.[64] During Reagan's first term, critics noted homelessness as a visible problem in U.S. urban centers.[65] In the closing weeks of his presidency, Reagan told David Brinkley that the homeless \"make it their own choice for staying out there,\" noting his belief that there \"are shelters in virtually every city, and shelters here, and those people still prefer out there on the grates or the lawn to going into one of those shelters\". He also stated that \"a large proportion\" of them are \"mentally impaired.\" A result (he believed) of ACLU (and similar organizations) lawsuits against institutions.[66] His policies became widely known as \"trickle-down economics\", due to the significant cuts in the upper tax brackets, as that extra money for the wealthy could trickle along to low-income groups.[67]\n",
        "\n",
        "Federal income tax and payroll tax levels\n",
        "\n",
        "During the Reagan administration, fiscal year federal receipts grew from $599 billion to $991 billion (an increase of 65%) while fiscal year federal outlays grew from $678 billion to $1144 billion (an increase of 69%).[68][69] According to a 1996 report of the Joint Economic Committee of the United States Congress, during Reagan's two terms, and through 1993, the top 10% of taxpayers paid an increased share of income taxes (not including payroll taxes) to the Federal government, while the lowest 50% of taxpayers paid a reduced share of income tax revenue.[70] Personal income tax revenues declined from 9.4% GDP in 1981 to 8.3% GDP in 1989, while payroll tax revenues increased from 6.0% GDP to 6.7% GDP during the same period.[25]\n",
        "\n",
        "Tax receipts\n",
        "\n",
        "According to a 2003 Treasury study, the tax cuts in the Economic Recovery Tax Act of 1981 resulted in a significant decline in revenue relative to a baseline without the cuts, approximately $111 billion (in 1992 dollars) on average during the first four years after implementation or nearly 3% GDP annually.[71][72] Other tax bills had neutral or, in the case of the Tax Equity and Fiscal Responsibility Act of 1982, a (~+1% of GDP) increase in revenue as a share of GDP. It should be noted, however, that the study did not examine the longer-term impact of Reagan tax policy, including sunset clauses and \"the long-run, fully-phased-in effect of the tax bills\".[72] The fact that tax receipts as a percentage of GDP fell following the Economic Recovery Tax Act of 1981 shows a decrease in tax burden as share of GDP and a commensurate increase in the deficit, as spending did not fall relative to GDP. Total federal tax receipts increased in every Reagan year except 1982, at an annual average rate of 6.2% compared to 10.8% during the preceding eight years.[73]\n",
        "\n",
        "The effect of Reagan's 1981 tax cuts (reduced revenue relative to a baseline without the cuts) were at least partially offset by phased in Social Security payroll tax increases that had been enacted by President Jimmy Carter and the 95th Congress in 1977, and further increases by Reagan in 1983[74] and following years, also to counter the uses of tax shelters.[75] An accounting indicated nominal tax receipts increased from $599 billion in 1981 to $1.032 trillion in 1990, an increase of 72% in current dollars. In 2005 dollars, the tax receipts in 1990 were $1.5 trillion, an increase of 20% above inflation.[76]\n",
        "\n",
        "Debt and government expenditures\n",
        "Reagan was inaugurated in January 1981, so the first fiscal year (FY) he budgeted was 1982 and the final year was 1989.\n",
        "\n",
        "During Reagan's presidency, the federal debt held by the public nearly tripled in nominal terms, from $738 billion to $2.1 trillion.[77] This led to the U.S. moving from the world's largest international creditor to the world's largest debtor nation.[5] Reagan described the new debt as the \"greatest disappointment\" of his presidency.[34]\n",
        "The federal deficit as percentage of GDP rose from 2.5% of GDP in fiscal year 1981 to a peak of 5.7% of GDP in 1983, then fell to 2.7% GDP in 1989.[78]\n",
        "Total federal outlays averaged of 21.8% of GDP from 198188, versus the 19741980 average of 20.1% of GDP. This was the highest of any President from Carter through Obama.[79]\n",
        "Total federal revenues averaged 17.7% of GDP from 198188, versus the 197480 average of 17.6% of GDP.[80]\n",
        "Federal individual income tax revenues fell from 8.7% of GDP in 1980 to a trough of 7.5% of GDP in 1984, then rose to 7.8% of GDP in 1988.[81]\n",
        "Business and market performance\n",
        "Nominal after-tax corporate profits grew at a compound annual growth rate of 3.0% during Reagan's eight years, compared to 13.0% during the preceding eight years.[82] The S&P 500 Index increased 113.3% during the 2024 trading days under Reagan, compared to 10.4% during the preceding 2024 trading days.[83] The business sector share of GDP, measured as gross private domestic investment, declined by 0.7 percentage points under Reagan, after increasing 0.7 percentage points during the preceding eight years.[84]\n",
        "\n",
        "Size of federal government\n",
        "The federal government's share of GDP increased 0.2 percentage points under Reagan, while it decreased 1.5 percentage points during the preceding eight years.[85] The number of federal civilian employees increased 4.2% during Reagan's eight years, compared to 6.5% during the preceding eight years.[86]\n",
        "\n",
        "As a candidate, Reagan asserted he would shrink government by abolishing the Cabinet-level departments of energy and education. He abolished neither, but elevated veterans affairs from independent agency status to Cabinet-level department status.[87][88]\n",
        "\n",
        "Income distribution\n",
        "Further information: Income inequality in the United States\n",
        "Continuing a trend that began in the 1970s, income inequality grew and accelerated in the 1980s. The Economist wrote in 2006: \"After the 1973 oil shocks, productivity growth suddenly slowed. A few years later, at the start of the 1980s, the gap between rich and poor began to widen.\"[89] According to the CBO:\n",
        "\n",
        "The top 1% of income earners' share of income before transfers and taxes rose from 9.0% in 1979 to a peak of 13.8% in 1986, before falling to 12.3% in 1989.\n",
        "The top 1% share of income earners' of income after transfers and taxes rose from 7.4% in 1979 to a peak of 12.8% in 1986, before falling to 11.0% in 1989.\n",
        "The bottom 90% had a lower share of the income in 1989 vs. 1979.[90]\n",
        "\n",
        "ANALYSIS\n",
        "\n",
        "According to a 1996 study[93] by the Cato Institute, a libertarian think tank, on 8 of the 10 key economic variables examined, the American economy performed better during the Reagan years than during the pre- and post-Reagan years. The study asserted that real median family income grew by $4,000 and during the eight Reagan years and experienced a loss of almost $1,500 in the post-Reagan years. Interest rates, inflation, and unemployment fell faster under Reagan than they did immediately before or after his presidency. The only economic variable that was lower during period than in both the pre- and post-Reagan years was the savings rate, which fell rapidly in the 1980s. The productivity rate was higher in the pre-Reagan years but lower in the post-Reagan years.[93] The Cato study was dismissive of any positive effects of tightening, and subsequent loosening, of Federal Reserve monetary policy under \"inflation hawk\" Paul Volcker, whom President Carter had appointed in 1979 to halt the persistent inflation of the 1970s.\n",
        "\n",
        "Economic analyst Stephen Moore stated in the Cato analysis, \"No act in the last quarter century had a more profound impact on the U.S. economy of the eighties and nineties than the Reagan tax cut of 1981.\" He argued that Reagan's tax cuts, combined with an emphasis on federal monetary policy, deregulation, and expansion of free trade created a sustained economic expansion, the greatest American sustained wave of prosperity ever. He also claims that the American economy grew by more than a third in size, producing a $15 trillion increase in American wealth. Consumer and investor confidence soared. Cutting federal income taxes, cutting the U.S. government spending budget, cutting useless programs, scaling down the government work force, maintaining low interest rates, and keeping a watchful inflation hedge on the monetary supply was Ronald Reagan's formula for a successful economic turnaround.[93]\n",
        "\n",
        "Milton Friedman stated, \"Reaganomics had four simple principles: Lower marginal tax rates, less regulation, restrained government spending, noninflationary monetary policy. Though Reagan did not achieve all of his goals, he made good progress.\"[94]\n",
        "\n",
        "The Tax Reform Act of 1986 and its impact on the alternative minimum tax (AMT) reduced nominal rates on the wealthy and eliminated tax deductions, while raising tax rates on lower-income individuals.[94][95][96][97] The across the board tax system reduced marginal rates and further reduced bracket creep from inflation. The highest income earners (with incomes exceeding $1,000,000) received a tax break, restoring a flatter tax system.[98] In 2006, the IRS's National Taxpayer Advocate's report characterized the effective rise in the AMT for individuals as a problem with the tax code.[99] Through 2007, the revised AMT had brought in more tax revenue than the former tax code, which has made it difficult for Congress to reform.[98][100]\n",
        "\n",
        "Economist Paul Krugman argued the economic expansion during the Reagan administration was primarily the result of the business cycle and the monetary policy by Paul Volcker.[101] Krugman argues that there was nothing unusual about the economy under Reagan because unemployment was reducing from a high peak and that it is consistent with Keynesian economics for the economy to grow as employment increases if inflation remains low.[102]\n",
        "\n",
        "The CBO Historical Tables indicate that federal spending during Reagan's two terms (FY 198188) averaged 22.4% GDP, well above the 20.6% GDP average from 1971 to 2009. In addition, the public debt rose from 26.1% GDP in 1980 to 41.0% GDP by 1988. In dollar terms, the public debt rose from $712 billion in 1980 to $2,052 billion in 1988, a three-fold increase.[25] Krugman argued in June 2012 that Reagan's policies were consistent with Keynesian stimulus theories, pointing to the significant increase in per-capita spending under Reagan.[103]\n",
        "\n",
        "William Niskanen noted that during the Reagan years, privately held federal debt increased from 22% to 38% of GDP, despite a long peacetime expansion. Second, the savings and loan problem led to an additional debt of about $125 billion. Third, greater enforcement of U.S. trade laws increased the share of U.S. imports subjected to trade restrictions from 12% in 1980 to 23% in 1988.[2]\n",
        "\n",
        "Economists Raghuram Rajan and Luigi Zingales pointed out that many deregulation efforts had either taken place or had begun before Reagan (note the deregulation of airlines and trucking under Carter, and the beginning of deregulatory reform in railroads, telephones, natural gas, and banking). They stated, \"The move toward markets preceded the leader [Reagan] who is seen as one of their saviors.\"[104] Economists Paul Joskow and Roger Noll made a similar contention.[105]\n",
        "\n",
        "Economist William A. Niskanen, a member of Reagan's Council of Economic Advisers wrote that deregulation had the \"lowest priority\" of the items on the Reagan agenda[2] given that Reagan \"failed to sustain the momentum for deregulation initiated in the 1970s\" and that he \"added more trade barriers than any administration since Hoover.\" By contrast, economist Milton Friedman has pointed to the number of pages added to the Federal Register each year as evidence of Reagan's anti-regulation presidency (the Register records the rules and regulations that federal agencies issue per year). The number of pages added to the Register each year declined sharply at the start of the Ronald Reagan presidency breaking a steady and sharp increase since 1960. The increase in the number of pages added per year resumed an upward, though less steep, trend after Reagan left office. In contrast, the number of pages being added each year increased under Ford, Carter, George H. W. Bush, Clinton, George W. Bush, and Obama.[106] The number of pages in Federal Register is however criticized as an extremely crude measure of regulatory activity, because it can be easily manipulated (e.g. font sizes have been changed to keep page count low).[107] The apparent contradiction between Niskanen's statements and Friedman's data may be resolved by seeing Niskanen as referring to statutory deregulation (laws passed by Congress) and Friedman to administrative deregulation (rules and regulations implemented by federal agencies). A 2016 study by the Congressional Research Service found that Reagan's average annual number of final federal regulatory rules published in the Federal Register was higher than during the Clinton, George W. Bush or Obama's administrations, even though the Reagan economy was considerably smaller than during those later presidents.[108] Another study by the QuantGov project of the libertarian Mercatus Center found that the Reagan administration added restrictive regulations  containing such terms as \"shall,\" \"prohibited\" or \"may not\"  at a faster average annual rate than did Clinton, Bush or Obama.[109]\n",
        "\n",
        "Greg Mankiw, a conservative Republican economist who served as chairman of the Council of Economic Advisors under President George W. Bush, wrote in 2007:\n",
        "\n",
        "I used the phrase \"charlatans and cranks\" in the first edition of my principles textbook to describe some of the economic advisers to Ronald Reagan, who told him that broad-based income tax cuts would have such large supply-side effects that the tax cuts would raise tax revenue. I did not find such a claim credible, based on the available evidence. I never have, and I still don't...My other work has remained consistent with this view. In a paper on dynamic scoring, written while I was working at the White House, Matthew Weinzierl and I estimated that a broad-based income tax cut (applying to both capital and labor income) would recoup only about a quarter of the lost revenue through supply-side growth effects. For a cut in capital income taxes, the feedback is larger  about 50 percent  but still well under 100 percent. A chapter on dynamic scoring in the 2004 Economic Report of the President says about the the [sic] same thing.[110]\n",
        "\n",
        "Glenn Hubbard, who preceded Mankiw as Bush's CEA chair, also disputed the assertion that tax cuts increase tax revenues, writing in his 2003 Economic Report of the President: \"Although the economy grows in response to tax reductions (because of higher consumption in the short run and improved incentives in the long run), it is unlikely to grow so much that lost tax revenue is completely recovered by the higher level of economic activity.\"[111]\n",
        "\n",
        "In 1986, Martin Feldstein  a self-described \"traditional supply sider\" who served as Reagan's chairman of the Council of Economic Advisors from 1982 to 1984  characterized the \"new supply siders\" who emerged circa 1980:\n",
        "\n",
        "What distinguished the new supply siders from the traditional supply siders as the 1980s began was not the policies they advocated but the claims that they made for those policies...The \"new\" supply siders were much more extravagant in their claims. They projected rapid growth, dramatic increases in tax revenue, a sharp rise in saving, and a relatively painless reduction in inflation. The height of supply side hyperbole was the \"Laffer curve\" proposition that the tax cut would actually increase tax revenue because it would unleash an enormously depressed supply of effort. Another remarkable proposition was the claim that even if the tax cuts did lead to an increased budget deficit, that would not reduce the funds available for investment in plant and equipment because tax changes would raise the saving rate by enough to finance the increased deficit...Nevertheless, I have no doubt that the loose talk of the supply side extremists gave fundamentally good policies a bad name and led to quantitative mistakes that not only contributed to subsequent budget deficits but that also made it more difficult to modify policy when those deficits became apparent.[112]\n",
        "\n",
        "FOOTNOTES\n",
        "\n",
        "https://en.wikipedia.org/wiki/Reaganomics#Footnotes"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Writing reagonomics.txt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KMSj8LN4rDGM",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "with open(\"reagonomics.txt\") as f:\n",
        "  doc3 = nlp(f.read())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7mc1uQNGrR9s",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "phrase_list = [\"voodoo economics\", \"supply-side economics\", \n",
        "               \"trickle-down economics\", \"free-market economics\"]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Kq6HIyeJrkZ2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "phrase_patterns = [nlp(text) for text in phrase_list]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hmF0oyEyrzEC",
        "colab_type": "code",
        "outputId": "0d48b3e8-b646-48d9-a297-43d1dbff1d1a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 156
        }
      },
      "source": [
        "matcher.add(\"EconMatcher\", None, *phrase_patterns)\n",
        "found_matches = matcher(doc3)\n",
        "for match_id, start, end in found_matches:\n",
        "  print(doc3[start-5:end+5])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "policies are commonly associated with supply-side economics, referred to as trickle\n",
            "economics, referred to as trickle-down economics or voodoo economics by political\n",
            "trickle-down economics or voodoo economics by political opponents, and\n",
            "by political opponents, and free-market economics by political advocates.\n",
            "\n",
            "\n",
            "attracted a following from the supply-side economics movement, which formed in\n",
            "became widely known as \"trickle-down economics\", due to the\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RP_49WUevV1v",
        "colab_type": "text"
      },
      "source": [
        "To print the sentence with matches"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ogxbhoRDvVZy",
        "colab_type": "code",
        "outputId": "ff393d99-86a4-42ee-daf4-6a2c0abd0677",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 454
        }
      },
      "source": [
        "sentences = [sentence for sentence in doc3.sents]\n",
        "for match in found_matches:\n",
        "  for sentence in sentences:\n",
        "    if match[1] < sentence.end:\n",
        "      print(f\"==============={doc3[match[1]:match[2]]}================\")\n",
        "      print(sentence.text)\n",
        "      break"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "===============supply-side economics================\n",
            "These policies are commonly associated with supply-side economics, referred to as trickle-down economics or voodoo economics by political opponents, and free-market economics by political advocates.\n",
            "\n",
            "\n",
            "===============trickle-down economics================\n",
            "These policies are commonly associated with supply-side economics, referred to as trickle-down economics or voodoo economics by political opponents, and free-market economics by political advocates.\n",
            "\n",
            "\n",
            "===============voodoo economics================\n",
            "These policies are commonly associated with supply-side economics, referred to as trickle-down economics or voodoo economics by political opponents, and free-market economics by political advocates.\n",
            "\n",
            "\n",
            "===============free-market economics================\n",
            "These policies are commonly associated with supply-side economics, referred to as trickle-down economics or voodoo economics by political opponents, and free-market economics by political advocates.\n",
            "\n",
            "\n",
            "===============supply-side economics================\n",
            "At the same time he attracted a following from the supply-side economics movement, which formed in opposition to Keynesian demand-stimulus economics.\n",
            "===============trickle-down economics================\n",
            "His policies became widely known as \"trickle-down economics\", due to the significant cuts in the upper tax brackets, as that extra money for the wealthy could trickle along to low-income groups.[67]\n",
            "\n",
            "Federal income tax and payroll tax levels\n",
            "\n",
            "During the Reagan administration, fiscal year federal receipts grew from $599 billion to $991 billion (an increase of 65%) while fiscal year federal outlays grew from $678 billion to $1144 billion (an increase of 69%).[68][69]\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}