4.1.0 Release (#125)

* Pull request for #92: Change executable-finding behavior in Stata builder (#104)

* #92 rm config checking of stata exec, unless stata packages are checked

* #92 edit the messages

* #92 fix conditional block for default exec

* #92 fix conditional block in build_stata after merge

* Fix missing paren

* Pull request for #98: R command line syntax (#103)

* #98 first pass

* #98 edits

* Pull request: Switch user_config.yaml to config_user.yaml (#106)

* switch name to config_user.yaml

* #61 more switchs

* Pull Request for #96: Write LaTeX builder (#105)

* #96 add latex builder

* #96 write test for latex builder

* #96 change to 4.0.2

* #96 small edit

* #96 delete log files

* #96 add nonstop option in latex builde

* Pull request for #93: add moving top level log to end_log  (#107)

* #93 first pass

* #93 second pass

* #93 add carriage return

* Pull request for #95: Check target files after each building step (#110)

* #95 first pass

* #95 second pass after testing

* fix PR

* small edits

* Pull Request for #108: Fix unit tests in the current builders (#109)

* #108 update test_build_stata

* #108 fix test_configuration_tests

* #108 fix test_log

* #108 fix test_build_r

* #108 change build_r and tests

* #108 modify test_build_r

* #108 fix some checks after adding check_targets

* #108 comment out target check in build_tables

* Pull request for #111: Fix release.py (#113)

* #111 fix

* #111 edit readme

* #111 fix PR

* Pull request for #97: Anything builder (#114)

* #97 write anything builder

* #97 change log_file to log_ext mechanism

* #97 new version, builder generator

* #97 add pretty comments

* #97 fix comment indent

* #97 respond to PR

* #97 warning edits

* Pull request for #94: allow folders to be excluded from search (#115)

* #94 allow folders to be exluded from search

* #94 add argument to upper level function

* #94 fix syntax

* Pull request: Bug in tablefill for latex (#102)

* LB Edits for latex tablefill

* #102 small edits and re-run tests

* #102 update python and re-run

* #102 update tests for tablefill latex

* #102 edit gitignore

* move import SCons.builder to builder instead of generator

* move import Scons.Builder outside action and into builder generator

* Pull request for migrating gencat (#116)

* first pass

* edit readme

* adding test

* uploading test

* fix test log

* Pull request for gslab-econ/template#60: Provenance (#120)

* #60 provenance.py mockup

* template#60 formatting

* #60 add pretty comments

* #60 add release mechanism

* template#60 respond to pr

* template#60 release PR

* gslab-econ/template#60 fix bug

* #60 respond to PR

* Pull request for #60: Provenance (round 2) (#121)

* #60 provenance.py mockup

* template#60 formatting

* #60 add pretty comments

* #60 add release mechanism

* template#60 respond to pr

* template#60 release PR

* gslab-econ/template#60 fix bug

* #60 respond to PR

* gslab-econ/template#60 add non-default sig

* gslab-econ/template#60 add external provenance option for release

* gslab-econ/template#60 pr

* gslab-econ/template#60 fix syntax

* Pull request for #91: size warning when git-lfs not present (#118)

* #91 first pass

* #91 improve warning message

* #91 second pass

* fix bug

* fix more bugs

* fix bugs and third pass

* fix bugs

* add in comments and options for path of git attributes

* first pass on debrief

* fix PR

* miss one file_name

* fix pr

* small edits

* implement debrief PR

* small edits

* Pull request for #122: Delete old local gslab_python versions upon both installation methods (#123)

* #122 test what pip does

* #122 cut some old code

* #122 abs path in setup.py

* #122 fix abs apth

* #122 Force uninstallation of all gslab tools at install time

* #123 #122 import site

* Pull request for template#60: update Provenance docs, update function args (#124)

* Allow version and readme args in function call instead of sysagrv, update docs

* respond to PR

* Pull request for #126: Fix lingering issues with release.py (#127)

* #126 hit all checks, no documentation

* #126 fix overwrite of scons_local_path

* #126 docs

* #127 #126 small readme changes for PR

* #126 small fixes

* #127 #126 abstract text search, better config checks

* #127 #126 search dir not output for SConstruct

* Pull request for #131: Change provenance to state_of_inputs (#132)

* #128 binding file limit on directory walk

* #131 purge provenance

* #131 first pass dir recording system

* #131 an SCons -> a SCons

* #131 change default log location

* #131 better naming, load into __init__, better prereq check

* #131 change default name for log

* #132 #131 change default log name

* #132 #131 gslab-econ/template#87 new log headding

* #132 #131 better path checks, fix walk-break rule, new path names

* #132 #131 fix heading creation of heading with hash

* Pull request for #130: Uncouple directory structure from gslab_python (#133)

* #130 fix log.py uncoupling

* #130 missing slash

* #133 add log.py optional argument

* #126 fix issue (#137)

* Pull request for 4.1.0: Change log exclusion syntax to prune (#138)

* change log exclusion syntax to proune

* #138 respond to pr

* gslab-econ/template#88 recursive fun to flatten dicts (#139)

* gslab-econ/template#88 recursive fun to flatten dicts

* #88 PR, don't clobber repeated keys

* #88 cleaner string manipulation in flatten_dict

* Pull request for #142: Allow parallelization in MATLAB builder (#143)

* #142 avoid collisions with sha1 hash of path

* #143 #142 fix source file removal

* Pull request for #140: Fix order of operations in release (#141)

* #140 remove superfluous assett upload

* #140 swap order of git and scons check; note behavior in readme

* #140 format readme

* #141 #140 split up up_to_date, raise error if not up-to-date

* #141 #140 put elif after if

* #141 #140 output -> log

* #141 #140 add function to determine if scons is a dry run

* #140 try some logging stuff

* #141 #140 loggers get own files, toggle on scons dry-run

* Pull request for #144: Do gslab_python side of MG review for 4.1.0 (#145)

* #144 gslab-econ/template#96 working versions of clean builders

* #144 fix recursive cases when checking vers and multiple cuaght exception

* #144 small adjustments to anything builder

* #145 #144 switch order of args in builders

* #145 #144 cleaner access to default stata executable

* #145 #144 docstrings in prereq checks

* #145 #144 unscramble more target/source args in the builders

gslab_fill/tablefill.py

@@ -17,7 +17,7 @@ def tablefill(**kwargs):
         write_to_lyx(args, lyx_text)
         exitmessage = args['template'] + ' filled successfully by tablefill'
         print exitmessage
-        return exitmessage	
+        return exitmessage    
     except:
         print 'Error Found'
         exitmessage = traceback.format_exc()
@@ -77,7 +77,13 @@ def parse_data(data):
     return tables    
 
 
-def insert_tables(args,tables):
+def insert_tables(args, tables):
+    if re.search('\.lyx', args['template']):
+        return insert_tables_lyx(args, tables)
+    elif re.search('\.tex', args['template']):
+        return insert_tables_latex(args, tables)
+
+def insert_tables_lyx(args, tables):
     lyx_text = open(args['template'], 'rU').readlines()
     for n in range( len(lyx_text) ):
         if lyx_text[n].startswith('name "tab:'):
@@ -86,14 +92,13 @@ def insert_tables(args,tables):
                 i = n
                 entry_count = 0
                 search_table = True
-                
+
                 while search_table is True:
                     i+=1
-
                     if re.match('^.*###', lyx_text[i]):
                         lyx_text[i] = lyx_text[i].replace('###', tables[tag][entry_count])
                         entry_count+=1
-                        
+
                     elif re.match('^.*#\d+#', lyx_text[i]) or re.match('^.*#\d+,#', lyx_text[i]):
                         entry_tag = re.split('#', lyx_text[i])[1]
                         if re.match('---', tables[tag][entry_count]):
@@ -104,12 +109,44 @@ def insert_tables(args,tables):
                                 rounded_entry = insert_commas(rounded_entry)
                         lyx_text[i] = lyx_text[i].replace('#' + entry_tag + '#', rounded_entry)
                         entry_count+=1
-
-                    elif lyx_text[i] == '</lyxtabular>\n' or lyx_text[i] == "end{tabular}":
+
+                    elif lyx_text[i] == '</lyxtabular>\n':
+                        search_table = False
+    return lyx_text
+
+def insert_tables_latex(args, tables):
+    lyx_text = open(args['template'], 'rU').readlines()
+    for n in range( len(lyx_text) ):
+        if re.search('label{tab:', lyx_text[n]):
+            tag = re.sub("[\}\"\n]", "", lyx_text[n].split(':')[1]).lower()
+            if tag in tables:
+                i = n
+                entry_count = 0
+                search_table = True
+
+                while search_table is True:
+                    i+=1
+                    lyx_text_i = lyx_text[i].split("&")
+                    for col in range(len(lyx_text_i)):
+                        if re.match('^.*###', lyx_text_i[col]):
+                            lyx_text_i[col] = lyx_text_i[col].replace('###', tables[tag][entry_count])
+                            entry_count+=1
+
+                        elif re.match('^.*#\d+#', lyx_text_i[col]) or re.match('^.*#\d+,#', lyx_text_i[col]):
+                            entry_tag = re.split('#', lyx_text_i[col])[1]
+                            if re.match('---', tables[tag][entry_count]):
+                                rounded_entry = '---'
+                            else:
+                                rounded_entry = round_entry(entry_tag, tables[tag][entry_count])
+                                if re.match('^.*#\d+,#', lyx_text_i[col]):
+                                    rounded_entry = insert_commas(rounded_entry)
+                            lyx_text_i[col] = lyx_text_i[col].replace('#' + entry_tag + '#', rounded_entry)
+                            entry_count+=1
+
+                    lyx_text[i] = "&".join(lyx_text_i)
+                    if re.search('end{tabular}', lyx_text[i]):
                         search_table = False
-
     return lyx_text
-
 
 def round_entry(entry_tag, entry):
     round_to = int(entry_tag.replace(',', ''))

gslab_fill/tests/input/tablefill_template.tex

@@ -0,0 +1,107 @@
+\documentclass[english]{article}
+\usepackage[T1]{fontenc}
+\usepackage[latin9]{inputenc}
+\usepackage{array}
+\usepackage{float}
+
+\makeatletter
+
+\providecommand{\tabularnewline}{\\}
+
+\makeatother
+
+\usepackage{babel}
+\begin{document}
+\begin{table}[H]
+\caption{Determinants of Newspaper Affiliation\label{tab:panel_supply}}
+\smallskip{}
+
+Dependent variable: Dummy for newspaper choosing Republican affiliation
+\begin{raggedright}
+\begin{tabular}{lccc}
+\hline 
+ & (1) & (2) & (3)\tabularnewline
+\hline 
+Republican vote share & #4,# & #4,# & #4,#\tabularnewline
+ & (#4,#) & (###) & (###)\tabularnewline
+Number of Republican incumbents & #0# & #4# & #4#\tabularnewline
+ & (#4#) & (#4#) & (#4#)\tabularnewline
+Number of Democratic incumbents & #4# & #4# & #4#\tabularnewline
+ & (#4#) & (#4#) & (#4#)\tabularnewline
+Lag Republican vote share &  &  & #4#\tabularnewline
+ &  &  & (#4#)\tabularnewline
+Instrument with lag vote share? &  & X & \tabularnewline
+\hline 
+R2 & #4# & #4# & #4#\tabularnewline
+Number of markets & #0# & #0# & #0#\tabularnewline
+Number of newspapers & #0# & #0# & #0#\tabularnewline
+\hline 
+\end{tabular}
+\par\end{raggedright}
+\raggedright{}{\footnotesize{}}%
+\begin{tabular}{>{\raggedright}p{6in}}
+\noalign{\vskip\doublerulesep}
+\raggedright{}\tabularnewline
+\end{tabular}{\footnotesize \par}
+\end{table}
+
+\begin{table}[H]
+\caption{Sensitivity of Parameter Estimates to Omitting Unobservables From
+Model\label{tab:unobservables}}
+\smallskip{}
+
+\centering{}%
+\begin{tabular}{lcc}
+\hline 
+ & Baseline & No Unobservables\tabularnewline
+\hline 
+\noalign{\vskip\doublerulesep}
+Demand parameters &  & \tabularnewline
+ &  & \tabularnewline
+\qquad{}$\underline{\beta}$ & ### & #2#\tabularnewline
+ & (#0#) & (#0#)\tabularnewline[\doublerulesep]
+\qquad{}$\overline{\beta}$ & #0# & #0#\tabularnewline
+ & (#10#) & (#10#)\tabularnewline[\doublerulesep]
+\qquad{}$\Gamma$ & #4# & #4#\tabularnewline
+ & (#4#) & (#4#)\tabularnewline[\doublerulesep]
+ &  & \tabularnewline
+Supply parameters &  & \tabularnewline
+ &  & \tabularnewline
+\qquad{}$a_{l}$ & #4# & #4#\tabularnewline
+ & (#4#) & (#4#)\tabularnewline[\doublerulesep]
+\qquad{}$\sigma_{\xi}$ & #4# & #4#\tabularnewline
+ & (#4#) & (#4#)\tabularnewline[\doublerulesep]
+\hline 
+\end{tabular}
+\end{table}
+
+\begin{table}
+\begin{centering}
+\caption{Determinants of Equilibrium Diversity\label{tab:Diversity}}
+\smallskip{}
+\par\end{centering}
+\centering{}%
+\begin{tabular}{lccc}
+\hline 
+ & Markets with & Share of hhlds & Share of hhlds\tabularnewline
+ & diverse & in mkt with & reading\tabularnewline
+ & papers & diverse papers & diverse papers\tabularnewline
+\hline 
+\noalign{\vskip0.01\textheight}
+Baseline & #0# & #2# & #3#\tabularnewline
+When choosing affiliation, newspapers: &  &  & \tabularnewline
+\noalign{\vskip\doublerulesep}
+\quad{}Ignore competitors' choices & #0# & #2# & #3#\tabularnewline[\doublerulesep]
+\noalign{\vskip\doublerulesep}
+\quad{}Ignore household ideology & #0# & #2# & #3#\tabularnewline[\doublerulesep]
+\noalign{\vskip\doublerulesep}
+\quad{}Ignore idiosyncratic cost shocks ($\xi$) & #0# & #2# & #3#\tabularnewline
+Owners chosen at random from &  &  & \tabularnewline
+local households and newspaper &  &  & \tabularnewline
+type equals owner type & #0# & #2# & #3#\tabularnewline[\doublerulesep]
+\hline 
+\noalign{\vskip0.01\textheight}
+\end{tabular}
+\end{table}
+
+\end{document}

gslab_fill/tests/input/tablefill_template_breaks.tex

@@ -0,0 +1,107 @@
+\documentclass[english]{article}
+\usepackage[T1]{fontenc}
+\usepackage[latin9]{inputenc}
+\usepackage{array}
+\usepackage{float}
+
+\makeatletter
+
+\providecommand{\tabularnewline}{\\}
+
+\makeatother
+
+\usepackage{babel}
+\begin{document}
+\begin{table}[H]
+\caption{Determinants of Newspaper Affiliation\label{tab:panel_supply}}
+\smallskip{}
+
+Dependent variable: Dummy for newspaper choosing Republican affiliation
+\begin{raggedright}
+\begin{tabular}{lccc}
+\hline 
+ & (1) & (2) & (3)\tabularnewline
+\hline 
+Republican vote share & #4,# & #4,# & #4,#\tabularnewline
+ & (#4,#) & (###) & (###)\tabularnewline
+Number of Republican incumbents & #0# & #4# & #4#\tabularnewline
+ & (#4#) & (#4#) & (#4#)\tabularnewline
+Number of Democratic incumbents & #4# & #4# & #4#\tabularnewline
+ & (#4#) & (#4#) & (#4#)\tabularnewline
+Lag Republican vote share &  &  & #4#\tabularnewline
+ &  &  & (#4#)\tabularnewline
+Instrument with lag vote share? &  & X & \tabularnewline
+\hline 
+R2 & #4# & #4# & #4#\tabularnewline
+Number of markets & #0# & #0# & #0#\tabularnewline
+Number of newspapers & #0# & #0# & #0#\tabularnewline
+\hline 
+\end{tabular}
+\par\end{raggedright}
+\raggedright{}{\footnotesize{}}%
+\begin{tabular}{>{\raggedright}p{6in}}
+\noalign{\vskip\doublerulesep}
+\raggedright{}\tabularnewline
+\end{tabular}{\footnotesize \par}
+\end{table}
+
+\begin{table}[H]
+\caption{Sensitivity of Parameter Estimates to Omitting Unobservables From
+Model\label{tab:unobservables}}
+\smallskip{}
+
+\centering{}%
+\begin{tabular}{lcc}
+\hline 
+ & Baseline & No Unobservables\tabularnewline
+\hline 
+\noalign{\vskip\doublerulesep}
+Demand parameters &  & \tabularnewline
+ &  & \tabularnewline
+\qquad{}$\underline{\beta}$ & #2# & #2#\tabularnewline
+ & (#0#) & (#0#)\tabularnewline[\doublerulesep]
+\qquad{}$\overline{\beta}$ & #0# & #0#\tabularnewline
+ & (#10#) & (#10#)\tabularnewline[\doublerulesep]
+\qquad{}$\Gamma$ & #4# & #4#\tabularnewline
+ & (#4#) & (#4#)\tabularnewline[\doublerulesep]
+ &  & \tabularnewline
+Supply parameters &  & \tabularnewline
+ &  & \tabularnewline
+\qquad{}$a_{l}$ & #4# & #4#\tabularnewline
+ & (#4#) & (#4#)\tabularnewline[\doublerulesep]
+\qquad{}$\sigma_{\xi}$ & #4# & #4#\tabularnewline
+ & (#4#) & (#4#)\tabularnewline[\doublerulesep]
+\hline 
+\end{tabular}
+\end{table}
+
+\begin{table}
+\begin{centering}
+\caption{Determinants of Equilibrium Diversity\label{tab:Diversity}}
+\smallskip{}
+\par\end{centering}
+\centering{}%
+\begin{tabular}{lccc}
+\hline 
+ & Markets with & Share of hhlds & Share of hhlds\tabularnewline
+ & diverse & in mkt with & reading\tabularnewline
+ & papers & diverse papers & diverse papers\tabularnewline
+\hline 
+\noalign{\vskip0.01\textheight}
+Baseline & #0# & #2# & #3#\tabularnewline
+When choosing affiliation, newspapers: &  &  & \tabularnewline
+\noalign{\vskip\doublerulesep}
+\quad{}Ignore competitors' choices & #0# & #2# & #3#\tabularnewline[\doublerulesep]
+\noalign{\vskip\doublerulesep}
+\quad{}Ignore household ideology & #0# & #2# & #3#\tabularnewline[\doublerulesep]
+\noalign{\vskip\doublerulesep}
+\quad{}Ignore idiosyncratic cost shocks ($\xi$) & #0# & #2# & #3#\tabularnewline
+Owners chosen at random from &  &  & \tabularnewline
+local households and newspaper &  &  & \tabularnewline
+type equals owner type & #0# & #2# & #3#\tabularnewline[\doublerulesep]
+\hline 
+\noalign{\vskip0.01\textheight}
+\end{tabular}
+\end{table}
+
+\end{document}