diff --git a/generate_index.py b/generate_index.py index 5222efb..8a52548 100644 --- a/generate_index.py +++ b/generate_index.py @@ -1,7 +1,6 @@ import collections import glob import re -import pdb PDF_META = 'PDF' MD_WILDCARD = 'summaries/*.md' @@ -10,12 +9,14 @@ md_files = sorted(glob.glob(MD_WILDCARD)) html_files = sorted(glob.glob(HTML_WILDCARD)) +assert(len(md_files) == len(html_files)) -ListItem = collections.namedtuple('ListItem', ['title', 'author', 'year', 'url']) +ListItem = collections.namedtuple( + 'ListItem', ['title', 'author', 'year', 'url']) list_items = [] for md_file, html_file in zip(md_files, html_files): - with open(md_file, 'rb') as f: + with open(md_file, 'r') as f: title_line = f.readline() match = re.search(TITLE_PATTERN, title_line) title = match.group(1) @@ -31,8 +32,8 @@ list_items = sorted(list_items, key=lambda t: t.year) -print '
    ' +print('
      ') for list_item in list_items: - print '
    1. %s (%s %d)
      2. ' % ( - list_item.url, list_item.title, list_item.author, list_item.year) -print '
    ' + print('
  1. %s (%s %d)
    2. ' % ( + list_item.url, list_item.title, list_item.author, list_item.year)) +print('
') diff --git a/html/chu2013socp-codegen.html b/html/chu2013socp-codegen.html index 76acd0a..5d02444 100644 --- a/html/chu2013socp-codegen.html +++ b/html/chu2013socp-codegen.html @@ -52,7 +52,7 @@

Novel contributions

  1. SOCPs encompass a large class of convex programs; previous canonicalization -suites targeted smaller problem clases, like QPs or QCQPs. Recall +suites targeted smaller problem classes, like QPs or QCQPs. Recall that linear programs, QPs, and QCQPs are all SOCPs.
  2. Problem parameters must enter through specific functions; this interface allows the code generator to circumvent all floating point operations.
    3. diff --git a/html/dunning2016jump.html b/html/dunning2016jump.html index e7b156a..015c0a3 100644 --- a/html/dunning2016jump.html +++ b/html/dunning2016jump.html @@ -46,7 +46,7 @@

    JuMP is a Julia-embedded modeling language for mathematical optimization that was written -with performance in mind; its salient features include automatic differentation +with performance in mind; its salient features include automatic differentiation of user-defined functions, efficient parsing of updated problems, support for user-defined problems and solve methods, and solver callbacks. JuMP targets linear, mixed-integer, quadratic, conic-quadratic, semidefinite, and nonlinear @@ -60,7 +60,7 @@

    Background

    a particular standard form, and it is the responsibility of the client to express her problem in an acceptable fashion. It is often not at all obvious to clients how to encode their problems using the solving -substrate's standard form; this encoding proces may require +substrate's standard form; this encoding proces may require clever re-expressions of the original problem or onerous stuffing of the problem's constraints into a rigid matrix structure.

    diff --git a/html/gatys2015neuralstyle.html b/html/gatys2015neuralstyle.html index 15fa3d1..4cf0dc6 100644 --- a/html/gatys2015neuralstyle.html +++ b/html/gatys2015neuralstyle.html @@ -43,8 +43,8 @@

    Papers

    A Neural Algorithm of Artistic Style (Gatys 2015)

    -

    Gatys et al. present a way to repurpose a deep network trained for image -classification to redraw images in the style of some reference image. +

    Gatys et al. present a way to repurpose a deep network trained for image +classification to redraw images in the style of some reference image. For example, this method can be used to render arbitrary photographs in the style of Van Gogh’s The Starry Night.

    @@ -75,7 +75,7 @@

    The Algorithm

    \end{array} \end{equation*} %]]> -

    where is fixed, is a weight that incorporates the the size +

    where is fixed, is a weight that incorporates the size of layer (see the paper for details), and is the optimization variable. Gram matrices are used above to capture correlations between the features within each layer. The -weighted expression measures the similarity diff --git a/html/gubin1966projections.html b/html/gubin1966projections.html index bb82a50..35ca033 100644 --- a/html/gubin1966projections.html +++ b/html/gubin1966projections.html @@ -199,12 +199,12 @@

    5. An Acceleration S

    John Von Neumann. The Geometry of Orthogonal Spaces. Functional Operators (AM-22), Vol. II. Princeton University Press, 1950. Reprint of lecture -notes originally compiled in 1933. ↩

    +notes originally compiled in 1933. ↩

  3. Boyd, S., & Dattorro, J. (2003). Alternating projections. EE392o, Stanford -University. ↩

    +University. ↩

diff --git a/html/gubin1996projections.html b/html/gubin1996projections.html deleted file mode 100644 index cd0b3ba..0000000 --- a/html/gubin1996projections.html +++ /dev/null @@ -1,165 +0,0 @@ - - - - - - - - - - - -
-
-

Papers

-
- - - -

The Method of Projections for Finding the Common Point of Convex Sets (Gubin 1966)

- -

1. Overview

- -
- -

Section Breakdown
-I. Methods of successive projection [4.5 pp]
-II. Proofs of [convergence results] theorems 1 and 2 [8.5 pp]
-III. Rate of convergence [3.5 pp]
-IV. Examples and Applications [7.5 pp]

- -
- -

This paper reviews methods of successive projections for finding a point in -the intersection of an arbitrary number of convex sets; this problem is known -as the convex feasibility problem. The family of methods under consideration -traces back to the alternating projections method of the 1930s, due to Johnn -von Neumann1. Though von Neumann cast it as a routine for finding a point -in the intersection of two subspaces, his method was later generalized to the -convex feasibility problem.

- -

Convex cone programs that admit strong duality can be reduced to instances of -the convex feasibility -problem via the -KKT system or the self-dual homogeneous embedding thereof.

- -

2. Key Contributions

-

(A) Strong convergence results for the method of projections, -proving that under certain conditions the sequences in question converge -linearly (presented in section II of the paper).

- -

(B) A cheap way to accelerate projection methods (presented in section -III of the paper).

- -

Simple examples show that projection methods are often painfully slow to -converge, so convergence results are not that interesting. But, because, as we -shall see, such methods are so simple, accelerating them with minor alterations -is an attractive prospect.

- -

3. The Method of Projections

- -

3.1 Definition: Projection

-

A projection of the point onto a set of a normed space is a -point that minimizes the distance between and , in the -following sense:

- - - -

3.2 The Convex Feasibility Problem & the Method of Projections

-

Let the sets convex, for . The convex -feasibility problem is the problem of finding a common point .

- -

The method of projections constructs a sequence of points where - is arbitrary and is chosen from by selecting a -set and stepping in the direction of projecting onto -the selected set, i.e.,

- - - -

Different policies for selecting the index yield different -instantiations of the method of projections. A simple, classical policy is -the cyclic one that sets as .

- -

Cyclic projections is an extension of von Neumman’s alternating -projections method: alternating projections is cyclic projections when the -number of sets . Perhaps surprisingly, the case of is not any -more general than : any convex feasibility problem that involves greater -than 2 sets can be reduced to a higher-dimensional problem with exactly two -sets: one set is the cartesian product of all the sets from the original -problem, and the other set is an affine set that enforces block-wise equality -of the vector.

- -

4. Proof ingredients

-

A proof of convergence for the alternating projections method can be found in -in [2]2, and a more general proof can be found in the reviewed paper. Here,

- - - -

Pathological Alternating Projections

- -

Accelerated Projection Method

- -
-
    -
  1. - -

    John Von Neumann. The Geometry of Orthogonal Spaces. Functional Operators -(AM-22), Vol. II. Princeton University Press, 1950. Reprint of lecture -notes originally compiled in 1933. ↩

    -
    2. -
  2. - -

    Boyd, S., & Dattorro, J. (2003). Alternating projections. EE392o, Stanford -University. ↩ ↩2

    -
    3. -
-
- - -
- - diff --git a/index.html b/index.html index 3224556..c9a3310 100644 --- a/index.html +++ b/index.html @@ -18,20 +18,20 @@

Papers

  1. The Method of Projections for Finding the Common Point of Convex Sets (Gubin 1966)
    2. -
  2. YALMIP: A Toolbox for Modeling and Optimization in MATLAB (Lofberg 2004)
    3. +
  3. YALMIP: A Toolbox for Modeling and Optimization in MATLAB (Lofberg 2004)
  4. Graph Implementations for Nonsmooth Convex Programs (Grant 2008)
  5. Code Generation for Embedded Second-Order Cone Programming (Chu 2013)
  6. A Neural Algorithm of Artistic Style (Gatys 2015)
  7. A Latent Variable Model Approach to PMI-based Word Embeddings (Arora 2016)
  8. CVXPY: A Python-Embedded Modeling Language for Convex Optimization (Diamond 2016)
  9. JuMP: A Modeling Language for Mathematical Optimization (Duninng 2016)
    10. -
  10. Train Faster, Generalize Better: Stability of Stochastic Gradient Descent (Hardt 2016)
    11. -
  11. TensorFlow: A System for Large-Scale Machine Learning (Mongat 2016)
    12. +
  12. Train Faster, Generalize Better: Stability of Stochastic Gradient Descent (Hardt 2016)
    13. +
  13. TensorFlow: A System for Large-Scale Machine Learning (Mongat 2016)
  14. Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding (O'Donoghue 2016)
  15. OptNet: Differentiable Optimization as a Layer in Neural Networks (Amos 2017)
  16. A Simple but Tough-to-Beat Baseline for Sentence Embeddings (Arora 2017)
    17. -
  17. Occupy the Cloud: Distributed Computing for the 99% (Jonas 2017)
    18. -
  18. The Mythos of Model Interpretability (Lipton 2017)
    19. +
  19. Occupy the Cloud: Distributed Computing for the 99% (Jonas 2017)
    20. +
  20. The Mythos of Model Interpretability (Lipton 2017)
diff --git a/reading_list.md b/reading_list.md index 54a79d0..3a6cb60 100644 --- a/reading_list.md +++ b/reading_list.md @@ -45,3 +45,14 @@ Recent Advances and Limitations * Lifted Neural Networks (El Ghaoui) * Minimizing Finite Sums with the Stochastic Average Gradient (Shmidt) * How to escape saddle points efficiently (jordan) +* Local Minima and Convergence in Low-Rank Semidefinite Programming (Burer, Monteiro) +* The non-convex Burer-Monteiro approach works on smooth semidefinite programs (Boumal) +* Fast Exact Multiplication by the Hessian (Pearlmutter 1994) +* The geometry of graphs and some of its algorithmic applications [LLR '94] +* Expander Flows, Geometric Embeddings and Graph Partitioning [Arora 09] +* Representation Tradeoffs for Hyperbolic Embeddings +* improving distributional similarity with lessons learned from word embeddings +* A La Carte Embedding: +Cheap but Effective Induction of Semantic Feature Vectors +* A CONVERGENCE +ANALYSIS OF GRADIENT DESCENT FOR DEEP LINEAR NEURAL NETWORKS diff --git a/style.css b/style.css index 8112017..0dcc0af 100644 --- a/style.css +++ b/style.css @@ -19,7 +19,7 @@ body { color: #333; margin: 0; padding: 0; - font-size: 18px; + font-size: 14px; } blockquote {