What are the steps for implementing a new syntax?

[jpfairbanks]

This came up in a biochemistry application I am working on for SemanticModels.

We want to model enzyme interactions where X is catalyzing the degradation of Y. In chemistry, this cleaving action is denoted:

    X + Y <=> XY
    XY -> X + degraded(Y)

The formation of the XY-complex is reversible but the degradation is not which is part of why this is interesting.

We want to build a category where for every object (species) X I have a special object degraded(X). And for every pair of objects (X,Y), I have another object complex(X,Y). And there are special homs that create and destroy these complexes. For example bind(X::Ob, Y::Ob)::Hom(otimes(X,Y),complex(X,Y)) creates the complex and degrade(X::Ob,Y::Ob)::Hom(complex(X,Y), otimes(X, degraded(Y))) converts the complex to the degraded form.

I thought this would work:

@signature BiproductCategory(Ob,Hom) => Chemistry(Ob,Hom) begin
    complex(X::Ob,Y::Ob)::Ob
    bind(X::Ob, Y::Ob)::Hom(otimes(X,Y),complex(X,Y))
    degraded(X::Ob)::Ob
    degrade(X::Ob,Y::Ob)::Hom(complex(X,Y), otimes(X, degraded(Y)))
    cleave(X::Ob, Y::Ob)::Hom(otimes(X,Y),otimes(X,degraded(Y)))
end

@syntax FreeChemistry(ObExpr,HomExpr) Chemistry begin
  otimes(A::Ob, B::Ob) = associate_unit(new(A,B), munit)
  otimes(f::Hom, g::Hom) = associate(new(f,g))
  compose(f::Hom, g::Hom) = associate(new(f,g; strict=true))

  pair(f::Hom, g::Hom) = compose(mcopy(dom(f)), otimes(f,g))
  copair(f::Hom, g::Hom) = compose(otimes(f,g), mmerge(codom(f)))
  proj1(A::Ob, B::Ob) = otimes(id(A), delete(B))
  proj2(A::Ob, B::Ob) = otimes(delete(A), id(B))
  incl1(A::Ob, B::Ob) = otimes(id(A), create(B))
  incl2(A::Ob, B::Ob) = otimes(create(A), id(B))
    
  # cleaving is not a fundamental operation so we normalize it out of the syntax
  cleave(A::Ob, B::Ob) = bind(A,B)⋅degrade(A,B)
end

I basically want this syntax to work just like a FreeBiproduct category but have the special constructors degraded(X), complex(X,Y), bind(X,Y), degrade(X,Y), and cleave(X,Y). What needs to happen to get that working for the following features of Catlab?

1. GATExprs
2. Presentantations
3. @parse_wiring_diagrams
4. to_wiring_diagram
5. x->to_graphviz(to_wiring_diagram(x))

Right now the first problem is

A,B = Ob.(FreeChemistry.Ob, [:A, :B])
r = Hom(:reaction, A,B)
to_wiring_diagram(r)

yields

MethodError: no method matching dom(::Main.FreeChemistry.Hom{:generator})
Closest candidates are:
  dom(!Matched::WiringLayer) at /Users/jfairbanks6/.julia/packages/Catlab/BhYS1/src/wiring_diagrams/Layers.jl:156
  dom(!Matched::Catlab.Programs.ExpressionTrees.Formulas) at /Users/jfairbanks6/.julia/packages/Catlab/BhYS1/src/programs/ExpressionTrees.jl:102
  dom(!Matched::Catlab.Doctrines.Category.Hom) at none:0
  ...

[epatters]

My guess is that you need to explicitly import the methods you are overriding:

import Catlab.Doctrines: dom, codom, compose, otimes, ...

This could (should?) be made part of the macro's syntactic sugar, but currently it is not.

Apart from that, you seem to be on the right track.

[jpfairbanks]

Thanks, that fixed a lot of it, but the wiring diagram code still doesn't know how to handle bind or degrade

to_wiring_diagram(bind(catK,catL))

MethodError: no method matching bind(::Ports{Main.Chemistry.Hom,Symbol}, ::Ports{Main.Chemistry.Hom,Symbol})

[jpfairbanks]

I don't know if this is "the right way" to solve this problem, but the diagrams show up.

degraded(p::Ports{Main.Chemistry.Hom,Symbol}) = begin
    p = Ports(map(p.ports) do x
        Symbol("$(x)ᵈᵉᵍ")
            end)
    return p
end
degraded(s::Symbol) = Symbol("$(s)ᵈᵉᵍ")
complex(p::Ports{Main.Chemistry.Hom,Symbol}, q::Ports{Main.Chemistry.Hom,Symbol}) = begin
    ps = prod(string.(p.ports))
    qs = prod(string.(q.ports))
    comp = ps*"⁺"*qs
    return Ports([Symbol(comp)])
end

complex(p::Symbol, q::Symbol) = begin
    return Symbol(string(p)*"⁺"*string(q))
end

degrade(p::Ports{Main.Chemistry.Hom,Symbol}, q::Ports{Main.Chemistry.Hom,Symbol}) = begin
    to_wiring_diagram(Hom(:degrade,
            Ob(FreeChemistry.Ob,
                complex(p.ports[1], q.ports[1])),
            Ob(FreeChemistry.Ob, degraded(q.ports[1]))))
end

bind(p::Ports{Main.Chemistry.Hom,Symbol}, q::Ports{Main.Chemistry.Hom,Symbol}) = begin
    to_wiring_diagram(Hom(:bind,
            Ob(FreeChemistry.Ob, p.ports[1])⊗Ob(FreeChemistry.Ob, q.ports[1]),
            Ob(FreeChemistry.Ob,
                complex(p.ports[1], q.ports[1])),
            ))
end

Is there a more elegant way to implement these functions?

[epatters]

I think so. One possibility is to retain the generator expressions and certain other expressions in the wiring diagrams. That way you don't have to introduce new types or perform hacky formatting logic.

For example, based on your current implementation, it looks like degraded should satisfy degraded(otimes(X,Y)) == otimes(degraded(X),degraded(Y)). So you could write:

degraded(p::Ports{Chemistry.Hom}) = Ports{Chemistry.Hom}(map(degraded, p.ports))

Then use the keep_exprs option when converting to wiring diagrams:

to_wiring_diagram(f, keep_exprs=true)

The result should be that you end up with ports of type Ports{Chemistry.Hom, FreeChemistry.Hom}. The same trick should work for the boxes.

[jpfairbanks]

Awesome! I’ll give that a try. I want elegant formatting instead of hacky formatting. Is there a way to tell Catlab what the latex expression is for an Ob/Hom?

[jpfairbanks]

It looks like the keep_exprs keyword argument is missing in v0.5.1. Was that an old API? I can’t find it with grep.

[epatters]

Oh sorry, coincidentally, I had added the keep_exprs option in a recent commit, after the v0.5.1 release. So it's only in master.

You can tell Catlab how to format expressions by adding a method for the generic functionSyntax.show_latex(io::IO, expr::GATExpr; kw...). E.g., you could dispatch on ObExpr{:degraded} or, if greater precision is desired, on FreeChemistry.Ob{:degraded}. There is also a Syntax.show_unicode function for Unicode formatting.

[epatters]

As noted in #86, keep_exprs has now been replaced by a more flexible API.

[jpfairbanks]

So I need to overload

import Syntax: show_latex
Syntax.show_latex(io::IO, expr::GATExpr; kw...) =   begin
  print(io, "\\mathop{\\mathrm{$(head(expr))}}")
  print(io, "\\left[")
  join(io, [sprint(show_latex, arg) for arg in args(expr)], ",")
  print(io, "\\right]")
end

What are the prefix, postfix, infix variants for?

[jpfairbanks]

This works for the jupyter notebook interface

Syntax.show_latex(io::IO, expr::ObExpr{:degraded}; kw...) =  begin
    if length(args(expr)) > 1
      print(io, "\\left[")
    end
    join(io, [sprint(show_latex, arg) for arg in args(expr)], ",")
    if length(args(expr)) > 1
        print(io, "\\right]")
    end
    print(io, "^{deg}")
    
end
Syntax.show_latex(io::IO, expr::ObExpr{:complex}; kw...) =  begin
    if length(args(expr)) > 1
      print(io, "\\left[")
    end
    join(io, [sprint(show_latex, arg) for arg in args(expr)], "")
    if length(args(expr)) > 1
        print(io, "\\right]")
    end
    
end

But that notation doesn't show up in the wiring diagram graphics. Do I need to overload something else?

[jpfairbanks]

Based on this code

""" Label for wire in wiring diagram.

Note: This function takes a port value, not a wire value.
"""
wire_label(value) = wire_label(MIME("text/plain"), value)
wire_label(mime::MIME, value) = diagram_element_label(mime, value)

diagram_element_label(::MIME, value) = string(value)
diagram_element_label(::MIME, ::Nothing) = ""

function diagram_element_label(::MIME"text/latex", expr::GATExpr)
  string("\$", sprint(show_latex, expr), "\$")
end

I would think this would just work. Does Graphviz use MIMEtype text/plain?

[jpfairbanks]

This got it working:

import Catlab.Graphics.WiringDiagramLayouts: diagram_element_label
function diagram_element_label(::MIME"text/plain", expr::ObExpr{:degraded})
    if length(args(expr)) > 1
        string("(", join(args(expr), ","),")", "ᵈ")
    else
        string(args(expr)[1], "ᵈ")
    end
end
function diagram_element_label(::MIME"text/plain", expr::ObExpr{:complex})
    if length(args(expr)) > 1
        string("(", join(args(expr), ""),")", )
    else
        string(args(expr)[1], "ᶜ")
    end
end

[epatters]

Right, Graphviz doesn't support LaTeX, so it has to use plain text.

[jpfairbanks]

Makes sense. Sorry for the near real time posting to gh issues. You are getting a livestream of my coding process.

[epatters]

No worries.

It might be worth adding a special MIME type for Graphviz, since Graphviz supports limited formatting (bold, italics, subscript, superscript) through its HTML-like labels.

[jpfairbanks]

It looks like going to a hom expr has revealed more steps to implementing a new syntax.

to_hom_expr(FreeChemistry, d)

 MethodError: no method matching id(::Main.FreeChemistry.Ob{:complex})
Closest candidates are:
  id(!Matched::NLayer) at /Users/jfairbanks6/.julia/dev/Catlab/src/wiring_diagrams/Layers.jl:160
  id(!Matched::Catlab.Programs.ExpressionTrees.NFormula) at /Users/jfairbanks6/.julia/dev/Catlab/src/programs/ExpressionTrees.jl:112
  id(!Matched::Catlab.Doctrines.Category.Ob) at none:0
  ...

[epatters]

My guess is that you need import Catlab.Doctrines: id.

[jpfairbanks]

always!

[jpfairbanks]

I think the number of things you need to import to use @signature, @syntax indicates that we should probably have the macros emit the imports

[epatters]

Probably so. It would seem to violate the spirit of Julia's module/namespace system, but I guess in the end convenience trumps everything :)

[jpfairbanks]

I think the idea of a macro creating a module violates the spirit of Julia. We already crossed that rubicon!

[epatters]

Fair enough. Once you're doing DSLs, all rules are out!

[epatters]

Going to close this for now. Feel free to reopen if necessary.