Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

-ppx could be made working in toplevel #5904

Closed
vicuna opened this Issue Jan 23, 2013 · 8 comments

Comments

Projects
None yet
2 participants
@vicuna
Copy link
Collaborator

commented Jan 23, 2013

Original bug ID: 5904
Reporter: meyer
Assigned to: @alainfrisch
Status: closed (set by @xavierleroy on 2016-12-07T10:36:48Z)
Resolution: fixed
Priority: low
Severity: feature
Fixed in version: 4.02.1+dev
Category: ~DO NOT USE (was: OCaml general)
Monitored by: @hcarty

Bug description

I'd like to test my -ppx code in toplevel. Currently -ppx works just offline, using the file system.

This is a feature wish, and probably requiring some effort, nevertheless I think it's worthwhile.

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

commented Jan 25, 2013

Comment author: @alainfrisch

Commit 13278 (trunk) adds support for -ppx to the toplevel. Each phrase (not directives) is passed to external rewriters. Not that when the toplevel is used to execute a script (passed on its command-line, or with #use), then each phrase is rewritten independently. It might be worth sending the whole script to the rewriter instead. It would also allow the rewriter to rewrite directives.

Actually, it would make sense to allow the rewriters to maintain their state across phrases. This could be done by letting the rewriters send (and read back) some extra custom data in addition to the ASTs; this requires the rewriter to maintain a clean notion of state, which can be easily serialized. Otherwise, we could keep the rewriters alive as external processes and talk with them for each phrase. But this requires more complex process control logic (from module Unix).

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

commented Jan 27, 2013

Comment author: meyer

That sounds awesome, thanks.

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 28, 2013

Comment author: @gasche

Is this issue resolved?

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 29, 2013

Comment author: @alainfrisch

Support for -ppx has been added to the toplevel, but it can be improved (see my note : "Actually, it would make sense to allow the rewriters to maintain their state across phrases...").

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

commented May 13, 2014

Comment author: @alainfrisch

A #ppx directive has been added, to add dynamically extra ppx rewriters during a toplevel session.

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 23, 2014

Comment author: @alainfrisch

Commit 15314 adds a notion of "persistent cookies", which can be set by ppx processors and retrieved on further invocations when called from the toplevel.
Cookies are identified by an arbitrary string and hold an arbitrary Parsetree expression.

It is the responsibility of each ppx processor to store a cookie that represent its state and then restore it (using Ast_mapper.get_cookie/set_cookie). Note that cookies are available when the function registered through Ast_mapper.register is executed, so it is possible to set some global variables there.

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

commented Oct 12, 2014

Comment author: @gasche

Sorry for reacting a bit late, but the current API for ppx
"environment" exposed in ast_mapper seems rather strange to me.

(** {2 Helper functions to call external mappers} *)

val add_ppx_context_str: tool_name:string -> Parsetree.structure -> Parsetree.structure
(** Extract information from the current environment and encode it
into an attribute which is prepended to the list of structure
items in order to pass the information to an external
processor. *)

val add_ppx_context_sig: tool_name:string -> Parsetree.signature -> Parsetree.signature
(** Same as [ad_ppx_context_str] for signatures. *)

val drop_ppx_context_str: restore:bool -> Parsetree.structure -> Parsetree.structure
(** Drop the ocaml.ppx.context attribute from a structure. If
[restore] is true, also restore the associated data in the current
process. *)

val drop_ppx_context_sig: restore:bool -> Parsetree.signature -> Parsetree.signature
(** Same as [drop_ppx_context_str] for signatures. *)

(** {2 Cookies} *)

(** Cookies are used to pass information from a ppx processor to
a further invocation of itself, when called from the OCaml
toplevel (or other tools that support cookies). *)

val set_cookie: string -> Parsetree.expression -> unit
val get_cookie: string -> Parsetree.expression option

I find this API quite indirect.

Here is what I understand it does:

  1. I can query and update some "information from the current environment"
  2. I can store and change the way this information is serialized as part of an AST

Aspects (1) and (2) are rather orthogonal, but the API only provide
direct actions for (2). If I want to change the current environment to
match a serialized one, I should do that by setting a an
optional ?drop parameter to true.

I would rather suggest an interface along the following lines:

  • an explicit type for "ppx contexts" (this is what is implemented
    under the hood) along a description of what it actually stores

  • functions to get and update that part of the currently-running
    compiler state

  • functions to serialize and deserialize that state as an attribute
    (in a standard way)

  • optionally, convenience functions to read and update this attribute
    from structure and signature items, as exposed here

Similarly, I would be interested in an immutable type for cookies with
pure get/set operations; the current {get,set}_cookie would still be
present as acting on a compiler-internal global reference.

As a concrete example of use for such immutable data structures,
Merlin has reimplemented the compiler Clflags module on top of
a pervasive record of booleans (src/ocaml_aux/clflags.mli) -- this is
used to handle having multiple Emacs/vim buffers with distinct
configurations talking to the same merlin daemon. The compiler exposes
overly imperative for historical reasons, but we shouldn't repeat that
mistake in new APIs.

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

commented Oct 13, 2014

Comment author: @alainfrisch

Hi Gabriel,

The API for cookies (set/get_cookie) is intended to be used by ppx rewriters.

The context function are "less public", and currently mostly intended to be used by the compiler itself or closely related tools. But don't hesitate to send a patch with a better API!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.