v12.0.0

Changelog

v12.0.0

Breaking Changes

• #2179 - @KronicDeth
  • Drop support for Elixir <= 1.6.
    Continuing support for Elixir <= 1.6 required special handling of the language level to support differences in precedence and operators. Removing the language level tracking allows dropping the Level and FilePropertyPusher classes and all their usages, including in the parser grammar and the special ifVersion external rule. It also eliminates the need for tests to setup the SDK since it was only needed to get the appropriate Level. This makes the tests run in 45 seconds instead of 7 minutes.
• #2339 - @KronicDeth
  • Remove code dependent on the Erlang plugin
    Erlang plugin is not 2021.3 compatible yet

Enhancements

• #2179 - @KronicDeth
  • Resolve more calls and bindings in Ecto.Query calls

    • Resolve bindings in Ecto.Query.lock/3

    • Resolve bindings in Ecto.Query.windows/3

    • Walk preload binding and expression

    • Resolve in update for Ecto.Query

    • Resolve fragment in with_cte

    • Resolve binding and expr in Ecto.Query.dynamic/1-2

    • Resolve field in join(..., on: field(..., ...) ...)
      For join/5, descend into the options to look for on: value and then walk value the same as the value to having or where in selects since they're all boolean conditions.

    • Extract ecto.query.Nested

    • Resolve Ecto.Query.WindowAPI functions

    • Resolve reference variable src in join(query, ..., [{src, counter}], ..., ...)
      Tuple lists in join have two forms:

      1. {^assoc, a}
      2. {src, counter}

      The pinned association form was already handled because the second element was checked for a declaration, but the first element was not, so src in (2) could not be resolved.

    • from([..] in ...)

    • Treat or_having the same as having

    • Treat or_where the same as where

    • Treat having: the same as where: in from

    • Treat select_merge the same as select for resolving Ecto.Query.API.

    • from(..., [elements])

    • Resolve Ecto reference variables in left in ...

  • ModuleWalker and NameArityRangeWalker
    Reduce code by abstracting common pattern for walking library APIs.
  • ExUnit
    • Find modules declared in tests.
    • Resolve call definitions inside describe blocks.
    • Resolve variables in assert_receive and assert_received.
    • Resolve alias to modules defined inside the enclosing describe block.
    • Walk assert expression for variable declarations
      • Check for earlier bindings of variables in right operand of = in assert.
  • Resolve require as: arguments as Aliases
  • Decompiler
    • Erlang
      • Decompile private Erlang functions
      • Decompile specs from Erlang DbgI
      • Decompile function bodies from Erlang DbgI
        • Escape "in" when an Erlang Var in type
      • Decompile types from Erlang DbgI
        Fixes #2017
    • Decompile Elixir function bodies using DbgI
      • :erlang./(a, b) -> a / b
      • :erlang.*(a, b) -> a * b
      • Convert :erlang.==(a,b) to a == b
      • Rewrite case to and when there is a badbool error too
      • Decompile %{struct: name, ...} as %name{...}
      • Rewrite more :erlang functions to Elixir
      • Rewrite case to ||
      • Rewrite case expr1 do pat1 -> true; _ -> false; end to match?(pat1, expr1)
      • Rewrite if var do false else true to !var
      • Rewrite case to or
      • Rewrite case to and
      • Rewrite :erlang.error(E.exception(M)) to raise E, M
      • Rewrite case statements to if
      • Rewrite case statements to &&
      • Indent all lines of spec macro string in case it is multiple @SPEC
  • Resolve module attributes defined outside the immediate modular lexical scope

    • Resolve module attributes registered in elixir_module.erl to decompiled source

      • after_compile
      • before_compile
      • behaviour
      • compile
      • derive
      • dialyzer
      • external_resource
      • on_definition

    • Index module attributes
      Use the index to resolve module attributes when it can't be found directly by tree walking.

      • Defined with Module.put_attribute/3
      • Defined with Module.register_attribute/3
      • Defined in quote blocks
  • Resolve variables to variables in any quote blocks
    If a variable can't be resolved in the scope, try resolving it to any variable declared at the top-level of a quote block. This helps with certain patterns of quote blocks used in Ecto where a variable's declaration and usage are not in the same quote block.
  • Simplify CallDefinitionClause resolver for modulars
    Due to the addition of skipping if the entrance is a child call, the check for only following siblings is no longer needed. Additionally, that check caused a bug because the ElixirStabBody skipped call definitions because they were new scopes.
  • Resolve functions declared with Mix.Generator.embed_template and embed_text.
    Also, new system for tracking resolves paths - imports, defdelegate, and use calls are added to the resolve results after the preferred elements are chosen for source in the same module. This prevents only the import showing because the actual declaration is in another module or the SDK.
  • Record a call as visited in its treeWalkUp instead of requiring the caller to do it in the pass ResolveState
    • Record quoteCall as visited in QuoteMacro.treeWalkUp
    • Record unquoteCall as visited in Unquote.treeWalkUp
    • Record unlessCall as visited in Unless.treeWalkUp
    • Record ifCall as visited in If.treeWalkUp
    • Record importCall as visited in Import.treeWalkUp
    • Record useCall as visited in Use.treeWalkUp
  • More macro specialized in Structure View
    • test
    • describe
  • Resolve Qualifer.unquote(variable)(...) to any definition with correct arity in Qualifier.
  • Only calculate element.resolvedFinalArity once in resolvedQualified
  • Implementations and Protocols
    • Redo icons
    • Implementations Go To Protocol line markers
    • Go to Super for calls to defimpl function/macro
      Goes to corresponding name/arity in the defprotocol that the defimpl implements.
    • Go to implementations line marker from defprotocol def
    • Go to implementations line marker from defprotocol
    • Go To Implementation from individual functions in defimpl
    • Go To Implementation from defimpl Alias
    • Resolve protocol function to def in defprotocol
    • Resolve defp inside of defimpl
      Process declarations inside of implementation the same as modules.
  • Stop prependQualifiers at top of file
  • Walk the false and true (else) branch of unless in Modules or Quote
  • Walk the true and false (else) branch of if in Modules or Quote
  • Port preferred and expand system from Callables to Modules.
  • Update CI build dependencies
• #2199 - @KronicDeth
  • Regression test for #2198.
• #2201 - @KronicDeth
  • Use callbacks as completions for calls.
• #2223 - @KronicDeth
  • Decompiler
    • Don't require MacroNameArity for accept, but use NameArity only because no decompiler cares about the macro.
  • Tests for Code.Identifier and String.Tokenizer
• #2226 - @KronicDeth
  • Structure View for EEx.function_from_(file|string)
  • Variants (completion) for functions declared by special macros.
    • Functions defined by EEx.function_from_(file|string)
    • exception/1 and message/1 defined by defexception
    • *_text/0 and *_template(assigns) functions defined by Mix.Generator.embed_text and Mix.Generator.embed_template.
• #2334 - @KronicDeth
  • Internal Tool for BEAM Bulk Decompilation
    Decompile all .beam files in the project, modules and SDKs to check for errors in the generated Elixir code
  • Decompiler
    • Erlang Abst
      • Log decompilation errors
  • Error Reports
    • Include system information in error reports
      Instead of just including the plugin version, also include the Application name, edition, and version; and the Operation System name and version as these are common follow-up questions I have.
    • Remove tab at start of location for title of issues
    • Don't include "java.lang.Throwable: " in title of issues
      The Throwable is necessary to get a stacktrace, but not a real error.
• #2339 - @KronicDeth
  • Build against 2021.3
  • runPluginVerifier in GitHub Actions
    • Update IDEA version range supports and verified
    • Fix reported compatibility warnings
      • Inline deprecated bundle messages
      • Don't bundle built-in markdown plugin, depend on it instead

Bug Fixes

• #2074 - @Thau
  • Alternative function clause for put_event with suite_finished
• #2179 - @KronicDeth
  • StackOverflow fixes

    • getElementDescription(ElixirAtom, ElementDescriptionLocation)
      Override getElementDescription for atoms to prevent StackOverflow while looking for a provider.

    • Don't descend into either branch of if or unless if entrance in either branch when resolving calls.
      If the definition were in one of the branch, it would already have been found on processing previous siblings in the ElixirStabBody.

    • Treat child of modulars as being at the same level if nested in if or unless
      Prevents test in if in supervisor_test.exs in ecto from stack overflowing.

    • Fix StackOverflow when looking for earlier bindings in parameters.

    • Don't check following siblings of modulars if entrance is a direct child

      Prevent StackOverflow when trying to resolve embed_template when more than one appears in the same module.

      In general, if the entrance is a child of modular then it can only be defined by a previous sibling, usually an import or use, but if the entrance is descendant of a child, then it child then it may be a call to a function or macro defined in the modular to following siblings of the entrance ancestor child needs to be checked if the entrance is a forward-call to a later declared function or macro.

    • Fix StackOverflowError in ifErlangRewriteTo
      Don't rewriter :erlang.* to a different :erlang.*

  • Adjust nameArityInterval in nameArityInAnyModule
    Ensures that fragment/1.. used in a quote can resolve to one in Ecto.Query.API.
  • Resolve variable that are the only child of quote
    Ecto loves doing quote do: query or other variable names in the code and tests, so record those as declarations to resolve as invalid results.
  • Find enclosing macro call when keyword do: is surrounded by parentheses
    Previously, only quote do: variable would work, but now quote(do: variable) also works to find the quote call.
  • Don't mark fields and keys that are not expected to be resolvable yet as unresolvable in Elixir References inspection.
    • QualifiedBracketOperation qualifier
    • StructOperation qualifier
    • Expect qualified unquote to only have invalid results.
    • Don't mark invalid only results for first chained function call.
      Don't report unquote(schema).__schema__(:source)
    • Don't mark invalid only results for chained function calls.
      Don't report Mix.shell().yes?("Are you sure you want to drop the database for repo #{inspect repo}?")
    • Don't mark invalid resolved function of call output
    • Don't mark invalid resolved keys or fields of call output
    • Don't mark invalid resolved function call on keys or fields
    • Don't mark parentheses calls on variables if the call does not resolve.
      Can't find exact valid resolves on variables yet.
    • Don't mark keys or fields on the output of a function call.
  • Fix some bugs with Ecto.Query calls.
    • Add missing state.put(Query.Call, call) for join/3-4 executeOnIn.
    • Walk the operands of |> in select expressions.
    • Resolve pinned variables as normal instead of as reference variables for Ecto.Query calls.
    • Don't walk keywords that cannot declare reference variables.
      • hints
      • lock
      • intersect
      • intersect_all
      • except
      • except_all
      • union
      • union_all
      • prefix
      • preload
      • offset
      • windows
      • limit
    • Don't treat signature for call definition as use of Ecto macro
  • Don't generate references to aliases, functions, or types that don't have declarations
    • assoc/2 in join: .. in assoc(_, _) in a no parentheses from call
    • var in type restrictions
      Related to elixir-ecto/ecto#3756
    • BitString
BitString is recognized in defimpl ..., for: BitString to define protocol implementations for <<..>>, but the BitString module itself does not exist, so can't be resolved.
  • Error reporting
    • Ignore at com.intellij.openapi.diagnostic.Logger when calculating location for error report titles
    • Improve error report format sent to GitHub
    • Fix the event message not being included, which meant that the excerpt wasn't included, so no reproducibility or element class was available.
    • Filter stacktrace to stop at last line from the plugin to limit their size and improve chance of URL being short enough for GitHub.
    • Don't include "What I was doing" section unless user actually fills in the additional information in the UI form.
      I'm sick of seeing the issue tracker full of "I don't know what I was doing", which is the default text when no additional info is given in the UI form.
    • Set title to the message at start of exception and first at that isn't from the errorreport.Logger instead of [auto-generated] as this is the pattern I follow when renaming manually.
  • Handle alias __MODULE__.{...} in prependQualifier
    Fixes #2153
  • Log error, but don't fail with TODO() for unknown strippedQualifier or null qualifier
    Fixes #2153
  • Go To Declaration for captures

    • Don't allow name to be acceptable named parent in &name/arity.
      Resolves #488
      Allows Go To Declaration on name and not just on /arity.

    • Don't allow Mod.name to be acceptable named parent in &Mod.name/arity.
      Resolves #488
      Fixes #2101

      Allows Go To Declaration on name and not just on /arity.

    • Resolve &name/arity and &Mod.name/arity using same code as callables.
      Fixes resolving &Mod.name.arity and ensures that special handling for weird definitions for callables also apply to captures.

  • Resolve __MODULE__ in quote to defmacro __MODULE__ in Kernel.SpecialForms
  • Performance

    • Fix String.Unicode decompiled being PlainText instead of Elixir
      String.Unicode when decompiled using all information from DbgI was 161,171 lines long, which made the JetBrains API treat it as plain text instead of Elixir. Being that long also made it freeze the UI while being decompiled.

      Now, don't even attempt to use the DbgI if the function has more than 10 clauses.

    • Don't decompile private macros and functions if > 500 definitions in one module.

    • If body cannot be decompiled, decompile as one-liner with ... body

    • Don't decompile Abst clause bodies that exceed 1024 bytes.

    • Decompile Erlang one clauses as Elixir one-liners

  • Fix resolving type specs

    • Find ancestorTypeSpec for qualified type used in parentheses in anonymous function type in an alternation

      @type run :: ((Ecto.Repo.t, changes) -> {:ok | :error, any}) | {module, atom, [any]}
      

    • Resolve type parameters used in inline anonymous function types

    • Resolve callback heads to themselves when they have type restrictions using when

    • Ignore literal parameters

      • Decimals
      • Aliases

    • Check left operand of \\ for type parameters as they could appear when copying def with defaults.

  • putInitialVisitedElement in variable.Variants
    Fixes #2002
  • Walk defdelegates when walking imports
    Fixes resolving config from use Mix.Config as it delegates to Config
  • Resolve variables used in match? guards to pattern declaration
    Resolves on_delete in match?(%{on_delete: on_delete} when on_delete != :nothing, reflection)
  • Implementations and Protocols
    • Fix calculating definition for stubs of defimpl with for:
      There was no clause for defimpl being arity 3, which is the case when there is the (1) protocol (2) for: and (3) do block. Not having a definition meant that the defimpl protocol, for: struct do would be in AllName index, but not ModularName.
    • Get name of enclosing modular for defimpl without for:
  • Store if stubs are guards
  • Decompiler
    • Surround case statements with parentheses when used in cond clause conditions
    • Convert OtpErlangString to OtpErlangList for tuple and call argument lists
    • Escape ESC character as \e
    • Handle Clause arguments being OtpErlangString
    • Handle tuple elements being an OtpErlangString
    • Add missing . after callee when it is a module or fn
    • Protect from Macro.toString(macro) StackOverflowError when decompiling body of function clauses
    • Don't print function names as atoms in captures
    • Escape \x to \\x in OtpErlangStr
    • Fix rewrite of :erlang calls
    • Surround type unions with parentheses
      Prevents parsing problems with unions in guard (when) clauses
    • Don't use prependIndent because it indents blank lines too.
      This doesn't match mix format or the IntelliJ Elixir formatting.
    • Erlang
      • Escape fn Erlang variable
      • Escape Erlang char \ as \
      • Don't append lines for clauses or after in Erlang receive when decompiling if empty.
      • Use Infix, Prefix, and Unquote decompolers for Erlang Abst chunk in addition to DbgI chunk
        • functions
        • typesepcs
      • Use function.macroNameArityMacro.macro when decompiling Erlang Abst clauses.
        Don't use def anymore when unexported and therefore private; use defp instead.
      • Remove space after ... in decompiled private types.
  • Process imports for calls
    Imports were previously only processed inside of Modules and not in general, which means that imports in the file were not processed, which is needed for association.ex in Ecto.
  • Classify ..// as OTHER instead of NOT_CALLABLE, so that it is escaped as a key.
  • Fix Macro.ifCaptureModuleNameArity
  • Resolve variable to parameter in %parameter{} patterns for struct names
  • Unquote.treeWalkUpUnquoted through tuples
  • Quote.treeWalkUp through case
  • Stop searching on numerical index in binding
  • Stop searching if atom in wrong place in binding
    Stops invalid binding test from erroring when resolving it.
  • Turn off tailrec because it doesn't work correctly for ElixirAccessExpression
  • Stop searching for qualifier when ElixirUnqualifiedNoParenthesesManyArgumentsCall.
• #2199 - @KronicDeth
  • Stop highlighting types when unquote_splicing/1 is reached.
    unquote_splicing is being used to splat arguments or fields of a struct into the type. The arguments to unquote_splicing are normal calls or variables, not types.
• #2201 - @KronicDeth
  • Implement call_definition_clause.Variants#executeOnCallback
• #2204 - @KronicDeth
  • CallDefinitionClause.time/1
    • Mark guards as runtime.
    • Mark anything unknown as runtime too.
    • Log unknown calls.
• #2207 - @KronicDeth
  • Check if Call isValid before using containingFile for locationString.
• #2208 - @KronicDeth
  • Check if project is not dumb in nameArityInAnyModule.
• #2209 - @KronicDeth
  • Take resolveInScope only if at least one valid
    Checking only for an empty collection allowed any prefixes in the scope to override exact matches in anywhere indexed, which meant that Ecto in defmodule Ecto.Adapter do resolved to itself instead of the exact defmodule Ecto do.
• #2214 - @KronicDeth
  • When regenerating the parser, ElixirVisitor is also regenerated. When it was regenerated it lost the bug fix for #visitLiteralSigileLine calling itself. Added a regression test, so that this can't happen again.
• #2223 - @KronicDeth
  • Ecto
    • Walk keyword keys as right operand of in in from
  • Resolving type references
    • Walk struct operations for type parameters
    • Check keyword values for type parameters
    • Check operands of two operations for type parameters
    • Stop looking for type parameters on qualified or unqualified alias
  • Decompiler
    • Only unquote in when an Erlang function, otherwise, use operators the same as Elixir for defs and calls.
    • Fix apply Erlang arguments, so that they are inside [].
    • Quote keyword keys containing -
      Fixes decompiling of Elixir.Phoenix.HTML.Tag.beam
    • Use apply with escaped atom when Erlang function call is an Elixir operator
  • Port String.Tokenizer.tokenize for use in Identifier.inspectAsKey
    I was putting off porting all of Identifer.inspectAsKey by adding special cases as needed, but the decompiler kept having bugs, so port all of it including String.Tokenizer.tokenize. It will also work for unicode characters now too.
  • Resolve calls that are unquoted values to search for quote blocks in those functions.
  • Stop looking for qualifiers to prepend when reaching =>
  • The parent argument to AccumulatorContinue.childExpressionsFoldWhile should be this and not parent
    When converting to an extension function I left parent in place because the argument is called parent, but since it is an extension function that value because this.parent when it really should have been this. Using this.parent meant it would ask for the parent's children and keep looping back to this.
  • Don't use tailrec in function with any body-recursion.
    It causes issues with ElixirAccessExpression recursion sometimes.
• #2226 - @KronicDeth
  • Implement completion for functions declared with defdelegate.
  • Fix LookupElementPresentation.putItemPresentation addTailText.
    Only append suffix of presentableText if it is prefixed by itemText.
• #2334 - @KronicDeth
  • Decompiler
    • Elixir
      • Decompile local function calls in Elixir DbgI using inspectAsFunction
        While remote calls used inspectAsFunction, local calls just used the atomValue, which meant names that needed to be unquoted weren't and caused parsing errors.
    • Erlang Abst
      • Decompile Erlang Abst string with OtpErlangList as strings with non-ASCII codepoints
        Fixes unknown string format in idna.beam
      • Always group for comprehensions in sequence even if there is only 1 element
        Some forms of for comprehensions cannot be used as the sole argument of a call unless surrounded by parentheses, so always add those parentheses.
      • Decompile Erlang Abst record empty record fields as [] for updates
      • Decompile Erlang Abst left xor right as :erlang.xor(left, right)
        Elixir does not have a logical xor infix operator, so have to decompile as normal function call
      • Decompile Erlang Abst named anonymous function as a macro
        Named anonymous functions are support in Erlang, but not Elixir, so fake it as a macro when decompiling.
      • Add builtin-types for Erlang Abst
        • bitstring
        • float
        • nonempty_improper_list
        • nonempty_maybe_improper_list
      • Decompile tagged atoms and other complex expression as function name in Abst capture
      • Decompile Erlang Abst float
      • Decompile Erlang Abst begin blocks as parenthesized groups separated by ;
      • Decompile empty OtpErlangList as "" in Erlang Abst string
      • Track if decompiled Erlang Abst contains do blocks so that they can be surrounded by parentheses when necessary
      • Fix decompiling Erlang Abst record_index when record name needs to be unquoted
      • Decompile map updates in Erlang Abst
      • Erlang Abst Function capture names are OtpErlangAtom and not tagged Atoms
      • Inspect local function names as atoms instead of as functions when apply/3 is used for operations and unquoted in Erlang Abst
        Stops :unquote(:"NAME") from happening
      • Surround anonymous function definitions that are called immediately with parentheses and call arguments with .( in Erlang Abst
      • Decompile field_type in Erlang Abst
        Fixes decompiling hipe_icode_call_elim.beam
      • Inspect type name usages as local functions to ensure invalid names are unquoted
      • Inspect type names as local functions to ensure invalid names are unquoted
  • References
    • Stop looking for qualifiers to prepend when exiting interpolation
    • Don't safeMultiResolve null call.reference in resolvesToModularName
  • Types
    • Fix highlighting types declared with unquote and no secondary parentheses
  • Performance
    • Don't error if a private function mirror cannot be found
      Private functions are not decompiled if there are too many public functions.
    • Fix CallDefinitionImpl.isExported
      Used to be hard-coded to return true, but this pre-dated decompiling private functions. Now with decompiling private functions, isExported needs to defer to the Definition and count as unexported if a private function, macro, or guard.
• #2337 - @KronicDeth
  • Walk map constructin arguments, associatons, and variables when resolving type parameters.
• #2339 - @KronicDeth
  • Don't use PluginId.findId that doesn't exist in 2021.1.X