v4.10.0-rc1

update RELEASES.md and CMakeLists.txt for release branch

v4.9.0

Language features, tactics, and metaprograms

• Definition transparency
  • #4053 adds the seal and unseal commands, which make definitions locally be irreducible or semireducible.
  • #4061 marks functions defined by well-founded recursion with @[irreducible] by default,
    which should prevent the expensive and often unfruitful unfolding of such definitions (see breaking changes below).
• Incrementality
  • #3940 extends incremental elaboration into various steps inside of declarations:
    definition headers, bodies, and tactics.
    .
  • 250994 and 67338b add @[incremental] attribute to mark an elaborator as supporting incremental elaboration.
  • #4259 improves resilience by ensuring incremental commands and tactics are reached only in supported ways.
  • #4268 adds special handling for := by so that stray tokens in tactic blocks do not inhibit incrementality.
  • #4308 adds incremental have tactic.
  • #4340 fixes incorrect info tree reuse.
  • #4364 adds incrementality for careful command macros such as set_option in theorem, theorem foo.bar, and lemma.
  • #4395 adds conservative fix for whitespace handling to avoid incremental reuse leading to goals in front of the text cursor being shown.
  • #4407 fixes non-incremental commands in macros blocking further incremental reporting.
  • #4436 fixes incremental reporting when there are nested tactics in terms.
• Functional induction
  • #4135 ensures that the names used for functional induction are reserved.
  • #4327 adds support for structural recursion on reflexive types.
    For example,

    inductive Many (α : Type u) where
      | none : Many α
      | more : α → (Unit → Many α) → Many α
    
    def Many.map {α β : Type u} (f : α → β) : Many α → Many β
      | .none => .none
      | .more x xs => .more (f x) (fun _ => (xs ()).map f)
    
    #check Many.map.induct
    /-
    Many.map.induct {α β : Type u} (f : α → β) (motive : Many α → Prop)
      (case1 : motive Many.none)
      (case2 : ∀ (x : α) (xs : Unit → Many α), motive (xs ()) → motive (Many.more x xs)) :
      ∀ (a : Many α), motive a
    -/
    

• #3903 makes the Lean frontend normalize all line endings to LF before processing.
  This lets Lean be insensitive to CRLF vs LF line endings, improving the cross-platform experience and making Lake hashes be faithful to what Lean processes.
• #4130 makes the tactic framework be able to recover from runtime errors (for example, deterministic timeouts or maximum recursion depth errors).
• split tactic
  • #4211 fixes split at h when h has forward dependencies.
  • #4349 allows split for if-expressions to work on non-propositional goals.
• apply tactic
  • #3929 makes error message for apply show implicit arguments in unification errors as needed.
    Modifies MessageData type (see breaking changes below).
• cases tactic
  • #4224 adds support for unification of offsets such as x + 20000 = 20001 in cases tactic.
• omega tactic
  • #4073 lets omega fall back to using classical Decidable instances when setting up contradiction proofs.
  • #4141 and #4184 fix bugs.
  • #4264 improves omega error message if no facts found in local context.
  • #4358 improves expression matching in omega by using match_expr.
• simp tactic

  • #4176 makes names of erased lemmas clickable.

  • #4208 adds a pretty printer for discrimination tree keys.

  • #4202 adds Simp.Config.index configuration option,
    which controls whether to use the full discrimination tree when selecting candidate simp lemmas.
    When index := false, only the head function is taken into account, like in Lean 3.
    This feature can help users diagnose tricky simp failures or issues in code from libraries
    developed using Lean 3 and then ported to Lean 4.

    In the following example, it will report that foo is a problematic theorem.

    opaque f : Nat → Nat → Nat
    
    @[simp] theorem foo : f x (x, y).2 = y := by sorry
    
    example : f a b ≤ b := by
      set_option diagnostics true in
      simp (config := { index := false })
    /-
    [simp] theorems with bad keys
      foo, key: f _ (@Prod.mk ℕ ℕ _ _).2
    -/

    With the information above, users can annotate theorems such as foo using no_index for problematic subterms. Example:

    opaque f : Nat → Nat → Nat
    
    @[simp] theorem foo : f x (no_index (x, y).2) = y := by sorry
    
    example : f a b ≤ b := by
      simp -- `foo` is still applied with `index := true`

  • #4274 prevents internal match equational theorems from appearing in simp trace.

  • #4177 and #4359 make simp continue even if a simp lemma does not elaborate, if the tactic state is in recovery mode.

  • #4341 fixes panic when applying @[simp] to malformed theorem syntax.

  • #4345 fixes simp so that it does not use the forward version of a user-specified backward theorem.

  • #4352 adds missing dsimp simplifications for fixed parameters of generated congruence theorems.

  • #4362 improves trace messages for simp so that constants are hoverable.

• Elaboration
  • #4046 makes subst notation (he ▸ h) try rewriting in both directions even when there is no expected type available.
  • #3328 adds support for identifiers in autoparams (for example, rfl in (h : x = y := by exact rfl)).
  • #4096 changes how the type in let and have is elaborated, requiring that any tactics in the type be evaluated before proceeding, improving performance.
  • #4215 ensures the expression tree elaborator commits to the computed "max type" for the entire arithmetic expression.
  • #4267 cases signature elaboration errors to show even if there are parse errors in the body.
  • #4368 improves error messages when numeric literals fail to synthesize an OfNat instance,
    including special messages warning when the expected type of the numeral can be a proposition.
• Metaprogramming
  • #4167 adds Lean.MVarId.revertAll to revert all free variables.
  • #4169 adds Lean.MVarId.ensureNoMVar to ensure the goal's target contains no expression metavariables.
  • #4180 adds cleanupAnnotations parameter to forallTelescope methods.
  • #4307 adds support for parser aliases in syntax quotations.
• Work toward implementing grind tactic
  • 0a515e and #4164 add grind_norm and grind_norm_proc attributes and @[grind_norm] theorems.
  • #4170, #4221, and #4249 create grind preprocessor and core module.
  • #4235 and d6709e add special cases tactic to grind along with @[grind_cases] attribute to mark types that this cases tactic should automatically apply to.
  • #4243 adds special injection? tactic to grind.
• Other fixes or improvements
  • #4065 fixes a bug in the Nat.reduceLeDiff simproc.
  • #3969 makes deprecation warnings activate even for general...

v4.9.0-rc3

chore: mark releases as prerelease

v4.9.0-rc2

feat: lake: reliably cache logs and hashes (#4402)

Moves the cached log into the trace file (no more `.log.json`). This
means logs are no longer cached on fatal errors and this ensures that an
out-of-date log is not associated with an up-to-date trace. Separately,
`.hash` file generation was changed to be more reliable as well. `.hash`
files are deleted as part of the build and always regenerate with
`--rehash`.

Closes #2751.

(cherry picked from commit db74ee9e83c631dd4bab9b7b6a10d0793cb0cbef)

v4.9.0-rc1

chore: set LEAN_VERSION_IS_RELEASE to 1

v4.8.0

Language features, tactics, and metaprograms

• Functional induction principles.
  #3432, #3620, #3754, #3762, #3738, #3776, #3898.

  Derived from the definition of a (possibly mutually) recursive function,
  a functional induction principle is created that is tailored to proofs about that function.

  For example from:

  def ackermann : Nat → Nat → Nat
    | 0, m => m + 1
    | n+1, 0 => ackermann n 1
    | n+1, m+1 => ackermann n (ackermann (n + 1) m)
  

  we get

  ackermann.induct (motive : Nat → Nat → Prop) (case1 : ∀ (m : Nat), motive 0 m)
    (case2 : ∀ (n : Nat), motive n 1 → motive (Nat.succ n) 0)
    (case3 : ∀ (n m : Nat), motive (n + 1) m → motive n (ackermann (n + 1) m) → motive (Nat.succ n) (Nat.succ m))
    (x x : Nat) : motive x x
  

  It can be used in the induction tactic using the using syntax:

  induction n, m using ackermann.induct
  

• The termination checker now recognizes more recursion patterns without an
  explicit termination_by. In particular the idiom of counting up to an upper
  bound, as in

  def Array.sum (arr : Array Nat) (i acc : Nat) : Nat :=
    if _ : i < arr.size then
      Array.sum arr (i+1) (acc + arr[i])
    else
      acc
  

  is recognized without having to say termination_by arr.size - i.

  • #3630 makes termination_by? not use sizeOf when not needed
  • #3652 improves the termination_by syntax.
  • #3658 changes how termination arguments are elaborated.
  • #3665 refactors GuessLex to allow inferring more complex termination arguments
  • #3666 infers termination arguments such as xs.size - i

• #3629, #3655, #3747:
  Adds @[induction_eliminator] and @[cases_eliminator] attributes to be able to define custom eliminators
  for the induction and cases tactics, replacing the @[eliminator] attribute.
  Gives custom eliminators for Nat so that induction and cases put goal states into terms of 0 and n + 1
  rather than Nat.zero and Nat.succ n.
  Added option tactic.customEliminators to control whether to use custom eliminators.
  Added a hack for rcases/rintro/obtain to use the custom eliminator for Nat.

• Shorter instances names. There is a new algorithm for generating names for anonymous instances.
  Across Std and Mathlib, the median ratio between lengths of new names and of old names is about 72%.
  With the old algorithm, the longest name was 1660 characters, and now the longest name is 202 characters.
  The new algorithm's 95th percentile name length is 67 characters, versus 278 for the old algorithm.
  While the new algorithm produces names that are 1.2% less unique,
  it avoids cross-project collisions by adding a module-based suffix
  when it does not refer to declarations from the same "project" (modules that share the same root). #3089 and #3934.

• 8d2adf Importing two different files containing proofs of the same theorem is no longer considered an error.
  This feature is particularly useful for theorems that are automatically generated on demand (e.g., equational theorems).

• 84b091 Lean now generates an error if the type of a theorem is not a proposition.

• Definition transparency. 47a343. @[reducible], @[semireducible], and @[irreducible] are now scoped and able to be set for imported declarations.

• simp/dsimp

  • #3607 enables kernel projection reduction in dsimp
  • b24fbf and acdb00: dsimproc command to define defeq-preserving simplification procedures.
  • #3624 makes dsimp normalize raw nat literals as OfNat.ofNat applications.
  • #3628 makes simp correctly handle OfScientific.ofScientific literals.
  • #3654 makes dsimp? report used simprocs.
  • dee074 fixes equation theorem handling in simp for non-recursive definitions.
  • #3819 improved performance when simp encounters a loop.
  • #3821 fixes discharger/cache interaction.
  • #3824 keeps simp from breaking Char literals.
  • #3838 allows Nat instances matching to be more lenient.
  • #3870 documentation for simp configuration options.
  • #3972 fixes simp caching.
  • #4044 improves cache behavior for "well-behaved" dischargers.

• omega

  • #3639, #3766, #3853, #3875: introduces a term canonicalizer.
  • #3736 improves handling of positivity for the modulo operator for Int.
  • #3828 makes it work as a simp discharger.
  • #3847 adds helpful error messages.

• rfl

  • #3671, #3708: upstreams the @[refl] attribute and the rfl tactic.
  • #3751 makes apply_rfl not operate on Eq itself.
  • #4067 improves error message when there are no goals.

• #3719 upstreams the rw? tactic, with fixes and improvements in #3783, #3794, #3911.

• conv

  • #3659 adds a conv version of the calc tactic.
  • #3763 makes conv clean up using try with_reducible rfl instead of try rfl.

• #guard_msgs

  • #3617 introduces whitespace protection using the ⏎ character.
  • #3883: The #guard_msgs command now has options to change whitespace normalization and sensitivity to message ordering.
    For example, #guard_msgs (whitespace := lax) in cmd collapses whitespace before checking messages,
    and #guard_msgs (ordering := sorted) in cmd sorts the messages in lexicographic order before checking.
  • #3931 adds an unused variables ignore function for #guard_msgs.
  • #3912 adds a diff between the expected and actual outputs. This feature is currently
    disabled by default, but can be enabled with set_option guard_msgs.diff true.
    Depending on user feedback, this option may default to true in a future version of Lean.

• do notation

  • #3820 makes it an error to lift (<- ...) out of a pure if ... then ... else ...

• Lazy discrimination trees

  • #3610 fixes a name collision for LazyDiscrTree that could lead to cache poisoning.
  • #3677 simplifies and fixes LazyDiscrTree handling for exact?/apply?.
  • #3685 moves general exact?/apply? functionality into LazyDiscrTree.
  • #3769 has lemma selection improvements for rw? and LazyDiscrTree.
  • #3818 improves ordering of matches.

• #3590 adds inductive.autoPromoteIndices option to be able to disable auto promotion of indices in the inductive command.

• Miscellaneous bug fixes and improvements

  • #3606 preserves cache and dischargeDepth fields in Lean.Meta.Simp.Result.mkEqSymm.
  • #3633 makes elabTermEnsuringType respect errToSorry, improving error recovery of the have tactic.
  • [#3647](https://github.com/leanprover/lean4/pull/...

v4.8.0-rc2

Update to v4.8.0-rc1 with improvements to lake's build monitor.

v4.8.0-rc1

chore: set release flag

v4.7.0

Changes since v4.6.0 (from RELEASES.md)

• simp and rw now use instance arguments found by unification,
  rather than always resynthesizing. For backwards compatibility, the original behaviour is
  available via set_option tactic.skipAssignedInstances false.
  #3507 and
  #3509.

• When the pp.proofs is false, now omitted proofs use ⋯ rather than _,
  which gives a more helpful error message when copied from the Infoview.
  The pp.proofs.threshold option lets small proofs always be pretty printed.
  #3241.

• pp.proofs.withType is now set to false by default to reduce noise in the info view.

• The pretty printer for applications now handles the case of over-application itself when applying app unexpanders.
  In particular, the | `($_ $a $b $xs*) => `(($a + $b) $xs*) case of an app_unexpander is no longer necessary.
  #3495.

• New simp (and dsimp) configuration option: zetaDelta. It is false by default.
  The zeta option is still true by default, but their meaning has changed.

  • When zeta := true, simp and dsimp reduce terms of the form
    let x := val; e[x] into e[val].
  • When zetaDelta := true, simp and dsimp will expand let-variables in
    the context. For example, suppose the context contains x := val. Then,
    any occurrence of x is replaced with val.

  See issue #2682 for additional details. Here are some examples:

  example (h : z = 9) : let x := 5; let y := 4; x + y = z := by
    intro x
    simp
    /-
    New goal:
    h : z = 9; x := 5 |- x + 4 = z
    -/
    rw [h]
  
  example (h : z = 9) : let x := 5; let y := 4; x + y = z := by
    intro x
    -- Using both `zeta` and `zetaDelta`.
    simp (config := { zetaDelta := true })
    /-
    New goal:
    h : z = 9; x := 5 |- 9 = z
    -/
    rw [h]
  
  example (h : z = 9) : let x := 5; let y := 4; x + y = z := by
    intro x
    simp [x] -- asks `simp` to unfold `x`
    /-
    New goal:
    h : z = 9; x := 5 |- 9 = z
    -/
    rw [h]
  
  example (h : z = 9) : let x := 5; let y := 4; x + y = z := by
    intro x
    simp (config := { zetaDelta := true, zeta := false })
    /-
    New goal:
    h : z = 9; x := 5 |- let y := 4; 5 + y = z
    -/
    rw [h]
  

• When adding new local theorems to simp, the system assumes that the function application arguments
  have been annotated with no_index. This modification, which addresses issue #2670,
  restores the Lean 3 behavior that users expect. With this modification, the following examples are now operational:

  example {α β : Type} {f : α × β → β → β} (h : ∀ p : α × β, f p p.2 = p.2)
    (a : α) (b : β) : f (a, b) b = b := by
    simp [h]
  
  example {α β : Type} {f : α × β → β → β}
    (a : α) (b : β) (h : f (a,b) (a,b).2 = (a,b).2) : f (a, b) b = b := by
    simp [h]
  

  In both cases, h is applicable because simp does not index f-arguments anymore when adding h to the simp-set.
  It's important to note, however, that global theorems continue to be indexed in the usual manner.

• Improved the error messages produced by the decide tactic. #3422

• Improved auto-completion performance. #3460

• Improved initial language server startup performance. #3552

• Changed call hierarchy to sort entries and strip private header from names displayed in the call hierarchy. #3482

• There is now a low-level error recovery combinator in the parsing framework, primarily intended for DSLs. #3413

• You can now write termination_by? after a declaration to see the automatically inferred
  termination argument, and turn it into a termination_by … clause using the “Try this” widget or a code action. #3514

• A large fraction of Std has been moved into the Lean repository.
  This was motivated by:

  1. Making universally useful tactics such as ext, by_cases, change at,
     norm_cast, rcases, simpa, simp?, omega, and exact?
     available to all users of Lean, without imports.
  2. Minimizing the syntactic changes between plain Lean and Lean with import Std.
  3. Simplifying the development process for the basic data types
     Nat, Int, Fin (and variants such as UInt64), List, Array,
     and BitVec as we begin making the APIs and simp normal forms for these types
     more complete and consistent.
  4. Laying the groundwork for the Std roadmap, as a library focused on
     essential datatypes not provided by the core langauge (e.g. RBMap)
     and utilities such as basic IO.
     While we have achieved most of our initial aims in v4.7.0-rc1,
     some upstreaming will continue over the coming months.

• The / and % notations in Int now use Int.ediv and Int.emod
  (i.e. the rounding conventions have changed).
  Previously Std overrode these notations, so this is no change for users of Std.
  There is now kernel support for these functions.
  #3376.

• omega, our integer linear arithmetic tactic, is now availabe in the core langauge.

  • It is supplemented by a preprocessing tactic bv_omega which can solve goals about BitVec
    which naturally translate into linear arithmetic problems.
    #3435.
  • omega now has support for Fin #3427,
    the <<< operator #3433.
  • During the port omega was modified to no longer identify atoms up to definitional equality
    (so in particular it can no longer prove id x ≤ x). #3525.
    This may cause some regressions.
    We plan to provide a general purpose preprocessing tactic later, or an omega! mode.
  • omega is now invoked in Lean's automation for termination proofs
    #3503 as well as in
    array indexing proofs #3515.
    This automation will be substantially revised in the medium term,
    and while omega does help automate some proofs, we plan to make this much more robust.

• The library search tactics exact? and apply? that were originally in
  Mathlib are now available in Lean itself. These use the implementation using
  lazy discrimination trees from Std, and thus do not require a disk cache but
  have a slightly longer startup time. The order used for selection lemmas has
  changed as well to favor goals purely based on how many terms in the head
  pattern match the current goal.

• The solve_by_elim tactic has been ported from Std to Lean so that library
  search can use it.

• New #check_tactic and #check_simp commands have been added. These are
  useful for checking tactics (particularly simp) behave as expected in test
  suites.

• Previously, app unexpanders would only be applied to entire applications. However, some notations produce
  functions, and these functions can be given additional arguments. The solution so far has been to write app unexpanders so that they can take an arbitrary number of additional arguments. However this leads to misleading hover information in the Infoview. For example, while HAdd.hAdd f g 1 pretty prints as (f + g) 1, hovering over f + g shows f. There is no way to fix the situation from within an app unexpander; the expression position for HAdd.hAdd f g is absent, and app unexpanders cannot register TermInfo.

  This commit changes the app delaborator to try running app unexpanders on every prefix of an application, from longest to shortest prefix. For efficiency, it is careful to only try this when app delaborators do in fact exist for the head constant, and it also ensures arguments are only delaborated once. Then, in (f + g) 1, the f + g gets TermInfo registered for that subexpression, making it properly hoverable.

  #3375

Breaking changes:

• Lean.withTraceNode and variants got a stronger MonadAlwaysExcept assumption to
  fix trace trees not being built on elaboration runtime exceptions. Instances for most elaboration
  monads built on EIO Exception should be synthesized automatically.
• The match ... with. and fun. notations previously in Std have been replaced by
  nomatch ... and nofun. #3279 and #3286

Other improvements:

• several bug fixes for simp:
  • we should not crash when simp loops #3269
  • simp gets stuck on autoParam #3315
  • simp fails when custom discharger makes no progress #3317
  • simp fails to discharge autoParam premises even when it can reduce them to True #3314
  • simp? suggests generated equations lemma names, fixes #3547 #3573
• fixes for match expressions:
  • fix regression with builtin literals #3521...

v4.7.0-rc2

Identical to v4.7.0-rc1, but with the backport of #3610, fixing a bug when exact? is used in Mathlib.