Auto merge of #139112 - m-ou-se:super-let, r=<try>

Implement `super let`

Tracking issue: #139076

This implements `super let` as proposed in #139080, based on the following two equivalence rules.

1. For all expressions `$expr` in any context, these are equivalent:
  - `& $expr`
  - `{ super let a = & $expr; a }`

2. And, additionally, these are equivalent in any context when `$expr` is a temporary (aka rvalue):
  - `& $expr`
  - `{ super let a = $expr; & a }`

So far, this experiment has a few interesting results:

## Interesting result 1

In this snippet:

```rust
super let a = f(&temp());
```

I originally expected temporary `temp()` would be dropped at the end of the statement (`;`), just like in a regular `let`, because `temp()` is not subject to temporary lifetime extension.

However, it turns out that that would break the fundamental equivalence rules.

For example, in

```rust
g(&f(&temp()));
```

the temporary `temp()` will be dropped at the `;`.

The first equivalence rule tells us this must be equivalent:

```rust
g({ super let a = &f(&temp()); a });
```

But that means that `temp()` must live until the last `;` (after `g()`), not just the first `;` (after `f()`).

While this was somewhat surprising to me at first, it does match the exact behavior we need for `pin!()`: The following _should work_. (See also #138718)

```rust
g(pin!(f(&mut temp())));
```

Here, `temp()` lives until the end of the statement. This makes sense from the perspective of the user, as no other `;` or `{}` are visible. Whether `pin!()` uses a `{}` block internally or not should be irrelevant.

This means that _nothing_ in a `super let` statement will be dropped at the end of that super let statement. It does not even need its own scope.

This raises questions that are useful for later on:

- Will this make temporaries live _too long_ in cases where `super let` is used not in a hidden block in a macro, but as a visible statement in code like the following?

    ```rust
    let writer = {
        super let file = File::create(&format!("/home/{user}/test"));
        Writer::new(&file)
    };
    ```

- Is a `let` statement in a block still the right syntax for this? Considering it has _no_ scope of its own, maybe neither a block nor a statement should be involved

This leads me to think that instead of `{ super let $pat = $init; $expr }`, we might want to consider something like `let $pat = $init in $expr` or `$expr where $pat = $init`. Although there are also issues with these, as it isn't obvious anymore if `$init` should be subject to temporary lifetime extension. (Do we want both `let _ = _ in ..` and `super let _ = _ in ..`?)

## Interesting result 2

What about `super let x;` without initializer?

```rust
let a = {
    super let x;
    x = temp();
    &x
};
```

This works fine with the implementation in this PR: `x` is extended to live as long as `a`.

While it matches my expectations, a somewhat interesting thing to realize is that these are _not_ equivalent:

- `super let x = $expr;`
- `super let x; x = $expr;`

In the first case, all temporaries in $expr will live at least as long as (the result of) the surrounding block.
In the second case, temporaries will be dropped at the end of the assignment statement. (Because the assignment statement itself "is not `super`".)

This difference in behavior might be confusing, but it _might_ be useful.
One might want to extend the lifetime of a variable without extending all the temporaries in the initializer expression.

On the other hand, that can also be expressed as:

- `let x = $expr; super let x = x;` (w/o temporary lifetime extension), or
- `super let x = { $expr };` (w/ temporary lifetime extension)

So, this raises these questions:

- Do we want to accept `super let x;` without initializer at all?

- Does it make sense for statements other than let statements to be "super"? An expression statement also drops temporaries at its `;`, so now that we discovered that `super let` basically disables that `;` (see interesting result 1), is there a use to having other statements without their own scope? (I don't think that's ever useful?)

## Interesting result 3

This works now:

```rust
super let Some(x) = a.get(i) else { return };
```

I didn't put in any special cases for `super let else`. This is just the behavior that 'naturally' falls out when implementing `super let` without thinking of the `let else` case.

- Should `super let else` work?

## Interesting result 4

This 'works':

```rust
fn main() {
    super let a = 123;
}
```

I didn't put in any special cases for `super let` at function scope. I had expected the code to cause an ICE or other weird failure when used at function body scope, because there's no way to let the variable live as long as the result of the function.

This raises the question:

- Does this mean that this behavior is the natural/expected behavior when `super let` is used at function scope? Or is this just a quirk and should we explicitly disallow `super let` in a function body? (Probably the latter.)

---

The questions above do not need an answer to land this PR. These questions should be considered when redesigning/rfc'ing/stabilizing the feature.

Author: bors

compiler/rustc_ast/src/ast.rs

@@ -1169,6 +1169,7 @@ pub enum MacStmtStyle {
 #[derive(Clone, Encodable, Decodable, Debug)]
 pub struct Local {
     pub id: NodeId,
+    pub super_: Option<Span>,
     pub pat: P<Pat>,
     pub ty: Option<P<Ty>>,
     pub kind: LocalKind,
@@ -3926,7 +3927,7 @@ mod size_asserts {
     static_assert_size!(Item, 144);
     static_assert_size!(ItemKind, 80);
     static_assert_size!(LitKind, 24);
-    static_assert_size!(Local, 80);
+    static_assert_size!(Local, 96);
     static_assert_size!(MetaItemLit, 40);
     static_assert_size!(Param, 40);
     static_assert_size!(Pat, 72);

compiler/rustc_ast/src/mut_visit.rs

@@ -704,7 +704,8 @@ fn walk_parenthesized_parameter_data<T: MutVisitor>(vis: &mut T, args: &mut Pare
 }
 
 fn walk_local<T: MutVisitor>(vis: &mut T, local: &mut P<Local>) {
-    let Local { id, pat, ty, kind, span, colon_sp, attrs, tokens } = local.deref_mut();
+    let Local { id, super_, pat, ty, kind, span, colon_sp, attrs, tokens } = local.deref_mut();
+    visit_opt(super_, |sp| vis.visit_span(sp));
     vis.visit_id(id);
     visit_attrs(vis, attrs);
     vis.visit_pat(pat);

compiler/rustc_ast/src/visit.rs

@@ -323,7 +323,7 @@ pub fn walk_crate<'a, V: Visitor<'a>>(visitor: &mut V, krate: &'a Crate) -> V::R
 }
 
 pub fn walk_local<'a, V: Visitor<'a>>(visitor: &mut V, local: &'a Local) -> V::Result {
-    let Local { id: _, pat, ty, kind, span: _, colon_sp: _, attrs, tokens: _ } = local;
+    let Local { id: _, super_: _, pat, ty, kind, span: _, colon_sp: _, attrs, tokens: _ } = local;
     walk_list!(visitor, visit_attribute, attrs);
     try_visit!(visitor.visit_pat(pat));
     visit_opt!(visitor, visit_ty, ty);

compiler/rustc_ast_lowering/src/block.rs

@@ -95,6 +95,7 @@ impl<'a, 'hir> LoweringContext<'a, 'hir> {
 
     fn lower_local(&mut self, l: &Local) -> &'hir hir::LetStmt<'hir> {
         // Let statements are allowed to have impl trait in bindings.
+        let super_ = l.super_;
         let ty = l.ty.as_ref().map(|t| {
             self.lower_ty(t, self.impl_trait_in_bindings_ctxt(ImplTraitPosition::Variable))
         });
@@ -109,7 +110,7 @@ impl<'a, 'hir> LoweringContext<'a, 'hir> {
         let span = self.lower_span(l.span);
         let source = hir::LocalSource::Normal;
         self.lower_attrs(hir_id, &l.attrs, l.span);
-        self.arena.alloc(hir::LetStmt { hir_id, ty, pat, init, els, span, source })
+        self.arena.alloc(hir::LetStmt { hir_id, super_, ty, pat, init, els, span, source })
     }
 
     fn lower_block_check_mode(&mut self, b: &BlockCheckMode) -> hir::BlockCheckMode {

compiler/rustc_ast_lowering/src/lib.rs

@@ -2223,6 +2223,7 @@ impl<'a, 'hir> LoweringContext<'a, 'hir> {
             self.attrs.insert(hir_id.local_id, a);
         }
         let local = hir::LetStmt {
+            super_: None,
             hir_id,
             init,
             pat,

compiler/rustc_ast_pretty/src/pprust/state.rs

@@ -1336,6 +1336,9 @@ impl<'a> State<'a> {
                 self.print_outer_attributes(&loc.attrs);
                 self.space_if_not_bol();
                 self.ibox(INDENT_UNIT);
+                if loc.super_.is_some() {
+                    self.word_nbsp("super");
+                }
                 self.word_nbsp("let");
 
                 self.ibox(INDENT_UNIT);

compiler/rustc_expand/src/build.rs

@@ -230,6 +230,7 @@ impl<'a> ExtCtxt<'a> {
             self.pat_ident(sp, ident)
         };
         let local = P(ast::Local {
+            super_: None,
             pat,
             ty,
             id: ast::DUMMY_NODE_ID,
@@ -245,6 +246,7 @@ impl<'a> ExtCtxt<'a> {
     /// Generates `let _: Type;`, which is usually used for type assertions.
     pub fn stmt_let_type_only(&self, span: Span, ty: P<ast::Ty>) -> ast::Stmt {
         let local = P(ast::Local {
+            super_: None,
             pat: self.pat_wild(span),
             ty: Some(ty),
             id: ast::DUMMY_NODE_ID,

compiler/rustc_feature/src/unstable.rs

@@ -630,7 +630,7 @@ declare_features! (
     /// Allows string patterns to dereference values to match them.
     (unstable, string_deref_patterns, "1.67.0", Some(87121)),
     /// Allows `super let` statements.
-    (incomplete, super_let, "CURRENT_RUSTC_VERSION", Some(139076)),
+    (unstable, super_let, "CURRENT_RUSTC_VERSION", Some(139076)),
     /// Allows subtrait items to shadow supertrait items.
     (unstable, supertrait_item_shadowing, "1.86.0", Some(89151)),
     /// Allows using `#[thread_local]` on `static` items.

compiler/rustc_hir/src/hir.rs

@@ -1817,6 +1817,8 @@ pub enum StmtKind<'hir> {
 /// Represents a `let` statement (i.e., `let <pat>:<ty> = <init>;`).
 #[derive(Debug, Clone, Copy, HashStable_Generic)]
 pub struct LetStmt<'hir> {
+    /// Span of `super` in `super let`.
+    pub super_: Option<Span>,
     pub pat: &'hir Pat<'hir>,
     /// Type annotation, if any (otherwise the type will be inferred).
     pub ty: Option<&'hir Ty<'hir>>,
@@ -4850,7 +4852,7 @@ mod size_asserts {
     static_assert_size!(ImplItemKind<'_>, 40);
     static_assert_size!(Item<'_>, 88);
     static_assert_size!(ItemKind<'_>, 64);
-    static_assert_size!(LetStmt<'_>, 64);
+    static_assert_size!(LetStmt<'_>, 72);
     static_assert_size!(Param<'_>, 32);
     static_assert_size!(Pat<'_>, 72);
     static_assert_size!(Path<'_>, 40);

compiler/rustc_hir_analysis/src/check/region.rs

@@ -8,6 +8,7 @@
 
 use std::mem;
 
+use rustc_data_structures::fx::FxHashMap;
 use rustc_hir as hir;
 use rustc_hir::def_id::DefId;
 use rustc_hir::intravisit::{self, Visitor};
@@ -44,6 +45,8 @@ struct ScopeResolutionVisitor<'tcx> {
     scope_tree: ScopeTree,
 
     cx: Context,
+
+    extended_super_lets: FxHashMap<hir::ItemLocalId, Option<Scope>>,
 }
 
 /// Records the lifetime of a local variable as `cx.var_parent`
@@ -214,18 +217,29 @@ fn resolve_stmt<'tcx>(visitor: &mut ScopeResolutionVisitor<'tcx>, stmt: &'tcx hi
     let stmt_id = stmt.hir_id.local_id;
     debug!("resolve_stmt(stmt.id={:?})", stmt_id);
 
-    // Every statement will clean up the temporaries created during
-    // execution of that statement. Therefore each statement has an
-    // associated destruction scope that represents the scope of the
-    // statement plus its destructors, and thus the scope for which
-    // regions referenced by the destructors need to survive.
+    if let hir::StmtKind::Let(LetStmt { super_: Some(_), .. }) = stmt.kind {
+        // `super let` statement does not start a new scope, such that
+        //
+        //     { super let x = identity(&temp()); &x }.method();
+        //
+        // behaves exactly as
+        //
+        //     (&identity(&temp()).method();
+        intravisit::walk_stmt(visitor, stmt);
+    } else {
+        // Every statement will clean up the temporaries created during
+        // execution of that statement. Therefore each statement has an
+        // associated destruction scope that represents the scope of the
+        // statement plus its destructors, and thus the scope for which
+        // regions referenced by the destructors need to survive.
 
-    let prev_parent = visitor.cx.parent;
-    visitor.enter_node_scope_with_dtor(stmt_id, true);
+        let prev_parent = visitor.cx.parent;
+        visitor.enter_node_scope_with_dtor(stmt_id, true);
 
-    intravisit::walk_stmt(visitor, stmt);
+        intravisit::walk_stmt(visitor, stmt);
 
-    visitor.cx.parent = prev_parent;
+        visitor.cx.parent = prev_parent;
+    }
 }
 
 fn resolve_expr<'tcx>(
@@ -481,14 +495,19 @@ fn resolve_expr<'tcx>(
     visitor.cx = prev_cx;
 }
 
+#[derive(Copy, Clone, PartialEq, Eq, Debug)]
+enum LetKind {
+    Regular,
+    Super,
+}
+
 fn resolve_local<'tcx>(
     visitor: &mut ScopeResolutionVisitor<'tcx>,
     pat: Option<&'tcx hir::Pat<'tcx>>,
     init: Option<&'tcx hir::Expr<'tcx>>,
+    let_kind: LetKind,
 ) {
-    debug!("resolve_local(pat={:?}, init={:?})", pat, init);
-
-    let blk_scope = visitor.cx.var_parent;
+    debug!("resolve_local(pat={:?}, init={:?}, let_kind={:?})", pat, init, let_kind);
 
     // As an exception to the normal rules governing temporary
     // lifetimes, initializers in a let have a temporary lifetime
@@ -546,14 +565,50 @@ fn resolve_local<'tcx>(
     // A, but the inner rvalues `a()` and `b()` have an extended lifetime
     // due to rule C.
 
+    if let_kind == LetKind::Super {
+        if let Some(scope) = visitor.extended_super_lets.remove(&pat.unwrap().hir_id.local_id) {
+            // This expression was lifetime-extended by a parent let binding. E.g.
+            //
+            //     let a = {
+            //         super let b = temp();
+            //         &b
+            //     };
+            //
+            // (Which needs to behave exactly as: let a = &temp();)
+            //
+            // Processing of `let a` will have already decided to extend the lifetime of this
+            // `super let` to its own var_scope. We use that scope.
+            visitor.cx.var_parent = scope;
+        } else {
+            // This `super let` is not subject to lifetime extension from a parent let binding. E.g.
+            //
+            //     identity({ super let x = temp(); &x }).method();
+            //
+            // (Which needs to behave exactly as: identity(&temp()).method();)
+            //
+            // Iterate up to the enclosing destruction scope to find the same scope that will also
+            // be used for the result of the block itself.
+            while let Some(s) = visitor.cx.var_parent {
+                let parent = visitor.scope_tree.parent_map.get(&s).cloned();
+                if let Some(Scope { data: ScopeData::Destruction, .. }) = parent {
+                    break;
+                }
+                visitor.cx.var_parent = parent;
+            }
+        }
+    }
+
     if let Some(expr) = init {
-        record_rvalue_scope_if_borrow_expr(visitor, expr, blk_scope);
+        record_rvalue_scope_if_borrow_expr(visitor, expr, visitor.cx.var_parent);
 
         if let Some(pat) = pat {
             if is_binding_pat(pat) {
                 visitor.scope_tree.record_rvalue_candidate(
                     expr.hir_id,
-                    RvalueCandidate { target: expr.hir_id.local_id, lifetime: blk_scope },
+                    RvalueCandidate {
+                        target: expr.hir_id.local_id,
+                        lifetime: visitor.cx.var_parent,
+                    },
                 );
             }
         }
@@ -565,6 +620,7 @@ fn resolve_local<'tcx>(
     if let Some(expr) = init {
         visitor.visit_expr(expr);
     }
+
     if let Some(pat) = pat {
         visitor.visit_pat(pat);
     }
@@ -642,6 +698,7 @@ fn resolve_local<'tcx>(
     ///        | [ ..., E&, ... ]
     ///        | ( ..., E&, ... )
     ///        | {...; E&}
+    ///        | { super let ... = E&; ... }
     ///        | if _ { ...; E& } else { ...; E& }
     ///        | match _ { ..., _ => E&, ... }
     ///        | box E&
@@ -678,6 +735,13 @@ fn resolve_local<'tcx>(
                 if let Some(subexpr) = block.expr {
                     record_rvalue_scope_if_borrow_expr(visitor, subexpr, blk_id);
                 }
+                for stmt in block.stmts {
+                    if let hir::StmtKind::Let(local) = stmt.kind
+                        && let Some(_) = local.super_
+                    {
+                        visitor.extended_super_lets.insert(local.pat.hir_id.local_id, blk_id);
+                    }
+                }
             }
             hir::ExprKind::If(_, then_block, else_block) => {
                 record_rvalue_scope_if_borrow_expr(visitor, then_block, blk_id);
@@ -803,7 +867,7 @@ impl<'tcx> Visitor<'tcx> for ScopeResolutionVisitor<'tcx> {
                     local_id: body.value.hir_id.local_id,
                     data: ScopeData::Destruction,
                 });
-                resolve_local(this, None, Some(body.value));
+                resolve_local(this, None, Some(body.value), LetKind::Regular);
             }
         })
     }
@@ -821,7 +885,11 @@ impl<'tcx> Visitor<'tcx> for ScopeResolutionVisitor<'tcx> {
         resolve_expr(self, ex, false);
     }
     fn visit_local(&mut self, l: &'tcx LetStmt<'tcx>) {
-        resolve_local(self, Some(l.pat), l.init)
+        let let_kind = match l.super_ {
+            Some(_) => LetKind::Super,
+            None => LetKind::Regular,
+        };
+        resolve_local(self, Some(l.pat), l.init, let_kind);
     }
     fn visit_inline_const(&mut self, c: &'tcx hir::ConstBlock) {
         let body = self.tcx.hir_body(c.body);
@@ -850,6 +918,7 @@ pub(crate) fn region_scope_tree(tcx: TyCtxt<'_>, def_id: DefId) -> &ScopeTree {
             cx: Context { parent: None, var_parent: None },
             pessimistic_yield: false,
             fixup_scopes: vec![],
+            extended_super_lets: Default::default(),
         };
 
         visitor.scope_tree.root_body = Some(body.value.hir_id);

compiler/rustc_hir_pretty/src/lib.rs

@@ -960,12 +960,16 @@ impl<'a> State<'a> {
 
     fn print_local(
         &mut self,
+        super_: bool,
         init: Option<&hir::Expr<'_>>,
         els: Option<&hir::Block<'_>>,
         decl: impl Fn(&mut Self),
     ) {
         self.space_if_not_bol();
         self.ibox(INDENT_UNIT);
+        if super_ {
+            self.word_nbsp("super");
+        }
         self.word_nbsp("let");
 
         self.ibox(INDENT_UNIT);
@@ -995,7 +999,9 @@ impl<'a> State<'a> {
         self.maybe_print_comment(st.span.lo());
         match st.kind {
             hir::StmtKind::Let(loc) => {
-                self.print_local(loc.init, loc.els, |this| this.print_local_decl(loc));
+                self.print_local(loc.super_.is_some(), loc.init, loc.els, |this| {
+                    this.print_local_decl(loc)
+                });
             }
             hir::StmtKind::Item(item) => self.ann.nested(self, Nested::Item(item)),
             hir::StmtKind::Expr(expr) => {
@@ -1488,7 +1494,7 @@ impl<'a> State<'a> {
 
                 // Print `let _t = $init;`:
                 let temp = Ident::from_str("_t");
-                self.print_local(Some(init), None, |this| this.print_ident(temp));
+                self.print_local(false, Some(init), None, |this| this.print_ident(temp));
                 self.word(";");
 
                 // Print `_t`:

compiler/rustc_hir_typeck/src/gather_locals.rs

@@ -43,7 +43,7 @@ pub(super) struct Declaration<'a> {
 
 impl<'a> From<&'a hir::LetStmt<'a>> for Declaration<'a> {
     fn from(local: &'a hir::LetStmt<'a>) -> Self {
-        let hir::LetStmt { hir_id, pat, ty, span, init, els, source: _ } = *local;
+        let hir::LetStmt { hir_id, super_: _, pat, ty, span, init, els, source: _ } = *local;
         Declaration { hir_id, pat, ty, span, init, origin: DeclOrigin::LocalDecl { els } }
     }
 }