Skip to content

Commit 01859da

Browse files
pcwaltonpcwalton
committed
[AliasAnalysis] Introduce getModRefInfoMask() as a generalization of pointsToConstantMemory().
The pointsToConstantMemory() method returns true only if the memory pointed to by the memory location is globally invariant. However, the LLVM memory model also has the semantic notion of *locally-invariant*: memory that is known to be invariant for the life of the SSA value representing that pointer. The most common example of this is a pointer argument that is marked readonly noalias, which the Rust compiler frequently emits. It'd be desirable for LLVM to treat locally-invariant memory the same way as globally-invariant memory when it's safe to do so. This patch implements that, by introducing the concept of a *ModRefInfo mask*. A ModRefInfo mask is a bound on the Mod/Ref behavior of an instruction that writes to a memory location, based on the knowledge that the memory is globally-constant memory (in which case the mask is NoModRef) or locally-constant memory (in which case the mask is Ref). ModRefInfo values for an instruction can be combined with the ModRefInfo mask by simply using the & operator. Where appropriate, this patch has modified uses of pointsToConstantMemory() to instead examine the mask. The most notable optimization change I noticed with this patch is that now redundant loads from readonly noalias pointers can be eliminated across calls, even when the pointer is captured. Internally, before this patch, AliasAnalysis was assigning Ref to reads from constant memory; now AA can assign NoModRef, which is a tighter bound. Differential Revision: https://reviews.llvm.org/D136659
1 parent 7af01fe commit 01859da

21 files changed

+198
-168
lines changed

‎llvm/docs/AliasAnalysis.rst

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -161,14 +161,24 @@ Other useful ``AliasAnalysis`` methods
161161
Several other tidbits of information are often collected by various alias
162162
analysis implementations and can be put to good use by various clients.
163163

164-
The ``pointsToConstantMemory`` method
165-
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
166-
167-
The ``pointsToConstantMemory`` method returns true if and only if the analysis
168-
can prove that the pointer only points to unchanging memory locations
169-
(functions, constant global variables, and the null pointer). This information
170-
can be used to refine mod/ref information: it is impossible for an unchanging
171-
memory location to be modified.
164+
The ``getModRefInfoMask`` method
165+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
166+
167+
The ``getModRefInfoMask`` method returns a bound on Mod/Ref information for
168+
the supplied pointer, based on knowledge about whether the pointer points to
169+
globally-constant memory (for which it returns ``NoModRef``) or
170+
locally-invariant memory (for which it returns ``Ref``). Globally-constant
171+
memory includes functions, constant global variables, and the null pointer.
172+
Locally-invariant memory is memory that we know is invariant for the lifetime
173+
of its SSA value, but not necessarily for the life of the program: for example,
174+
the memory pointed to by ``readonly`` ``noalias`` parameters is known-invariant
175+
for the duration of the corresponding function call. Given Mod/Ref information
176+
``MRI`` for a memory location ``Loc``, ``MRI`` can be refined with a statement
177+
like ``MRI &= AA.getModRefInfoMask(Loc);``. Another useful idiom is
178+
``isModSet(AA.getModRefInfoMask(Loc))``; this checks to see if the given
179+
location can be modified at all. For convenience, there is also a method
180+
``pointsToConstantMemory(Loc)``; this is synonymous with
181+
``isNoModRef(AA.getModRefInfoMask(Loc))``.
172182

173183
.. _never access memory or only read memory:
174184

‎llvm/include/llvm/Analysis/AliasAnalysis.h

Lines changed: 39 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -359,7 +359,9 @@ class AAResults {
359359

360360
/// Checks whether the given location points to constant memory, or if
361361
/// \p OrLocal is true whether it points to a local alloca.
362-
bool pointsToConstantMemory(const MemoryLocation &Loc, bool OrLocal = false);
362+
bool pointsToConstantMemory(const MemoryLocation &Loc, bool OrLocal = false) {
363+
return isNoModRef(getModRefInfoMask(Loc, OrLocal));
364+
}
363365

364366
/// A convenience wrapper around the primary \c pointsToConstantMemory
365367
/// interface.
@@ -372,6 +374,22 @@ class AAResults {
372374
/// \name Simple mod/ref information
373375
/// @{
374376

377+
/// Returns a bitmask that should be unconditionally applied to the ModRef
378+
/// info of a memory location. This allows us to eliminate Mod and/or Ref
379+
/// from the ModRef info based on the knowledge that the memory location
380+
/// points to constant and/or locally-invariant memory.
381+
///
382+
/// If IgnoreLocals is true, then this method returns NoModRef for memory
383+
/// that points to a local alloca.
384+
ModRefInfo getModRefInfoMask(const MemoryLocation &Loc,
385+
bool IgnoreLocals = false);
386+
387+
/// A convenience wrapper around the primary \c getModRefInfoMask
388+
/// interface.
389+
ModRefInfo getModRefInfoMask(const Value *P, bool IgnoreLocals = false) {
390+
return getModRefInfoMask(MemoryLocation::getBeforeOrAfter(P), IgnoreLocals);
391+
}
392+
375393
/// Get the ModRef info associated with a pointer argument of a call. The
376394
/// result's bits are set to indicate the allowed aliasing ModRef kinds. Note
377395
/// that these bits do not necessarily account for the overall behavior of
@@ -614,6 +632,8 @@ class AAResults {
614632
AAQueryInfo &AAQI);
615633
bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
616634
bool OrLocal = false);
635+
ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
636+
bool IgnoreLocals = false);
617637
ModRefInfo getModRefInfo(Instruction *I, const CallBase *Call2,
618638
AAQueryInfo &AAQIP);
619639
ModRefInfo getModRefInfo(const CallBase *Call, const MemoryLocation &Loc,
@@ -681,6 +701,10 @@ class BatchAAResults {
681701
bool pointsToConstantMemory(const MemoryLocation &Loc, bool OrLocal = false) {
682702
return AA.pointsToConstantMemory(Loc, AAQI, OrLocal);
683703
}
704+
ModRefInfo getModRefInfoMask(const MemoryLocation &Loc,
705+
bool IgnoreLocals = false) {
706+
return AA.getModRefInfoMask(Loc, AAQI, IgnoreLocals);
707+
}
684708
ModRefInfo getModRefInfo(const CallBase *Call, const MemoryLocation &Loc) {
685709
return AA.getModRefInfo(Call, Loc, AAQI);
686710
}
@@ -743,16 +767,19 @@ class AAResults::Concept {
743767
virtual AliasResult alias(const MemoryLocation &LocA,
744768
const MemoryLocation &LocB, AAQueryInfo &AAQI) = 0;
745769

746-
/// Checks whether the given location points to constant memory, or if
747-
/// \p OrLocal is true whether it points to a local alloca.
748-
virtual bool pointsToConstantMemory(const MemoryLocation &Loc,
749-
AAQueryInfo &AAQI, bool OrLocal) = 0;
750-
751770
/// @}
752771
//===--------------------------------------------------------------------===//
753772
/// \name Simple mod/ref information
754773
/// @{
755774

775+
/// Returns a bitmask that should be unconditionally applied to the ModRef
776+
/// info of a memory location. This allows us to eliminate Mod and/or Ref from
777+
/// the ModRef info based on the knowledge that the memory location points to
778+
/// constant and/or locally-invariant memory.
779+
virtual ModRefInfo getModRefInfoMask(const MemoryLocation &Loc,
780+
AAQueryInfo &AAQI,
781+
bool IgnoreLocals) = 0;
782+
756783
/// Get the ModRef info associated with a pointer argument of a callsite. The
757784
/// result's bits are set to indicate the allowed aliasing ModRef kinds. Note
758785
/// that these bits do not necessarily account for the overall behavior of
@@ -801,9 +828,9 @@ template <typename AAResultT> class AAResults::Model final : public Concept {
801828
return Result.alias(LocA, LocB, AAQI);
802829
}
803830

804-
bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
805-
bool OrLocal) override {
806-
return Result.pointsToConstantMemory(Loc, AAQI, OrLocal);
831+
ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
832+
bool IgnoreLocals) override {
833+
return Result.getModRefInfoMask(Loc, AAQI, IgnoreLocals);
807834
}
808835

809836
ModRefInfo getArgModRefInfo(const CallBase *Call, unsigned ArgIdx) override {
@@ -856,9 +883,9 @@ class AAResultBase {
856883
return AliasResult::MayAlias;
857884
}
858885

859-
bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
860-
bool OrLocal) {
861-
return false;
886+
ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
887+
bool IgnoreLocals) {
888+
return ModRefInfo::ModRef;
862889
}
863890

864891
ModRefInfo getArgModRefInfo(const CallBase *Call, unsigned ArgIdx) {

‎llvm/include/llvm/Analysis/BasicAliasAnalysis.h

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -75,9 +75,15 @@ class BasicAAResult : public AAResultBase {
7575
ModRefInfo getModRefInfo(const CallBase *Call1, const CallBase *Call2,
7676
AAQueryInfo &AAQI);
7777

78-
/// Chases pointers until we find a (constant global) or not.
79-
bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
80-
bool OrLocal);
78+
/// Returns a bitmask that should be unconditionally applied to the ModRef
79+
/// info of a memory location. This allows us to eliminate Mod and/or Ref
80+
/// from the ModRef info based on the knowledge that the memory location
81+
/// points to constant and/or locally-invariant memory.
82+
///
83+
/// If IgnoreLocals is true, then this method returns NoModRef for memory
84+
/// that points to a local alloca.
85+
ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
86+
bool IgnoreLocals = false);
8187

8288
/// Get the location associated with a pointer argument of a callsite.
8389
ModRefInfo getArgModRefInfo(const CallBase *Call, unsigned ArgIdx);

‎llvm/include/llvm/Analysis/ObjCARCAliasAnalysis.h

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,8 +52,8 @@ class ObjCARCAAResult : public AAResultBase {
5252

5353
AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB,
5454
AAQueryInfo &AAQI);
55-
bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
56-
bool OrLocal);
55+
ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
56+
bool IgnoreLocals);
5757

5858
using AAResultBase::getMemoryEffects;
5959
MemoryEffects getMemoryEffects(const Function *F);

‎llvm/include/llvm/Analysis/TypeBasedAliasAnalysis.h

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,8 +40,8 @@ class TypeBasedAAResult : public AAResultBase {
4040

4141
AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB,
4242
AAQueryInfo &AAQI);
43-
bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
44-
bool OrLocal);
43+
ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
44+
bool IgnoreLocals);
4545
MemoryEffects getMemoryEffects(const CallBase *Call, AAQueryInfo &AAQI);
4646
MemoryEffects getMemoryEffects(const Function *F);
4747
ModRefInfo getModRefInfo(const CallBase *Call, const MemoryLocation &Loc,

‎llvm/lib/Analysis/AliasAnalysis.cpp

Lines changed: 35 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -147,19 +147,25 @@ AliasResult AAResults::alias(const MemoryLocation &LocA,
147147
return Result;
148148
}
149149

150-
bool AAResults::pointsToConstantMemory(const MemoryLocation &Loc,
151-
bool OrLocal) {
150+
ModRefInfo AAResults::getModRefInfoMask(const MemoryLocation &Loc,
151+
bool IgnoreLocals) {
152152
SimpleAAQueryInfo AAQIP(*this);
153-
return pointsToConstantMemory(Loc, AAQIP, OrLocal);
153+
return getModRefInfoMask(Loc, AAQIP, IgnoreLocals);
154154
}
155155

156-
bool AAResults::pointsToConstantMemory(const MemoryLocation &Loc,
157-
AAQueryInfo &AAQI, bool OrLocal) {
158-
for (const auto &AA : AAs)
159-
if (AA->pointsToConstantMemory(Loc, AAQI, OrLocal))
160-
return true;
156+
ModRefInfo AAResults::getModRefInfoMask(const MemoryLocation &Loc,
157+
AAQueryInfo &AAQI, bool IgnoreLocals) {
158+
ModRefInfo Result = ModRefInfo::ModRef;
161159

162-
return false;
160+
for (const auto &AA : AAs) {
161+
Result &= AA->getModRefInfoMask(Loc, AAQI, IgnoreLocals);
162+
163+
// Early-exit the moment we reach the bottom of the lattice.
164+
if (isNoModRef(Result))
165+
return ModRefInfo::NoModRef;
166+
}
167+
168+
return Result;
163169
}
164170

165171
ModRefInfo AAResults::getArgModRefInfo(const CallBase *Call, unsigned ArgIdx) {
@@ -253,10 +259,11 @@ ModRefInfo AAResults::getModRefInfo(const CallBase *Call,
253259

254260
Result &= ArgMR | OtherMR;
255261

256-
// If Loc is a constant memory location, the call definitely could not
262+
// Apply the ModRef mask. This ensures that if Loc is a constant memory
263+
// location, we take into account the fact that the call definitely could not
257264
// modify the memory location.
258-
if (isModSet(Result) && pointsToConstantMemory(Loc, AAQI, /*OrLocal*/ false))
259-
Result &= ModRefInfo::Ref;
265+
if (!isNoModRef(Result))
266+
Result &= getModRefInfoMask(Loc);
260267

261268
return Result;
262269
}
@@ -510,9 +517,11 @@ ModRefInfo AAResults::getModRefInfo(const StoreInst *S,
510517
if (AR == AliasResult::NoAlias)
511518
return ModRefInfo::NoModRef;
512519

513-
// If the pointer is a pointer to constant memory, then it could not have
514-
// been modified by this store.
515-
if (pointsToConstantMemory(Loc, AAQI))
520+
// Examine the ModRef mask. If Mod isn't present, then return NoModRef.
521+
// This ensures that if Loc is a constant memory location, we take into
522+
// account the fact that the store definitely could not modify the memory
523+
// location.
524+
if (!isModSet(getModRefInfoMask(Loc)))
516525
return ModRefInfo::NoModRef;
517526
}
518527

@@ -529,10 +538,11 @@ ModRefInfo AAResults::getModRefInfo(const FenceInst *S,
529538
ModRefInfo AAResults::getModRefInfo(const FenceInst *S,
530539
const MemoryLocation &Loc,
531540
AAQueryInfo &AAQI) {
532-
// If we know that the location is a constant memory location, the fence
533-
// cannot modify this location.
534-
if (Loc.Ptr && pointsToConstantMemory(Loc, AAQI))
535-
return ModRefInfo::Ref;
541+
// All we know about a fence instruction is what we get from the ModRef
542+
// mask: if Loc is a constant memory location, the fence definitely could
543+
// not modify it.
544+
if (Loc.Ptr)
545+
return getModRefInfoMask(Loc);
536546
return ModRefInfo::ModRef;
537547
}
538548

@@ -552,10 +562,9 @@ ModRefInfo AAResults::getModRefInfo(const VAArgInst *V,
552562
if (AR == AliasResult::NoAlias)
553563
return ModRefInfo::NoModRef;
554564

555-
// If the pointer is a pointer to constant memory, then it could not have
565+
// If the pointer is a pointer to invariant memory, then it could not have
556566
// been modified by this va_arg.
557-
if (pointsToConstantMemory(Loc, AAQI))
558-
return ModRefInfo::NoModRef;
567+
return getModRefInfoMask(Loc, AAQI);
559568
}
560569

561570
// Otherwise, a va_arg reads and writes.
@@ -572,10 +581,9 @@ ModRefInfo AAResults::getModRefInfo(const CatchPadInst *CatchPad,
572581
const MemoryLocation &Loc,
573582
AAQueryInfo &AAQI) {
574583
if (Loc.Ptr) {
575-
// If the pointer is a pointer to constant memory,
584+
// If the pointer is a pointer to invariant memory,
576585
// then it could not have been modified by this catchpad.
577-
if (pointsToConstantMemory(Loc, AAQI))
578-
return ModRefInfo::NoModRef;
586+
return getModRefInfoMask(Loc, AAQI);
579587
}
580588

581589
// Otherwise, a catchpad reads and writes.
@@ -592,10 +600,9 @@ ModRefInfo AAResults::getModRefInfo(const CatchReturnInst *CatchRet,
592600
const MemoryLocation &Loc,
593601
AAQueryInfo &AAQI) {
594602
if (Loc.Ptr) {
595-
// If the pointer is a pointer to constant memory,
603+
// If the pointer is a pointer to invariant memory,
596604
// then it could not have been modified by this catchpad.
597-
if (pointsToConstantMemory(Loc, AAQI))
598-
return ModRefInfo::NoModRef;
605+
return getModRefInfoMask(Loc, AAQI);
599606
}
600607

601608
// Otherwise, a catchret reads and writes.

‎llvm/lib/Analysis/BasicAliasAnalysis.cpp

Lines changed: 30 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -676,33 +676,46 @@ BasicAAResult::DecomposeGEPExpression(const Value *V, const DataLayout &DL,
676676
return Decomposed;
677677
}
678678

679-
/// Returns whether the given pointer value points to memory that is local to
680-
/// the function, with global constants being considered local to all
681-
/// functions.
682-
bool BasicAAResult::pointsToConstantMemory(const MemoryLocation &Loc,
683-
AAQueryInfo &AAQI, bool OrLocal) {
679+
ModRefInfo BasicAAResult::getModRefInfoMask(const MemoryLocation &Loc,
680+
AAQueryInfo &AAQI,
681+
bool IgnoreLocals) {
684682
assert(Visited.empty() && "Visited must be cleared after use!");
685-
auto _ = make_scope_exit([&]{ Visited.clear(); });
683+
auto _ = make_scope_exit([&] { Visited.clear(); });
686684

687685
unsigned MaxLookup = 8;
688686
SmallVector<const Value *, 16> Worklist;
689687
Worklist.push_back(Loc.Ptr);
688+
ModRefInfo Result = ModRefInfo::NoModRef;
689+
690690
do {
691691
const Value *V = getUnderlyingObject(Worklist.pop_back_val());
692692
if (!Visited.insert(V).second)
693693
continue;
694694

695-
// An alloca instruction defines local memory.
696-
if (OrLocal && isa<AllocaInst>(V))
695+
// Ignore allocas if we were instructed to do so.
696+
if (IgnoreLocals && isa<AllocaInst>(V))
697697
continue;
698698

699-
// A global constant counts as local memory for our purposes.
699+
// If the location points to memory that is known to be invariant for
700+
// the life of the underlying SSA value, then we can exclude Mod from
701+
// the set of valid memory effects.
702+
//
703+
// An argument that is marked readonly and noalias is known to be
704+
// invariant while that function is executing.
705+
if (const Argument *Arg = dyn_cast<Argument>(V)) {
706+
if (Arg->hasNoAliasAttr() && Arg->onlyReadsMemory()) {
707+
Result |= ModRefInfo::Ref;
708+
continue;
709+
}
710+
}
711+
712+
// A global constant can't be mutated.
700713
if (const GlobalVariable *GV = dyn_cast<GlobalVariable>(V)) {
701714
// Note: this doesn't require GV to be "ODR" because it isn't legal for a
702715
// global to be marked constant in some modules and non-constant in
703716
// others. GV may even be a declaration, not a definition.
704717
if (!GV->isConstant())
705-
return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
718+
return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
706719
continue;
707720
}
708721

@@ -718,16 +731,20 @@ bool BasicAAResult::pointsToConstantMemory(const MemoryLocation &Loc,
718731
if (const PHINode *PN = dyn_cast<PHINode>(V)) {
719732
// Don't bother inspecting phi nodes with many operands.
720733
if (PN->getNumIncomingValues() > MaxLookup)
721-
return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
734+
return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
722735
append_range(Worklist, PN->incoming_values());
723736
continue;
724737
}
725738

726739
// Otherwise be conservative.
727-
return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
740+
return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
728741
} while (!Worklist.empty() && --MaxLookup);
729742

730-
return Worklist.empty();
743+
// If we hit the maximum number of instructions to examine, be conservative.
744+
if (!Worklist.empty())
745+
return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
746+
747+
return Result;
731748
}
732749

733750
static bool isIntrinsicCall(const CallBase *Call, Intrinsic::ID IID) {

‎llvm/lib/Analysis/MemoryDependenceAnalysis.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -525,7 +525,7 @@ MemDepResult MemoryDependenceResults::getSimplePointerDependencyFrom(
525525
}
526526

527527
// Stores don't alias loads from read-only memory.
528-
if (BatchAA.pointsToConstantMemory(LoadLoc))
528+
if (!isModSet(BatchAA.getModRefInfoMask(LoadLoc)))
529529
continue;
530530

531531
// Stores depend on may/must aliased loads.

0 commit comments

Comments
 (0)