Skip to content

Commit

Permalink
Sync to upstream/release/566 (#853)
Browse files Browse the repository at this point in the history
* Fixed incorrect lexeme generated for string parts in the middle of an
interpolated string (Fixes #744)
* DeprecatedApi lint can report some issues without type inference
information
* Fixed performance of autocomplete requests when suggestions have large
intersection types (Solves
#847)
* Marked `table.getn`/`foreach`/`foreachi` as deprecated ([RFC:
Deprecate
table.getn/foreach/foreachi](https://github.com/Roblox/luau/blob/master/rfcs/deprecate-table-getn-foreach.md))
* With -O2 optimization level, we now optimize builtin calls based on
known argument/return count.
Note that this change can be observable if `getfenv/setfenv` is used to
substitute a builtin, especially if arity is different.
Fastcall heavy tests show a 1-2% improvement.
* Luau can now be built with clang-cl (Fixes
#736)

We also made many improvements to our experimental components.

For our new type solver:
* Overhauled data flow analysis system, fixed issues with 'repeat'
loops, global variables and type annotations
* Type refinements now work on generic table indexing with a string
literal
* Type refinements will properly track potentially 'nil' values (like
t[x] for a missing key) and their further refinements
* Internal top table type is now isomorphic to `{}` which fixes issues
when `typeof(v) == 'table'` type refinement is handled
* References to non-existent types in type annotations no longer resolve
to 'error' type like in old solver
* Improved handling of class unions in property access expressions
* Fixed default type packs
* Unsealed tables can now have metatables
* Restored expected types for function arguments

And for native code generation:
* Added min and max IR instructions mapping to vminsd/vmaxsd on x64
* We now speculatively extract direct execution fast-paths based on
expected types of expressions which provides better optimization
opportunities inside a single basic block
* Translated existing math fastcalls to IR form to improve tag guard
removal and constant propagation
  • Loading branch information
vegorov-rbx authored Mar 3, 2023
1 parent 48172dd commit 140e5a1
Show file tree
Hide file tree
Showing 99 changed files with 3,356 additions and 1,554 deletions.
75 changes: 75 additions & 0 deletions Analysis/include/Luau/Breadcrumb.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details
#pragma once

#include "Luau/Def.h"
#include "Luau/NotNull.h"
#include "Luau/Variant.h"

#include <string>
#include <optional>

namespace Luau
{

using NullableBreadcrumbId = const struct Breadcrumb*;
using BreadcrumbId = NotNull<const struct Breadcrumb>;

struct FieldMetadata
{
std::string prop;
};

struct SubscriptMetadata
{
BreadcrumbId key;
};

using Metadata = Variant<FieldMetadata, SubscriptMetadata>;

struct Breadcrumb
{
NullableBreadcrumbId previous;
DefId def;
std::optional<Metadata> metadata;
std::vector<BreadcrumbId> children;
};

inline Breadcrumb* asMutable(NullableBreadcrumbId breadcrumb)
{
LUAU_ASSERT(breadcrumb);
return const_cast<Breadcrumb*>(breadcrumb);
}

template<typename T>
const T* getMetadata(NullableBreadcrumbId breadcrumb)
{
if (!breadcrumb || !breadcrumb->metadata)
return nullptr;

return get_if<T>(&*breadcrumb->metadata);
}

struct BreadcrumbArena
{
TypedAllocator<Breadcrumb> allocator;

template<typename... Args>
BreadcrumbId add(NullableBreadcrumbId previous, DefId def, Args&&... args)
{
Breadcrumb* bc = allocator.allocate(Breadcrumb{previous, def, std::forward<Args>(args)...});
if (previous)
asMutable(previous)->children.push_back(NotNull{bc});
return NotNull{bc};
}

template<typename T, typename... Args>
BreadcrumbId emplace(NullableBreadcrumbId previous, DefId def, Args&&... args)
{
Breadcrumb* bc = allocator.allocate(Breadcrumb{previous, def, Metadata{T{std::forward<Args>(args)...}}});
if (previous)
asMutable(previous)->children.push_back(NotNull{bc});
return NotNull{bc};
}
};

} // namespace Luau
1 change: 0 additions & 1 deletion Analysis/include/Luau/Constraint.h
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
#pragma once

#include "Luau/Ast.h" // Used for some of the enumerations
#include "Luau/Def.h"
#include "Luau/DenseHash.h"
#include "Luau/NotNull.h"
#include "Luau/Type.h"
Expand Down
13 changes: 10 additions & 3 deletions Analysis/include/Luau/ConstraintGraphBuilder.h
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ struct ConstraintGraphBuilder
* @param inTypeArguments whether we are resolving a type that's contained within type arguments, `<...>`.
* @return the type of the AST annotation.
**/
TypeId resolveType(const ScopePtr& scope, AstType* ty, bool inTypeArguments);
TypeId resolveType(const ScopePtr& scope, AstType* ty, bool inTypeArguments, bool replaceErrorWithFresh = false);

/**
* Resolves a type pack from its AST annotation.
Expand All @@ -233,7 +233,7 @@ struct ConstraintGraphBuilder
* @param inTypeArguments whether we are resolving a type that's contained within type arguments, `<...>`.
* @return the type pack of the AST annotation.
**/
TypePackId resolveTypePack(const ScopePtr& scope, AstTypePack* tp, bool inTypeArguments);
TypePackId resolveTypePack(const ScopePtr& scope, AstTypePack* tp, bool inTypeArguments, bool replaceErrorWithFresh = false);

/**
* Resolves a type pack from its AST annotation.
Expand All @@ -242,7 +242,7 @@ struct ConstraintGraphBuilder
* @param inTypeArguments whether we are resolving a type that's contained within type arguments, `<...>`.
* @return the type pack of the AST annotation.
**/
TypePackId resolveTypePack(const ScopePtr& scope, const AstTypeList& list, bool inTypeArguments);
TypePackId resolveTypePack(const ScopePtr& scope, const AstTypeList& list, bool inTypeArguments, bool replaceErrorWithFresh = false);

/**
* Creates generic types given a list of AST definitions, resolving default
Expand Down Expand Up @@ -282,10 +282,17 @@ struct ConstraintGraphBuilder
* initial scan of the AST and note what globals are defined.
*/
void prepopulateGlobalScope(const ScopePtr& globalScope, AstStatBlock* program);

/** Given a function type annotation, return a vector describing the expected types of the calls to the function
* For example, calling a function with annotation ((number) -> string & ((string) -> number))
* yields a vector of size 1, with value: [number | string]
*/
std::vector<std::optional<TypeId>> getExpectedCallTypesForFunctionOverloads(const TypeId fnType);
};

/** Borrow a vector of pointers from a vector of owning pointers to constraints.
*/
std::vector<NotNull<Constraint>> borrowConstraints(const std::vector<ConstraintPtr>& constraints);


} // namespace Luau
4 changes: 2 additions & 2 deletions Analysis/include/Luau/ConstraintSolver.h
Original file line number Diff line number Diff line change
Expand Up @@ -132,8 +132,8 @@ struct ConstraintSolver
bool tryDispatchIterableFunction(
TypeId nextTy, TypeId tableTy, TypeId firstIndexTy, const IterableConstraint& c, NotNull<const Constraint> constraint, bool force);

std::optional<TypeId> lookupTableProp(TypeId subjectType, const std::string& propName);
std::optional<TypeId> lookupTableProp(TypeId subjectType, const std::string& propName, std::unordered_set<TypeId>& seen);
std::pair<std::vector<TypeId>, std::optional<TypeId>> lookupTableProp(TypeId subjectType, const std::string& propName);
std::pair<std::vector<TypeId>, std::optional<TypeId>> lookupTableProp(TypeId subjectType, const std::string& propName, std::unordered_set<TypeId>& seen);

void block(NotNull<const Constraint> target, NotNull<const Constraint> constraint);
/**
Expand Down
120 changes: 75 additions & 45 deletions Analysis/include/Luau/DataFlowGraph.h
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

// Do not include LValue. It should never be used here.
#include "Luau/Ast.h"
#include "Luau/Breadcrumb.h"
#include "Luau/DenseHash.h"
#include "Luau/Def.h"
#include "Luau/Symbol.h"
Expand All @@ -17,39 +18,44 @@ struct DataFlowGraph
DataFlowGraph(DataFlowGraph&&) = default;
DataFlowGraph& operator=(DataFlowGraph&&) = default;

// TODO: AstExprLocal, AstExprGlobal, and AstLocal* are guaranteed never to return nullopt.
// We leave them to return an optional as we build it out, but the end state is for them to return a non-optional DefId.
std::optional<DefId> getDef(const AstExpr* expr) const;
std::optional<DefId> getDef(const AstLocal* local) const;
NullableBreadcrumbId getBreadcrumb(const AstExpr* expr) const;

/// Retrieve the Def that corresponds to the given Symbol.
///
/// We do not perform dataflow analysis on globals, so this function always
/// yields nullopt when passed a global Symbol.
std::optional<DefId> getDef(const Symbol& symbol) const;
BreadcrumbId getBreadcrumb(const AstLocal* local) const;
BreadcrumbId getBreadcrumb(const AstExprLocal* local) const;
BreadcrumbId getBreadcrumb(const AstExprGlobal* global) const;

BreadcrumbId getBreadcrumb(const AstStatDeclareGlobal* global) const;
BreadcrumbId getBreadcrumb(const AstStatDeclareFunction* func) const;

private:
DataFlowGraph() = default;

DataFlowGraph(const DataFlowGraph&) = delete;
DataFlowGraph& operator=(const DataFlowGraph&) = delete;

DefArena arena;
DenseHashMap<const AstExpr*, const Def*> astDefs{nullptr};
DenseHashMap<const AstLocal*, const Def*> localDefs{nullptr};
DefArena defs;
BreadcrumbArena breadcrumbs;

DenseHashMap<const AstExpr*, NullableBreadcrumbId> astBreadcrumbs{nullptr};

// Sometimes we don't have the AstExprLocal* but we have AstLocal*, and sometimes we need to extract that DefId.
DenseHashMap<const AstLocal*, NullableBreadcrumbId> localBreadcrumbs{nullptr};

// There's no AstStatDeclaration, and it feels useless to introduce it just to enforce an invariant in one place.
// All keys in this maps are really only statements that ambiently declares a symbol.
DenseHashMap<const AstStat*, NullableBreadcrumbId> declaredBreadcrumbs{nullptr};

friend struct DataFlowGraphBuilder;
};

struct DfgScope
{
DfgScope* parent;
DenseHashMap<Symbol, const Def*> bindings{Symbol{}};
};
DenseHashMap<Symbol, NullableBreadcrumbId> bindings{Symbol{}};
DenseHashMap<const Def*, std::unordered_map<std::string, NullableBreadcrumbId>> props{nullptr};

struct ExpressionFlowGraph
{
std::optional<DefId> def;
NullableBreadcrumbId lookup(Symbol symbol) const;
NullableBreadcrumbId lookup(DefId def, const std::string& key) const;
};

// Currently unsound. We do not presently track the control flow of the program.
Expand All @@ -65,23 +71,19 @@ struct DataFlowGraphBuilder
DataFlowGraphBuilder& operator=(const DataFlowGraphBuilder&) = delete;

DataFlowGraph graph;
NotNull<DefArena> arena{&graph.arena};
struct InternalErrorReporter* handle;
std::vector<std::unique_ptr<DfgScope>> scopes;
NotNull<DefArena> defs{&graph.defs};
NotNull<BreadcrumbArena> breadcrumbs{&graph.breadcrumbs};

// Does not belong in DataFlowGraphBuilder, but the old solver allows properties to escape the scope they were defined in,
// so we will need to be able to emulate this same behavior here too. We can kill this once we have better flow sensitivity.
DenseHashMap<const Def*, std::unordered_map<std::string, const Def*>> props{nullptr};
struct InternalErrorReporter* handle = nullptr;
DfgScope* moduleScope = nullptr;

DfgScope* childScope(DfgScope* scope);
std::vector<std::unique_ptr<DfgScope>> scopes;

std::optional<DefId> use(DfgScope* scope, Symbol symbol, AstExpr* e);
DefId use(DefId def, AstExprIndexName* e);
DfgScope* childScope(DfgScope* scope);

void visit(DfgScope* scope, AstStatBlock* b);
void visitBlockWithoutChildScope(DfgScope* scope, AstStatBlock* b);

// TODO: visit type aliases
void visit(DfgScope* scope, AstStat* s);
void visit(DfgScope* scope, AstStatIf* i);
void visit(DfgScope* scope, AstStatWhile* w);
Expand All @@ -97,24 +99,52 @@ struct DataFlowGraphBuilder
void visit(DfgScope* scope, AstStatCompoundAssign* c);
void visit(DfgScope* scope, AstStatFunction* f);
void visit(DfgScope* scope, AstStatLocalFunction* l);

ExpressionFlowGraph visitExpr(DfgScope* scope, AstExpr* e);
ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprLocal* l);
ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprGlobal* g);
ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprCall* c);
ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprIndexName* i);
ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprIndexExpr* i);
ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprFunction* f);
ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprTable* t);
ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprUnary* u);
ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprBinary* b);
ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprTypeAssertion* t);
ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprIfElse* i);
ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprInterpString* i);

// TODO: visitLValue
// TODO: visitTypes (because of typeof which has access to values namespace, needs unreachable scope)
// TODO: visitTypePacks (because of typeof which has access to values namespace, needs unreachable scope)
void visit(DfgScope* scope, AstStatTypeAlias* t);
void visit(DfgScope* scope, AstStatDeclareGlobal* d);
void visit(DfgScope* scope, AstStatDeclareFunction* d);
void visit(DfgScope* scope, AstStatDeclareClass* d);
void visit(DfgScope* scope, AstStatError* error);

BreadcrumbId visitExpr(DfgScope* scope, AstExpr* e);
BreadcrumbId visitExpr(DfgScope* scope, AstExprLocal* l);
BreadcrumbId visitExpr(DfgScope* scope, AstExprGlobal* g);
BreadcrumbId visitExpr(DfgScope* scope, AstExprCall* c);
BreadcrumbId visitExpr(DfgScope* scope, AstExprIndexName* i);
BreadcrumbId visitExpr(DfgScope* scope, AstExprIndexExpr* i);
BreadcrumbId visitExpr(DfgScope* scope, AstExprFunction* f);
BreadcrumbId visitExpr(DfgScope* scope, AstExprTable* t);
BreadcrumbId visitExpr(DfgScope* scope, AstExprUnary* u);
BreadcrumbId visitExpr(DfgScope* scope, AstExprBinary* b);
BreadcrumbId visitExpr(DfgScope* scope, AstExprTypeAssertion* t);
BreadcrumbId visitExpr(DfgScope* scope, AstExprIfElse* i);
BreadcrumbId visitExpr(DfgScope* scope, AstExprInterpString* i);
BreadcrumbId visitExpr(DfgScope* scope, AstExprError* error);

void visitLValue(DfgScope* scope, AstExpr* e);
void visitLValue(DfgScope* scope, AstExprLocal* l);
void visitLValue(DfgScope* scope, AstExprGlobal* g);
void visitLValue(DfgScope* scope, AstExprIndexName* i);
void visitLValue(DfgScope* scope, AstExprIndexExpr* i);
void visitLValue(DfgScope* scope, AstExprError* e);

void visitType(DfgScope* scope, AstType* t);
void visitType(DfgScope* scope, AstTypeReference* r);
void visitType(DfgScope* scope, AstTypeTable* t);
void visitType(DfgScope* scope, AstTypeFunction* f);
void visitType(DfgScope* scope, AstTypeTypeof* t);
void visitType(DfgScope* scope, AstTypeUnion* u);
void visitType(DfgScope* scope, AstTypeIntersection* i);
void visitType(DfgScope* scope, AstTypeError* error);

void visitTypePack(DfgScope* scope, AstTypePack* p);
void visitTypePack(DfgScope* scope, AstTypePackExplicit* e);
void visitTypePack(DfgScope* scope, AstTypePackVariadic* v);
void visitTypePack(DfgScope* scope, AstTypePackGeneric* g);

void visitTypeList(DfgScope* scope, AstTypeList l);

void visitGenerics(DfgScope* scope, AstArray<AstGenericType> g);
void visitGenericPacks(DfgScope* scope, AstArray<AstGenericTypePack> g);
};

} // namespace Luau
8 changes: 0 additions & 8 deletions Analysis/include/Luau/Def.h
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,6 @@ namespace Luau
struct Def;
using DefId = NotNull<const Def>;

struct FieldMetadata
{
DefId parent;
std::string propName;
};

/**
* A cell is a "single-object" value.
*
Expand All @@ -29,7 +23,6 @@ struct FieldMetadata
*/
struct Cell
{
std::optional<struct FieldMetadata> field;
};

/**
Expand Down Expand Up @@ -83,7 +76,6 @@ struct DefArena
TypedAllocator<Def> allocator;

DefId freshCell();
DefId freshCell(DefId parent, const std::string& prop);
// TODO: implement once we have cases where we need to merge in definitions
// DefId phi(const std::vector<DefId>& defs);
};
Expand Down
21 changes: 8 additions & 13 deletions Analysis/include/Luau/Frontend.h
Original file line number Diff line number Diff line change
Expand Up @@ -162,8 +162,7 @@ struct Frontend
ScopePtr getGlobalScope();

private:
ModulePtr check(const SourceModule& sourceModule, Mode mode, std::vector<RequireCycle> requireCycles,
bool forAutocomplete = false);
ModulePtr check(const SourceModule& sourceModule, Mode mode, std::vector<RequireCycle> requireCycles, bool forAutocomplete = false, bool recordJsonLog = false);

std::pair<SourceNode*, SourceModule*> getSourceNode(const ModuleName& name);
SourceModule parse(const ModuleName& name, std::string_view src, const ParseOptions& parseOptions);
Expand Down Expand Up @@ -202,16 +201,12 @@ struct Frontend
ScopePtr globalScope;
};

ModulePtr check(
const SourceModule& sourceModule,
const std::vector<RequireCycle>& requireCycles,
NotNull<BuiltinTypes> builtinTypes,
NotNull<InternalErrorReporter> iceHandler,
NotNull<ModuleResolver> moduleResolver,
NotNull<FileResolver> fileResolver,
const ScopePtr& globalScope,
NotNull<UnifierSharedState> unifierState,
FrontendOptions options
);
ModulePtr check(const SourceModule& sourceModule, const std::vector<RequireCycle>& requireCycles, NotNull<BuiltinTypes> builtinTypes,
NotNull<InternalErrorReporter> iceHandler, NotNull<ModuleResolver> moduleResolver, NotNull<FileResolver> fileResolver,
const ScopePtr& globalScope, FrontendOptions options);

ModulePtr check(const SourceModule& sourceModule, const std::vector<RequireCycle>& requireCycles, NotNull<BuiltinTypes> builtinTypes,
NotNull<InternalErrorReporter> iceHandler, NotNull<ModuleResolver> moduleResolver, NotNull<FileResolver> fileResolver,
const ScopePtr& globalScope, FrontendOptions options, bool recordJsonLog);

} // namespace Luau
Loading

0 comments on commit 140e5a1

Please sign in to comment.