api.specification

= Introduction =

The API discovery component of xsbt analyzes the source code of a project and extracts the API.  This discovery component operates as a phase during normal compilation.  This phase exists in the 'compiler-interface' subproject and is discussed in section 'API Compiler Phase'.  The API of a source file is discussed in 'API Specification'.

The discovery phase provides a representation of the API in the form of a data structure, which is discussed in the section 'API Data Structure'.  This data structure is a simple hierarchy of Java classes and enums generated from a definition file.  The generated data structure exists in the 'interface' project under the 'xsbti.api' package.

The file that defines the hierarchy is processed by a small datatype generating subproject called 'datatype'.  This subproject defines a basic syntax for creating an immutable data structure hierarchy in Java.  The output is in Java in order to use the data structure across Scala versions.

Finally, the data structure needs to be persisted.  This is currently implemented with standard Java serialization, but will likely use SBinary for a custom implementation for efficiency.

= API Specification =

This section specifies what the API of a source file is.  Familiarity with the specification for Scala is assumed.  For this specification, let 'classes' include classes, traits, objects, and package objects (2.8+ only).

Loosely, the API consists of the signatures in a source file.  This includes packages and types of classes in a source.  The type of a class includes its fully qualified name, type parameters, base classes, and member signatures.

Equality of APIs is determined by straightforward structural equality except for type parameter definitions and references.
 * The ID of a type parameter does not affect equality between two type parameters.  The mapping between IDs of equal type parameters is recorded, however.
 * When comparing two type parameter references, the comparison is deferred until initial equality checking (all other equality checking) is done.  Then, the IDs are compared according to the mapping determined for type parameters in the initial check.

== Top Level Constructs ==
The top level contributions to the API in a source file are packages and top-level classes.  Packages in Scala can be nested, but how they are nested is not important for the API of a source file.  That only affects scope within that source file [VERIFY].  A top level class is a class that is not enclosed in another class or a method.

The public API of a source file is then:
 * The set of fully qualified package names defined in the file.  This includes packages without any classes in them or package objects defined for them.
 * The API of the set of top level classes.  The API of a class is defined in 'Class API' below.

== Definitions ==
A definition introduces a new named class or a class member.  Class members include vars, vals, defs, and types.  A definition has modifiers, including access restrictions.  Other attributes of definitions are specific to the type of definition.  The access modifiers are treated here separately and the effective modifiers are considered.  For example, traits and abstract members are always considered to have the 'abstract' modifier.  Synthetic refers to whether a definition was generated by the compiler.

The API of a definition is
 * The definition's name, which is fully-qualified if it is a class.
 * The list of annotations on the definition
 * The state of the following modifiers: abstract, override, final, sealed, implicit, lazy, synthetic
 * The access restrictions of the definition, one of:
  * public
  * protected with qualifier
  * private with qualifier
  A qualifier part is either empty, 'this', or the fully qualified name of an enclosing package or class.
 * The API specific to the definition, described below

=== Field-like API ===
A field is a val or a var.  Fields contribute setter and getter methods, but the API they contribute cannot be completely defined by these.  The presence of a val or var affect the rules for overriding.
 
The field-specific API (contributed in addition to the parts common to all definitions) is then:
 * Whether the field is a val or var.
 * The type of the field.
 * The API of associated setters and getters.

=== Type Parameters ===
 Type parameters are not a definition, but they are used by methods, classes, and type members.  The important aspects of a type parameter are its position in its defining parameter list and its upper and lower bounds, which are types.  The actual name of a type parameter is not important, but it needs to be uniquely referenced within a source file.
 
The API contributed by a type parameter section is:
 * The API of each type parameter in the type parameter section, in order.  The number of type parameters is implicit here.
 * The API of a type parameter is:
  * A unique ID, which is not significant on its own, but is used to refer to this parameter in types.  (the ID is relative, not absolute)
  * The lower bound, a type.
  * The upper bound, a type.

=== Method API ===
A method is a def.  A def consists of type parameters, value parameters and a return type.  The value parameters come in sections (parameter lists) and the last section may be declared 'implicit'.  A value parameter consists of a name (2.8+ only), a type, and whether a default is specified (2.8+ only).  Additionally, a value parameter may be repeated or by-name.

The API specific to a method (contributed in addition to the parts common to all definitions) is:
 * The API contributed by its type parameter section, described in the Type Parameters section previously.
 * The return type.
 * The API contributed by its value parameter lists, in order.  The number of lists is implicit in this definition.
 * The API of a value parameter list is:
  * Whether the list is marked implicit.
  * The API of its value parameters, in order.  The number of value parameters is implicit here.
  * The API of a value parameter is:
   * The parameter name (2.8+ only)
	* The parameter type.  Types are described later
	* The modifier for the parameter type.  This can be either 'no modifier', 'by-name', or 'repeated'.
	* Whether the parameter has default defined (2.8+ only)

=== Type Member API ===
Type members consist of alias types and abstract types.  Each can have type parameters.

The API specific to an alias type (contributed in addition to the parts common to all definitions) is:
 * The API contributed by its type parameter section, described in the Type Parameters section previously.
 * The type aliased.
 
The API specific to an abstract type (contributed in addition to the parts common to all definitions) is:
 * The API contributed by its type parameter section, described in the Type Parameters section previously.
 * The lower bound, a type.
 * The upper bound, a type.

=== Class API ===
A class has a self type, whether it is a trait, class, object, or package object, and a structure.  Its structure includes its base classes and its members, both declared and inherited.

The API specific to a class (contributed in addition to the parts common to all definitions) is:
 * The self type of the class.
 * Whether the class is a trait, class, object, or package object.
 * The structure, which is described by a structural type.

== Types ==

Types can be described as simple, annotated, structural, or existential.  Within simple types are projections, singletons, type variable references, parameterized types.  Types are well described in the Scala specification.  They are described here because they are essential to defining the API of a source file.

The following sections briefly describe the contents of the different types.

=== Projection ===
 * A prefix, which is a simple type
 * An ID, which is a String

=== Type Variable Reference ===
 * The unique ID of the referenced type variable

=== Singleton ===
 * The path underlying the singleton.
 
=== Parameterized ===
 * The type arguments, which is an ordered list of simple types.
 * The base type, which is a simple type.
 
=== Annotated ===
 * A list of annotations
 * The base type, which is a simple type
 
=== Structural ===
 * The base class types, in linearization order.
 * The declared members.  These are definitions, described above.
 * The inherited members.  These are definitions, described above.
 
=== Existential ===
 * The base type
 * The clause, which is a type parameter section, described previously

=== Polymorphic ===
 * The base type
 * The type parameter section, described previously

= API Data Structure =

This section and explains the data structure representing the API of a source file described in the previous section.

As mentioned in the introduction, the data structure is defined by the 'definition' file in the 'interface' subproject.  Code from the 'datatype' subproject processes this file and generates Java sources in 'src_managed' in the 'interface' subproject.  The explanation includes listings from the definition file.  These listings are current as of November 24, 2009.

The top-level datatype is `Source`.  It represents the API in a single source file.  Packages are listed separately from definitions for two reasons.  First, packages cannot be inferred from definitions because a source file can introduce a package without a top-level definition in that package.  A reasonable alternative would have been including Definitions in Packages.  However, it is anticipated that the most common operation will be to iterate directly over definitions.
{{{
Source
	packages : Package*
	definitions: Definition*
}}}

A package is represented as a simple String.  More structure is not required as explained in the API specification section.
{{{
Package
	name: String
}}}

Definitions are represented by straightforward subclasses of a Definition class.  vals and vars are FieldLike subclasses that differ only by their class name.  The remaining definition variants all have type parameters and so they share a common parent that provides this information.  Class definitions are represented by a single class and the variants (traits, objects, ...) are distinguished by the 'definitionType' field.  The 'selfType' field can be an instance of the special Type EmptyType, which indicates that the self type is not different from the normal self type.
{{{
Definition
	name: String
	access: Access
	modifiers: Modifiers
	annotations: Annotation*
	FieldLike
		tpe : Type
		Val
		Var
	ParameterizedDefinition
		typeParameters: TypeParameter*
		Def
			valueParameters: ParameterList*
			returnType: Type
		ClassLike
			definitionType: DefinitionType
			selfType: Type
			structure: Structure
		TypeMember
			TypeAlias
				tpe: Type
			TypeDeclaration
				lowerBound: Type
				upperBound: Type
}}}

Types are fairly straightforward representations as well.  They are intended to correspond closely to the types defined in the Scala specification.  Note that function, infix, and tuple types translate to parameterized types and type designators translate to type projections.  Compound types and refinements are represented by the Structure class.
{{{
Type
	SimpleType
		Projection
			prefix : SimpleType
			id : String
		ParameterRef
			id: Int
		Singleton
			path: Path
		EmptyType
		Parameterized
			baseType : SimpleType
			typeArguments: Type*
	Annotated
		baseType : SimpleType
		annotations : Annotation*
	Structure
		parents : Type*
		declarations: Definition*
		inherited: Definition*
	Existential
		baseType : Type
		clause: TypeParameter*
	Polymorphic
		baseType : Type
		clause: TypeParameter*
}}}

There is one point to note about the access hierarchy.  It includes a Pkg class, which represents Java's package private access.  This is only used for Java sources.
{{{
Access
	Public
	Qualified
		qualifier: Qualifier
		Protected
		Private
		Pkg
Qualifier
	Unqualified
	ThisQualifier
	IdQualifier
		value: String
}}}

Modifiers are straightforward except that abstract is split into abstract and deferred, according to the Scala compiler's internal representation.  Abstract applies to traits and abstract classes, while deferred applies to class members.
{{{
Modifiers
	isAbstract: Boolean
	isDeferred: Boolean
	isOverride: Boolean
	isFinal: Boolean
	isSealed: Boolean
	isImplicit: Boolean
	isLazy: Boolean
	isSynthetic: Boolean
	isMacro: Boolean
}}}

{{{
ParameterList
	parameters: MethodParameter*
	isImplicit: Boolean
MethodParameter
	name: String
	tpe: Type
	hasDefault: Boolean
	modifier: ParameterModifier

TypeParameter
	id: Int
	typeParameters : TypeParameter*
	variance: Variance
	lowerBound: Type
	upperBound: Type
}}}

The representation of annotations does not express the full information that the compiler has about annotations.  The arguments to an annotation are constant expressions.  This could be represented in this data structure, but this is not currently done.  Rather, the arguments are converted to a String in an unspecified manner.
{{{
Annotation
	base: SimpleType
	arguments: String*
}}}

enum Variance : Contravariant, Covariant, Invariant
enum ParameterModifier : Repeated, Plain, ByName
enum DefinitionType : Trait, ClassDef, Module, PackageModule

Path
	components: PathComponent*

PathComponent
	Super
		qualifier: Path
	This
	Id
		id: String
}}}

= API Compiler Phase =

= Data Structure Generator =

= API Data Structure Persistence =

The data structure is persisted by standard Java serialization on Source.  This is done for simplicity and will be changed to use a more efficient encoding.