Skip to content

Latest commit

 

History

History
219 lines (135 loc) · 5.36 KB

development.rst

File metadata and controls

219 lines (135 loc) · 5.36 KB
Raw

Jedi Development

jedi

Note

This documentation is for Jedi developers who want to improve Jedi itself, but have no idea how Jedi works. If you want to use Jedi for your IDE, look at the plugin api. It is also important to note that it's a pretty old version and some things might not apply anymore.

Introduction

This page tries to address the fundamental demand for documentation of the internals. Understanding a dynamic language is a complex task. Especially because type inference in Python can be a very recursive task. Therefore couldn't get rid of complexity. I know that simple is better than complex, but unfortunately it sometimes requires complex solutions to understand complex systems.

In six chapters I'm trying to describe the internals of :

  • The Jedi Core <core>
  • Core Extensions <core-extensions>
  • Imports & Modules <imports-modules>
  • Stubs & Annotations <stubs>
  • Caching & Recursions <caching-recursions>
  • Helper modules <dev-helpers>

Note

Testing is not documented here, you'll find that right here.

The Jedi Core

The core of Jedi consists of three parts:

  • Parser <parser>
  • Python type inference <inference>
  • API <dev-api>

Most people are probably interested in type inference <inference>, because that's where all the magic happens. I need to introduce the parser <parser> first, because jedi.inference uses it extensively.

Parser

Jedi used to have its internal parser, however this is now a separate project and is called parso.

The parser creates a syntax tree that analyses and tries to understand. The grammar that this parser uses is very similar to the official Python grammar files.

Type inference of python code (inference/__init__.py)

jedi.inference

Inference Values (inference/base_value.py)

jedi.inference.base_value

jedi.inference.value.instance.TreeInstance jedi.inference.value.klass.ClassValue jedi.inference.value.function.FunctionValue jedi.inference.value.function.FunctionExecutionContext

Name resolution (inference/finder.py)

jedi.inference.finder

API (api/__init__.py and api/classes.py)

The API has been designed to be as easy to use as possible. The API documentation can be found here. The API itself contains little code that needs to be mentioned here. Generally I'm trying to be conservative with the API. I'd rather not add new API features if they are not necessary, because it's much harder to deprecate stuff than to add it later.

Core Extensions

Core Extensions is a summary of the following topics:

  • Iterables & Dynamic Arrays <iterables>
  • Dynamic Parameters <dynamic_params>
  • Docstrings <docstrings>
  • Refactoring <refactoring>

These topics are very important to understand what Jedi additionally does, but they could be removed from Jedi and Jedi would still work. But slower and without some features.

Iterables & Dynamic Arrays (inference/value/iterable.py)

To understand Python on a deeper level, needs to understand some of the dynamic features of Python like lists that are filled after creation:

jedi.inference.value.iterable

Parameter completion (inference/dynamic_params.py)

jedi.inference.dynamic_params

Docstrings (inference/docstrings.py)

jedi.inference.docstrings

Refactoring (inference/api/refactoring.py)

jedi.api.refactoring

Imports & Modules

  • Modules <modules>
  • Builtin Modules <builtin>
  • Imports <imports>

Compiled Modules (inference/compiled.py)

jedi.inference.compiled

Imports (inference/imports.py)

jedi.inference.imports

Stubs & Annotations (inference/gradual)

jedi.inference.gradual

Caching & Recursions

  • Caching <cache>
  • Recursions <recursion>

Caching (cache.py)

jedi.cache

Recursions (recursion.py)

jedi.inference.recursion

Helper Modules

Most other modules are not really central to how Jedi works. They all contain relevant code, but you if you understand the modules above, you pretty much understand Jedi.