Skip to content

Commit

Permalink
[docs] update documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
@jauhien
jauhien committed Apr 20, 2015
1 parent 4b58d04 commit e77fc9e
Show file tree
Hide file tree
Showing 9 changed files with 260 additions and 111 deletions.
130 changes: 106 additions & 24 deletions docs/developer_instructions.html
View file
Expand Up @@ -3,13 +3,13 @@
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.10: http://docutils.sourceforge.net/" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>Developer Instructions</title>
<style type="text/css">

/*
:Author: David Goodger (goodger@python.org)
:Id: $Id: html4css1.css 7514 2012-09-14 14:27:12Z milde $
:Id: $Id: html4css1.css 7614 2013-02-21 15:55:51Z milde $
:Copyright: This stylesheet has been placed in the public domain.

Default cascading style sheet for the HTML output of Docutils.
Expand Down Expand Up @@ -313,6 +313,21 @@
white-space: nowrap ;
padding-left: 0 }

/* "booktabs" style (no vertical lines) */
table.docutils.booktabs {
border: 0px;
border-top: 2px solid;
border-bottom: 2px solid;
border-collapse: collapse;
}
table.docutils.booktabs * {
border: 0px;
}
table.docutils.booktabs th {
border-bottom: thin solid;
text-align: left;
}

h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
font-size: 100% }
Expand Down Expand Up @@ -412,7 +427,7 @@ <h1>g-sorcery overview</h1>
</ul>
<p>There are two ways of using backend:</p>
<ul class="simple">
<li>run it as a CLI tool manually</li>
<li>run it as a CLI tool manually (not recommended)</li>
<li>use its integration with layman</li>
</ul>
</div>
Expand Down Expand Up @@ -443,7 +458,7 @@ <h1>Backend structure</h1>
side: generate the entire overlay tree (possibly by layman) or generate a given ebuild
and its dependencies. In a first case it would be very bad idea to have sources in ebuild's
SRC_URI as during manifest generation for an overlay all the sources would be downloaded
to the user's comuter that inevitably would made user really happy. So one ebuild generator
to the user's computer that inevitably would made user really happy. So one ebuild generator
generates ebuild with empty SRC_URI. Note that a mechanism for downloading of sources during
ebuild merging should be provided. For an example see <strong>git-2</strong> eclass from the main tree or
any eclass from backends provided with g-sorcery if you want to implement such a mechanism or
Expand Down Expand Up @@ -513,8 +528,19 @@ <h2>Backend config</h2>
</div>
<div class="section" id="package-database">
<h1>Package database</h1>
<div class="section" id="directory-layout">
<h2>Directory layout</h2>
<p>The package is an in memory structure that describes available
packages and to this structure corresponding files layout.</p>
<div class="section" id="directory-layout-versions">
<h2>Directory layout versions</h2>
<p>There are two directory layouts at the moment:</p>
<ul class="simple">
<li>v.0 legacy layout</li>
<li>v.1 layout that supports different DB structure versions and
different file formats.</li>
</ul>
</div>
<div class="section" id="v-0-legacy-layout">
<h2>v.0 legacy layout</h2>
<p>Package database is a directory tree with JSON files. The layout of this tree looks like:</p>
<pre class="code literal-block">
db dir
Expand All @@ -526,30 +552,74 @@ <h2>Directory layout</h2>
...
</pre>
</div>
<div class="section" id="v-1-layout">
<h2>v.1 layout</h2>
<p>Metadata file contains information about layout and DB versions as
well as information about file format used to store packages
information. At the moment JSON and BSON are supported.</p>
<pre class="code literal-block">
db dir
manifest.json: database manifest
categories.json: information about categories
metadata.json: DB metadata
category1
packages.[b|j]son: information about available packages
category2
...
</pre>
</div>
<div class="section" id="database-structure-versions">
<h2>Database structure versions</h2>
<p>Database structure has two versions: legacy v.0 and v.1. With
directory layout v.0 only DB structure v.0 is supported. DB structure
is internal and shouldn't be relied on by any external tools (including
backends). PackageDB class API should be used instead.</p>
</div>
<div class="section" id="packagedb-class">
<h2>PackageDB class</h2>
<p>PackageDB class is aimed for interaction with package database. It has methods that allow
to add categories and packages and to do queries on them. Usually you do not want to customize this
class. But in case you want there is number of methods that can be redifend.</p>
<p>First of all if you have a database that should be synced with another already generate database
class.</p>
<p>If you have a database that should be synced with another already generate database
you can redifine URI to be used for syncing using <strong>get_real_db_uri</strong> method.</p>
<p>There is a number of hooks that are called after package, category or the whole database is
written/read:</p>
<ul class="simple">
<li>additional_write_version</li>
<li>additional_write_package</li>
<li>additional_write_category</li>
<li>additional_write</li>
<li>additional_read_version</li>
<li>additional_read_package</li>
<li>additional_read_category</li>
<li>additional_read</li>
</ul>
<p>Note that before add any package you should add a category for it using <strong>add_category</strong>.
Then packages can be added using <strong>add_package</strong>. PackageDB currently does not write changes
automatically, so you should call <strong>write_and_manifest</strong> after changes are done. This is not relevant
automatically, so you should call <strong>write</strong> after changes are done. This is not relevant
for database changing in <strong>process_data</strong> method of database generator as there all changes
are written by other methods it calls internally after <strong>process_data</strong>.</p>
are written by other methods it calls internally after
<strong>process_data</strong>.</p>
<p>If you have some fields that are common to all ebuilds in a given
category, it's better to split them to common data, that can be set for
category. This data will be added to ebuild data in results of package
queries automatically.</p>
<p>Public API that should be used for manipulating packages data:</p>
<ul class="simple">
<li>add_category(self, category, description=None) -- add new category.</li>
<li>set_common_data(self, category, common_data) -- set common ebuild
data for a category.</li>
<li>get_common_data(self, category) -- get common ebuild data for a
category.</li>
<li>add_package(self, package, ebuild_data=None) -- add new packages
(characterized by category, package name and version) with given
ebuild data.</li>
<li>list_categories(self) -- list categories.</li>
<li>in_category(self, category, name) -- test whether a package is in a
given category.</li>
<li>list_package_names(self, category) -- list package names in a
category.</li>
<li>list_catpkg_names(self) -- list category/package name.</li>
<li>list_package_versions(self, category, name) -- list package
versions.</li>
<li>list_all_packages(self) -- list all packages.</li>
<li>get_package_description(self, package) -- get ebuild data (it
returns a dict that contains both ebuild data for a given package
and fields from common data for a given category).</li>
<li>get_max_version(self, category, name) -- get the recent available
version of a package.</li>
<li>iterator -- PackageDB class defines an iterator that iterates
through all available package/ebuild data pairs.</li>
</ul>
<p>To see description of these methods look in g_sorcery/package_db.py file.</p>
</div>
<div class="section" id="json-serializable-objects">
<h2>JSON serializable objects</h2>
Expand All @@ -561,6 +631,7 @@ <h2>JSON serializable objects</h2>
<li>class method <strong>deserialize</strong> that takes a value returned by <strong>serialize</strong> and constructs new instance
of your class using it</li>
</ul>
<p>This holds true for other supported file formats (BSON at the moment).</p>
</div>
<div class="section" id="dependency-handling">
<h2>Dependency handling</h2>
Expand Down Expand Up @@ -633,6 +704,15 @@ <h2>Customizing DBGenerator</h2>
</li>
</ul>
<p>Generally speaking these are all the method you should implement.</p>
<p>Both PackageDB and DBGenerator constructors accept these fields that
are used to control preferred DB version/layout and file format (used
during writing DB to disk):</p>
<ul class="simple">
<li>preferred_layout_version, 1 by default</li>
<li>preferred_db_version, 1 by default</li>
<li>preferred_category_format, json by default</li>
</ul>
<p>To see how to use them look at the gs-pypi backend.</p>
</div>
<div class="section" id="value-convertion">
<h2>Value convertion</h2>
Expand Down Expand Up @@ -943,8 +1023,10 @@ <h1>Summary</h1>
<p>Before doing it you should have defined classes you pass to it as parameters. They should be database
generator, two ebuild generators, eclass and metadata generators.</p>
<p>Also you should write an executable that calls g-sorcery and some configs.</p>
<p>To have better understanding you can look at gs-elpa and gs-pypi backends available
in g-sorcery repository. Also available tests could be usefull.</p>
<p>To have better understanding you can look at
gs-elpa (<a class="reference external" href="https://github.com/jauhien/gs-elpa">https://github.com/jauhien/gs-elpa</a>) and gs-pypi
(<a class="reference external" href="https://github.com/jauhien/gs-pypi">https://github.com/jauhien/gs-pypi</a>) backends. Also available tests
could be usefull.</p>
<p>Note that there is a tool for editing generated database named <strong>gs-db-tool</strong>.</p>
</div>
</div>
Expand Down

0 comments on commit e77fc9e

Please sign in to comment.