Merge pull request #272 from lzpap/docs_addresses

docs: improve `Generating Addresses` page

docs/addresses.rst

@@ -5,9 +5,10 @@ In IOTA, addresses are generated deterministically from seeds. This
 ensures that your account can be accessed from any location, as long as
 you have the seed.
 
-Note that this also means that anyone with access to your seed can spend
-your IOTAs! Treat your seed(s) the same as you would the password for
-any other financial service.
+.. warning::
+  Note that this also means that anyone with access to your seed can spend
+  your iotas! Treat your seed(s) the same as you would the password for
+  any other financial service.
 
 .. note::
 
@@ -22,6 +23,41 @@ any other financial service.
 
         To install the extension, run ``pip install pyota[ccurl]``.
 
+Algorithm
+---------
+
+.. figure:: images/address_gen.svg
+   :scale: 100 %
+   :alt: Process of address generation in IOTA.
+
+   Deriving addresses from a seed.
+
+The following process takes place when you generate addresses in IOTA:
+
+1. First, a sub-seed is derived from your seed by adding ``index`` to it,
+   and hashing it once with the `Kerl`_ hash function.
+2. Then the sub-seed is absorbed and squeezed in a `sponge function`_ 27 times
+   for each security level. The result is a private key that varies in length
+   depending on security level.
+
+   .. note::
+     A private key with ``security_level = 1`` consists of 2187 trytes, which is
+     exactly 27 x 81 trytes. As the security level increases, so does the length
+     of the private key: 2 x 2187 trytes for ``security_level = 2``, and 3 x 2187
+     trytes for ``security_level = 3``.
+
+3. A private key is split into 81-tryte segments, and these segments are hashed
+   26 times. A group of 27 hashed segments is called a key fragment. Observe,
+   that a private key has one key fragment for each security level.
+4. Each key fragment is hashed once more to generate key digests, that are
+   combined and hashed once more to get the 81-tryte address.
+
+   .. note::
+     An address is the public key pair of the corresponding private key. When
+     you spend iotas from an address, you need to sign the transaction with a
+     key digest that was generated from the address's corresponing private key.
+     This way you prove that you own the funds on that address.
+
 PyOTA provides two methods for generating addresses:
 
 Using the API
@@ -35,6 +71,7 @@ Using the API
 
     # Generate 5 addresses, starting with index 0.
     gna_result = api.get_new_addresses(count=5)
+    # Result is a dict that contains a list of addresses.
     addresses = gna_result['addresses']
 
     # Generate 1 address, starting with index 42:
@@ -45,26 +82,55 @@ Using the API
     gna_result = api.get_new_addresses(index=86, count=None)
     addresses = gna_result['addresses']
 
-To generate addresses using the API, invoke its ``get_new_addresses``
+To generate addresses using the API, invoke its :py:meth:`iota.Iota.get_new_addresses`
 method, using the following parameters:
 
 -  ``index: int``: The starting index (defaults to 0). This can be used
    to skip over addresses that have already been generated.
 -  ``count: Optional[int]``: The number of addresses to generate
    (defaults to 1).
--  If ``None``, the API will generate addresses until it finds one that
-   has not been used (has no transactions associated with it on the
-   Tangle, and was not spent from). This makes the command safer to use after
-   a snapshot has been taken. It will then return the unused address and
-   discard the rest.
+
+    - If ``None``, the API will generate addresses until it finds one that
+      has not been used (has no transactions associated with it on the
+      Tangle and was never spent from). It will then return the unused address
+      and discard the rest.
 -  ``security_level: int``: Determines the security level of the
    generated addresses. See `Security Levels <#security-levels>`__
    below.
 
-``get_new_addresses`` returns a dict with the following items:
+Depending on the ``count`` parameter, :py:meth:`Iota.get_new_addresses` can be
+operated in two modes.
+
+Offline mode
+~~~~~~~~~~~~
 
--  ``addresses: List[Address]``: The generated address(es). Note that
-   this value is always a list, even if only one address was generated.
+  When ``count`` is greater than 0, the API generates ``count`` number of
+  addresses starting from ``index``. It does not check the Tangle if
+  addresses were used or spent from before.
+
+Online mode
+~~~~~~~~~~~
+
+  When ``count`` is ``None``, the API starts generating addresses starting
+  from ``index``. Then, for each generated address, it checks the Tangle
+  if the address has any transactions associated with it, or if the address
+  was ever spent from. If both of the former checks return "no", address
+  generation stops and the address is returned (a new address is found).
+
+.. warning::
+    Take care when using the online mode after a snapshot. Transactions referencing
+    a generated address may have been pruned from a node's ledger, therefore the
+    API could return an already-used address as "new" (note: The snapshot has
+    no effect on the "was ever spent from" check).
+
+    To make your application more robust to handle snapshots, it is recommended
+    that you keep a local database with at least the indices of your used addresses.
+    After a snapshot, you could specify ``index`` parameter as the last
+    index in your local used addresses database, and keep on generating truly
+    new addresses.
+
+    PyOTA is planned to receive the `account module`_ in the future, that makes
+    the library stateful and hence would solve the issue mentioned above.
 
 Using AddressGenerator
 ----------------------
@@ -87,7 +153,7 @@ Using AddressGenerator
       ...
 
 If you want more control over how addresses are generated, you can use
-the ``AddressGenerator`` class.
+:py:class:`iota.crypto.addresses.AddressGenerator`.
 
 ``AddressGenerator`` can create iterators, allowing your application to
 generate addresses as needed, instead of having to generate lots of
@@ -97,17 +163,23 @@ You can also specify an optional ``step`` parameter, which allows you to
 skip over multiple addresses between iterations... or even iterate over
 addresses in reverse order!
 
-``AddressGenerator`` provides two methods:
+AddressGenerator
+~~~~~~~~~~~~~~~~
 
--  ``get_addresses: (int, int, int) -> List[Address]``: Returns a list
-   of addresses. This is the same method that the ``get_new_addresses``
-   API command uses internally.
--  ``create_iterator: (int, int) -> Generator[Address]``: Returns an
-   iterator that will create addresses endlessly. Use this if you have a
-   feature that needs to generate addresses "on demand".
+.. autoclass:: iota.crypto.addresses.AddressGenerator
+
+**get_addresses**
+^^^^^^^^^^^^^^^^^
+
+.. automethod:: iota.crypto.addresses.AddressGenerator.get_addresses
+
+**create_iterator**
+^^^^^^^^^^^^^^^^^^^
+
+.. automethod:: iota.crypto.addresses.AddressGenerator.create_iterator
 
 Security Levels
-===============
+---------------
 
 .. code:: python
 
@@ -120,8 +192,10 @@ Security Levels
       )
 
 If desired, you may change the number of iterations that
-``AddressGenerator`` uses internally when generating new addresses, by
-specifying a different ``security_level`` when creating a new instance.
+:py:class:`iota.crypto.addresses.AddressGenerator` or
+:py:class:`iota.Iota.get_new_addresses` uses internally when generating new
+addresses, by specifying a different ``security_level`` when creating a new
+instance.
 
 ``security_level`` should be between 1 and 3, inclusive. Values outside
 this range are not supported by the IOTA protocol.
@@ -134,3 +208,7 @@ Use the following guide when deciding which security level to use:
    security.
 -  ``security_level=3``: Most secure; results in longer signatures in
    transactions.
+
+.. _Kerl: https://github.com/iotaledger/kerl
+.. _sponge function: https://keccak.team/sponge_duplex.html
+.. _account module: https://docs.iota.org/docs/client-libraries/0.1/account-module/introduction/overview

docs/index.rst

@@ -6,10 +6,10 @@
    basic_concepts
    types
    adapters
-   addresses
    api
    core_api
    extended_api
+   addresses
    multisig
 
 .. include:: ../README.rst

iota/api.py

@@ -1217,9 +1217,18 @@ def get_new_addresses(
                 inside a loop.
 
             If ``None``, this method will progressively generate
-            addresses and scan the Tangle until it finds one that is unused.
-            This is if no transactions are referencing it and it was not spent
-            from before.
+            addresses and scan the Tangle until it finds one that has no
+            transactions referencing it and was never spent from.
+
+            .. note::
+                A snapshot removes transactions from the Tangle. As a
+                consequence, after a snapshot, it may happen that when ``count``
+                is ``None``, this API call returns a "new" address that used to
+                have transactions before the snapshot.
+                As a workaround, you can save your used addresses and their
+                ``key_index`` attribute in a local database. Use the
+                ``index`` parameter to tell the API from where to start
+                generating and checking new addresses.
 
         :param int security_level:
             Number of iterations to use when generating new addresses.

iota/crypto/addresses.py

@@ -19,13 +19,14 @@ class AddressGenerator(Iterable[Address]):
     """
     Generates new addresses using a standard algorithm.
 
-    Note: This class does not check if addresses have already been used;
-    if you want to exclude used addresses, invoke
-    :py:meth:`iota.api.IotaApi.get_new_addresses` instead.
+    .. note::
+        This class does not check if addresses have already been used;
+        if you want to exclude used addresses, invoke
+        :py:meth:`iota.Iota.get_new_addresses` instead.
 
-    Note also that :py:meth:`iota.api.IotaApi.get_new_addresses` uses
-    ``AddressGenerator`` internally, so you get the best of both worlds
-    when you use the API (:
+        Note also that :py:meth:`iota.Iota.get_new_addresses` uses
+        ``AddressGenerator`` internally, so you get the best of both worlds
+        when you use the API (:
 
     :param TrytesCompatible seed:
         The seed to derive addresses from.
@@ -44,6 +45,8 @@ class AddressGenerator(Iterable[Address]):
 
     :param bool checksum:
         Whether to generate address with or without checksum.
+
+    :returns: :py:class:`iota.crypto.addresses.AddressGenerator` object.
     """
     DEFAULT_SECURITY_LEVEL = 2
     """
@@ -91,19 +94,19 @@ def get_addresses(self, start, count=1, step=1):
         :py:meth:`create_iterator` and sharing the resulting generator
         object instead.
 
-        Warning: This method may take awhile to run if the starting
-        index and/or the number of requested addresses is a large
-        number!
+        .. warning::
+            This method may take awhile to run if the starting
+            index and/or the number of requested addresses is a large number!
 
-        :param start:
+        :param int start:
             Starting index.
             Must be >= 0.
 
-        :param count:
+        :param int count:
             Number of addresses to generate.
             Must be > 0.
 
-        :param step:
+        :param int step:
             Number of indexes to advance after each address.
             This may be any non-zero (positive or negative) integer.
 
@@ -115,6 +118,10 @@ def get_addresses(self, start, count=1, step=1):
             The returned list will contain ``count`` addresses, except
             when ``step * count < start`` (only applies when ``step`` is
             negative).
+
+        :raises ValueError:
+            - if ``count`` is lower than 1.
+            - if ``step`` is zero.
         """
         if count < 1:
             raise with_context(
@@ -157,17 +164,27 @@ def create_iterator(self, start=0, step=1):
         Creates an iterator that can be used to progressively generate new
         addresses.
 
-        :param start:
+        Returns an iterator that will create addresses endlessly.
+        Use this if you have a feature that needs to generate addresses
+        “on demand”.
+
+        :param int start:
             Starting index.
 
-            Warning: This method may take awhile to reset if ``start``
-            is a large number!
+            .. warning::
+                This method may take awhile to reset if ``start`` is a large
+                number!
 
-        :param step:
+        :param int step:
             Number of indexes to advance after each address.
 
-            Warning: The generator may take awhile to advance between
-            iterations if ``step`` is a large number!
+            .. warning::
+                The generator may take awhile to advance between
+                iterations if ``step`` is a large number!
+
+        :return:
+            ``Generator[Address, None, None]`` object that you can iterate to
+            generate addresses.
         """
         key_iterator = (
             KeyGenerator(self.seed).create_iterator(