Something I wrote that you might be interested in.

[kdschlosser]

I was poking about in your commit history to see what changes you have made in the past few months. I didn't know if you had added the multi threaded progress bar or not.. But I had noticed a bunch of PR's for sphinx and thought this may be of use to you.

https://github.com/kdschlosser/sphinx-distutils-extension

It allows for the setup program to handle building the documentation. It also builds the config file as well. All options for the config file are provided as well as all of the sphinx command line arguments are extended into the build class. It is very simple to use it is also cross platform and does not require anything else to be installed to run it (make for example). Works out of the box with all of the various CI utilities and you can also use it in a chained command...

python setup.py install build_docs

this will install your program as well as build the documentation. There are code examples of how to install any of the extensions as well as sphinx using setup_requires. this is a nice way to go about it so that any modules needed by the doc builder do not get installed into the user site-packages but get placed in a temporary directory ".eggs" where the setup.py file is located. It is not not to install onetime use modules into the users python interpreter

[wolph]

At the moment everything is working good enough but it's definitely a good idea for future projects and the backlog :)

The building of the Sphinx config file has always bothered me honestly. The current hack I have with the __about__.py feels horrible and leaves a lot of room for error.

As soon as I get a bunch of time again (most likely not this year though) I'll look into converting my existing projects: https://pypi.org/user/WoLpH/

[kdschlosser]

I wanted to point it out to ya because I thought you might feel the same as I as far as sphinx goes and having to use make to build the python documentation just doesn't seem right. That is why I made the thing. and I wanted something that was pretty much plug and play and was very easily added to an existing project. it is pretty much a copy and paste thing. open your config file and copy then copy and paste the settings into your setup file. it really cannot get any easier then that. it handles the mundane things like author and project name, copyright internally but can be overridden

The best part about it is because of how i made the thing you can have it install anything related to the building of the documentation using setup_requires which does not install anything into the site-packages folder. everything is stashed into a temp folder instead. so no installing packages for what could be a one time use.

It is there if you decide to use it on exiting or future projects. I also have examples of how to add it to setup_requires.

[kdschlosser]

might consider merging this project Sphinx-PyPI-upload-2 0.2.2 with the sphinx builder possibly. I can see the value in Sphinx-PyPI-upload-2 0.2.2. sphinx does already have a builder added to distutils. but it is very minimal. I am going to at some point try to figure out how to get around writing the config file. I am going to look into that now actually.

[kdschlosser]

OK so I think I have removed writing the config file. I still have to test it but i did a commit anywho.

I never liked writing that file I knew there had to be a way to inject it into Sphinx.. unfortunately there was no direct way without sub classing the builder it's self. I ended up creating a property for the config file that simply ignores the empty one that sphinx creates and creates the one we want to use.. kind of crafty actually. This is because of how much of the setup of the builder is done in __init__... no way to override it without having to copy a bunch of code from it. and I wanted to avoid doing that. so the property is what I came up with.

[kdschlosser]

the only time the file gets written is if you are using the ConfigOptions class and pecify the makefile argument. this is because the config file MUST be there because the build is being executed by make.

[wolph]

It sounds like a huge improvement already. Definitely when comparing it to my current approach of copying the docs/conf.py and a bunch of other files between projects and modifying them to be "correct".

Very nice work indeed!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[kdschlosser]

@wolph

I have another "toy" for you... you may want to try it.. It should help you greatly if you get into dealing with threads. and making sure everything is thread safe https://github.com/kdschlosser/angry_debugger, it's an in depth logging routine. it is just a decorator.. but it works with methods, functions, nested functions. it also work with properties and you can individually put the decorator on set, get and delete. it can also be used for class attributes and instance variables for getting and setting.

It provides detailed information like the file and line number where the call was made from. the file and line number where the decorator is.. any arguements passed including the keyword names and default values (if any) it also displays any returned data. and you can time the execution of a function...

you can log a complete data path through a program on a per thread basis and it supports multiple threads doing this at the same time. it keeps the output grouped together for that thread.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[wolph]

It's been a long time but I've taken a look at it. I'm really liking the project in general but one thing that's holding me back is the way it's currently set up with being a requirement for the setup.py. I currently have the sphinx requirement set up as an extra (extras_require) which is something I would like to keep. No need for the setup to require a documentation building package :)

Also, is it correct that you haven't pushed it to PyPI yet?

Love the angry debugger btw. I've written so many similar functions in the past but it's so useful having something like that as a package. I've already used it several times while debugging asyncio code using multiple threads, quite useful :)

[kdschlosser]

Nothing has been pushed to pypi.

Here is an example of how to use the sphinx-distutils package I wrote.

I make use of setup_requires to install any one time use packages, like Sphinx and related components. using the setup_requires downloads and installs the packages into the same folder that the setup file is located. It does not install the packages into the users site-packages. This is a nice thing because it keeps a users site-packages nice and tidy. and if the setup program is getting run from pip then the packages will be deleted when it deletes the temporary directory.

Here is the example code. There are a couple of other little ding dongs and ho ho's in there along with how to use the document builder. Just some things I like to use in my setup programs.

# -*- coding: utf-8 -*-

__project__ = 'project'
__author__ = 'author'
__author_email__ = 'some@email.com'
__version__ = '0.0.0'
__release__ = 'alpha2'
__url__ = 'http://some_url'
__description__ = 'short description'
__long_description__ = 'long description'
__download_url__ = 'http://some_download_url'
__github_user__ = 'some_user'
__github_repo__ = 'some_repo'



from __future__ import print_function
import os
import sys

# added a command line argument that allows for setting
# distutils into a CRAZY debug output.
if '--DISTUTILS_DEBUG' in sys.argv:
    os.environ['DISTUTILS_DEBUG'] = '1'
    sys.argv.remove('--DISTUTILS_DEBUG')


# This is a monkey patch that corrects file copying output.
# this is normally always shown which is something I hated.
# with this patch it now only gets shown when the --verbose
# command line switch is used. This is how it should have
# been made.

_verbose = '--verbose' in sys.argv

import distutils.file_util # NOQA

_copy_file = distutils.file_util.copy_file


def copy_file(
    src,
    dst,
    preserve_mode=1,
    preserve_times=1,
    update=0,
    link=None,
    verbose=1,
    dry_run=0
):
    return _copy_file(
        src,
        dst,
        preserve_mode=preserve_mode,
        preserve_times=preserve_times,
        update=update,
        link=link,
        verbose=int(_verbose),
        dry_run=dry_run
    )


distutils.file_util.copy_file = copy_file

from setuptools import setup, find_packages # NOQA


# any additional lines of code that you want to have sphinx run.
# this is the same as adding code into the config file.

sphinx_extra_lines = '''
def setup(app):
    app.add_stylesheet('css/some_stylesheet.css')
'''

if 'build_docs' in sys.argv:
    # ----------------------------------------------------------------------
    # this will trick distutils/setuptools into downloading and installing
    # the documentation builder. we need to have this downloaded and installed
    # before we use it. all other sphinx related components will get downloaded
    # and installed by the "real" build program
    
    
    # This is a little trick to get setuptools to use github instead of pypi
    dependency_links = [
        'https://github.com/kdschlosser/sphinx-distutils-extension/tarball'
        '/master#egg=sphinx_distutils',
    ]

    setup_requires = ['sphinx_distutils']

    from distutils.command import build as _build

    class FakeBuild(_build.build):

        def run(self):
            return

    setup(
        script_args=['build'],
        name=__project__,
        author=__author__,
        author_email=__author_email__,
        version=__version__,
        setup_requires=setup_requires,
        dependency_links=dependency_links,
        zip_safe=False,
        url=__url__,
        cmdclass=dict(
            build=FakeBuild
        ),
        description=__description__,
        long_description=__long_description__,
        download_url=__download_url__
    )

    import sphinx_distutils

    cmdclass = dict(
        build_docs=sphinx_distutils.build_docs
    )

    sphinx_conf = sphinx_distutils.ConfigOptions()
    sphinx_distutils.build_docs.sphinx_conf = sphinx_conf

    # -- GENERAL options ------------------------------------------------------
    sphinx_conf.suppress_warnings = ['image.nonlocal_uri']
    sphinx_conf.extensions = [
        'sphinx.ext.autodoc',
        'sphinx.ext.todo',
        'sphinx.ext.doctest',
        'sphinx.ext.viewcode',
        'sphinx.ext.autosummary',
        'sphinxcontrib.blockdiag',
        'sphinxcontrib.nwdiag',
        'sphinxcontrib.actdiag',
        'sphinxcontrib.seqdiag',
        'sphinxcontrib.fulltoc',
        'sphinx_sitemap',
        'sphinx.ext.graphviz',
        'sphinx.ext.inheritance_diagram'
    ]

    if sys.platform.startswith('win'):
        sphinx_conf.blockdiag_fontpath = os.path.join(
            os.path.expandvars('%WINDIR%'),
            'Fonts',
            'Arial.ttf'
        )
    else:
        sphinx_conf.blockdiag_fontpath = (
            '/usr/share/fonts/truetype/ubuntu-font-family/Ubuntu-B.ttf'
        )
    sphinx_conf.templates_path = ['_templates']
    sphinx_conf.source_suffix = '.rst'
    sphinx_conf.master_doc = 'index'
    sphinx_conf.version = __version__
    sphinx_conf.release = __release__
    sphinx_conf.exclude_patterns = ['_build']
    sphinx_conf.add_function_parentheses = True
    sphinx_conf.show_authors = True
    sphinx_conf.pygments_style = 'sphinx'
    sphinx_conf.add_module_names = True

    # -- Options for HTML output ----------------------------------------------
    sphinx_conf.html_baseurl = '/docs/'
    sphinx_conf.html_theme = 'groundwork'
    sphinx_conf.html_theme_options = {
        "sidebar_width": '240px',
        "stickysidebar": True,
        "stickysidebarscrollable": True,
        "contribute": True,
        "github_fork": __github_user__ + '/' + __github_repo__,
        "github_user": __github_user__,
    }

    sphinx_conf.html_theme_path = ["_themes"]
    sphinx_conf.html_title = __project__
    sphinx_conf.html_logo = '_static/images/logo.png'
    sphinx_conf.html_static_path = ['_static']
    sphinx_conf.html_domain_indices = True
    sphinx_conf.html_use_index = True
    sphinx_conf.html_split_index = False
    sphinx_conf.html_show_sourcelink = True
    sphinx_conf.html_sourcelink_suffix = '.py'
    sphinx_conf.html_experimental_html5_writer = True
    sphinx_conf.html_show_sphinx = False
    sphinx_conf.html_copy_source = True
    sphinx_conf.html_show_copyright = True
    sphinx_conf.htmlhelp_basename = __project__
    sphinx_conf.html_sidebars = {
        '**': [
            'globaltoc.html',
            'sourcelink.html',
            'searchbox.html'
        ],
        'using/windows': [
            'windowssidebar.html',
            'searchbox.html'
        ],
    }

    # -- Options for LATEX output ---------------------------------------------
    sphinx_conf.latex_elements = {}
    sphinx_conf.latex_documents = [
      (
          'index',
          __project__ + '.tex',
          __project__ + ' Documentation',
          __author__,
          'manual'
      )
    ]

    # -- Options for MAN PAGES output -----------------------------------------
    sphinx_conf.man_pages = [
        (
            'index',
            __project__,
            __project__ + ' Documentation',
            [__author__],
            1
        )
    ]

    # -- Options for TEXT INFO output -----------------------------------------
    sphinx_conf.texinfo_documents = [
        (
            'index',
            __project__,
            __project__ + ' Documentation',
            __author__,
            __project__,
            __description__,
            'Miscellaneous'
        ),
    ]

    sphinx_conf.additional_lines = sphinx_extra_lines

    setup_requires = [
        'Sphinx==1.8.0', #
        'groundwork-sphinx-theme', #
        'sphinx-sitemap>=1.0.2', #
        'sphinxcontrib-blockdiag>=1.5.5', #
        'sphinxcontrib-nwdiag>=0.9.5', #
        'sphinxcontrib-actdiag>=0.8.5', #
        'sphinxcontrib-seqdiag>=0.8.5', #
        'sphinxcontrib-fulltoc>=1.2.0', #
    ]

    dependency_links = [
        'https://github.com/kdschlosser/groundwork-sphinx-theme/tarball'
        '/sphinx_2.0_support#egg=groundwork_sphinx_theme',
    ]

else:
    setup_requires = []
    dependency_links = []
    cmdclass = {}

setup(
    name=__project__,
    author=__author__,
    author_email=__author_email__,
    version=__version__,
    setup_requires=setup_requires,
    dependency_links=dependency_links,
    zip_safe=False,
    url=__url__,
    cmdclass=cmdclass,
    description=__description__,
    long_description=__long_description__,
    download_url=__download_url__
)

[kdschlosser]

as far as that angry debugger program.. I do have to say that I did a pretty good job with that thing. It has been a HUGE help for me personally. when you are debugging a program that has a bunch of threads it is a bear to track the data path if you use the builtin logging module. I basically extended the logging module so it provides a heap more information and will keep the logging output in order by thread.

I have another whizbang package I wrote for handling deprecated items.
It works on methods, functions. properties and class level attributes.
It also supports nested functions inside of methods or functions or properties.
You can deprecate each of the property types individually (fset, fget, fdel)
The class level attributes will only produce a warning if the attribute gets changed or has it's data collected.

custom message can also be supplied

[wolph]

Unfortunately with the current direction if distutils and setuptools it seems that any building through the setup.py has become highly discouraged and largely broken...

If I'm going to update and rewrite everything, it will be a rewrite to a setup.cfg or pyproject.toml file and being fully PEP 517 compliant.

[kdschlosser]

Trust you me I know all about the broken crap in distutils and in setup tools. The broken MSVC compiler detection for Python 3.10 is biting a lot of people. I wrote a solution for it and I also dropped an issue in setuptools bug tracker only for it to fall upon deaf ears

[wolph]

Similarly... try and release a package with cython support these days. That's a minefield of deprecated and hacky stuff to get working :(

[kdschlosser]

If you use my MSVC script when compiling on Windows Cython and also distutils works properly to compile c extension modules.

What package are you having problems with that uses Cython?

[wolph]

It's not that I'm having problems now, but it took me a while to figure out how to get this package to build reliably: https://github.com/WoLpH/speedups/

[kdschlosser]

I am going to open an issue in that repository.