Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optional TOML configs support #2457

Merged
merged 27 commits into from Jul 28, 2018

Conversation

Projects
None yet
5 participants
@orsinium
Copy link
Contributor

orsinium commented Jul 13, 2018

Description

  • Added parser for TOML configs.
  • TOML config parser can read files from luigi/base.toml and luigi/local.toml. This is useful for effective secrets management via VCS.
  • You can select config parser via LUIGI_CONFIG_PARSER env var.

Motivation and Context

  1. TOML standardized, modern format which supports data types.
  2. It is useful setup common default options in config files, not in tasks.
  3. Now we can write new config parsers for other formats like YAML without pain. Luigi has been more flexible and extendable.

Have you tested this? If so, how?

  1. I have included unit tests.
  2. I ran my jobs with this code and it works for me.
@dlstadther
Copy link
Collaborator

dlstadther left a comment

@orsinium , Thanks for your submission!

I like the idea here (enabling other styles of configuration formats), but I'm not yet sold on why we'd want other formats.

Do other configuration formats allow features that ConfigParser doesn't? If so, what are they and how are they applicable to luigi?

(I anticipated your clarification to my concern above and I've gone ahead and reviewed your code. Those comments and questions are below.)

* ``/etc/luigi/luigi.toml``
* ``luigi.toml``
* ``luigi/base.toml``
* ``luigi/local.toml``

This comment has been minimized.

@dlstadther

dlstadther Jul 13, 2018

Collaborator

Could you explain why base.toml and local.toml are proposed as valid configuration file names?

This comment has been minimized.

@orsinium

orsinium Jul 14, 2018

Author Contributor

Because implicit better than explicit. Now we can use explicit inheritance via luigi.cfg and LUIGI_CONFIG_PATH=luigi_local.cfg. Motivation of inheritance I describe in Pull Request: this is useful for configuring default settings, that can be overridden inlocal config.

This comment has been minimized.

@Tarrasch

Tarrasch Jul 15, 2018

Collaborator

You mean that it uses local first and then base. Which makes sense so you can have local changes that don't conflict with the base config.

This comment has been minimized.

@orsinium

orsinium Jul 16, 2018

Author Contributor

I'm removing base and local configs. This need be discussed in future for better realization.

@@ -23,6 +41,17 @@ The config file is broken into sections, each controlling a different part of th
[core]
scheduler_host=luigi-host.mycompany.foo

Example toml config:

.. code:: python

This comment has been minimized.

@dlstadther

dlstadther Jul 13, 2018

Collaborator

why is python used here for code syntax highlighting?

This comment has been minimized.

@orsinium

orsinium Jul 14, 2018

Author Contributor

Because Pygments doesn't have available lexerfor TOML. This is already contributed in some Pygments versions:

But Pygments which used for travis checks doesn't have this lexer yet. I choose python as alternative lexer because toml syntax very similar to hybrid of python and ini :)

This comment has been minimized.

@Tarrasch

Tarrasch Jul 15, 2018

Collaborator

And a TODO then please. :)

streaming-jar = "/usr/lib/hadoop-xyz/hadoop-streaming-xyz-123.jar"
[core]
scheduler_host = "luigi-host.mycompany.foo"

This comment has been minimized.

@dlstadther

dlstadther Jul 13, 2018

Collaborator

are single quotes acceptable too?

This comment has been minimized.

@orsinium

orsinium Jul 14, 2018

Author Contributor

Yes, see TOML spec for more details about syntax

This comment has been minimized.

@Tarrasch

Tarrasch Jul 15, 2018

Collaborator

Maybe link to the spec in the docs too?

@@ -0,0 +1,22 @@
# -*- coding: utf-8 -*-
#
# Copyright 2012-2015 Spotify AB

This comment has been minimized.

@dlstadther

dlstadther Jul 13, 2018

Collaborator

Shouldn't these dates include 2018. Probably not that important

This comment has been minimized.

@orsinium

orsinium Jul 14, 2018

Author Contributor

I just copied Apache license notice from original configuration.py. You can improve it, I don't now how it must be look in right way.

This comment has been minimized.

@Tarrasch

Tarrasch Jul 15, 2018

Collaborator

Yea I think this is fine ...

if cls._instance is None:
cls._instance = cls(*args, **kwargs)
loaded = cls._instance.reload()
logging.getLogger('luigi-interface').info('Loaded %r', loaded)

This comment has been minimized.

@dlstadther

dlstadther Jul 13, 2018

Collaborator

side note: am i correct that neither the client nor the server's logging config names are configurable? I've found that interesting - why statically set to luigi-interface. Perhaps we don't want that changed in general.

This comment has been minimized.

@orsinium

orsinium Jul 14, 2018

Author Contributor

I don't change .instance() logic, just moved it from cfg parser to base.

This comment has been minimized.

@dlstadther

dlstadther Jul 23, 2018

Collaborator

Yeah, I was just raising an unrelated question - a topic for future possible PR (configurable logging names).

Nothing for you to do here.

return cls._instance

@classmethod
def add_config_path(cls, path):

This comment has been minimized.

@dlstadther

dlstadther Jul 13, 2018

Collaborator

Any reason for changing the order of these classmethods?

This comment has been minimized.

@orsinium

orsinium Jul 14, 2018

Author Contributor

What did you mean? I just moved this method from cfg parser to base without any changes.


class BaseParser:
_instance = None
_config_paths = []

This comment has been minimized.

@dlstadther

dlstadther Jul 13, 2018

Collaborator

Shouldn't this be an abstract property that is required to be overwritten?

This comment has been minimized.

@orsinium

orsinium Jul 14, 2018

Author Contributor

Yeah, this can be confusable if somebody try to use different parsers in one project. I fix it. Thank you for review :)

# limitations under the License.
#
import os
import os.path

This comment has been minimized.

@dlstadther

dlstadther Jul 13, 2018

Collaborator

why import both os and os.path? import os.path can be removed

This comment has been minimized.

@orsinium

orsinium Jul 14, 2018

Author Contributor

Nearly. import os can be removed, not os.path. Fixed. Thank you.

}


def get_config(parser=os.environ.get('LUIGI_CONFIG_PARSER', 'cfg')):

This comment has been minimized.

@dlstadther

dlstadther Jul 13, 2018

Collaborator

Am i correct that if you want to use a config parser other than cfg, you have to define the LUIGI_CONFIG_PARSER environmental variable?

This comment has been minimized.

@orsinium

orsinium Jul 14, 2018

Author Contributor

Yes, you're right. I plan to contribute ability to set up config parser as argument for worker's run. But now Luigi supports config parser chainging only via env var. I write it into documentation.

setup.py Outdated
@@ -39,6 +39,7 @@ def get_static_files(path):
install_requires = [
'tornado>=4.0,<5',
'python-daemon<3.0',
'toml',

This comment has been minimized.

@dlstadther

dlstadther Jul 13, 2018

Collaborator

No version specification?

This comment has been minimized.

@orsinium

orsinium Jul 14, 2018

Author Contributor

Right, I think, we don't need specify library version. This library exposes an API familiar to users of the standard library json, marshal and pickle, and authors will not change it in future.

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Jul 14, 2018

Yes, TOML, opposite to ini, supports data types like integer, float, boolean, list, dict. Ini supports only strings.

In future, I plan to contribute pyproject.toml support, when tools like pip and setuptools will migrate on it. Also you can read in this article other benefits of migration from ini to toml.

Thank you for good question :)

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Jul 14, 2018

By the way, thank you for useful library, guys. Luigi is save many hours for our team :)

@Tarrasch
Copy link
Collaborator

Tarrasch left a comment

Looks like a nice contribution. How long have you used this in prod?


in increasing order of preference. The order only matters in case of key conflicts (see docs for ConfigParser.read_). These files are meant for both the client and ``luigid``. If you decide to specify your own configuration you should make sure that both the client and ``luigid`` load it properly.
You can choose right parser via ``LUIGI_CONFIG_PARSER`` environment variable. For example, ``LUIGI_CONFIG_PARSER=toml``.

This comment has been minimized.

@Tarrasch

Tarrasch Jul 15, 2018

Collaborator

Can we instead require that the file must end in .toml? It's very little magic for more convenience.

This comment has been minimized.

@orsinium

orsinium Jul 16, 2018

Author Contributor

I think about it, but this is very explicit while we have many possible places for configs. What if we have luigi.cfg and luigi.toml? We can't combine it into one config.

This comment has been minimized.

@dlstadther

dlstadther Jul 26, 2018

Collaborator

I think we can add an improvement down the road to define a hierarchical level of config extensions. I.e. (.cfg, .toml, .yml, <etc.>). I'm fine with requiring an ENV variable for now

* ``/etc/luigi/luigi.toml``
* ``luigi.toml``
* ``luigi/base.toml``
* ``luigi/local.toml``

This comment has been minimized.

@Tarrasch

Tarrasch Jul 15, 2018

Collaborator

You mean that it uses local first and then base. Which makes sense so you can have local changes that don't conflict with the base config.

@@ -23,6 +41,17 @@ The config file is broken into sections, each controlling a different part of th
[core]
scheduler_host=luigi-host.mycompany.foo

Example toml config:

.. code:: python

This comment has been minimized.

@Tarrasch

Tarrasch Jul 15, 2018

Collaborator

And a TODO then please. :)

streaming-jar = "/usr/lib/hadoop-xyz/hadoop-streaming-xyz-123.jar"
[core]
scheduler_host = "luigi-host.mycompany.foo"

This comment has been minimized.

@Tarrasch

Tarrasch Jul 15, 2018

Collaborator

Maybe link to the spec in the docs too?

@@ -0,0 +1,22 @@
# -*- coding: utf-8 -*-
#
# Copyright 2012-2015 Spotify AB

This comment has been minimized.

@Tarrasch

Tarrasch Jul 15, 2018

Collaborator

Yea I think this is fine ...

import logging


class BaseParser:

This comment has been minimized.

@Tarrasch

Tarrasch Jul 15, 2018

Collaborator

inherit from object?

This comment has been minimized.

@orsinium

orsinium Jul 16, 2018

Author Contributor

No-no-no, don't do it! In Python2 cfg parser fails if BaseParser inherit from object. This is strange, yes. I will add comment for it

This comment has been minimized.

@orsinium

orsinium Jul 16, 2018

Author Contributor

Added

@@ -38,8 +37,10 @@
except ImportError:
from configparser import ConfigParser, NoOptionError, NoSectionError

from .base_parser import BaseParser

This comment has been minimized.

@Tarrasch

Tarrasch Jul 15, 2018

Collaborator

I recall relative imports are discouraged, no?

This comment has been minimized.

@orsinium

orsinium Jul 16, 2018

Author Contributor

Why? We already have from .tools import range in project, and it's works. Relative imports little faster and better recognized some tools like isort.

setup.py Outdated
@@ -39,6 +39,7 @@ def get_static_files(path):
install_requires = [
'tornado>=4.0,<5',
'python-daemon<3.0',
'toml',

This comment has been minimized.

@Tarrasch

Tarrasch Jul 15, 2018

Collaborator

Can this be excluded? Eventually we want to populate this with dependencies needed but it's important that you can run luigi without toml installed.

This comment has been minimized.

@orsinium

orsinium Jul 16, 2018

Author Contributor

Fixed

@@ -0,0 +1,34 @@
# -*- coding: utf-8 -*-

This comment has been minimized.

@Tarrasch

Tarrasch Jul 15, 2018

Collaborator

Is there a better name than getter.py?

This comment has been minimized.

@orsinium

orsinium Jul 16, 2018

Author Contributor

fixed

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Jul 15, 2018

Good question, thank you :) I used it only one day on the dev. I make this feature for our team and send this Pull Request immediately. I can give here feedback after 1-2 weeks of active usage. I think, this is a good idea, thank you.

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Jul 16, 2018

Improved some things:

  • Dropped luigi/base.toml and luigi/local.toml paths. Inheritance must be discussed in future.
  • Added function for adding config in paths list of current active config parser.
  • Now TOML-parser's requirements optional for luigi and can be installed via extras: pip install luigi[toml]
  • For TOML parser we can get sections and options as attributes: config['mongo']['port'].
  • FIX catching config option getting in whole project: cfg parser raises NoSectionError, but TOML parser raises KeyError. Raise from new parsers exceptions from configparser -- bad idea.

Also you can run example project as proof-of-concept. For example, I am getting config['mongo']['port'] and this is returns int, not str. Soooo beautiful :)

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Jul 16, 2018

codecov shows strange things. Don't trust it.

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Jul 23, 2018

I'm fine. Codecov good and fair again :)

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Jul 23, 2018

Today I'll try to run our new pipeline with this improvements. I'll give you feedback soon.

@dlstadther
Copy link
Collaborator

dlstadther left a comment

Let us know the results of your pipeline execution as it relates to this PR :)

* ``luigi.toml``
* ``LUIGI_CONFIG_PATH`` environment variable

Both config lists reversordered by priotity (from low to high). The order only matters in case of key conflicts (see docs for ConfigParser.read_). These files are meant for both the client and ``luigid``. If you decide to specify your own configuration you should make sure that both the client and ``luigid`` load it properly.

This comment has been minimized.

@dlstadther

dlstadther Jul 23, 2018

Collaborator

reversordered is not a word. Perhaps we reword something to the effect of the below:

Both config lists increase in priority (from low to high).


# IMPORTANT: don't inherit from `object`!
# ConfigParser have some troubles in this case.
class BaseParser:

This comment has been minimized.

@dlstadther

dlstadther Jul 23, 2018

Collaborator

The issue is explained better on SO

if cls._instance is None:
cls._instance = cls(*args, **kwargs)
loaded = cls._instance.reload()
logging.getLogger('luigi-interface').info('Loaded %r', loaded)

This comment has been minimized.

@dlstadther

dlstadther Jul 23, 2018

Collaborator

Yeah, I was just raising an unrelated question - a topic for future possible PR (configurable logging names).

Nothing for you to do here.

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Jul 25, 2018

All is fine, our pipeline works right with new configs. I'm planning to commit dlstadther's improvements tomorrow.

Future improvements for other pull requests:

1. Local configs

I want to have file for local settings for storing some config in env

2. Configure logging via main config

Config example:

[logging]
version = 1
disable_existing_loggers = true

[logging.formatters.simple]
format = "{levelname:8} {asctime} {module}:{lineno} {message}"
style = "{"
datefmt = "%Y-%m-%d %H:%M:%S"

[logging.handlers.console]
level = "DEBUG"
class = "logging.StreamHandler"
formatter = "simple"

[logging.loggers.parser]
handlers = ["console"]
level = "DEBUG"
disabled = false
propagate = false

How we use it in our projects:

import os.path
import logging
from logging.config import dictConfig
import toml

path = os.path.dirname(os.path.abspath(__file__))
CONFIG = toml.load(os.path.join(path, 'config.toml'))

dictConfig(CONFIG['logging'])
logger = logging.getLogger('parser')

@orsinium orsinium requested a review from ulzha as a code owner Jul 26, 2018

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Jul 26, 2018

Added all actual @dlstadther 's improvements and merged spotify/master

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Jul 26, 2018

@ulzha, sorry for mentioning. This is from my branch actualization

@dlstadther
Copy link
Collaborator

dlstadther left a comment

Just a couple comments and clarifying questions


in increasing order of preference. The order only matters in case of key conflicts (see docs for ConfigParser.read_). These files are meant for both the client and ``luigid``. If you decide to specify your own configuration you should make sure that both the client and ``luigid`` load it properly.
You can choose right parser via ``LUIGI_CONFIG_PARSER`` environment variable. For example, ``LUIGI_CONFIG_PARSER=toml``.

This comment has been minimized.

@dlstadther

dlstadther Jul 26, 2018

Collaborator

I think we can add an improvement down the road to define a hierarchical level of config extensions. I.e. (.cfg, .toml, .yml, <etc.>). I'm fine with requiring an ENV variable for now

@@ -486,7 +486,7 @@ def _get_s3_config(self, key=None):
defaults = dict(configuration.get_config().defaults())
try:
config = dict(configuration.get_config().items('s3'))
except NoSectionError:
except (NoSectionError, KeyError):

This comment has been minimized.

@dlstadther

dlstadther Jul 26, 2018

Collaborator

Am i correct to assume that ConfigParser returns the NoSectionError and toml returns the KeyError when a config section heading is missing?

This comment has been minimized.

@orsinium

orsinium Jul 27, 2018

Author Contributor

Yes. I think, this is incorrect raise from toml parser any exceptions from ConfigParser. Maybe, we can make custom common exception:

class SectionError(NoSectionError, KeyError):
    pass

This comment has been minimized.

@dlstadther

dlstadther Jul 27, 2018

Collaborator

I'm fine with it as it is currently written; I was just verifying that they raised different errors for the same problem.

setup.py Outdated
@@ -75,6 +76,9 @@ def get_static_files(path):
]
},
install_requires=install_requires,
extras_require={
'toml': ['toml'],

This comment has been minimized.

@dlstadther

dlstadther Jul 26, 2018

Collaborator

What are your thoughts on limiting this to the current milestone version? i.e. 'toml<1.0' (since the current is 0.9.4) I just don't want version 1.0.0 to break this. I'd rather tread with caution, even with an optional configurable feature.

This comment has been minimized.

@orsinium

orsinium Jul 27, 2018

Author Contributor

Python toml package version related to toml spec version. TOML spec v1.0 will be backward compatible. Some quotes:

1.0.0 will be the first version to support TOML 1.0.0. TOML still has yet to hit TOML 1.0.0 so its release is postponed indefinitely.

As of version 0.5.0, TOML should be considered extremely stable. The goal is for version 1.0.0 to be backwards compatible (as much as humanly possible) with version 0.5.0. All implementations are strongly encouraged to become 0.5.0 compatible so that the transition to 1.0.0 will be simple when that happens.

This comment has been minimized.

@dlstadther

dlstadther Jul 27, 2018

Collaborator

In that case, why don't we put 'toml<2.0'?

This comment has been minimized.

@orsinium

orsinium Jul 27, 2018

Author Contributor

Ok. This don't break anything. This fix added. Thank you for review :)

tox.ini Outdated
@@ -35,6 +35,7 @@ deps=
hypothesis[datetime]
selenium==3.0.2
pymongo==3.4.0
toml

This comment has been minimized.

@dlstadther

dlstadther Jul 26, 2018

Collaborator

If we agree to a version requirement in setup.py, it'll need to be reflected here.

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Jul 27, 2018

I think, this PR is ready to merge. What do you think? :)

@dlstadther
Copy link
Collaborator

dlstadther left a comment

I'm good with this now

@Tarrasch Tarrasch merged commit 7d2c557 into spotify:master Jul 28, 2018

4 checks passed

codecov/changes No unexpected coverage changes found.
Details
codecov/patch 83.33% of diff hit (target 79.74%)
Details
codecov/project/core 92.68% (target 92%)
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@Tarrasch

This comment has been minimized.

Copy link
Collaborator

Tarrasch commented Jul 28, 2018

Good then! Let's see if this gets well used by othres. 🙌

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Jul 28, 2018

Thank you! We've done great work :)

cabouffard added a commit to glossier/luigi that referenced this pull request Aug 3, 2018

Optional TOML configs support (spotify#2457)
See the added docs for usage.

cabouffard added a commit to glossier/luigi that referenced this pull request Aug 8, 2018

Optional TOML configs support (spotify#2457)
See the added docs for usage.

dlstadther added a commit to dlstadther/luigi that referenced this pull request Aug 14, 2018

Merge branch 'upstream-master' into feature/email-attachment
* upstream-master: (82 commits)
  S3 client refactor (spotify#2482)
  Rename to rpc_log_retries, and make it apply to all the logging involved
  Factor log_exceptions into a configuration parameter
  Fix attribute forwarding for tasks with dynamic dependencies (spotify#2478)
  Add a visiblity level for luigi.Parameters (spotify#2278)
  Add support for multiple requires and inherits arguments (spotify#2475)
  Add metadata columns to the RDBMS contrib (spotify#2440)
  Fix race condition in luigi.lock.acquire_for (spotify#2357) (spotify#2477)
  tests: Use RunOnceTask where possible (spotify#2476)
  Optional TOML configs support (spotify#2457)
  Added default port behaviour for Redshift (spotify#2474)
  Add codeowners file with default and specific example (spotify#2465)
  Add Data Revenue to the `blogged` list (spotify#2472)
  Fix Scheduler.add_task to overwrite accepts_messages attribute. (spotify#2469)
  Use task_id comparison in Task.__eq__. (spotify#2462)
  Add stale config
  Move github templates to .github dir
  Fix transfer config import (spotify#2458)
  Additions to provide support for the Load Sharing Facility (LSF) job scheduler (spotify#2373)
  Version 2.7.6
  ...

dlstadther added a commit to dlstadther/luigi that referenced this pull request Aug 14, 2018

Merge branch 'upstream-master' into feature/moto-version-update
* upstream-master:
  S3 client refactor (spotify#2482)
  Rename to rpc_log_retries, and make it apply to all the logging involved
  Factor log_exceptions into a configuration parameter
  Fix attribute forwarding for tasks with dynamic dependencies (spotify#2478)
  Add a visiblity level for luigi.Parameters (spotify#2278)
  Add support for multiple requires and inherits arguments (spotify#2475)
  Add metadata columns to the RDBMS contrib (spotify#2440)
  Fix race condition in luigi.lock.acquire_for (spotify#2357) (spotify#2477)
  tests: Use RunOnceTask where possible (spotify#2476)
  Optional TOML configs support (spotify#2457)
  Added default port behaviour for Redshift (spotify#2474)
  Add codeowners file with default and specific example (spotify#2465)
  Add Data Revenue to the `blogged` list (spotify#2472)

dlstadther added a commit to dlstadther/luigi that referenced this pull request Aug 16, 2018

Merge branch 'upstream-master' into feature/py37-support
* upstream-master:
  Remove long-deprecated scheduler config variable alternatives (spotify#2491)
  Bump tornado milestone version (spotify#2490)
  Update moto to 1.x milestone version (spotify#2471)
  Use passed password when create a redis connection (spotify#2489)
  S3 client refactor (spotify#2482)
  Rename to rpc_log_retries, and make it apply to all the logging involved
  Factor log_exceptions into a configuration parameter
  Fix attribute forwarding for tasks with dynamic dependencies (spotify#2478)
  Add a visiblity level for luigi.Parameters (spotify#2278)
  Add support for multiple requires and inherits arguments (spotify#2475)
  Add metadata columns to the RDBMS contrib (spotify#2440)
  Fix race condition in luigi.lock.acquire_for (spotify#2357) (spotify#2477)
  tests: Use RunOnceTask where possible (spotify#2476)
  Optional TOML configs support (spotify#2457)
  Added default port behaviour for Redshift (spotify#2474)
  Add codeowners file with default and specific example (spotify#2465)
  Add Data Revenue to the `blogged` list (spotify#2472)
  Fix Scheduler.add_task to overwrite accepts_messages attribute. (spotify#2469)
  Use task_id comparison in Task.__eq__. (spotify#2462)
try:
import toml
except ImportError:
toml = False

This comment has been minimized.

@adamist521

adamist521 Aug 25, 2018

@orsinium
This will end up following error if you have toml library not installed...
Works great after toml installation!

File "/Users/*****/.pyenv/versions/miniconda-latest/envs/ddhr-rjb/lib/python3.6/site-packages/luigi/configuration/toml_parser.py", line 53, in read
    self.data = self._update_data(self.data, toml.load(path))
AttributeError: 'bool' object has no attribute 'load'

This comment has been minimized.

@Tarrasch

Tarrasch Aug 25, 2018

Collaborator

Oh noes, @orsinium can you fix this asap?

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Aug 25, 2018

@adamist521, please, can you provide full traceback?

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Aug 25, 2018

@Tarrasch, don't panic. This is unusual usage or something like this, because get_config function checks reader's enabled option. Also this error possible when you install Luigi without toml support, but want to use toml parser.

Hmmmm... I need to add info about toml support in installation section :)

@honnix

This comment has been minimized.

Copy link
Contributor

honnix commented Sep 6, 2018

This PR effectively fails from luigi.configuration import NoSectionError. Was this intended?

@orsinium

This comment has been minimized.

Copy link
Contributor Author

orsinium commented Sep 6, 2018

Yes. Please, import NoSectionError from configparser from stdlib:

from configparser import NoSectionError```

tanmay2893 added a commit to lexentbio/luigi that referenced this pull request Oct 25, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.