[FEATURE] Argument Parser: new member function that checks if an option was set. #1859

smehringer · 2020-05-28T15:32:53Z

Current look:

int main(int argc, char ** argv)
{
    int i{5}; // default
    seqan3::argument_parser parser{"test_parser", argc, argv};
    // parser.add_option(i, 'f', "foo", "description");
    // parser.parse(); 
    // return parser.option_overwritten('f');
    // return parser.default_overwritten('f');
    // return parser.default_overwridden('f');
    // return parser.overwritten_by_user('f');
    // return parser.overwritten_on_cmd('f');
    // return parser.option_is_set('f'); // tloka's fav
    return parser.is_option_set('f');
    // return parser.option_is_set_by_user('f');
    // return parser.option_is_set_on_cmd('f');
    // return parser.set_by_user('f');
}

int main(int argc, char ** argv)
{
    std::optional<int> i{std::nullopt}; // default

    seqan3::argument_parser parser{"test_parser", argc, argv};
    parser.add_option(i, 'f', "foo", "Default is computed.");
    parser.parse(); 

    return i.has_value();
}

would return 1 if called with ./test_parser -f 1
would return 0 if called with ./test_parser -g 1

There are open design questions:

Naming: option_was_set, option_is_set, is_set, was_set, option_is_overwritten ... suggestions?
By design, this function could be called without the necessity of
(a) a respective option was added (parser.add_option('f', ...))
(b) parser.parse() was called and all options values were parsed
Do we want to allow this? We can easily introduce these semantic requirements.

Resolution: parser.is_option_set

The function will return an enum that can be implicitly cast to bool.

codecov · 2020-05-28T19:03:55Z

Codecov Report

Merging #1859 into master will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1859      +/-   ##
==========================================
+ Coverage   98.08%   98.09%   +0.01%     
==========================================
  Files         262      262              
  Lines       10679    10744      +65     
==========================================
+ Hits        10474    10539      +65     
  Misses        205      205

Impacted Files	Coverage Δ
include/seqan3/argument_parser/argument_parser.hpp	`98.78% <100.00%> (+0.07%)`	⬆️
...ude/seqan3/argument_parser/detail/format_parse.hpp	`96.80% <100.00%> (ø)`
...lude/seqan3/search/fm_index/bi_fm_index_cursor.hpp	`100.00% <0.00%> (ø)`
...qan3/alignment/pairwise/alignment_configurator.hpp	`97.67% <0.00%> (ø)`
...nt/pairwise/detail/policy_affine_gap_recursion.hpp	`100.00% <0.00%> (ø)`
...ise/detail/pairwise_alignment_algorithm_banded.hpp	`100.00% <0.00%> (ø)`
include/seqan3/io/alignment_file/format_bam.hpp	`95.58% <0.00%> (+0.01%)`	⬆️
include/seqan3/argument_parser/validators.hpp	`90.45% <0.00%> (+0.04%)`	⬆️
include/seqan3/io/alignment_file/format_sam.hpp	`98.13% <0.00%> (+0.07%)`	⬆️
...clude/seqan3/io/alignment_file/format_sam_base.hpp	`98.74% <0.00%> (+0.09%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8140b10...b801b4f. Read the comment docs.

include/seqan3/argument_parser/argument_parser.hpp

rrahn · 2020-06-24T12:16:35Z

Resolution 24.06.2020

We have a preference for the member function design.
The OP of the issue should be asked for feedback on this as well, to come to a final solution.

include/seqan3/argument_parser/detail/format_parse.hpp

eseiler · 2020-09-01T09:12:45Z

include/seqan3/argument_parser/argument_parser.hpp

+        auto end = std::find(cmd_arguments.begin(), cmd_arguments.end(), "--");
+        return detail::format_parse::find_option_id(cmd_arguments.begin(), end, id) != end;


I'm having trouble understanding what you are doing.

Why search for -- when you are looking at short ids?

Wouldn't you want to use std::find_end since you surely don't want end to point to the first occurrence of --?

Why don't you just pass begin and end of cmd_arguments?

I just noticed that you answered yesterday under my month old comment, maybe you can add the reasoning for the -- as a comment, because it is really confusing.
Also, where does the -- come from?

No this is something different.
I actually do want to search for the first occurrence of "--". Note: Not as a prefix (or other -fix) of another word but ONLY "--" because it signals the end of options and marks the beginning of arguments for positional options. It is special in its meaning. And since I am searching for option names I must only search before the arguments for positional options.
Was that explained in an understandable way? Otherwise I'll try again.

-- is a POSIX convention to singal that there are no more option names after this. That way the user can use dash in arguments without confusing the argument parser. This done like this in most argument parsers.

See POSIX conventions

So to your questions: I explicitly do NOT want to search the whole command line but only that part where option names can be found.

Do you add the -- somewhere or is it magically there?
Or is this user input that may be there?

It is user input on the command line that may be there

And while looking at it. You should make -- a static constexpr variable somewhere in the class's scope such that it also gets a proper name: end_of_options_marker or something like that.

some more thoughts about that. If I understand this correctly, then we have to call parse before we can ask whether an option was set by the user. So should it not throw in the case parse wasn't called before? I think we have a variable that caches this?

The other thing I'd like to reconsider from the implementation is the duplication of this code.
First, I observed that an extra copy is made to store the parsed arguments in the argument parser.
Second, the arguments are only necessary if format_parse was called so also this option can only be available if format_parse was called.

So the code should be refactored that I can ask for the set option in the format_parse class. Calling it for any other parsing option can then throw, right? So since all formats inherit from a base class this can be achieved easily. Then only the format_parse class overloads this and inside of the argument_parser you call std::visit on the selected format.

some more thoughts about that. If I understand this correctly, then we have to call parse before we can ask whether an option was set by the user. So should it not throw in the case parse wasn't called before? I think we have a variable that caches this?

No, we do not have to call parse before checking this. It is independent. (see test case)
This is the case because we have the list argv already on construction with all the information we need.
We can of course enforce this restriction anyway if we think it is confusing for the user or more intuitive 🤷.

The other thing I'd like to reconsider from the implementation is the duplication of this code.

Which duplication? Do you mean the extra argv copy?

First, I observed that an extra copy is made to store the parsed arguments in the argument parser.

Yes. We now have to copies of argv. One in the argument_parser and one in format_parse.
But this is needed either way, even if I move this list into format_parse because format_parse is stripping it's copy of argv of arguments while parsing the information s.t. it can afterwards detect "too many" or "unknown option identifiers". So in order to enable is_option_set I need to store the original argv in some place anyway.

Second, the arguments are only necessary if format_parse was called so also this option can only be available if format_parse was called.

Technically no, see above.

So the code should be refactored that I can ask for the set option in the format_parse class. Calling it for any other parsing option can then throw, right? So since all formats inherit from a base class this can be achieved easily. Then only the format_parse class overloads this and inside of the argument_parser you call std::visit on the selected format.

IMHO I think that is_option_set is not strongly attached to format_parse. It's a feature of the argument parser that doesn't involve actual parsing of argument values but only checking if an id can be found. Is this such a strong design decision, whether we call a static function of format_parse or use std::visit?

No, we do not have to call parse before checking this. It is independent. (see test case)
IMHO I think that is_option_set is not strongly attached to format_parse. It's a feature of the argument parser that doesn't involve actual parsing of argument values but only checking if an id can be found. Is this such a strong design decision, whether we call a static function of format_parse or use std::visit?

But the design doesn't add up in my opinion. If I understand it correctly, then right now the API requires to call parse in order to validate and set the parameters with the options from the command line or to print the help page. So the design deliberately forces us to call parse after we added the options to the parser. Everything is deferred. Accordingly, in the proper design the check if an option was set should be put after the parse method was called. The reason is that the interface should only work for valid options and throw if you try to ask for an option that wasn't added to the parser. But this can only be known after the option was added, and parse functions here as the terminator. So that's why I think that the current solution does not reflect the design of the argument parser.

test/unit/argument_parser/format_parse_test.cpp

smehringer · 2020-09-02T08:25:37Z

I hope I have answered all your questions. If I missed a requested change please tell me again :)

eseiler

One small thing for the changelog, otherwise lgtm

CHANGELOG.md

rrahn

Ok, I have some general considerations about the implementation of this feature. Please find my comments below.

CHANGELOG.md

rrahn · 2020-09-07T07:58:23Z

include/seqan3/argument_parser/argument_parser.hpp

+        auto end = std::find(cmd_arguments.begin(), cmd_arguments.end(), "--");
+        return detail::format_parse::find_option_id(cmd_arguments.begin(), end, id) != end;


Yes, we should definitely have a documentation entry that points to the POSIX conventions.
And second, it would be good to annotate the intention on the line above with a comment. This avoids confusion later on, because from the code it is not immediately clear why you are looking for --.

rrahn · 2020-09-07T07:59:31Z

include/seqan3/argument_parser/argument_parser.hpp

+        auto end = std::find(cmd_arguments.begin(), cmd_arguments.end(), "--");
+        return detail::format_parse::find_option_id(cmd_arguments.begin(), end, id) != end;


On the second though, this should also throw an exception if the id has a dash, since you explicitly requiring it not to.

include/seqan3/argument_parser/argument_parser.hpp

rrahn · 2020-09-07T08:41:16Z

include/seqan3/argument_parser/argument_parser.hpp

+        auto end = std::find(cmd_arguments.begin(), cmd_arguments.end(), "--");
+        return detail::format_parse::find_option_id(cmd_arguments.begin(), end, id) != end;


some more thoughts about that. If I understand this correctly, then we have to call parse before we can ask whether an option was set by the user. So should it not throw in the case parse wasn't called before? I think we have a variable that caches this?

rrahn · 2020-09-07T08:59:33Z

include/seqan3/argument_parser/argument_parser.hpp

+        auto end = std::find(cmd_arguments.begin(), cmd_arguments.end(), "--");
+        return detail::format_parse::find_option_id(cmd_arguments.begin(), end, id) != end;


The other thing I'd like to reconsider from the implementation is the duplication of this code.
First, I observed that an extra copy is made to store the parsed arguments in the argument parser.
Second, the arguments are only necessary if format_parse was called so also this option can only be available if format_parse was called.

So the code should be refactored that I can ask for the set option in the format_parse class. Calling it for any other parsing option can then throw, right? So since all formats inherit from a base class this can be achieved easily. Then only the format_parse class overloads this and inside of the argument_parser you call std::visit on the selected format.

eseiler · 2020-10-07T08:16:12Z

Can you rebase?

smehringer · 2020-10-07T09:19:26Z

Since there are several open discussion points I will wait with the work on this PR.

rrahn · 2020-10-08T09:48:03Z

test/unit/argument_parser/format_parse_test.cpp

+    EXPECT_TRUE(parser.is_option_set('l'));
+    EXPECT_TRUE(parser.is_option_set("foobar"));
+
+    EXPECT_FALSE(parser.is_option_set("foo"));


Can you please add a test with an empty string.

rrahn · 2020-10-08T10:32:52Z

include/seqan3/argument_parser/argument_parser.hpp

+        auto end = std::find(cmd_arguments.begin(), cmd_arguments.end(), "--");
+        return detail::format_parse::find_option_id(cmd_arguments.begin(), end, id) != end;


No, we do not have to call parse before checking this. It is independent. (see test case)
IMHO I think that is_option_set is not strongly attached to format_parse. It's a feature of the argument parser that doesn't involve actual parsing of argument values but only checking if an id can be found. Is this such a strong design decision, whether we call a static function of format_parse or use std::visit?

But the design doesn't add up in my opinion. If I understand it correctly, then right now the API requires to call parse in order to validate and set the parameters with the options from the command line or to print the help page. So the design deliberately forces us to call parse after we added the options to the parser. Everything is deferred. Accordingly, in the proper design the check if an option was set should be put after the parse method was called. The reason is that the interface should only work for valid options and throw if you try to ask for an option that wasn't added to the parser. But this can only be known after the option was added, and parse functions here as the terminator. So that's why I think that the current solution does not reflect the design of the argument parser.

rrahn

I am still a little bit unhappy with the design. This is merely because I believe that the argument parser design might not be reflected properly. I tried to explain my concerns in the comments. Please feel free to put it into core discussion if you have a different opinion or if I did not clear things up. Thank you!

smehringer · 2020-10-15T06:28:03Z

@rrahn polite ping

rrahn

Nicely done. Some suggestions to make the code more concise.

rrahn · 2020-10-15T13:31:00Z

include/seqan3/argument_parser/argument_parser.hpp

+    /*!\brief Checks whether the option distinguished by `id` was set on the command line by the user.
+     * \param[in] id The option identifier to search for (must not contain dashes!).
+     * \returns `true` if option identifier `id` was set on the command line by the user.
+     */


The documentation is part of the public API and should be a little bit more verbose. For example it is not clear that the function might throw. Second the documentation says that id should not have dashes. But the overloaded function would get the same information, while it is not true in this case.

rrahn · 2020-10-15T13:36:49Z

include/seqan3/argument_parser/argument_parser.hpp

+    bool is_option_set(char const id)
+    {
+        if (!parse_was_called)
+            throw design_error("You can only asked which options have been set after calling the function `parse()`.");
+
+        auto constexpr allowed = is_alnum || is_char<'_'> || is_char<'@'>;
+
+        if (!allowed(id))
+            throw design_error("Short option identifiers may only contain alphanumeric characters, '_', or '@'.");
+
+        // we only need to search for an option before the `end_of_options_indentifier` (`--`)
+        auto end_of_options = std::find(cmd_arguments.begin(), cmd_arguments.end(), end_of_options_indentifier);
+        return detail::format_parse::find_option_id(cmd_arguments.begin(), end_of_options, id) != end_of_options;
+    }
+
+    // allows all types that are implicitly convertible to std::string
+    //!\overload
+    bool is_option_set(std::string const & id)
+    {
+        if (!parse_was_called)
+            throw design_error("You can only asked which options have been set after calling the function `parse()`.");
+
+        if (id.size() <= 1)
+            throw design_error("Long IDs must be longer than one character.");
+        else if (is_char<'-'>(id[0]))
+            throw design_error("First character of long ID cannot be '-'.");
+
+        auto constexpr allowed = is_alnum || is_char<'_'> || is_char<'@'> || is_char<'-'>;
+        std::for_each(id.begin(), id.end(), [&allowed] (char c)
+        {
+            if (!allowed(c))
+                throw design_error("Long identifiers may only contain alphanumeric characters, '_', '-', or '@'.");
+        });
+
+        // we only need to search for an option before the `end_of_options_indentifier` (`--`)
+        auto end_of_options = std::find(cmd_arguments.begin(), cmd_arguments.end(), end_of_options_indentifier);
+        return detail::format_parse::find_option_id(cmd_arguments.begin(), end_of_options, id) != end_of_options;
+    }


Second, what happens if I call is_option_set("a")? is that really a design error? It would be a user error, right? But it should also just work. And is is_option_set('_') allowed as an option?
In addition, I see a lot of potential to DRY this function. For example, you could write a detail function that get's the option identifier always as a string and in addition the set of valid characters. Then you can only need to handle the really different things in the tpo-level functions.

I thought about this again and I propose the following: Since we do enforce that parse() is called before this function can be used, I can now just check if the requested option identifier is in the list of used_option_identifiers. Those have already been tested in the add_option calls beforehand so I can omit all the testing completely?

But you specify the option by a string, which could contain invalid characters, aren't you? So you would need to have some checks but you could resue what is already there to test them properly, which is probably the same code as done when adding the options?

include/seqan3/argument_parser/argument_parser.hpp

test/unit/argument_parser/format_parse_test.cpp

smehringer · 2020-10-16T11:55:04Z

Sorry I had to rebase because of conflicts so I rebased all the commits correctly right away

… iterator.

…rser.

include/seqan3/argument_parser/argument_parser.hpp

rrahn · 2020-10-23T07:50:37Z

include/seqan3/argument_parser/argument_parser.hpp

+    bool is_option_set(std::string const & id) const
+    {
+        if (!parse_was_called)
+            throw design_error{"You can only asked which options have been set after calling the function `parse()`."};
+
+        if (id.size() == 1)
+            throw design_error{"Long option identifiers must be longer than one character! If " + id + "' was meant to "
+                               "be a short identifier, please pass it as a char ('') not a string (\"\")!"};
+
+        if (std::find(used_option_ids.begin(), used_option_ids.end(), id) == used_option_ids.end())
+            throw design_error{"You can only ask for option identifiers that you added with add_option() before."};
+
+        // we only need to search for an option before the `end_of_options_indentifier` (`--`)
+        auto end_of_options = std::find(cmd_arguments.begin(), cmd_arguments.end(), end_of_options_indentifier);
+        return detail::format_parse::find_option_id(cmd_arguments.begin(), end_of_options, id) != end_of_options;
+    }


Suggested change

bool is_option_set(std::string const & id) const

{

if (!parse_was_called)

throw design_error{"You can only asked which options have been set after calling the function `parse()`."};

if (id.size() == 1)

throw design_error{"Long option identifiers must be longer than one character! If " + id + "' was meant to "

"be a short identifier, please pass it as a char ('') not a string (\"\")!"};

if (std::find(used_option_ids.begin(), used_option_ids.end(), id) == used_option_ids.end())

throw design_error{"You can only ask for option identifiers that you added with add_option() before."};

// we only need to search for an option before the `end_of_options_indentifier` (`--`)

auto end_of_options = std::find(cmd_arguments.begin(), cmd_arguments.end(), end_of_options_indentifier);

return detail::format_parse::find_option_id(cmd_arguments.begin(), end_of_options, id) != end_of_options;

}

bool is_option_set(char const id) const

{

return is_option_set_impl(std::string{id}, false);

}

bool is_option_set(std::string const & id) const

{

return is_option_set_impl(id, true);

}

bool is_option_set_impl(std::string const & id, bool const is_long_option_id) const

{

if (!parse_was_called)

throw design_error{"You can only ask which options have been set after calling the function `parse()`."};

if (is_long_option_id && id.size() == 1)

throw design_error{"Long option identifiers must be longer than one character! If " + id + "' was meant to "

"be a short identifier, please pass it as a char ('') not a string (\"\")!"};

if (std::find(used_option_ids.begin(), used_option_ids.end(), id) == used_option_ids.end())

throw design_error{"You can only ask for option identifiers that you added with add_option() before."};

// we only need to search for an option before the `end_of_options_indentifier` (`--`)

auto end_of_options = std::find(cmd_arguments.begin(), cmd_arguments.end(), end_of_options_indentifier);

return detail::format_parse::find_option_id(cmd_arguments.begin(), end_of_options, id) != end_of_options;

}

This also also reduces the typo to only one occurrence :)

This does not work because

is_option_set_impl(std::string{id}, false);

will trigger the exception of long ids, that they must be longer than one char.
I'll try to refactor otherwise

Wait what? I did care of this by passing this additional bool parameter to the internal interface. See, the check whether the long_id has only one element is only evaluate if is_long_option_id is set to true. But it is not in the case of a short id. Other than that the code was exactly the same.

Oh, no. This goes on in the parse operations as well. I see.

rrahn · 2020-10-23T07:54:42Z

include/seqan3/argument_parser/argument_parser.hpp

+     * \details
+     *
+     * You can only ask for option identifiers that were added to the parser beforehand via
+     * seqan3::argument_parser::add_option.


The API documentation is missing the throw statement and a description when and what this function throws.

include/seqan3/argument_parser/argument_parser.hpp

smehringer · 2020-10-29T09:24:22Z

@rrahn polite ping :)

rrahn

lgtm! only renaming of the id_ variable. You can already rebase. Thank you!

rrahn · 2020-10-29T15:17:13Z

include/seqan3/argument_parser/argument_parser.hpp

-                               "be a short identifier, please pass it as a char ('') not a string (\"\")!"};
+        // the detail::format_parse::find_option_id call in the end expects either a char or std::string
+        using char_or_string_t = std::conditional_t<std::same_as<id_type, char>, char, std::string>;
+        char_or_string_t id_ = {id}; // e.g. convert char * to string here if necessary


Suggested change

char_or_string_t id_ = {id}; // e.g. convert char * to string here if necessary

char_or_string_t short_or_long_id{id}; // e.g. convert char * to string here if necessary

smehringer marked this pull request as draft May 28, 2020 15:33

smehringer force-pushed the argument_parser_option_is_set branch from 31297e0 to cc66b9a Compare May 28, 2020 18:02

eseiler reviewed Jun 2, 2020

View reviewed changes

include/seqan3/argument_parser/argument_parser.hpp Outdated Show resolved Hide resolved

smehringer force-pushed the argument_parser_option_is_set branch from cc66b9a to 47c1534 Compare August 31, 2020 16:18

smehringer requested a review from eseiler August 31, 2020 16:19

smehringer marked this pull request as ready for review September 1, 2020 06:59

eseiler requested changes Sep 1, 2020

View reviewed changes

smehringer requested a review from eseiler September 2, 2020 08:25

eseiler approved these changes Sep 2, 2020

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

eseiler requested a review from rrahn September 2, 2020 10:18

smehringer mentioned this pull request Mar 21, 2022

[Argument Parser] Follow up: Document POSIX conventions that are implemented in the argument parser seqan/sharg-parser#79

Closed

4 tasks

smehringer force-pushed the argument_parser_option_is_set branch from 11669f1 to 4d76bdc Compare September 3, 2020 06:27

rrahn requested changes Sep 7, 2020

View reviewed changes

smehringer requested a review from rrahn September 10, 2020 08:52

rrahn reviewed Oct 8, 2020

View reviewed changes

rrahn requested changes Oct 8, 2020

View reviewed changes

smehringer force-pushed the argument_parser_option_is_set branch 2 times, most recently from d6b52c8 to bd7aa56 Compare October 9, 2020 11:59

smehringer requested a review from rrahn October 9, 2020 11:59

rrahn requested changes Oct 15, 2020

View reviewed changes

smehringer added 3 commits October 16, 2020 13:47

[MISC] Make find_option_id of format_parse public.

e5daded

[MISC] Move member variables of argument_parser together.

70fed35

[MISC] Make prepend_dash member functions of format_parse static.

e1ddb16

smehringer force-pushed the argument_parser_option_is_set branch from 8f6be37 to 58a5ffb Compare October 16, 2020 11:54

smehringer requested a review from rrahn October 16, 2020 11:55

smehringer added 2 commits October 16, 2020 14:09

[MISC] Adapt find_option_id member function to accept an explicit end…

29e60e2

… iterator.

[MISC] Add a private member variable cmd_arguments to the argument_pa…

272dccd

…rser.

smehringer force-pushed the argument_parser_option_is_set branch from 58a5ffb to 1a37788 Compare October 16, 2020 12:09

rrahn reviewed Oct 23, 2020

View reviewed changes

include/seqan3/argument_parser/argument_parser.hpp Outdated Show resolved Hide resolved

rrahn requested changes Oct 23, 2020

View reviewed changes

smehringer requested a review from rrahn October 26, 2020 10:32

rrahn requested changes Oct 29, 2020

View reviewed changes

smehringer added 2 commits October 29, 2020 19:43

[FEATURE] add option_was_set member function.

cc3d1a0

[MISC] Use braces for initialisation of return value in prepend_dash.

b801b4f

smehringer force-pushed the argument_parser_option_is_set branch from ae80f7c to b801b4f Compare October 29, 2020 18:44

smehringer requested a review from rrahn October 29, 2020 18:46

rrahn approved these changes Oct 30, 2020

View reviewed changes

rrahn merged commit 945b12b into seqan:master Oct 30, 2020

smehringer deleted the argument_parser_option_is_set branch November 4, 2020 10:02

		auto end = std::find(cmd_arguments.begin(), cmd_arguments.end(), "--");
		return detail::format_parse::find_option_id(cmd_arguments.begin(), end, id) != end;

	char_or_string_t id_ = {id}; // e.g. convert char * to string here if necessary
	char_or_string_t short_or_long_id{id}; // e.g. convert char * to string here if necessary

[FEATURE] Argument Parser: new member function that checks if an option was set. #1859

[FEATURE] Argument Parser: new member function that checks if an option was set. #1859

Conversation

smehringer commented May 28, 2020 • edited Loading

codecov bot commented May 28, 2020 • edited Loading

Codecov Report

rrahn commented Jun 24, 2020

Resolution 24.06.2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smehringer Sep 1, 2020 • edited Loading

Choose a reason for hiding this comment

eseiler Sep 1, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smehringer Sep 10, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smehringer commented Sep 2, 2020

eseiler left a comment

Choose a reason for hiding this comment

rrahn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eseiler commented Oct 7, 2020

smehringer commented Oct 7, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rrahn left a comment

Choose a reason for hiding this comment

smehringer commented Oct 15, 2020

rrahn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smehringer commented Oct 16, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smehringer Oct 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smehringer commented Oct 29, 2020

rrahn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smehringer commented May 28, 2020 •

edited

Loading

codecov bot commented May 28, 2020 •

edited

Loading

smehringer Sep 1, 2020 •

edited

Loading

eseiler Sep 1, 2020 •

edited

Loading

smehringer Sep 10, 2020 •

edited

Loading

smehringer Oct 26, 2020 •

edited

Loading