Octo 1616 no experimentdata #134

gbordyugov · 2017-07-17T15:26:47Z

PLEASE DON'T MERGE YET *

coveralls · 2017-07-20T11:59:25Z

Coverage increased (+2.8%) to 82.062% when pulling 4b9484e on gbordyugov:OCTO-1616-no-experimentdata into dfbf3ed on zalando:dev.

coveralls · 2017-07-20T13:02:35Z

Coverage increased (+2.9%) to 82.133% when pulling 63d9a46 on gbordyugov:OCTO-1616-no-experimentdata into dfbf3ed on zalando:dev.

coveralls · 2017-07-20T13:02:35Z

Coverage increased (+2.9%) to 82.133% when pulling 63d9a46 on gbordyugov:OCTO-1616-no-experimentdata into dfbf3ed on zalando:dev.

daryadedik · 2017-07-20T16:56:54Z

expan/core/experiment.py

-        }
-
-        if not method in method_table:
+        if not method in workerTable:


if method not in workerTable may be a bit more readable.

daryadedik · 2017-07-21T08:23:18Z

tests/tests_core/test_early_stopping.py

+        self.assertEqual               (res['n_x'],             1000)
+        self.assertEqual               (res['n_y'],             1000)
+        self.assertAlmostEqual         (res['mu_x'],           -0.045256707490195384)
+        self.assertAlmostEqual         (res['mu_y'],            0.11361694031616358)


to my opinion this structure is rather difficult to maintain if there are some changes (you need to keep in mind spaces all the time).
https://www.python.org/dev/peps/pep-0008/#whitespace-in-expressions-and-statements

I agree, it would be easier if we could agree on a specific style.

I agree that writing this might be a bit cumbersome, but reading is much easier - and we tend to read more often than write.

mkolarek

looks great, we could potentially merge these changes with the dev branch, but not with master. I believe this would ease further collaboration without breaking and outside dependencies (yet).

mkolarek · 2017-07-21T08:59:03Z

expan/core/early_stopping.py

@@ -159,6 +165,8 @@ def get_or_compile_stan_model(model_file, distribution):
            pickle.dump(sm, f)
    return sm

+cacheSamplingResults = False
+samplingResults = {} # memoized sampling results


+1 for PEP8

daryadedik · 2017-07-21T09:29:39Z

expan/core/experiment.py


-        res += '\n {:d} variants: {}'.format(len(variants),


"res" is unresolved variable, there are no declaration (or was commented?)

I think it's part of the old (removed) code, hence in red.

got it! thanks!

daryadedik · 2017-07-21T13:56:34Z

expan/core/experiment.py


+        res += '\n {:d} variants: {}'.format(len(variants),


Ah, here the res was commented and now 'res' has no declaration. Comment before actually was about that, I just missed the place.

gbordyugov · 2017-07-24T10:02:34Z

variable names style change from camelCase to snake_case

coveralls · 2017-07-24T10:08:47Z

Coverage increased (+2.9%) to 82.133% when pulling 943442f on gbordyugov:OCTO-1616-no-experimentdata into dfbf3ed on zalando:dev.

coveralls · 2017-07-24T13:53:48Z

Coverage increased (+2.9%) to 82.133% when pulling c3507be on gbordyugov:OCTO-1616-no-experimentdata into dfbf3ed on zalando:dev.

coveralls · 2017-07-24T15:01:42Z

Coverage increased (+2.9%) to 82.133% when pulling 8ccc1af on gbordyugov:OCTO-1616-no-experimentdata into dfbf3ed on zalando:dev.

…group_sequential_delta in expan/core/early_stoppy.py

coveralls · 2017-07-26T10:09:30Z

Coverage increased (+2.8%) to 82.072% when pulling c8a330d on gbordyugov:OCTO-1616-no-experimentdata into dfbf3ed on zalando:dev.

coveralls · 2017-07-26T11:31:33Z

Coverage increased (+2.8%) to 82.072% when pulling e5a4367 on gbordyugov:OCTO-1616-no-experimentdata into dfbf3ed on zalando:dev.

coveralls · 2017-07-26T12:07:17Z

Coverage increased (+2.8%) to 82.072% when pulling bb1e087 on gbordyugov:OCTO-1616-no-experimentdata into dfbf3ed on zalando:dev.

coveralls · 2017-07-26T13:51:38Z

Coverage increased (+2.9%) to 82.166% when pulling 1f6f080 on gbordyugov:OCTO-1616-no-experimentdata into dfbf3ed on zalando:dev.

gbordyugov added 30 commits July 7, 2017 16:04

added expan/core/jsonable.py and expan/core/.py

6b725a9

shadowing workers in early_stopping.py

3d37f15

shadowing workers in statistics.py

3c45575

OCTO-1616 early stopping returning EarlyStoppingStatistics

100c01d

OCTO-1616 little formatting issues

dfd6cb8

typos in comments

e0b145a

OCTO-1616: statistics.delta() returns DeltaStatistics object

2ce868c

OCTO-1616 removed import statistic

3251aae

OCTO-1616 removed import statistic again

e04f8fd

rotated lines in experiment.py

64e33c7

OCTO-1616 just a backup at the end of the day, nothing`s working

5ecc734

OCTO-1616 get rid of lambda

3b0014c

OCTO-1616 at the end of the day

c1bad28

fixed_horizon_delta() could apply delta to all variants / all KPIs

68d957f

fixed_horizon_delta() returns things

2c9c7c8

early stopping workers return dicts

a02f1e9

delta() returning dict with statistics

f8dc45d

early_stopping.py doesnt need to import jsonable anymore

5209595

Experiment.fhd(), Experiment._getWeights()

5bcd55f

no need for defaultdict in experiment.py

d323b4c

added make_* functions for early stopping workers

172eee8

towards Experiment.newDelta()

8fbdbf4

typo in Experiment.newDelta()

a9165ce

formula in newDelta

f9b8363

addDerivedKpi

92b1543

removed experimentdata.py results.py test_results.py test_data_check.py

b070072

before lunch

a31764f

just to compare

4e1dd70

deltas working, need unit tests on numerical values

b76c479

fixed csv_fetcher and its unit test

b26217a

back to np.float64 from np.number as default KPI column type

4b9484e

bettern handling of KPI names in ctor of Experiment

63d9a46

daryadedik reviewed Jul 20, 2017

View reviewed changes

daryadedik reviewed Jul 21, 2017

View reviewed changes

mkolarek approved these changes Jul 21, 2017

View reviewed changes

daryadedik reviewed Jul 21, 2017

View reviewed changes

camel case -> underscores

943442f

removed space

c3507be

camel -> snake again

8ccc1af

estimated_sample_size instead of information_fraction as argument of …

c8a330d

…group_sequential_delta in expan/core/early_stoppy.py

added docstring to estimated_sample_size in expan/core/early_stopping.py

e5a4367

added cap to information fraction in group_sequential_delta()

bb1e087

errors, warnings, and expan_version in results dict

1f6f080

gbordyugov merged commit c6e39e5 into zalando:dev Jul 26, 2017

This was referenced Aug 1, 2017

Optimizing the control flow from Experiment to Results #82

Closed

Results.to_json() implementation not flexible #65

Closed

reenable means and bounds functions on Results object #9

Closed

Results.to_json() doesn't support trend() results #64

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Octo 1616 no experimentdata #134

Octo 1616 no experimentdata #134

gbordyugov commented Jul 17, 2017

coveralls commented Jul 20, 2017 •

edited

Loading

coveralls commented Jul 20, 2017

coveralls commented Jul 20, 2017

daryadedik Jul 20, 2017

daryadedik Jul 21, 2017

mkolarek Jul 21, 2017

gbordyugov Jul 24, 2017

mkolarek left a comment

mkolarek Jul 21, 2017

daryadedik Jul 21, 2017

gbordyugov Jul 21, 2017

daryadedik Jul 21, 2017

daryadedik Jul 21, 2017 •

edited

Loading

gbordyugov commented Jul 24, 2017

coveralls commented Jul 24, 2017 •

edited

Loading

coveralls commented Jul 24, 2017 •

edited

Loading

coveralls commented Jul 24, 2017 •

edited

Loading

coveralls commented Jul 26, 2017 •

edited

Loading

coveralls commented Jul 26, 2017 •

edited

Loading

coveralls commented Jul 26, 2017 •

edited

Loading

coveralls commented Jul 26, 2017 •

edited

Loading

Octo 1616 no experimentdata #134

Octo 1616 no experimentdata #134

Conversation

gbordyugov commented Jul 17, 2017

coveralls commented Jul 20, 2017 • edited Loading

coveralls commented Jul 20, 2017

coveralls commented Jul 20, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mkolarek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daryadedik Jul 21, 2017 • edited Loading

Choose a reason for hiding this comment

gbordyugov commented Jul 24, 2017

coveralls commented Jul 24, 2017 • edited Loading

coveralls commented Jul 24, 2017 • edited Loading

coveralls commented Jul 24, 2017 • edited Loading

coveralls commented Jul 26, 2017 • edited Loading

coveralls commented Jul 26, 2017 • edited Loading

coveralls commented Jul 26, 2017 • edited Loading

coveralls commented Jul 26, 2017 • edited Loading

coveralls commented Jul 20, 2017 •

edited

Loading

daryadedik Jul 21, 2017 •

edited

Loading

coveralls commented Jul 24, 2017 •

edited

Loading

coveralls commented Jul 24, 2017 •

edited

Loading

coveralls commented Jul 24, 2017 •

edited

Loading

coveralls commented Jul 26, 2017 •

edited

Loading

coveralls commented Jul 26, 2017 •

edited

Loading

coveralls commented Jul 26, 2017 •

edited

Loading

coveralls commented Jul 26, 2017 •

edited

Loading