Skip to content

Commit

Permalink
minor changes to Distributions
Browse files Browse the repository at this point in the history
  • Loading branch information
MatthewReid854 committed Dec 16, 2021
1 parent 9fafba7 commit 515e71a
Show file tree
Hide file tree
Showing 7 changed files with 635 additions and 409 deletions.
7 changes: 5 additions & 2 deletions docs/Changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,14 @@ There are also several minor changes, mainly to the documentation and a minor bu

**New features**

- Extracting confidence bounds on CDF, SF, and CHF for bounds on time or bounds on reliability can now be done directly from the distribution object that is created by the fitter, as shown `here <https://reliability.readthedocs.io/en/latest/Working%20with%20fitted%20distributions.html>`_. This required a large number of functions to be rewritten and resulted in several API changes (see below).
- Extracting confidence bounds on CDF, SF, and CHF for bounds on time or bounds on reliability can now be done directly from the distribution object that is created by the fitter, as shown `here <https://reliability.readthedocs.io/en/latest/Working%20with%20fitted%20distributions.html>`_. This required a large number of functions to be modified and resulted in several API changes (see below).
- Repairable_systems.reliability growth has been completely rewritten. This function now includes both the Duane and Crow-AMSAA reliability growth models. The parametrisation of the Duane model has been modified to match what reliasoft uses.
- New dataset called system_growth has been added to the Datasets module.

**API Changes**

- Repairable_systems.reliability growth has been completely rewritten, so the inputs and outputs are completely different. The old version of the Duane model has been replaced without deprecation. Users needing to use the old version should use v0.7.1 of reliability.
- All references to percentiles have now been replaced by quantiles. Note that the previous percentiles argument in the Fitters mandated values between 0 and 100. The quantiles argument mandates values between 0 and 1. They are otherwise the same, just a factor of 100 different. This has been changed without deprecation, so it may cause your code to break if you are using the percentiles argument.
- All references to percentiles have now been replaced by quantiles. Note that the previous percentiles argument in the Fitters mandated values between 0 and 100. The quantiles argument mandates values between 0 and 1. They are otherwise the same, just a factor of 100 different. This has been changed without deprecation, so it may cause your code to break if you are using the percentiles argument. This change was made for simplicity, since the plots show quantiles so there was no real need to have a new argument for the same thing multiplied by 100.
- The subfunctions (.CDF(), .SF(), .CHF()) for each Distribution (that has confidence intervals) now have all relevant arguments visible as args rather than kwargs. This refers to plot_CI, CI_type, CI, CI_y, CI_x. Previously plot_CI, CI_type, and CI were kwargs so your IDE would not show you these. They have been converted to args for ease of use. The arguments CI_x and CI_y are new, and are used to extract the confidence bounds from the plot of a fitted distribution object.

**Bug Fixes**
Expand All @@ -36,6 +36,9 @@ There are also several minor changes, mainly to the documentation and a minor bu
- Theory documents are finished for `censored data <https://reliability.readthedocs.io/en/latest/What%20is%20censored%20data.html>`_, `plotting positions <https://reliability.readthedocs.io/en/latest/How%20are%20the%20plotting%20positions%20calculated.html>`_, `Least Squares Estimation <https://reliability.readthedocs.io/en/latest/How%20does%20Least%20Squares%20Estimation%20work.html>`_, `Maximum Likelihood Estimation <https://reliability.readthedocs.io/en/latest/How%20does%20Maximum%20Likelihood%20Estimation%20work.html>`_, and `Confidence Intervals <https://reliability.readthedocs.io/en/latest/How%20are%20the%20confidence%20intervals%20calculated.html>`_.
- Updates pytests for new reliability_growth function.
- New document on `working with fitted distributions <https://reliability.readthedocs.io/en/latest/Working%20with%20fitted%20distributions.html>`_.
- Added several new utils functions including, distributions_input_checking, extract_CIs, unpack_single_arrays
- Within the Distributions module, the returns from each PDF, CDF, SF, HF, CHF, quantile, inverse_SF function will automatically unpack arrays of length 1. This means that if given an array of length 1 as input, you will now get a float instead of an array as output. This makes it easier for users to avoid the need to manually unpack single values for later use.
- Updated API documentation for Distributions to reflect the numerous changes to the inputs and outputs.

**Version: 0.7.1 --- Released: 26 Oct 2021**
''''''''''''''''''''''''''''''''''''''''''''
Expand Down
4 changes: 2 additions & 2 deletions docs/Fitting a specific distribution to data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -212,10 +212,10 @@ To obtain details of the quantiles (y-values from the CDF) which include the low
dist = Weibull_Distribution(alpha=500, beta=6)
data = dist.random_samples(50, seed=1) # generate some data
# this will produce the large table of percentiles below the first table of results
# this will produce the large table of quantiles below the first table of results
Fit_Weibull_2P(failures=data, quantiles=True, CI=0.8, show_probability_plot=False)
print('----------------------------------------------------------')
# repeat the process but using specified percentiles.
# repeat the process but using specified quantiles.
output = Fit_Weibull_2P(failures=data, quantiles=[0.05, 0.5, 0.95], CI=0.8)
# these points have been manually annotated on the plot using crosshairs
crosshairs()
Expand Down
51 changes: 26 additions & 25 deletions docs/Working with fitted distributions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ The confidence interval is also required, though this defaults to 0.95 for 95% c

The confidence bounds available are bounds on time or bounds on reliability. For an detailed explaination of how these are calculated, please see the `theory document <https://reliability.readthedocs.io/en/latest/How%20are%20the%20confidence%20intervals%20calculated.html#confidence-intervals-on-the-plots>`_ on confidence intervals.
The important thing to note here is that they are different, and which one you should use depends on what you want to know.
If you want to know the system reliability at a certain time, then you are specifying time (CI_x) and seeking bounds on reliability (CI_type='reliability).
If you want to know the time that the system will reach a certain reliability, then you are specifying reliability (CI_y) and seeking bounds on time (CI_type='time).
If you want to know the system reliability at a certain time, then you are specifying time (CI_x) and seeking bounds on reliability (CI_type='reliability').
If you want to know the time that the system will reach a certain reliability, then you are specifying reliability (CI_y) and seeking bounds on time (CI_type='time').
Note that these are paired, so CI_y only works with CI_type='time' and CI_x only works with CI_type='reliability'. The only exception to this is the Exponential Distribution which does not accept CI_type since bounds on time and reliability are identical.

Example 1
Expand All @@ -38,31 +38,31 @@ In this example, we see how the confidence bounds on time or reliability should
arrow_x = 25
arrow_y = 0.025
X_lower,X_point,X_upper = fit.distribution.CDF(CI_type='time',CI_y=[0.7])
plt.arrow(x=0,y=0.7,dx=X_upper[0],dy=0,color='red',head_width=arrow_y,head_length=arrow_x,length_includes_head=True)
plt.arrow(x=X_lower[0],y=0.7,dx=0,dy=-0.7,color='red',head_width=arrow_x,head_length=arrow_y,length_includes_head=True)
plt.arrow(x=X_point[0],y=0.7,dx=0,dy=-0.7,color='red',head_width=arrow_x,head_length=arrow_y,length_includes_head=True)
plt.arrow(x=X_upper[0],y=0.7,dx=0,dy=-0.7,color='red',head_width=arrow_x,head_length=arrow_y,length_includes_head=True)
X_lower,X_point,X_upper = fit.distribution.CDF(CI_type='time',CI_y=0.7)
plt.arrow(x=0,y=0.7,dx=X_upper,dy=0,color='red',head_width=arrow_y,head_length=arrow_x,length_includes_head=True)
plt.arrow(x=X_lower,y=0.7,dx=0,dy=-0.7,color='red',head_width=arrow_x,head_length=arrow_y,length_includes_head=True)
plt.arrow(x=X_point,y=0.7,dx=0,dy=-0.7,color='red',head_width=arrow_x,head_length=arrow_y,length_includes_head=True)
plt.arrow(x=X_upper,y=0.7,dx=0,dy=-0.7,color='red',head_width=arrow_x,head_length=arrow_y,length_includes_head=True)
plt.xlim(0,dist.quantile(0.99))
plt.ylim(0,1.1)
plt.text(x=0,y=0.705,s='CI_x=[0.7]',va='bottom')
plt.text(x=X_lower[0],y=0.035,s='lower',va='bottom',ha='right',rotation=90)
plt.text(x=X_point[0],y=0.035,s='point',va='bottom',ha='right',rotation=90)
plt.text(x=X_upper[0],y=0.035,s='upper',va='bottom',ha='right',rotation=90)
plt.text(x=0,y=0.705,s='CI_x=0.7',va='bottom')
plt.text(x=X_lower,y=0.035,s='lower',va='bottom',ha='right',rotation=90)
plt.text(x=X_point,y=0.035,s='point',va='bottom',ha='right',rotation=90)
plt.text(x=X_upper,y=0.035,s='upper',va='bottom',ha='right',rotation=90)
plt.title('Weibull CDF\nConfidence bounds on time')
plt.subplot(122)
Y_lower,Y_point,Y_upper = fit.distribution.CDF(CI_type='reliability',CI_x=[300])
plt.arrow(x=300,y=0,dx=0,dy=Y_upper[0],color='red',head_width=arrow_x,head_length=arrow_y,length_includes_head=True)
plt.arrow(x=300,y=Y_lower[0],dx=-300,dy=0,color='red',head_width=arrow_y,head_length=arrow_x,length_includes_head=True)
plt.arrow(x=300,y=Y_point[0],dx=-300,dy=0,color='red',head_width=arrow_y,head_length=arrow_x,length_includes_head=True)
plt.arrow(x=300,y=Y_upper[0],dx=-300,dy=0,color='red',head_width=arrow_y,head_length=arrow_x,length_includes_head=True)
Y_lower,Y_point,Y_upper = fit.distribution.CDF(CI_type='reliability',CI_x=300)
plt.arrow(x=300,y=0,dx=0,dy=Y_upper,color='red',head_width=arrow_x,head_length=arrow_y,length_includes_head=True)
plt.arrow(x=300,y=Y_lower,dx=-300,dy=0,color='red',head_width=arrow_y,head_length=arrow_x,length_includes_head=True)
plt.arrow(x=300,y=Y_point,dx=-300,dy=0,color='red',head_width=arrow_y,head_length=arrow_x,length_includes_head=True)
plt.arrow(x=300,y=Y_upper,dx=-300,dy=0,color='red',head_width=arrow_y,head_length=arrow_x,length_includes_head=True)
plt.xlim(0,dist.quantile(0.99))
plt.ylim(0,1.1)
plt.text(x=301,y=0.001,s='CI_y=[300]',va='bottom')
plt.text(x=40,y=Y_lower[0]+0.002,s='lower')
plt.text(x=40,y=Y_point[0]+0.002,s='point')
plt.text(x=40,y=Y_upper[0]+0.002,s='upper')
plt.text(x=301,y=0.001,s='CI_y=300',va='bottom')
plt.text(x=40,y=Y_lower+0.002,s='lower')
plt.text(x=40,y=Y_point+0.002,s='point')
plt.text(x=40,y=Y_upper+0.002,s='upper')
plt.title('Weibull CDF\nConfidence bounds on reliability')
plt.tight_layout()
Expand Down Expand Up @@ -175,18 +175,19 @@ We will fit a Weibull Distribution, and then from the fitted distribution, we wa
fit = Fit_Weibull_2P(failures=automotive().failures,right_censored=automotive().right_censored,show_probability_plot=False)
# we want to know the system reliability after 100000 miles
lower,point,upper = fit.distribution.SF(CI_x=[100000],CI_type='reliability',CI=0.8)
plt.scatter([100000,100000,100000],[lower[0],point[0],upper[0]],color='black')
lower,point,upper = fit.distribution.SF(CI_x=100000,CI_type='reliability',CI=0.8)
plt.scatter([100000,100000,100000],[lower,point,upper],color='black')
print('Failures:',automotive().failures)
print('Right censored:',automotive().right_censored)
print('')
print("The 80% reliability estimates at 100000 miles are:")
print("lower bound: "+"{:.2%}".format(lower[0]))
print("point estimate: "+"{:.2%}".format(point[0]))
print("upper bound: "+"{:.2%}".format(upper[0]))
print("lower bound: "+"{:.2%}".format(lower))
print("point estimate: "+"{:.2%}".format(point))
print("upper bound: "+"{:.2%}".format(upper))
plt.show()
'''
Results from Fit_Weibull_2P (95% CI):
Analysis method: Maximum Likelihood Estimation (MLE)
Expand Down
Binary file modified docs/images/confidence_bounds_trace.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 515e71a

Please sign in to comment.