/
Changelog.txt
187 lines (136 loc) · 5.34 KB
/
Changelog.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.7.5.2]
Fix for issue 220
removed numpy. distutils since it is deprecated in pythonn 3.12
## [0.7.5.1]
Fixed bug 306
## [0.7.5]
Allow detX and inverse to both be set
## [0.7.4.8]
## [0.7.4.9]
Added analyze(... invere=True ... ) to support original Double Descent experiment
## [0.7.4.7]
Implementing torch.svd_lowrank
default SVD_METHOD = FAST_SVD *(not accurate)
## [0.7.4.6]
LASER release
Fixed bugs in SVDSmoothing implementation
changed default to method=DETX and also percent=0.8 (very conservative) for any SVD
also added an AUTO option, which is not working yet (i.e poor results)
## [0.7.4.3]
Changed peft=True tp default to only analyzing the lora_BA matrices
## [0.7.4.2]
Added ability to skip layers by "-name"
## [0.7.4.1]
Added support for PEFT / LORA models
Reduces memory for models in safetensors format
## [0.7.3.1]
minor Bug fix in Keras layers for SVDSmoothing
## [0.7.3]
Can process safetensors without layer_map (for testing(
## [0.7.2.6]
trying to fix bug in analyze_eigenvectors
## [0.7.2.3]
## [0.7.2.4]
minor bug fixes in the 0.7.x release
## [0.7.2]
support for base_model
## [0.7.1.8]
added support for safetensors format
rename test: Test_VGG11_StateDict -> Test_VGG11_PyStateDict
Issue 261 fixed bug in reading model config
Issue 260 add min singular value to the dataframe
## [0.7.1.7]
Issue 259, fixed torch.load() -> cpu
and start_ids for Test_VGG11_StateDict test
## [0.7.1.6]
removed restriction on start_ids , issue #255
## [0.7.1.5]
Problem with 0.7.1.4. deploy to pypi ?
## [0.7.1.4]
Warnings mixed up. Fixed now. Unit test pending
## [0.7.1.3] #245 stil working on this, debugging on Google Colab
## [0.7.1.2]
Addressing issue #245, making loading of PyStateDictFileLayer weights lazy
## [0.7.1.1]
Fixed issue #243, add ability to extract state_dict from model
## [0.7.1]
Fixed critical bug in ONNX models (#issue 233)
## [0.7.0.9]
Fixed critical bugs in the error handling
## [0.7]
Sorry, this is a bit out of date; will update shortly [chm]
## [0.6.6]
could not deploy 0.6.5 to testpypi, upgraded to 0.6.6
## [0.6.5]
Preparation for release 0.7
Refactored WWLayer to WWLayer, WWFrameworkLayer, WWKerasLayer, ...
Added support for pyTorch StateDict format (in memory)
SVD Fast method added (GPU enabled, on by default) #199
PowerLaw fits now have default xmax=None
WW_PowerLaw method is now the default PL fit
Simple CKA distance metric added (#148)
## [0.6.4.1 (testpypi)]]
added plot esd5 to analyze() , plot of xmins vs alphas
added fit_entropy() metric to fit_powerlaw()
## [0.6.4 (pypi)] - 2023-01-22]
new fix_fingers='clip_xmax' implementation #160
num_fingers added to dataframe
added sigma to Log-log Plots #162
detX value is now on same scale as xmin
params no longer modified as side effect #156
more extensive unit tests
check longname now part of unit tests
distances NOT working yet in full
removed explicit tensorflow and torch dependences #154
Q in details data frame #147
new method signature: rescaled_evals, Wscale = RMT_Util.rescale_eigenvalues(evals)
removed telly
## [0.6.3.4 (testpypi)] - 2023-01-20
fixed bug in fix_fingers
## [0.6.3.3 (testpypi)] - 2023-01-17
Fixed unit tests, removed CKA and EUCLIDIAN tests for now
Note: min_evals default different for analyze() and describe()
## [0.6.3.1 (testpypi)] - 2023-01-16
minor bug fixes in the unit tests associated with issue 156
## [0.6.3.1 (testpypi)] - 2023-01-15
issue 160: updated fix_fingers='clip_xmax', now only checks alpha decreases by 1 or more
issue 156: implemented everywhere: params = DEFAULT_PARAMS.copy()
added examples/WW-Albert.ipynb
## [0.6.3 (in progress on trunk)] - 2023-01-05
removed explcit dependencies on tensorflow pytoch, and onnx
infers framework from model , then imports (tf | torch |onnx)
add ability to extract and read pytorch statedict files directly
removed combine_weights_and_bias as default, not implemented anywhere
fixed unit tests, moved model and watcher to object constructor
## [0.6.2 (in progress on trunk)] - 2022-11-27
### Added
Q in details dataframe
raw_distances implemented as distances(..., method=RAW)
distances(): added method=EUCLIDEAN | CKA
unit test(s) for distances
## [0.6.x (in progress on trunk)] - 2022-10-05
### Added
examples directory with notebooks and README
start_ids=1 added to analyze(), describe(), SVDSmoothing(), SVDSharpness() for issue #65
tqdm to setup.py
matplotlib-inline to setup.py
### Removed
watcher.analyze(..., normalize=...).
pyRMT
### Changed
powerlaw.Fit() wrapped in method to supress stdout and stderr
analyze(vectors=True) is turned back on as default
setup.py to use metaclass , not import ww, to allow better handling of dependencies
details.append() changed to details=pd.contact([details, data]) #137
models.vgg11(pretrained=True) to models.vgg11(weights='VGG11_Weights.IMAGENET1K_V1')
same for resnet
### Fixed
TPL and E_TPL fits use xmin_distribution='trunctated_power_law'
replaced np.linalg.eig and np.linalg.svd with scipy version because they are faster and more numerically stable
analyze(savefig=False) now works in all cases
Fixed minor issues: 65, #69, #72, #82
tests/test.py Test_TFBert.test_num_layers added (but #74 kept)