TensorFlow Model Analysis 0.46.0

Major Features and Improvements

• Removes the metrics modules from experimental now that it is migrated to
  py-ml-metrics package.
• Adds Constituent Flip Rate Metrics: SymmetricFlipRate, NegToNegFlipRate,
  NegToPosFlipRate, PosToNegFlipRate, PosToPosFlipRate.
• Depend on tensorflow-estimator package explicitly.

Bug fixes and other Changes

• Fix the bug about batching unsized numpy arrays.

Breaking Changes

• Removes attrs requirement.
• Consolidate Matrix definition for semantic segmentation confusion matrix
  metrics.
• Provide AggregateFn and interface and default call impl to adapt TFMA
  metrics combiner for in-process call.
• Move Mean metrics from experimental to metrics.
• Fix the bug of size estimator failure.
• Depends on tensorflow>=2.15.0,<2.16.
• Fix the failure in testMeanAttributions.
• Fix the input type mismatch in metric_specs_tests between bool and None.
• Fix the failure in the slice test due to beam type hints check.
• Fix the failure in metric_specs test, all TFMA deps on keras are
  keras 2.
• Depends on apache-beam[gcp]>=2.53.0,<3 for Python 3.11 and on
  apache-beam[gcp]>=2.47.0,<3 for 3.9 and 3.10.
• Depends on protobuf>=4.25.2,<5 for Python 3.11 and on protobuf>3.20.3,<5
  for 3.9 and 3.10.
• Update the minimum Bazel version required to build TFMA to 6.1.0
• Refactors BooleanFlipRates computations to a combiner (flip_counts) and a
  DerivedMetricComputation (flip_rates).

Deprecations

• Deprecated python 3.8 support.

TensorFlow Model Analysis 0.45.0

Major Features and Improvements

• Add F1, False positive rate, and Accuracy into the confusion matrix plot.
• Add support for setting top_k and class_id at the same time for confusion
  matrix metrics.
• Add the false positive for semantic segmentation metrics.
• Add Mean Metric (experimental) which calculates the mean of any feature. *.
  Adds support of output_keypath to ModelSignatureDoFn to explicitly set a
  chain of output keys in the multi-level dict (extracts). Adds output_keypath
  to common prediction extractors.

Bug fixes and other Changes

• Fix the bug that SetMatchRecall is always 1 when top_k is set.
• Add 'tfma_eval' model_type in model_specs as the identifier for
  eval_saved_model, allowing signature='eval' to now be used with other model
  types.
• Add "materialized_prediction" model type to allow users bypassing model
  inference explicitly.
• Depends on pyarrow>=10,<11.
• Depends on apache-beam>=2.47,<3.
• Depends on numpy>=1.23.0.
• Depends on tensorflow>=2.13.0,<3.

Breaking Changes

• Depend on PIL for image related metrics.
• Separate extract_key from signature names in ModelSignaturesDoFn.

Deprecations

• N/A

TensorFlow Model Analysis 0.44.0

Major Features and Improvements

• Add BinaryCrossEntropy and CategoricalCrossEntropy.
• Add MeanAbsolutePercentageError and MeanSquaredLogarithmicError
• Add SetMatchPrecision and SetMatchRecall

Bug fixes and other Changes

• Fix for jupiter notebook
• Fix element dimension inconsistency when some of the extracts have missing
  key.
• Add public visibility to the servo beam extractor.
• Fix for bug where binary_confusion_matrices with different class_weights are
  considered identical and deduplicated.
• Fixes bug where positive / negative axes labels are reversed in prediction
  distribution plot.
• Depends on numpy~=1.22.0.
• Modify ExampleCount to not depend on labels and predictions.
• Add class_id info into sub_key in metric_key for object detection confusion
  matrix metrics.
• Add class_id info into sub_key in plot_key for object detection confusion
  matrix plot.
• Fix a bug that auto_pivot dropped nan when deciding which columns are
  multivalent for pivoting.
• Depends on tensorflow>=2.12.0,<2.13.
• Depends on protobuf>=3.20.3,<5.
• Depends on tfx-bsl>=1.13.0,<1.14.0.
• Depends on tensorflow-metadata>=1.13.1,<1.14.0.

Breaking Changes

• N/A

Deprecations

Deprecated python3.7 support.

TensorFlow Model Analysis 0.43.0

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Depends on tensorflow>=2.11,<3
• Depends on tfx-bsl>=1.2.0,<1.13.0.
• Depends on tensorflow-metadata>=1.12.0,<1.13.0.

Breaking Changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.42.0

Major Features and Improvements

• This is the last version that supports TensorFlow 1.15.x. TF 1.15.x support
  will be removed in the next version. Please check the
  TF2 migration guide to migrate
  to TF2.
• Add BooleanFlipRate metric for comparing thresholded predictions between
  multiple models.
• Add CounterfactualPredictionsExtractor for computing predictions on modified
  inputs.

Bug fixes and other Changes

• Add support for parsing the Predict API prediction log output to the
  experimental TFX-BSL PredictionsExtractor implementation.

• Add support for parsing the Classification API prediction log output to the
  experimental TFX-BSL PredictionsExtractor implementation.

• Update remaining predictions_extractor_test.py tests to cover
  PredictionsExtractorOSS. Fixes a pytype bug related to multi tensor output.

• Depends on tensorflow>=1.15.5,<2 or tensorflow>=2.10,<3

• Apply changes in the latest Chrome browser

• Add InferneceInterface to experimental PredictionsExtractor implementation.

• Stop returning empty example_ids metric from binary_confusion_matrices
  derived computations when example_id_key is not set but use_histrogam is
  true.

• Add transformed features lookup for NDCG metrics query key and gain key.

• Deprecate BoundedValue and TDistribution in ConfusionMatrixAtThresholds.

• Fix a bug that dataframe auto_pivot fails if there is only Overall slice.

• Use SavedModel PB to determine default signature instead of loading the
  model.

• Reduce clutter in the multi-index columns and index in the experimental
  dataframe auto_pivot util.

Breaking Changes

• N/A

Deprecations

• N/A

Version 0.41.1

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Move the version to top of init.py since the original "from
  tensorflow_model_analysis.sdk import *" will not import private symbol.

Breaking Changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.41.1

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Move the version to top of init.py since the original "from tensorflow_model_analysis.sdk import *" will not import private symbol.

Breaking Changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.41.0

Major Features and Improvements

• Add COCO object detection metrics, object detection related utilities,
  objection detection opitons in binary confusion matrix, Precision At Recall,
  and AUC. Add MaxRecall metric.
• Add support for parsing sparse tensors with explicit tensor representations
  via TFXIO.

Bug fixes and other Changes

• Add score_distribution_plot.
• Separate the Predictions Extractor into two extractors.
• Update PredictionsExtractor to support backwards compatibility with the
  Materialized Predictions Extractor.
• Depends on apache-beam[gcp]>=2.40,<3.
• Depends on pyarrow>=6,<7.
• Update merge_extracts with an option to skip squeezing one-dim arrays.
  Update split_extracts with an option to expand zero-dim arrays.
• Added experimental bulk inference implementation to PredictionsExtractor.
  Currently only supports the RegressionAPI.

Breaking Changes

• Adds multi-index columns for view.experimental.metrics_as_dataframe util.
• Changes SymmetricPredictionDifference output type from array to scalar.

Deprecations

• N/A

TensorFlow Model Analysis 0.40.0

Major Features and Improvements

• Add object detection related utilities.

Bug fixes and other Changes

• Depends on tensorflow>=1.15.5,<2 or tensorflow>=2.9,<3
• Fix issue where labels with -1 values are one-hot encoded when they
  shouldn't be ## Breaking Changes
• Depends on tfx-bsl>=1.9.0,<1.10.0.
• Depends on tensorflow-metadata>=1.9.0,<1.10.0.

Breaking Changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.39.0

Major Features and Improvements

• SqlSliceKeyExtractor now supports slicing on transformed features.

Bug fixes and other Changes

• Depends on tfx-bsl>=1.8.0,<1.9.0.
• Depends on tensorflow-metadata>=1.8.0,<1.9.0.
• Depends on apache-beam[gcp]>=2.38,<3.

Breaking Changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.38.0

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Fixes issue attempting to parse metrics, plots, and attributions without a
  format suffix.
• Fixes the non-deterministic key ordering caused by proto string
  serialization in metrics validator.
• Update variable name to respectful terminology, rebuild JS
• Fixes issues preventing standard preprocessors from being applied.
• Allow merging extracts including sparse tensors with different dense shapes.

Breaking Changes

• MetricsPlotsAndValidationsWriter will now write files with an explicit
  output format suffix (".tfrecord" by default). This should only affect
  pipelines which directly construct MetricsPlotsAndValidationWriter
  instances and do not set output_file_format. Those which use
  default_writers() should be unchanged.
• Batched based extractors previously worked off of either lists of dicts of
  single tensor values or arrow record batches. These have been updated to be
  based on dicts with batched tensor values at the leaves.
• Depends on
  tensorflow>=1.15.5,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.
• Depends on tfx-bsl>=1.7.0,<1.8.0.
• Depends on tensorflow-metadata>=1.7.0,<1.8.0.
• Depends on apache-beam[gcp]>=2.36,<3.

Deprecations

• N/A