Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend image blending module #556

Merged
merged 1 commit into from Jan 11, 2020
Merged

Extend image blending module #556

merged 1 commit into from Jan 11, 2020

Conversation

aleju
Copy link
Owner

@aleju aleju commented Jan 9, 2020

With this patch the available augmenters for
alpha-blending of images are significantly extended.
There are now new blending augmenters available to
alpha-blend acoording to:

  • Some randomly chosen colors. (BlendAlphaSomeColors)
  • Linear gradients. (BlendAlphaHorizontalLinearGradient,
    BlendAlphaVerticalLinearGradient)
  • Regular grids and checkerboard patterns. (BlendAlphaRegularGrid,
    BlendAlphaCheckerboard)
  • Only at locations that overlap with specific segmentation class
    IDs (or the inverse of that). (BlendAlphaSegMapClassIds)
  • Only within bounding boxes with specific labels (or the inverse
    of that). (BlendAlphaBoundingBoxes)

This allows to e.g. randomly remove some colors while leaving
other colors unchanged (BlendAlphaSomeColors(Grayscale(1.0))),
to change the color of some objects
(BlendAlphaSegMapClassIds(AddToHue((-256, 256)))), to add
cloud-patterns only to the top of images
(BlendAlphaVerticalLinearGradient(Clouds())) or to apply
augmenters in some coarse rectangular areas (e.g.
BlendAlphaRegularGrid(Multiply(0.0)) to achieve a similar
effect to CoarseDropout or
BlendAlphaRegularGrid(AveragePooling(8)) to pool in equally
coarse image sub-regions).

Other mask-based alpha blending techniques can be achieved by
subclassing IBatchwiseMaskGenerator and providing an
instance of such a class to BlendAlphaMask.

This patch also changes the naming of the blending augmenters
as follows:

  • Alpha -> BlendAlpha
  • AlphaElementwise -> BlendAlphaElementwise
  • SimplexNoiseAlpha -> BlendAlphaSimplexNoise
  • FrequencyNoiseAlpha -> BlendAlphaFrequencyNoise
    The old names are now deprecated.
    Furthermore, the parameters first and second, which were
    used by all blending augmenters, have now the names foreground
    and background.

List of changes:

  • Add imgaug.augmenters.blend.BlendAlphaMask, which uses
    a mask generator instance to generate per batch alpha masks and
    then alpha-blends using these masks.
  • Add imgaug.augmenters.blend.BlendAlphaSomeColors.
  • Add imgaug.augmenters.blend.BlendAlphaHorizontalLinearGradient.
  • Add imgaug.augmenters.blend.BlendAlphaVerticalLinearGradient.
  • Add imgaug.augmenters.blend.BlendAlphaRegularGrid.
  • Add imgaug.augmenters.blend.BlendAlphaCheckerboard.
  • Add imgaug.augmenters.blend.BlendAlphaSegMapClassIds.
  • Add imgaug.augmenters.blend.BlendAlphaBoundingBoxes.
  • Add imgaug.augmenters.blend.IBatchwiseMaskGenerator,
    an interface for classes generating masks on a batch-by-batch
    basis.
  • Add imgaug.augmenters.blend.StochasticParameterMaskGen,
    a helper to generate masks from StochasticParameter instances.
  • Add imgaug.augmenters.blend.SomeColorsMaskGen, a generator
    that produces masks marking randomly chosen colors in images.
  • Add imgaug.augmenters.blend.HorizontalLinearGradientMaskGen,
    a linear gradient mask generator.
  • Add imgaug.augmenters.blend.VerticalLinearGradientMaskGen,
    a linear gradient mask generator.
  • Add imgaug.augmenters.blend.RegularGridMaskGen,
    a checkerboard-like mask generator where every grid cell has
    a random alpha value.
  • Add imgaug.augmenters.blend.CheckerboardMaskGen,
    a checkerboard-like mask generator where every grid cell has
    the opposite alpha value of its 4-neighbours.
  • Add imgaug.augmenters.blend.SegMapClassIdsMaskGen, a
    segmentation map-based mask generator.
  • Add imgaug.augmenters.blend.BoundingBoxesMaskGen, a bounding
    box-based mask generator.
  • Add imgaug.augmenters.blend.InvertMaskGen, an mask generator
    that inverts masks produces by child generators.
  • Change imgaug.parameters.SimplexNoise and
    imgaug.parameters.FrequencyNoise to also accept (H, W, C)
    sampling shapes, instead of only (H, W).
  • Refactor AlphaElementwise to be a wrapper around
    BlendAlphaMask.
  • Rename Alpha to BlendAlpha.
    Alpha is now deprecated.
  • Rename AlphaElementwise to BlendAlphaElementwise.
    AlphaElementwise is now deprecated.
  • Rename SimplexNoiseAlpha to BlendAlphaSimplexNoise.
    SimplexNoiseAlpha is now deprecated.
  • Rename FrequencyNoiseAlpha to BlendAlphaFrequencyNoise.
    FrequencyNoiseAlpha is now deprecated.
  • Rename arguments first and second to foreground and
    background in BlendAlpha, BlendAlphaElementwise,
    BlendAlphaSimplexNoise and BlendAlphaFrequencyNoise.
  • Change imgaug.parameters.handle_categorical_string_param()
    to allow parameter valid_values to be None.
  • Fix a wrong error message in
    imgaug.augmenters.color.change_colorspace_().
  • Remove imgaug.augmenters.color.GrayscaleColorwise
    and imgaug.augmenters.color.RemoveSaturationColorwise
    as these are now covered by BlendAlphaSomeColors.

@codecov-io
Copy link

codecov-io commented Jan 11, 2020

Codecov Report

Merging #556 into master will increase coverage by 0.04%.
The diff coverage is 95.24%.

@@            Coverage Diff            @@
##           master    #556      +/-   ##
=========================================
+ Coverage   96.45%   96.5%   +0.04%     
=========================================
  Files          40      40              
  Lines       14230   14496     +266     
=========================================
+ Hits        13725   13988     +263     
- Misses        505     508       +3

With this patch the available augmenters for
alpha-blending of images are significantly extended.
There are now new blending augmenters available to
alpha-blend acoording to:
* Some randomly chosen colors. (`BlendAlphaSomeColors`)
* Linear gradients. (`BlendAlphaHorizontalLinearGradient`,
  `BlendAlphaVerticalLinearGradient`)
* Regular grids and checkerboard patterns. (`BlendAlphaRegularGrid`,
  `BlendAlphaCheckerboard`)
* Only at locations that overlap with specific segmentation class
  IDs (or the inverse of that). (`BlendAlphaSegMapClassIds`)
* Only within bounding boxes with specific labels (or the inverse
  of that). (`BlendAlphaBoundingBoxes`)

This allows to e.g. randomly remove some colors while leaving
other colors unchanged (`BlendAlphaSomeColors(Grayscale(1.0))`),
to change the color of some objects
(`BlendAlphaSegMapClassIds(AddToHue((-256, 256)))`), to add
cloud-patterns only to the top of images
(`BlendAlphaVerticalLinearGradient(Clouds())`) or to apply
augmenters in some coarse rectangular areas (e.g.
`BlendAlphaRegularGrid(Multiply(0.0))` to achieve a similar
effect to `CoarseDropout` or
`BlendAlphaRegularGrid(AveragePooling(8))` to pool in equally
coarse image sub-regions).

Other mask-based alpha blending techniques can be achieved by
subclassing `IBatchwiseMaskGenerator` and providing an
instance of such a class to `BlendAlphaMask`.

This patch also changes the naming of the blending augmenters
as follows:
* `Alpha` -> `BlendAlpha`
* `AlphaElementwise` -> `BlendAlphaElementwise`
* `SimplexNoiseAlpha` -> `BlendAlphaSimplexNoise`
* `FrequencyNoiseAlpha` -> `BlendAlphaFrequencyNoise`
The old names are now deprecated.
Furthermore, the parameters `first` and `second`, which were
used by all blending augmenters, have now the names `foreground`
and `background`.

List of changes:
* Add `imgaug.augmenters.blend.BlendAlphaMask`, which uses
  a mask generator instance to generate per batch alpha masks and
  then alpha-blends using these masks.
* Add `imgaug.augmenters.blend.BlendAlphaSomeColors`.
* Add `imgaug.augmenters.blend.BlendAlphaHorizontalLinearGradient`.
* Add `imgaug.augmenters.blend.BlendAlphaVerticalLinearGradient`.
* Add `imgaug.augmenters.blend.BlendAlphaRegularGrid`.
* Add `imgaug.augmenters.blend.BlendAlphaCheckerboard`.
* Add `imgaug.augmenters.blend.BlendAlphaSegMapClassIds`.
* Add `imgaug.augmenters.blend.BlendAlphaBoundingBoxes`.
* Add `imgaug.augmenters.blend.IBatchwiseMaskGenerator`,
  an interface for classes generating masks on a batch-by-batch
  basis.
* Add `imgaug.augmenters.blend.StochasticParameterMaskGen`,
  a helper to generate masks from `StochasticParameter` instances.
* Add `imgaug.augmenters.blend.SomeColorsMaskGen`, a generator
  that produces masks marking randomly chosen colors in images.
* Add `imgaug.augmenters.blend.HorizontalLinearGradientMaskGen`,
  a linear gradient mask generator.
* Add `imgaug.augmenters.blend.VerticalLinearGradientMaskGen`,
  a linear gradient mask generator.
* Add `imgaug.augmenters.blend.RegularGridMaskGen`,
  a checkerboard-like mask generator where every grid cell has
  a random alpha value.
* Add `imgaug.augmenters.blend.CheckerboardMaskGen`,
  a checkerboard-like mask generator where every grid cell has
  the opposite alpha value of its 4-neighbours.
* Add `imgaug.augmenters.blend.SegMapClassIdsMaskGen`, a
  segmentation map-based mask generator.
* Add `imgaug.augmenters.blend.BoundingBoxesMaskGen`, a bounding
  box-based mask generator.
* Add `imgaug.augmenters.blend.InvertMaskGen`, an mask generator
  that inverts masks produces by child generators.
* Change `imgaug.parameters.SimplexNoise` and
  `imgaug.parameters.FrequencyNoise` to also accept `(H, W, C)`
  sampling shapes, instead of only `(H, W)`.
* Refactor `AlphaElementwise` to be a wrapper around
  `BlendAlphaMask`.
* Rename `Alpha` to `BlendAlpha`.
  `Alpha` is now deprecated.
* Rename `AlphaElementwise` to `BlendAlphaElementwise`.
  `AlphaElementwise` is now deprecated.
* Rename `SimplexNoiseAlpha` to `BlendAlphaSimplexNoise`.
  `SimplexNoiseAlpha` is now deprecated.
* Rename `FrequencyNoiseAlpha` to `BlendAlphaFrequencyNoise`.
  `FrequencyNoiseAlpha` is now deprecated.
* Rename arguments `first` and `second` to `foreground` and
  `background` in `BlendAlpha`, `BlendAlphaElementwise`,
  `BlendAlphaSimplexNoise` and `BlendAlphaFrequencyNoise`.
* Change `imgaug.parameters.handle_categorical_string_param()`
  to allow parameter `valid_values` to be `None`.
* Fix a wrong error message in
  `imgaug.augmenters.color.change_colorspace_()`.
* Remove `imgaug.augmenters.color.GrayscaleColorwise`
  and `imgaug.augmenters.color.RemoveSaturationColorwise`
  as these are now covered by `BlendAlphaSomeColors`.
@aleju aleju changed the title Extend blending augmenters Extend image blending module Jan 11, 2020
@aleju aleju merged commit 744db3e into master Jan 11, 2020
@aleju aleju deleted the blend_mask branch January 11, 2020 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants