Fix TD3 DDPG Implementation: Move Sampling Inside Gradient Step LoopFix implementation mismatch

[alessandroassirelli98]

This pull request addresses a discrepancy between the original TD3 and DDPG paper's algorithm and the current implementation in the repository. Specifically, the original implementation performs the sampling step outside of the gradient step loop, which diverges from the methodology outlined in the paper. We have corrected this by moving the sampling process inside the gradient step loop, aligning the implementation more closely with the intended algorithmic procedure described in the original paper and SpinningUp description.

[Toni-SM]

Hi @alessandroassirelli98

Great!

Please, also update:

• docs

  • skrl/docs/source/api/agents/ddpg.rst

    Lines 50 to 51 in 631613a

    	| :green:`# sample a batch from memory`
    	| [:math:`s, a, r, s', d`] :math:`\leftarrow` states, actions, rewards, next_states, dones of size :guilabel:`batch_size`

  • skrl/docs/source/api/agents/td3.rst

    Lines 50 to 51 in 631613a

    	| :green:`# sample a batch from memory`
    	| [:math:`s, a, r, s', d`] :math:`\leftarrow` states, actions, rewards, next_states, dones of size :guilabel:`batch_size`

• CHANGELOG under the following version (create it if it doesn't exist)

  ## [1.2.0] - Unreleased
  ### Fixed
  - YOUR_FIX_IMPLEMENTATION_SHORT_DESCRIPTION
  

[alessandroassirelli98]

Hi @Toni-SM !
I see the current version is 1.1.0.
Should I just add a comment on that version in the changelog, like so?

[Toni-SM]

I see the current version is 1.1.0.

Version 1.1.0 is already released (see: https://github.com/Toni-SM/skrl/releases)

New additions/changes/fixes (like the one you proposed) will be released under version 1.2.0. Please, use this for CHANGELOG:

## [1.2.0] - Unreleased
### Fixed
- YOUR_FIX_IMPLEMENTATION_SHORT_DESCRIPTION