Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Importer tweaks #14800

Merged
merged 17 commits into from
Jun 12, 2024
Merged

Importer tweaks #14800

merged 17 commits into from
Jun 12, 2024

Conversation

snipe
Copy link
Owner

@snipe snipe commented May 31, 2024

This PR:

  • Adds additional fields for importing
  • Adds mutators and accessors for the date fields on assets
  • Added some additional info to the validation feedback on the importer
  • Fixed an issue where the $defaultStatusLabelId could be a non-deployable status type
  • Moved asset-only date handling into the AssetImporter instead of the generic ItemImporter, since no other objects have those fields
  • Removed some pretty noisy debugging

I know we've gone back and forth on the getters/setters stuff with dates, but this makes handling bad data in imports much easier. I'm expecting a fight tho ;)

Copy link

what-the-diff bot commented May 31, 2024

PR Summary

  • Enhancements to Importer
    The importer file has been modified to include new fields and translations. Logging statements and certain imports have been eliminated for cleaner code. The deletion process has been updated to utilize a specific directory.

  • Extended AssetImporter Functionalities
    Additional fields have been integrated into the __construct method and the createAssetIfNotExists method in AssetImporter. Date handling for the latter has been improved.

  • Importer.php Updates
    A new method parseOrNullDate has been implemented for parsing dates. Additionally, the date field has been removed from the createAssetIfNotExists method.

  • Changes to ItemImporter
    The handle method can now manage new date fields. The asset_eol_date field has been removed from the createOrFetchAssetModel method for smoother operation.

  • Asset.php Adjustments
    Accessors and mutators for last_checkin, last_checkout, last_audit_date, and asset_eol_date fields have been added to increase flexibility in managing them.

  • Updates to Hardware View
    The expected_checkin field has been removed from the view to streamline the user interface.

  • Refinement in Importer View Blade
    Extraneous debugging statements have been removed for a cleaner codebase.

  • Test Case Improvements
    The test case in AssetStoreTest.php has been updated to use the new last_audit_date field format, better reflecting the recent changes made.

Copy link
Collaborator

@marcusmoore marcusmoore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't done a full review of this but I'm going to "request changes" just to get the first questions posted and also say that the addition of the accessors/mutators probably warrants checking the places they are used throughout the codebase to make sure we're not causing any unexpected issues. I'm thinking we might be double-parsing a date here or there now which shouldn't cause an issue but should still be double-checked.

app/Importer/Importer.php Outdated Show resolved Hide resolved
app/Importer/Importer.php Outdated Show resolved Hide resolved
Comment on lines 986 to 1024
public function getLastCheckoutAttribute($value)
{
return $this->attributes['last_checkout'] = $value ? Carbon::parse($value)->format('Y-m-d H:i:s') : null;
}

public function setLastCheckoutAttribute($value)
{
$this->attributes['last_checkout'] = $value ? Carbon::parse($value)->format('Y-m-d H:i:s') : null;
}

public function getLastCheckinAttribute($value)
{
return $this->attributes['last_checkin'] = $value ? Carbon::parse($value)->format('Y-m-d H:i:s') : null;
}

public function setLastCheckinAttribute($value)
{
$this->attributes['last_checkin'] = $value ? Carbon::parse($value)->format('Y-m-d H:i:s') : null;
}

public function getLastAuditDateAttribute($value)
{
return $this->attributes['last_audit_date'] = $value ? Carbon::parse($value)->format('Y-m-d H:i:s') : null;
}

public function setLastAuditDateAttribute($value)
{
$this->attributes['last_audit_date'] = $value ? Carbon::parse($value)->format('Y-m-d H:i:s') : null;
}

public function getAssetEolDateAttribute($value)
{
return $this->attributes['asset_eol_date'] = $value ? Carbon::parse($value)->format('Y-m-d') : null;
}

public function setAssetEolDateAttribute($value)
{
$this->attributes['asset_eol_date'] = $value ? Carbon::parse($value)->format('Y-m-d') : null;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this gets merged we should remove these fields from $casts since we're overwriting the functionality. I wouldn't be surprised if keeping them in $casts while additionally handling the accessors and mutators ourselves causes a bug down the line because the framework (and/or libraries) assume it is in control of casting the included properties.

Additionally, if this is getting merged, you may want to consider using the new accessor/mutator syntax that groups them into one method for each field so it is easier to read.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Casting has never worked the way we expected anyway tbh, since it only kicks in when the data is serialized IIRC. It's why we've gone back and forth so many times :(

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dates have especially always behaved weirdly in the importer (since that ignores the casting), and I can't help but think that using accessors and mutators is the most consistent path.

Copy link
Collaborator

@spencerrlongg spencerrlongg Jun 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really wish we wouldn't do this. A few reasons:

  1. Framework expectations, 90% of the time I'm grabbing a date from a model I'm expecting it to be cast so that I have carbon already available. I'd love to see us work with the framework more often than against it.
    We should follow framework convention as much as we possibly can - I think that's what will help us iterate more quickly and possibly even see more involvement from the FOSS community.
  2. The getters I feel are actively or have the potential to be actively breaking casting, as Marcus said above.
  3. If in the future we want to use a date-time-picker a little more tightly integrated it likely wouldn't work with this out of the box (think something like filament forms: https://filamentphp.com/docs/3.x/forms/fields/date-time-picker)
  4. On the setter front, with casts, as long as we receive something that's parsable from Carbon we can save it directly to the model instead of having to format it manually.

Casting has never worked the way we expected anyway tbh, since it only kicks in when the data is serialized IIRC. It's why we've gone back and forth so many times :(

It's cast as a Carbon object, then the formatting kicks in when an object doesn't make sense (when it's serialized).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Casting has never worked the way we expected anyway tbh, since it only kicks in when the data is serialized IIRC

I was caught off guard by this but it makes sense to me now: you get all the niceties of a carbon object without having to do the casting yourself. So we can compare against other objects or format differently while the model is being passed around the application (etc) and then it falls back to default formatting when leaving the application layer.

Overall, I agree with Spencer's points and think we should lean into the framework as much as possible.

Copy link
Owner Author

@snipe snipe Jun 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The getters I feel are actively or have the potential to be actively breaking casting, as Marcus said above.

The casting has literally NEVER worked. It's only invoked in JSON and serialization, and we already use a formatting helper for that.

On the setter front, with casts, as long as we receive something that's parsable from Carbon we can save it directly to the model instead of having to format it manually.

This is literally not what my own tests with this PR have shown. Please, by all means, prove me wrong. Regardless of how I tried to parse the date, if it was blank, it came through as 0000-00-00 or 0000-00-00 00:00:00, even while checking for nulls, try/catching whether or not Carbon could parse it, etc.

If I was missing something, I'm glad to take the correction, but this has never worked the way we wanted/expected, and I need stuff that actually works, versus what's "correct".

Copy link
Owner Author

@snipe snipe Jun 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we can compare against other objects or format differently while the model is being passed around the application (etc) and then it falls back to default formatting when leaving the application layer.

@marcusmoore The only time we really compare dates, we've already got helpers for that (since we always want to display the date according to the user's own preferences in the settings.) Otherwise, it's not a carbon object, just a date or date time field in the database and we're comparing those at a SQL level.

@snipe snipe requested a review from marcusmoore June 5, 2024 09:11
snipe added 17 commits June 5, 2024 10:28
Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
I have no idea why this is necessary

Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
Signed-off-by: snipe <snipe@snipe.net>
@snipe
Copy link
Owner Author

snipe commented Jun 5, 2024

If one of the two of you want to take a crack at this yourself, I'm happy to retire this PR, but I don't make decisions like mutators etc lightly. If you can make it work without them, go nuts.

@marcusmoore
Copy link
Collaborator

If one of the two of you want to take a crack at this yourself, I'm happy to retire this PR, but I don't make decisions like mutators etc lightly. If you can make it work without them, go nuts.

I'd like to give it a go but don't want to hold up a legitimate bug fix. Maybe we just go ahead with this PR as is and me/Spencer can take a swing at it afterwards?

@snipe snipe merged commit cacb5d6 into develop Jun 12, 2024
8 checks passed
@snipe snipe deleted the fixes/importer_tweaks branch June 12, 2024 11:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants