Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

check ranking command and jobs #82

Merged
merged 30 commits into from
Apr 13, 2021
Merged

Conversation

arwaawan3
Copy link
Contributor

Would appreciate some feedback to see what needs to be changed. Thanks!

@arwaawan3 arwaawan3 marked this pull request as draft April 1, 2021 19:22
- fix psalm errors
- add serp api key
- move nova resource tests into nova test folder
- move nova resource tests into nova test folder
@@ -23,6 +24,11 @@ protected static function boot()
static::saving(function ($keyword) {
$keyword->phrase = strtolower($keyword->phrase);
});

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we sure we want to add this as a global scope? It seems like this should just be a customer query builder or local scope since the user is going to want to see their inactive keywords somewhere

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying to do it like the locations package. But I think you are allowed to do something like static:: withoutGlobalScope('active')->get() right?

joshtorres and others added 9 commits April 3, 2021 23:39
- add is_active to keyword table
- add active scope to keyword model
- add check keyword rankings commands
- add shorcuts to actions inside of keyword and ranking models
- update CheckRankingCommand.php to use action
- add an exception to GetOrganicResults.php so we know when it doesn't return any data
- add setActive method to Keyword, which updates appropriate date fields
- change tests to fail so we know we need to update them
- add test comments so tests are run
@arwaawan3
Copy link
Contributor Author

arwaawan3 commented Apr 11, 2021

I have removed the unique constraint from the results table, it doesn't work for sitelinks which would all be the same as technically (e.g. attached image - rows 150-153 are all parent and sitelinks to wikipedia results). Also, I was able to test the local and organic results individually but the chain isn't working so the local results dont get executed after the organic. Maybe it's too late and I need to look at it again in the morning 😅
Screen Shot 2021-04-11 at 12 53 36 AM

@arwaawan3 arwaawan3 marked this pull request as ready for review April 12, 2021 05:32
@@ -14,6 +14,7 @@ public function up()
$table->id();
$table->string('phrase')->index()->unique();
$table->string('type')->index(); // Example: 'Branded', 'Generic', 'Local'
$table->boolean('is_active')->index();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to not have this in the database. Can we just create a scope for the active keywords based on the datetime fields for tracking_requested_at and tracking_stopped_at?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I will put that back.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Let's discuss with the team and we may want to add this back if there are performance gains.

CC: @pdbreen @jasonmccreary @joshtorres @phuclh

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @drewroberts I had asked Arwa to add this since we are going to be using that active scope for everything in regards to keywords. Yes I was thinking it would be better for performance but that just depends on how big the keyword database will get. If it's not going to get that big then it probably wouldn't make much of a difference. I am curious to know the others' opinions on it though.

@@ -22,7 +22,7 @@ public function up()

// Don't need timestamps since can use created_at timestamp of the ranking class

$table->unique(['ranking_id', 'type', 'position'], 'result_unique');
//$table->unique(['ranking_id', 'type', 'position'], 'result_unique');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think sitelinks should prevent the use of unique results. They should have a different type value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I understand. All the sitelinks of a website would have the same ranking id and position which they would get from the parent webpage. Should we have a "Sitelink" type enum for the "type" column? But even so, all sitelinks would have that same type. So the combination of the 3 would still be duplicated for the sitelinks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'm going to push this through and then let's take a look at how it runs in a Zoom meeting and I'll know more then what we should do. My thought is that I want to know what order the sitelinks are in. They should have a parent_id of the main result and be a different type with their own ranking.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh! That makes sense! I think I get it now. However, sitelinks results don't have a "position" unlike local, organic, etc. We could use their position in the result set itself, or the "key" of the array element.
Screen Shot 2021-04-12 at 11 18 23 PM

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's run some examples on their website and see if the key matches the order in which they are shown in the search results.

Comment on lines 14 to 17
$table->id();
$table->foreignIdFor(app('keyword'))->index();
$table->foreignIdFor(app('search_locale'))->index();
$table->timestamps();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add creator & updater to this so we know who added them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member

@drewroberts drewroberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! 🚎

I'm excited to see this code in action with our SerpAPI key. It looks great and I look forward to a demonstration to see it in action.

@drewroberts drewroberts merged commit 57b044a into main Apr 13, 2021
@drewroberts drewroberts deleted the omnia/feature/Check-Ranking-Command branch April 13, 2021 02:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create Ranking Command to pull in rankings based on active keywords
4 participants