Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Sitemaps] Upgrade Valid / Legal / Strict SitemapUrls #82

Merged
merged 2 commits into from
Sep 7, 2015

Conversation

jnioche
Copy link
Contributor

@jnioche jnioche commented Jun 11, 2015

Fixes #60

SitemapUrls can be not valid when they are referenced in a sitemap which
it's
directory is on a completely different path than the referenced
SitemapUrl.

All as indicated here:
http://www.sitemaps.org/protocol.html#location

In order to clarify the validity aspect we need to upgrade the following

  1. Add a little more explanations as javadocs and as logs
  2. Rename "Legal" (I think only one occurrence) to "valid" (in the
    parser)
  3. Add to the Sitemap class a new method to get all valid SitemapUrls
  4. When dropping a URL due to invalidity a log should be shown, a URL
    shouldn't
    be dropped quietly.

Chaiavi and others added 2 commits June 8, 2015 23:41
SitemapUrls can be not valid when they are referenced in a sitemap which
it's
directory is on a completely different path than the referenced
SitemapUrl.

All as indicated here:
http://www.sitemaps.org/protocol.html#location

In order to clarify the validity aspect we need to upgrade the following
1. Add a little more explanations as javadocs and as logs
2. Rename "Legal" (I think only one occurrence) to "valid" (in the
parser)
3. Add to the Sitemap class a new method to get all *valid* SitemapUrls
4. When dropping a URL due to invalidity a log should be shown, a URL
shouldn't
be dropped quietly.

sb.append("url = \"").append(url).append("\", lastMod = ").append(lastModStr).append(", type = ").append(getType()).append(", processed = ").append(isProcessed()).append(", urlListSize = ")
.append(urlList.size());
sb.append("url = \"").append(url).append("\", lastMod = ").append((getLastModified() == null) ? "null" : SiteMap.getFullDateFormat().format(getLastModified())).append(", type = ")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing code is easier to read and debug. Inlining it does not add anything, does it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would agree @jnioche. The one line concatenation is almost cryptic.

@lewismc
Copy link
Member

lewismc commented Aug 26, 2015

It would also be nice to have a test case which asserts the statement "A SitemapUrl(s) can be not valid when they are referenced in a sitemap with which it's directory is on a completely different path than the referenced SitemapUrl."
After all, this is what seemed to cause the bug previously.

@Chaiavi
Copy link
Member

Chaiavi commented Aug 26, 2015

The cryptiness comes from our automatic styling plugin we run now in order
to have consistent code.

That ugly line, was styled like all other chained methods, where each
method was started on a different line as can be seen here:
feb40af

In the old style it made sense as I didn't see any added value in those two
temporary variables.

That being said, now I agree that it is less readable.

Will revert it.

A. Does it mean that all other code was reviewed ?
B. I think it is a good test case, we just need to have in mind the future
implementation of the cross submit feature

On Wed, Aug 26, 2015 at 7:54 PM, Lewis John McGibbney <
notifications@github.com> wrote:

It would also be nice to have a test case which asserts the statement "A
SitemapUrl(s) can be not valid when they are referenced in a sitemap with
which it's directory is on a completely different path than the referenced
SitemapUrl."
Any thoughts?


Reply to this email directly or view it on GitHub
#82 (comment)
.

@lewismc
Copy link
Member

lewismc commented Sep 5, 2015

@Chaiavi
bq. A. Does it mean that all other code was reviewed ?
Yes
bq. I think it is a good test case, we just need to have in mind the future implementation of the cross submit feature
Agreed.

@Chaiavi can you commit this to master?

@Chaiavi
Copy link
Member

Chaiavi commented Sep 6, 2015

Yes, I will push it forward.

On Sat, Sep 5, 2015 at 6:39 PM, Lewis John McGibbney <
notifications@github.com> wrote:

@Chaiavi https://github.com/Chaiavi
bq. A. Does it mean that all other code was reviewed ?
Yes
bq. I think it is a good test case, we just need to have in mind the
future implementation of the cross submit feature
Agreed.

@Chaiavi https://github.com/Chaiavi can you commit this to master?


Reply to this email directly or view it on GitHub
#82 (comment)
.

@lewismc lewismc mentioned this pull request Sep 6, 2015
Chaiavi added a commit that referenced this pull request Sep 7, 2015
[Sitemaps] Upgrade Valid / Legal / Strict SitemapUrls
@Chaiavi Chaiavi merged commit 478a7d7 into master Sep 7, 2015
@jnioche
Copy link
Contributor Author

jnioche commented Sep 7, 2015

in Sitemap - why wasn't the line reverted as you said you would? Could you please do that in separate commit?
Thanks

@Chaiavi
Copy link
Member

Chaiavi commented Sep 8, 2015

Didn't forget it, i just hoped the "Merge pull request" on github will give
me more control (where i could do some manual code change) - I was wrong.

I will change it back in a seperate small commit.

On Mon, Sep 7, 2015 at 9:47 PM, Julien Nioche notifications@github.com
wrote:

in Sitemap - why wasn't the line reverted as you said you would? Could you
please do that in separate commit?
Thanks


Reply to this email directly or view it on GitHub
#82 (comment)
.

@jnioche jnioche deleted the validSitemaps branch July 1, 2016 13:32
@lewismc lewismc added this to the crawler-commons-0.7 milestone Sep 25, 2016
Chaiavi added a commit that referenced this pull request Jul 15, 2019
* Upgraded to Junit v5.5
Updated the annotations and assertions accordingly

* Removed unneeded before and after

* This is a technical debt

Fixing a styling issue I caused about 4 years ago

Details can be found here: #82

* Fixed according to @sebastian-nagel code review
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants