Can't Edit Old Blogger Posts (>500) #247

Closed
FixarFarsan opened this Issue Dec 25, 2015 · 22 comments

Projects

None yet

6 participants

@FixarFarsan

Hi OLW developers and thanks for the working blogger release!

I am now and then editing my old posts and the current version does not allow me to open more than 500 posts. Could you please remove this limit when you choose "Show All".

Thanks in advance!

@Crypt32
Crypt32 commented Dec 25, 2015

I'm using version 0.5.1.4 already have this option. Did you try the most recent version of Live Writer?

@ScottIsAFool
Member

@fixarfarsan which blog type are you using?

@FixarFarsan

Oh, sorry @ScottIsAFool - I am using Blogger platform and I have the "show all" option but it gives me an error, "Show 500" works but not "Show 1000" , 3000 and All.

I have about 3500 published blog posts on my blog.

@ScottIsAFool
Member

Can you post the error and your log files please.

@FixarFarsan

The service blogger has thrown an exception: Google.GoogleApiException: Google.Apis.Requests.RequestError
We're sorry, but the value for field max_results is larger than the maximum allowed page size of 500. [400]
Errors [
Message[We're sorry, but the value for field max_results is larger than the maximum allowed page size of 500.] Location[ - ] Reason[invalid] Domain[global]
]

at Google.Apis.Requests.ClientServiceRequest1.Execute() at OpenLiveWriter.BlogClient.Clients.GoogleBloggerv3Client.GetRecentPosts(String blogId, Int32 maxPosts, Boolean includeCategories, Nullable1 now)
at OpenLiveWriter.BlogClient.Blog.GetRecentPosts(Int32 maxPosts, Boolean includeCategories)
at OpenLiveWriter.PostEditor.RemoteWeblogBlogPostSource.GetPosts(RecentPostRequest request, Boolean getPages)
at OpenLiveWriter.PostEditor.RemoteWeblogBlogPostSource.GetRecentPosts(RecentPostRequest request)
at OpenLiveWriter.PostEditor.OpenPost.BlogPostListBox.GetRecentPostsAsyncOperation.DoWork()
at OpenLiveWriter.CoreServices.AsyncOperation.InternalStart()

@Crypt32
Crypt32 commented Dec 25, 2015

This is a Blogger limitation. They do not allow to retrieve more than 500 posts at one request (maybe due to large traffic). There is nothing you or OLW team can do.

@FixarFarsan

Ok, but with the latest version of WLW I could open my first posts and edit them by selecting "Show ALL" and wait.. Is it possible to load the posts in batches of 500 when selecting "show all" in OLW?

@Crypt32
Crypt32 commented Dec 25, 2015

nope. Metaweblog API do not allow this. There is only metaweblog.getRecentPosts() method where you specify the blog and post count to retrieve and server behavior is to fetch top N posts from database. There is no way to tell server something like "hey server, give me another 500 posts after top 500". This is how databases work, but not Metaweblog. Server always assumes TOP N.

Although, Metaweblog API is simple, it lacks in some important things. Wordpress made data structure extensions, but no one made changes in methods (or, at least, made them a community standard).

For example, WLW/OLW uses metaweblog.getRecentPosts() to fetch a list of posts (and shows 1-2 lines of the post). Method return value is an array of full posts. Think about this: you retrieve 5 000 posts and each post is 5kb in average (for example). It will result in 25MB of wasted network traffic and server load. This is why most shared blog providers apply restrictions. And client should store them somewhere in the memory.

When you select desired post, WLW/OLW calls metaweblog.getPost() method to download this particular post. It would be great If there would be a standard method like metaweblog.getPostTitles() that would return a list of post titles and metadata (except content). Result size will be much-much smaller (less then megabyte). In this case I could afford to query even 10k post titles without significant overhead. Select desired one and only then download full post.

@FixarFarsan

Interesting but still.. in WLW it worked to open all 3000+ posts so could OLW get the same function for "show all"?

@Crypt32
Crypt32 commented Dec 25, 2015

As I said, it is Google's limitation. They set retrieval post count limit on their servers. If you try to do this with WLW, it will fail now as well.

Sent from Outlook Mobilehttps://aka.ms/sdimjr

On Fri, Dec 25, 2015 at 2:21 PM -0800, "FixarFarsan" <notifications@github.commailto:notifications@github.com> wrote:

Interesting but still.. in WLW it worked to open all 3000+ posts so could OLW get the same function for "show all"?

Reply to this email directly or view it on GitHubhttps://github.com/OpenLiveWriter/OpenLiveWriter/issues/247#issuecomment-167264968.

@jtbworld

This is too bad. Then we need to have a solution like this soon to avoid having to edit posts directly in Blogger something that often messes up the formatting.
http://aovestdipaperino.com/posts/windows-live-writer-post-download-plugin.aspx

@ScottIsAFool
Member

@jtbworld That plugin will be available soon for OLW.

@FixarFarsan

I am so looking forward to be able to edit and update older posts so this is really good news!

Hope to see this plugin very soon, thanks!

@FixarFarsan

@ScottIsAFool - Do you know anything about the plugin?

I am trying to update old posts by copying the post html to OLW, update it and paste it back using blogger.com editor but it does not look good (extra rows for example) so the plugin would be a huge help!

Thanks in advance!

@FixarFarsan

@ScottIsAFool Ok, thanks I have sent him the same question 👍

@willduff
Member

I have a fix to support loading more than 500 Google Blogger posts (pull request #371) but I'd like to get a few volunteers to test the changes. I have a version of Open Live Writer that I've built that contains the fix. Please follow these instructions and then report back if it worked or didn't work for you. Thanks!

  1. Close Open Live Writer if you have it running
  2. Download and run my fixed version here: https://ci.appveyor.com/api/buildjobs/eqm9fps0deaxkk5k/artifacts/Releases/OpenLiveWriterSetup.exe (note: you may get a Windows Smartscreen dialog complaining that this app is unrecognized, you'll need to click More info > Run anyway)
  3. Go to File > Open recent post, and click on one of your Google Blogger blogs and Show 1,000, 3,000 or All posts. You can also switch and look at Pages.
    image
  4. Scroll through the list and make sure it looks right (no duplicates? nothing obviously missing?)

/cc
@FixarFarsan @jtbworld @Crypt32 @kathweaver @ibonobo

@willduff
Member

@Crypt32 I was lucky in this case, Google provided an easy way to fetch another 500 posts after top 500 posts. Each Blogger response returns a NextPageToken to get the next 500 items, so I added support to loop and download all posts and pages, or stop early at a given maximum. Docs for Blogger v3 are here: https://developers.google.com/blogger/docs/3.0/reference/posts/list#response

@Crypt32
Crypt32 commented Feb 10, 2016

this is interesting. I wasn't aware about this (I'm not using blogger). Maybe, it is something I could implement this in my weblog engine if your fix will be accepted.

@willduff willduff referenced this issue Feb 10, 2016
Closed

Nightlies #366

@FixarFarsan

THANK YOU @willduff - You are are the best!!

I have tested to open ALL and it loads all my 3500+ posts I have written since 2009.
No duplicates in the list
It was also much faster to load the full list than with WLW.
I also tested to open one of my first posts, update it and re-publish

Everything went Perfect so please add this update into the next public OLW release.
Great work! Very impressed that you managed to fix something that was "impossible" ;)

FYI: @jtbworld @Crypt32 @kathweaver @ibonobo

@willduff
Member

Just checked this in, so it will be in the next OLW release. Closing as fixed with pull request #371.

@willduff willduff closed this Feb 10, 2016
@banders03

Curious as to when the next release will be? Is there somewhere I can
get notifications of it?

Thanks and thanks to ALL of you working on this project!!!

Barbara

On 2/10/2016 11:33 AM, Will Duff wrote:

Just checked this in, so it will be in the next OLW release. Closing
as fixed with pull request #371
#371.


Reply to this email directly or view it on GitHub
#247 (comment).

@willduff willduff changed the title from Edit Old Posts (>500) to Can't Edit Old Blogger Posts (>500) Feb 13, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment