Turkish Mirror Doesn't have Page about the referendum

[simplymathematics]

The Turkish referendum was the inciting incident that led Erdogan to block wikipedia. That is because it mentioned how he stole the election. It would be excellent if you could update that single page if at all possible before this goes live

[ghost]

Where do we get that page?

[ghost]

This has already gone live ~18 hours ago

[simplymathematics]

https://en.wikipedia.org/wiki/Turkish_constitutional_referendum,_201

https://tr.wikipedia.org/wiki/2017_T%C3%BCrkiye_anayasa_de%C4%9Fi%C5%9Fikli%C4%9Fi_referandumu

[ghost]

Mh I wonder why that's not part of the dump. We can try patching it in.

[Kubuxu]

It is because it was created at the end of April, the Kiwix dump was created at the mid or April.

I accidentally looked at en wiki.

[victorb]

@Kubuxu seems the turkish version of the page was first made on January 21, 2017 though, https://tr.wikipedia.org/w/index.php?title=2017_T%C3%BCrkiye_anayasa_de%C4%9Fi%C5%9Fikli%C4%9Fi_referandumu&offset=20170128113837&limit=500&action=history

[Kubuxu]

Hmm, interesting.

[simplymathematics]

The most important part is about the controversy surrounding the referendum, which only came after the votes were 'counted' on April 16th. The ipns hash has the information, but the URL does not.

[flyingzumwalt]

Sounds like we need to talk to kiwix about their dumps - see if there are more complete or more current versions available.

[flyingzumwalt]

Update: we are still trying to get a more recent dump from kiwix. The turkish snapshot is definitely outdated -- the styling was updated for all of the new dumps but hasn't been updated for the turkish dump.

[simplymathematics]

Oh man. I hope I don't have to pin a whole new snapshot. It took 72 hours of grueling downloads.

[flyingzumwalt]

Note: this is off topic. If you want to discuss bandwidth needs and improvements, please redirect the conversation to https://discuss.ipfs.io

We have a lot of bandwidth optimizations coming. In the long run we intend for IPFS to be at least as fast as HTTP and in many cases faster. We are all super motivated to work on this. This is part of what motivated us to create https://github.com/ipfs/test-lab -- it lets us spin up test networks and test performance in various scenarios.

[JanZerebecki]

It seems not only does it not contain all articles that existed as of the date 2017-04, also some articles are multiple month older version than existed when other articles where last edited that made it.

[kelson42]

The solution here seems indeed to be pretty simple: download last version of the ZIM file. Here is the permalink, a new version has been published a few days ago: download.kiwix.org/zim/wikipedia_tr_all.zim. So far I have seen last time I checked, you were using a ZIM file from January 2017. @meyerscr The file is only 3GB big, so it should take a few hours max. If you have the felling the mirror is slow, better use BitTorrent.

[Kubuxu]

Was the file updated? I was downloading it from there 1.5months ago.

[kelson42]

@Kubuxu Yes this is a permalink and a new version has been published a few days ago.

[victorb]

The solution here seems indeed to be pretty simple: download last version of the ZIM file

Not sure if it's that simple. Even the ZIM file from January 2017 doesn't seem to include all the pages, or we are not including all pages from it, as noted in the above comments.

[kelson42]

@victorbjelkholm I do not follow you. Can you be more specific please? This ticket is about missing https://tr.wikipedia.org/wiki/2017_T%C3%BCrkiye_anayasa_de%C4%9Fi%C5%9Fikli%C4%9Fi_referandumu. Yourself has identified that it was created the 21 January and the ZIM file used has been published the January 13th. Case closed!?

[flyingzumwalt]

Let's pull down the new ZIM and see if it's there. Fingers crossed...

…

On Sun, Jun 11, 2017 at 6:41 AM Kelson ***@***.***> wrote: @victorbjelkholm <https://github.com/victorbjelkholm> I do not follow you. Can you be more specific please? This ticket is about missing https://tr.wikipedia.org/wiki/2017_T%C3%BCrkiye_anayasa_de%C4%9Fi%C5%9Fikli%C4%9Fi_referandumu. Yourself has identified that it was created the 21 January and the ZIM file used has been published the January 13th. Case closed!? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#24 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIesnYdii629IBUjXg59bLtD3H43a9Eks5sC8RygaJpZM4NUua3> .

[Kubuxu]

Doing it right now.

[Kubuxu]

Extraction took: 1m 30s
ipfs add took: 4m

I am using the old search graph. I might ask @magik6k to create script in this repo for rebuilding the graph.

Finishing up as we speak.

[Kubuxu]

The article exists in new mirror: https://ipfs.io/ipfs/QmeuuJnfJoXfWnJPj4wcNe2sZTXqrAzVKQ7ThK56aG5dNw/wiki/2017_T%C3%BCrkiye_anayasa_de%C4%9Fi%C5%9Fikli%C4%9Fi_referandumu.html

[kelson42]

Ticket could probably be closed.