Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Wikipedia with discussion page #34

Closed
AnselmD opened this issue Dec 23, 2015 · 5 comments
Closed

Wikipedia with discussion page #34

AnselmD opened this issue Dec 23, 2015 · 5 comments
Labels
setup - general

Comments

@AnselmD
Copy link

AnselmD commented Dec 23, 2015

I whish you a nice Christmas :-)

I would like to use xowas import/script feature to download and import the wikipedia version which includes the discussion page. If you find some time to do it, this will be very nice for me.

Everytime i do no work a longer time not with xowa, i am asking myself, what is the difference between category 1 / 2. Can you please make a tooltip to category system (import/script).

I whish you a nice Christmas :-)
@gnosygnu
Copy link
Owner

I whish you a nice Christmas :-)

And a Merry Christmas to you too. :)

I would like to use xowas import/script feature to download and import the wikipedia version which includes the discussion page. If you find some time to do it, this will be very nice for me.
Everytime i do no work a longer time not with xowa,

Sure! What problem are you running into though? Is XOWA failing or just hanging? What does the log say: home/wiki/Special:XowaSystemData?type=log_session

In the meantime, I can build one on my side and upload it. Which one do you need: English Wikipedia or German Wikipedia?

i am asking myself, what is the difference between category 1 / 2. Can you please make a tooltip to category system (import/script).

In brief, category 1 is an incomplete category system, but it's generated with very little space. category 2 is a complete category system, but requires more space

There's more info here: home/wiki/Help:Category

@AnselmD
Copy link
Author

AnselmD commented Dec 24, 2015

Sure! What problem are you running into though? Is XOWA failing or just hanging? What does the log say: home/wiki/Special:XowaSystemData?type=log_session

No, no problem. My intention was to set up a new xowa version with new data (nearly all wikis in German and in English). And i did not want to spent much time (my time not machine time) and so i wanted to use import/script to download and install them in a batch.. I remember the files without discussion page was pages-articles and i forgot about the name of the one with discussion pages. So my wish (feature request) is to have a checkbox in import/script to switch to the bigger ones with dicussion page.

Sure! What problem are you running into though? Is XOWA failing or just hanging? What does the l>og say: home/wiki/Special:XowaSystemData?type=log_session

In the meantime, I can build one on my side and upload it. Which one do you need: English >Wikipedia or German Wikipedia?

Thank you, you do not need to. I did not run in an import issue.

In brief, category 1 is an incomplete category system, but it's generated with very little space. >category 2 is a complete category system, but requires more space

There's more info here: home/wiki/Help:Category

I could remember i always used category 2. But every time i am asking myself, why? Did i remember correctly? And i start up to looking in the help pages and there i can find it. And i go back to the import/script and i had to start again, because the input is resetted.

You have two nice tooltips at the import/script: "Import", can you make anotherone for category?

@gnosygnu
Copy link
Owner

So my wish (feature request) is to have a checkbox in import/script to switch to the bigger ones with dicussion page.

Ah! Understood. I basically need to edit the Wikitext for home/wiki/Help:Import/Script. If you want to do it manually, do the following:

  • Go to home/wiki/Help:Import/Script
  • Click on Edit
  • Find this line: "app.setup.cmds.dump_add_many('" + domain + "', 'latest', 'pages-articles', 'wiki.download'"
  • Replace 'pages-articles' with 'pages-meta-current'

However, if you want to set up a system, you should really use the command-line. I use them for all my builds, so they're very reliable. I'll post the script I used for German Wikis in the next comment

You have two nice tooltips at the import/script: "Import", can you make anotherone for category?

Sure! I added it now. Text is below. Thanks for the suggestion.

Also, Merry Christmas!

* V1 is a category system from using just the wikitext in "pages-articles". It is quick and does not take up much space, but it will be incomplete
* V2 is a category system that reproduces the exact category system in Wikipedia. However it requires additional files ("categorylinks") and takes up more space (as much as 10 GB for English Wikipedia)

For more info, see [[Help:Category]]

@gnosygnu
Copy link
Owner

For more info, see home/wiki/Help:Import/Command-line

Full text of script for German follows

app.bldr.pause_at_end_('n');
app.scripts.run_file_by_type('xowa_cfg_app');
app.bldr.cmds {
  // de.wikipedia.org
  add     ('de.wikipedia.org' , 'util.cleanup')          {delete_all = 'y';}
  add     ('de.wikipedia.org' , 'util.download')         {dump_type = 'pages-articles';}
  add     ('de.wikipedia.org' , 'util.download')         {dump_type = 'categorylinks';}
  add     ('de.wikipedia.org' , 'util.download')         {dump_type = 'page_props';}
  add     ('de.wikipedia.org' , 'util.download')         {dump_type = 'image';}
  add     ('de.wikipedia.org' , 'text.init');
  add     ('de.wikipedia.org' , 'text.page')             {}
  add     ('de.wikipedia.org' , 'text.search');
  add     ('de.wikipedia.org' , 'text.css');
  add     ('de.wikipedia.org' , 'text.cat.core');
  add     ('de.wikipedia.org' , 'text.cat.link');
  add     ('de.wikipedia.org' , 'text.cat.hidden');
  add     ('de.wikipedia.org' , 'text.term');
  add     ('de.wikipedia.org' , 'wiki.page_dump.make');
  add     ('de.wikipedia.org' , 'wiki.redirect')         {commit_interval = 1000; progress_interval = 100; cleanup_interval = 100;}
  add     ('de.wikipedia.org' , 'wiki.image');
  add     ('de.wikipedia.org' , 'util.cleanup')          {delete_tmp = 'y'; delete_by_match('*.xml|*.sql|*.bz2|*.gz');}
  add     ('de.wikipedia.org' , 'file.lnki_temp') {
    commit_interval = 10000; progress_interval = 50; cleanup_interval = 50; select_size = 25;
    ns_ids = '0|4|14|100';
    // hdump_bldr {enabled = 'y'; hzip_enabled = 'y'; hzip_diff = 'y';}
  }  
  add     ('de.wikipedia.org' , 'file.lnki_regy');
  add     ('de.wikipedia.org' , 'file.page_regy')        {build_commons = 'n';}
  add     ('de.wikipedia.org' , 'file.orig_regy');
  add     ('de.wikipedia.org' , 'file.xfer_temp.thumb');
  add     ('de.wikipedia.org' , 'file.xfer_regy');
  add     ('de.wikipedia.org' , 'file.xfer_regy_update');
  add     ('de.wikipedia.org' , 'file.fsdb_make') {
    commit_interval = 1000; progress_interval = 200; select_interval = 10000;
    ns_ids = '0|4|14|100';
    src_bin_mgr__fsdb_version = 'v2';
    src_bin_mgr__fsdb_skip_wkrs = 'page_gt_1|small_size';
    src_bin_mgr__wmf_enabled = 'y';
    // poll_mgr.poll_interval = 10000;
    // resume_enabled = 'y';
  }
  add     ('de.wikipedia.org' , 'file.orig_reg');
  add     ('de.wikipedia.org' , 'wiki.page_dump.drop');

  // de.wiktionary.org
  add     ('de.wiktionary.org' , 'util.cleanup')          {delete_all = 'y';}
  add     ('de.wiktionary.org' , 'util.download')         {dump_type = 'pages-articles';}
  add     ('de.wiktionary.org' , 'util.download')         {dump_type = 'categorylinks';}
  add     ('de.wiktionary.org' , 'util.download')         {dump_type = 'page_props';}
  add     ('de.wiktionary.org' , 'util.download')         {dump_type = 'image';}
  add     ('de.wiktionary.org' , 'text.init');
  add     ('de.wiktionary.org' , 'text.page')             {}
  add     ('de.wiktionary.org' , 'text.search');
  add     ('de.wiktionary.org' , 'text.css');
  add     ('de.wiktionary.org' , 'text.cat.core');
  add     ('de.wiktionary.org' , 'text.cat.link');
  add     ('de.wiktionary.org' , 'text.cat.hidden');
  add     ('de.wiktionary.org' , 'text.term');
  add     ('de.wiktionary.org' , 'wiki.page_dump.make');
  add     ('de.wiktionary.org' , 'wiki.redirect')         {commit_interval = 1000; progress_interval = 100; cleanup_interval = 100;}
  add     ('de.wiktionary.org' , 'wiki.image');
  add     ('de.wiktionary.org' , 'util.cleanup')          {delete_tmp = 'y'; delete_by_match('*.xml|*.sql|*.bz2|*.gz');}
  add     ('de.wiktionary.org' , 'file.lnki_temp') {
    commit_interval = 10000; progress_interval = 50; cleanup_interval = 50; select_size = 25;
    ns_ids = '0|4|14|102|104|106';
    // hdump_bldr {enabled = 'y'; hzip_enabled = 'y'; hzip_diff = 'y';}
  }  
  add     ('de.wiktionary.org' , 'file.lnki_regy');
  add     ('de.wiktionary.org' , 'file.page_regy')        {build_commons = 'n';}
  add     ('de.wiktionary.org' , 'file.orig_regy');
  add     ('de.wiktionary.org' , 'file.xfer_temp.thumb');
  add     ('de.wiktionary.org' , 'file.xfer_regy');
  add     ('de.wiktionary.org' , 'file.xfer_regy_update');
  add     ('de.wiktionary.org' , 'file.fsdb_make') {
    commit_interval = 1000; progress_interval = 200; select_interval = 10000;
    ns_ids = '0|4|14|102|104|106';
    src_bin_mgr__fsdb_version = 'v2';
    src_bin_mgr__fsdb_skip_wkrs = 'page_gt_1|small_size';
    src_bin_mgr__wmf_enabled = 'y';
    // poll_mgr.poll_interval = 10000;
    // resume_enabled = 'y';
  }
  add     ('de.wiktionary.org' , 'file.orig_reg');
  add     ('de.wiktionary.org' , 'wiki.page_dump.drop');

  // de.wikisource.org
  add     ('de.wikisource.org' , 'util.cleanup')          {delete_all = 'y';}
  add     ('de.wikisource.org' , 'util.download')         {dump_type = 'pages-articles';}
  add     ('de.wikisource.org' , 'util.download')         {dump_type = 'categorylinks';}
  add     ('de.wikisource.org' , 'util.download')         {dump_type = 'page_props';}
  add     ('de.wikisource.org' , 'util.download')         {dump_type = 'image';}
  add     ('de.wikisource.org' , 'text.init');
  add     ('de.wikisource.org' , 'text.page')             {}
  add     ('de.wikisource.org' , 'text.search');
  add     ('de.wikisource.org' , 'text.css');
  add     ('de.wikisource.org' , 'text.cat.core');
  add     ('de.wikisource.org' , 'text.cat.link');
  add     ('de.wikisource.org' , 'text.cat.hidden');
  add     ('de.wikisource.org' , 'text.term');
  add     ('de.wikisource.org' , 'wiki.page_dump.make');
  add     ('de.wikisource.org' , 'wiki.redirect')         {commit_interval = 1000; progress_interval = 100; cleanup_interval = 100;}
  add     ('de.wikisource.org' , 'wiki.image');
  add     ('de.wikisource.org' , 'util.cleanup')          {delete_tmp = 'y'; delete_by_match('*.xml|*.sql|*.bz2|*.gz');}
  add     ('de.wikisource.org' , 'file.lnki_temp') {
    commit_interval = 10000; progress_interval = 50; cleanup_interval = 50; select_size = 25;
    ns_ids = '0|4|14|102|104';
    // hdump_bldr {enabled = 'y'; hzip_enabled = 'y'; hzip_diff = 'y';}
  }  
  add     ('de.wikisource.org' , 'file.lnki_regy');
  add     ('de.wikisource.org' , 'file.page_regy')        {build_commons = 'n';}
  add     ('de.wikisource.org' , 'file.orig_regy');
  add     ('de.wikisource.org' , 'file.xfer_temp.thumb');
  add     ('de.wikisource.org' , 'file.xfer_regy');
  add     ('de.wikisource.org' , 'file.xfer_regy_update');
  add     ('de.wikisource.org' , 'file.fsdb_make') {
    commit_interval = 1000; progress_interval = 200; select_interval = 10000;
    ns_ids = '0|4|14|102|104';
    src_bin_mgr__fsdb_version = 'v2';
    src_bin_mgr__fsdb_skip_wkrs = 'page_gt_1|small_size';
    src_bin_mgr__wmf_enabled = 'y';
    // poll_mgr.poll_interval = 10000;
    // resume_enabled = 'y';
  }
  add     ('de.wikisource.org' , 'file.orig_reg');
  add     ('de.wikisource.org' , 'wiki.page_dump.drop');

  // de.wikivoyage.org
  add     ('de.wikivoyage.org' , 'util.cleanup')          {delete_all = 'y';}
  add     ('de.wikivoyage.org' , 'util.download')         {dump_type = 'pages-articles';}
  add     ('de.wikivoyage.org' , 'util.download')         {dump_type = 'categorylinks';}
  add     ('de.wikivoyage.org' , 'util.download')         {dump_type = 'page_props';}
  add     ('de.wikivoyage.org' , 'util.download')         {dump_type = 'image';}
  add     ('de.wikivoyage.org' , 'text.init');
  add     ('de.wikivoyage.org' , 'text.page')             {}
  add     ('de.wikivoyage.org' , 'text.search');
  add     ('de.wikivoyage.org' , 'text.css');
  add     ('de.wikivoyage.org' , 'text.cat.core');
  add     ('de.wikivoyage.org' , 'text.cat.link');
  add     ('de.wikivoyage.org' , 'text.cat.hidden');
  add     ('de.wikivoyage.org' , 'text.term');
  add     ('de.wikivoyage.org' , 'wiki.page_dump.make');
  add     ('de.wikivoyage.org' , 'wiki.redirect')         {commit_interval = 1000; progress_interval = 100; cleanup_interval = 100;}
  add     ('de.wikivoyage.org' , 'wiki.image');
  add     ('de.wikivoyage.org' , 'util.cleanup')          {delete_tmp = 'y'; delete_by_match('*.xml|*.sql|*.bz2|*.gz');}
  add     ('de.wikivoyage.org' , 'file.lnki_temp') {
    commit_interval = 10000; progress_interval = 50; cleanup_interval = 50; select_size = 25;
    ns_ids = '0|4|14|100|102|104|106';
    // hdump_bldr {enabled = 'y'; hzip_enabled = 'y'; hzip_diff = 'y';}
  }  
  add     ('de.wikivoyage.org' , 'file.lnki_regy');
  add     ('de.wikivoyage.org' , 'file.page_regy')        {build_commons = 'n';}
  add     ('de.wikivoyage.org' , 'file.orig_regy');
  add     ('de.wikivoyage.org' , 'file.xfer_temp.thumb');
  add     ('de.wikivoyage.org' , 'file.xfer_regy');
  add     ('de.wikivoyage.org' , 'file.xfer_regy_update');
  add     ('de.wikivoyage.org' , 'file.fsdb_make') {
    commit_interval = 1000; progress_interval = 200; select_interval = 10000;
    ns_ids = '0|4|14|100|102|104|106';
    src_bin_mgr__fsdb_version = 'v2';
    src_bin_mgr__fsdb_skip_wkrs = 'page_gt_1|small_size';
    src_bin_mgr__wmf_enabled = 'y';
    // poll_mgr.poll_interval = 10000;
    // resume_enabled = 'y';
  }
  add     ('de.wikivoyage.org' , 'file.orig_reg');
  add     ('de.wikivoyage.org' , 'wiki.page_dump.drop');

  // de.wikiquote.org
  add     ('de.wikiquote.org' , 'util.cleanup')          {delete_all = 'y';}
  add     ('de.wikiquote.org' , 'util.download')         {dump_type = 'pages-articles';}
  add     ('de.wikiquote.org' , 'util.download')         {dump_type = 'categorylinks';}
  add     ('de.wikiquote.org' , 'util.download')         {dump_type = 'page_props';}
  add     ('de.wikiquote.org' , 'util.download')         {dump_type = 'image';}
  add     ('de.wikiquote.org' , 'text.init');
  add     ('de.wikiquote.org' , 'text.page')             {}
  add     ('de.wikiquote.org' , 'text.search');
  add     ('de.wikiquote.org' , 'text.css');
  add     ('de.wikiquote.org' , 'text.cat.core');
  add     ('de.wikiquote.org' , 'text.cat.link');
  add     ('de.wikiquote.org' , 'text.cat.hidden');
  add     ('de.wikiquote.org' , 'text.term');
  add     ('de.wikiquote.org' , 'wiki.page_dump.make');
  add     ('de.wikiquote.org' , 'wiki.redirect')         {commit_interval = 1000; progress_interval = 100; cleanup_interval = 100;}
  add     ('de.wikiquote.org' , 'wiki.image');
  add     ('de.wikiquote.org' , 'util.cleanup')          {delete_tmp = 'y'; delete_by_match('*.xml|*.sql|*.bz2|*.gz');}
  add     ('de.wikiquote.org' , 'file.lnki_temp') {
    commit_interval = 10000; progress_interval = 50; cleanup_interval = 50; select_size = 25;
    ns_ids = '0|4|14|100';
    // hdump_bldr {enabled = 'y'; hzip_enabled = 'y'; hzip_diff = 'y';}
  }  
  add     ('de.wikiquote.org' , 'file.lnki_regy');
  add     ('de.wikiquote.org' , 'file.page_regy')        {build_commons = 'n';}
  add     ('de.wikiquote.org' , 'file.orig_regy');
  add     ('de.wikiquote.org' , 'file.xfer_temp.thumb');
  add     ('de.wikiquote.org' , 'file.xfer_regy');
  add     ('de.wikiquote.org' , 'file.xfer_regy_update');
  add     ('de.wikiquote.org' , 'file.fsdb_make') {
    commit_interval = 1000; progress_interval = 200; select_interval = 10000;
    ns_ids = '0|4|14|100';
    src_bin_mgr__fsdb_version = 'v2';
    src_bin_mgr__fsdb_skip_wkrs = 'page_gt_1|small_size';
    src_bin_mgr__wmf_enabled = 'y';
    // poll_mgr.poll_interval = 10000;
    // resume_enabled = 'y';
  }
  add     ('de.wikiquote.org' , 'file.orig_reg');
  add     ('de.wikiquote.org' , 'wiki.page_dump.drop');

  // de.wikibooks.org
  add     ('de.wikibooks.org' , 'util.cleanup')          {delete_all = 'y';}
  add     ('de.wikibooks.org' , 'util.download')         {dump_type = 'pages-articles';}
  add     ('de.wikibooks.org' , 'util.download')         {dump_type = 'categorylinks';}
  add     ('de.wikibooks.org' , 'util.download')         {dump_type = 'page_props';}
  add     ('de.wikibooks.org' , 'util.download')         {dump_type = 'image';}
  add     ('de.wikibooks.org' , 'text.init');
  add     ('de.wikibooks.org' , 'text.page')             {}
  add     ('de.wikibooks.org' , 'text.search');
  add     ('de.wikibooks.org' , 'text.css');
  add     ('de.wikibooks.org' , 'text.cat.core');
  add     ('de.wikibooks.org' , 'text.cat.link');
  add     ('de.wikibooks.org' , 'text.cat.hidden');
  add     ('de.wikibooks.org' , 'text.term');
  add     ('de.wikibooks.org' , 'wiki.page_dump.make');
  add     ('de.wikibooks.org' , 'wiki.redirect')         {commit_interval = 1000; progress_interval = 100; cleanup_interval = 100;}
  add     ('de.wikibooks.org' , 'wiki.image');
  add     ('de.wikibooks.org' , 'util.cleanup')          {delete_tmp = 'y'; delete_by_match('*.xml|*.sql|*.bz2|*.gz');}
  add     ('de.wikibooks.org' , 'file.lnki_temp') {
    commit_interval = 10000; progress_interval = 50; cleanup_interval = 50; select_size = 25;
    ns_ids = '0|4|14|102';
    // hdump_bldr {enabled = 'y'; hzip_enabled = 'y'; hzip_diff = 'y';}
  }  
  add     ('de.wikibooks.org' , 'file.lnki_regy');
  add     ('de.wikibooks.org' , 'file.page_regy')        {build_commons = 'n';}
  add     ('de.wikibooks.org' , 'file.orig_regy');
  add     ('de.wikibooks.org' , 'file.xfer_temp.thumb');
  add     ('de.wikibooks.org' , 'file.xfer_regy');
  add     ('de.wikibooks.org' , 'file.xfer_regy_update');
  add     ('de.wikibooks.org' , 'file.fsdb_make') {
    commit_interval = 1000; progress_interval = 200; select_interval = 10000;
    ns_ids = '0|4|14|102';
    src_bin_mgr__fsdb_version = 'v2';
    src_bin_mgr__fsdb_skip_wkrs = 'page_gt_1|small_size';
    src_bin_mgr__wmf_enabled = 'y';
    // poll_mgr.poll_interval = 10000;
    // resume_enabled = 'y';
  }
  add     ('de.wikibooks.org' , 'file.orig_reg');
  add     ('de.wikibooks.org' , 'wiki.page_dump.drop');

  // de.wikiversity.org
  add     ('de.wikiversity.org' , 'util.cleanup')          {delete_all = 'y';}
  add     ('de.wikiversity.org' , 'util.download')         {dump_type = 'pages-articles';}
  add     ('de.wikiversity.org' , 'util.download')         {dump_type = 'categorylinks';}
  add     ('de.wikiversity.org' , 'util.download')         {dump_type = 'page_props';}
  add     ('de.wikiversity.org' , 'util.download')         {dump_type = 'image';}
  add     ('de.wikiversity.org' , 'text.init');
  add     ('de.wikiversity.org' , 'text.page')             {}
  add     ('de.wikiversity.org' , 'text.search');
  add     ('de.wikiversity.org' , 'text.css');
  add     ('de.wikiversity.org' , 'text.cat.core');
  add     ('de.wikiversity.org' , 'text.cat.link');
  add     ('de.wikiversity.org' , 'text.cat.hidden');
  add     ('de.wikiversity.org' , 'text.term');
  add     ('de.wikiversity.org' , 'wiki.page_dump.make');
  add     ('de.wikiversity.org' , 'wiki.redirect')         {commit_interval = 1000; progress_interval = 100; cleanup_interval = 100;}
  add     ('de.wikiversity.org' , 'wiki.image');
  add     ('de.wikiversity.org' , 'util.cleanup')          {delete_tmp = 'y'; delete_by_match('*.xml|*.sql|*.bz2|*.gz');}
  add     ('de.wikiversity.org' , 'file.lnki_temp') {
    commit_interval = 10000; progress_interval = 50; cleanup_interval = 50; select_size = 25;
    ns_ids = '0|4|14|106|108|446';
    // hdump_bldr {enabled = 'y'; hzip_enabled = 'y'; hzip_diff = 'y';}
  }  
  add     ('de.wikiversity.org' , 'file.lnki_regy');
  add     ('de.wikiversity.org' , 'file.page_regy')        {build_commons = 'n';}
  add     ('de.wikiversity.org' , 'file.orig_regy');
  add     ('de.wikiversity.org' , 'file.xfer_temp.thumb');
  add     ('de.wikiversity.org' , 'file.xfer_regy');
  add     ('de.wikiversity.org' , 'file.xfer_regy_update');
  add     ('de.wikiversity.org' , 'file.fsdb_make') {
    commit_interval = 1000; progress_interval = 200; select_interval = 10000;
    ns_ids = '0|4|14|106|108|446';
    src_bin_mgr__fsdb_version = 'v2';
    src_bin_mgr__fsdb_skip_wkrs = 'page_gt_1|small_size';
    src_bin_mgr__wmf_enabled = 'y';
    // poll_mgr.poll_interval = 10000;
    // resume_enabled = 'y';
  }
  add     ('de.wikiversity.org' , 'file.orig_reg');
  add     ('de.wikiversity.org' , 'wiki.page_dump.drop');

  // de.wikinews.org
  add     ('de.wikinews.org' , 'util.cleanup')          {delete_all = 'y';}
  add     ('de.wikinews.org' , 'util.download')         {dump_type = 'pages-articles';}
  add     ('de.wikinews.org' , 'util.download')         {dump_type = 'categorylinks';}
  add     ('de.wikinews.org' , 'util.download')         {dump_type = 'page_props';}
  add     ('de.wikinews.org' , 'util.download')         {dump_type = 'image';}
  add     ('de.wikinews.org' , 'text.init');
  add     ('de.wikinews.org' , 'text.page')             {}
  add     ('de.wikinews.org' , 'text.search');
  add     ('de.wikinews.org' , 'text.css');
  add     ('de.wikinews.org' , 'text.cat.core');
  add     ('de.wikinews.org' , 'text.cat.link');
  add     ('de.wikinews.org' , 'text.cat.hidden');
  add     ('de.wikinews.org' , 'text.term');
  add     ('de.wikinews.org' , 'wiki.page_dump.make');
  add     ('de.wikinews.org' , 'wiki.redirect')         {commit_interval = 1000; progress_interval = 100; cleanup_interval = 100;}
  add     ('de.wikinews.org' , 'wiki.image');
  add     ('de.wikinews.org' , 'util.cleanup')          {delete_tmp = 'y'; delete_by_match('*.xml|*.sql|*.bz2|*.gz');}
  add     ('de.wikinews.org' , 'file.lnki_temp') {
    commit_interval = 10000; progress_interval = 50; cleanup_interval = 50; select_size = 25;
    ns_ids = '0|4|14|100|102';
    // hdump_bldr {enabled = 'y'; hzip_enabled = 'y'; hzip_diff = 'y';}
  }  
  add     ('de.wikinews.org' , 'file.lnki_regy');
  add     ('de.wikinews.org' , 'file.page_regy')        {build_commons = 'n';}
  add     ('de.wikinews.org' , 'file.orig_regy');
  add     ('de.wikinews.org' , 'file.xfer_temp.thumb');
  add     ('de.wikinews.org' , 'file.xfer_regy');
  add     ('de.wikinews.org' , 'file.xfer_regy_update');
  add     ('de.wikinews.org' , 'file.fsdb_make') {
    commit_interval = 1000; progress_interval = 200; select_interval = 10000;
    ns_ids = '0|4|14|100|102';
    src_bin_mgr__fsdb_version = 'v2';
    src_bin_mgr__fsdb_skip_wkrs = 'page_gt_1|small_size';
    src_bin_mgr__wmf_enabled = 'y';
    // poll_mgr.poll_interval = 10000;
    // resume_enabled = 'y';
  }
  add     ('de.wikinews.org' , 'file.orig_reg');
  add     ('de.wikinews.org' , 'wiki.page_dump.drop');

}
app.bldr.run;

@gnosygnu
Copy link
Owner

Sorry for the lack of follow-up. I think there aren't any outstanding questions on this issue. Let me know if there's something else. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
setup - general
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@gnosygnu @AnselmD