Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP_Server: Handle edge cases for /fsys (Image and link problem in http_server mode) #524

Closed
lawnowner opened this issue Jul 28, 2019 · 15 comments

Comments

@lawnowner
Copy link

Hello, thanks for the new release, much appreciated. However, it is buggy on my end for some reason. Here is the problem: I've been using Xowa version 4.5.21.1808 on Windows 10 64-bit in http_server mode with offline databases for two Wikis, one of them being en.wikipedia.org, without any problems whatsoever. After the release of version 4.6.2.1907, I downloaded the new version as well as the new databases for en.wikipedia.org, namely the 'en.wikipedia.org Articles (2019-05)' and 'en.wikipedia.org Images (2019-05)', via the URLs listed on xowa.org/home/wiki/Wiki_setup/English_wikis. After downloading and extracting, I verified md5 checksums of all databases. Now everything is fine when Xowa's built-in browser is used, but when Firefox or Chrome is used via Xowa's http_server mode, images in a page will load as empty boxes (although correctly sized), and all links in a rendered page are relative to wiki/ whereas they should be relative to the en.wikipedia.org/wiki/ (e.g. 127.0.0.1:8080/wiki/Navigable instead of 127.0.0.1:8080/en.wikipedia.org/wiki/Navigable) and thus links on a page don't work once the page is opened by typing in the address bar. The link problem exists only for en.wikipedia.org, not the other wiki, but also note that I'm using old databases for the other wiki. Thanks.

@gnosygnu
Copy link
Owner

Hi! Thanks for creating the issue. Unfortunately, I'm not able to reproduce the issue.

When you get a chance, can you try my steps below? If they still fail, please upload the session log file and I'll review it. The session.txt file would be at C:\xowa\user\anonymous\app\tmp\log\current or wherever you put XOWA.

Also, one random thought: since you said only en.wikipedia.org is affected, can you also navigate to this page and paste the output here? home/wiki/Special:XowaDiag?type=fs.check&wiki=en.wikipedia.org

Thanks


  • Opened up command prompt
  • Ran cd C:\xowa
  • Ran java -jar xowa_windows_64.jar --app_mode http_server
  • Opened up Chrome and pasted following: http://localhost:8080/en.wikipedia.org/wiki/Earth
    • -> Images showed up
  • Clicked on Sun link
    • -> Navigated to Sun page and images still showed up

@lawnowner
Copy link
Author

lawnowner commented Jul 29, 2019

Following the steps you described, the images were not displayed in either of the pages, and also I was not able to follow the link to Sun since it referred to http://localhost:8080/wiki/Sun instead of http://localhost:8080/en.wikipedia.org/wiki/Sun, which I could visit only after modifying the URL manually at the address bar. I can visit both en.wikipedia.org/wiki/Earth and en.wikipedia.org/wiki/Sun using the Xowa's built-in browser without any problem; images show up and links refer to correct addresses and can be followed.

The two files I attached to this message are:
F:\Xowa__4.6.2.1907__Windows__Portable__64\user\anonymous\app\tmp\log\20190729_043432.956\session.txt
F:\Xowa__4.6.2.1907__Windows__Portable__64\user\anonymous\app\tmp\log\20190729_043432.956\err.txt

The output of home/wiki/Special:XowaDiag?type=fs.check&wiki=en.wikipedia.org is as follows:

scanning: F:\Xowa__4.6.2.1907__Windows__Portable__64\wiki\en.wikipedia.org\ fil|en.wikipedia.org-core.xowa|2417680384|20190605 183208.000 fil|en.wikipedia.org-file-core.xowa|2351001600|20190605 192520.000 fil|en.wikipedia.org-file-ns.000-db.001.xowa|1209802752|20190605 192652.000 fil|en.wikipedia.org-file-ns.000-db.002.xowa|1179803648|20190605 192732.000 fil|en.wikipedia.org-file-ns.000-db.003.xowa|1237504000|20190605 192812.000 fil|en.wikipedia.org-file-ns.000-db.004.xowa|1291579392|20190605 192852.000 fil|en.wikipedia.org-file-ns.000-db.005.xowa|1237331968|20190605 192936.000 fil|en.wikipedia.org-file-ns.000-db.006.xowa|1201020928|20190605 193016.000 fil|en.wikipedia.org-file-ns.000-db.007.xowa|1226440704|20190605 193056.000 fil|en.wikipedia.org-file-ns.000-db.008.xowa|1222787072|20190605 193136.000 fil|en.wikipedia.org-file-ns.000-db.009.xowa|1219133440|20190605 193216.000 fil|en.wikipedia.org-file-ns.000-db.010.xowa|1228005376|20190605 193258.000 fil|en.wikipedia.org-file-ns.000-db.011.xowa|1226473472|20190605 193338.000 fil|en.wikipedia.org-file-ns.000-db.012.xowa|1205567488|20190605 193418.000 fil|en.wikipedia.org-file-ns.000-db.013.xowa|1229557760|20190605 193458.000 fil|en.wikipedia.org-file-ns.000-db.014.xowa|1232433152|20190605 193540.000 fil|en.wikipedia.org-file-ns.000-db.015.xowa|1243824128|20190605 193620.000 fil|en.wikipedia.org-file-ns.000-db.016.xowa|1257852928|20190605 193702.000 fil|en.wikipedia.org-file-ns.000-db.017.xowa|1245675520|20190605 193742.000 fil|en.wikipedia.org-file-ns.000-db.018.xowa|1253552128|20190605 193824.000 fil|en.wikipedia.org-file-ns.000-db.019.xowa|1242554368|20190605 193906.000 fil|en.wikipedia.org-file-ns.000-db.020.xowa|1257054208|20190605 193946.000 fil|en.wikipedia.org-file-ns.000-db.021.xowa|1265352704|20190605 194028.000 fil|en.wikipedia.org-file-ns.000-db.022.xowa|1235468288|20190605 194110.000 fil|en.wikipedia.org-file-ns.000-db.023.xowa|1241759744|20190605 194150.000 fil|en.wikipedia.org-file-ns.000-db.024.xowa|1282670592|20190605 194232.000 fil|en.wikipedia.org-file-ns.000-db.025.xowa|1301094400|20190605 194314.000 fil|en.wikipedia.org-file-ns.000-db.026.xowa|1287753728|20190605 194356.000 fil|en.wikipedia.org-file-ns.000-db.027.xowa|1269346304|20190605 194440.000 fil|en.wikipedia.org-file-ns.000-db.028.xowa|1364275200|20190605 194522.000 fil|en.wikipedia.org-file-ns.000-db.029.xowa|1317740544|20190605 194608.000 fil|en.wikipedia.org-file-ns.000-db.030.xowa|1349767168|20190605 194652.000 fil|en.wikipedia.org-file-ns.000-db.031.xowa|1299247104|20190605 194736.000 fil|en.wikipedia.org-file-ns.000-db.032.xowa|1294561280|20190605 194820.000 fil|en.wikipedia.org-file-ns.000-db.033.xowa|1280729088|20190605 194902.000 fil|en.wikipedia.org-file-ns.000-db.034.xowa|1320607744|20190605 194944.000 fil|en.wikipedia.org-file-ns.000-db.035.xowa|1334083584|20190605 195028.000 fil|en.wikipedia.org-file-ns.000-db.036.xowa|1343270912|20190605 195112.000 fil|en.wikipedia.org-file-ns.000-db.037.xowa|1329614848|20190605 195158.000 fil|en.wikipedia.org-file-ns.000-db.038.xowa|1314816000|20190605 195242.000 fil|en.wikipedia.org-file-ns.000-db.039.xowa|1316134912|20190605 195324.000 fil|en.wikipedia.org-file-ns.000-db.040.xowa|1294426112|20190605 195408.000 fil|en.wikipedia.org-file-ns.000-db.041.xowa|1301991424|20190605 195452.000 fil|en.wikipedia.org-file-ns.000-db.042.xowa|1305870336|20190605 195534.000 fil|en.wikipedia.org-file-ns.000-db.043.xowa|1304928256|20190605 195618.000 fil|en.wikipedia.org-file-ns.000-db.044.xowa|1278537728|20190605 195702.000 fil|en.wikipedia.org-file-ns.000-db.045.xowa|1301450752|20190605 195744.000 fil|en.wikipedia.org-file-ns.000-db.046.xowa|1222250496|20190605 195826.000 fil|en.wikipedia.org-file-ns.000-db.047.xowa|1170137088|20190605 195908.000 fil|en.wikipedia.org-file-ns.000-db.048.xowa|1102032896|20190605 200158.000 fil|en.wikipedia.org-file-ns.000-db.049.xowa|1003167744|20190605 200234.000 fil|en.wikipedia.org-file-ns.000-db.050.xowa|1054916608|20190605 200308.000 fil|en.wikipedia.org-file-ns.000-db.051.xowa|1001807872|20190605 200344.000 fil|en.wikipedia.org-file-ns.000-db.052.xowa|966377472|20190605 200418.000 fil|en.wikipedia.org-file-ns.000-db.053.xowa|983969792|20190605 200450.000 fil|en.wikipedia.org-file-ns.000-db.054.xowa|1158103040|20190605 200524.000 fil|en.wikipedia.org-file-ns.000-db.055.xowa|1132494848|20190605 200602.000 fil|en.wikipedia.org-file-ns.000-db.056.xowa|1019633664|20190605 200640.000 fil|en.wikipedia.org-file-ns.000-db.057.xowa|1037049856|20190605 200714.000 fil|en.wikipedia.org-file-ns.000-db.058.xowa|1208664064|20190605 200750.000 fil|en.wikipedia.org-file-ns.000-db.059.xowa|1069068288|20190605 200830.000 fil|en.wikipedia.org-file-ns.000-db.060.xowa|1105129472|20190605 200954.000 fil|en.wikipedia.org-file-ns.000-db.061.xowa|1086648320|20190605 201030.000 fil|en.wikipedia.org-file-ns.000-db.062.xowa|1520484352|20190605 201108.000 fil|en.wikipedia.org-file-ns.000-db.063.xowa|1560170496|20190605 201202.000 fil|en.wikipedia.org-file-ns.000-db.064.xowa|1350778880|20190605 201256.000 fil|en.wikipedia.org-file-ns.000-db.065.xowa|1093328896|20190605 201342.000 fil|en.wikipedia.org-file-ns.000-db.066.xowa|1166213120|20190605 201418.000 fil|en.wikipedia.org-file-ns.000-db.067.xowa|992858112|20190605 201458.000 fil|en.wikipedia.org-file-ns.000-db.068.xowa|828715008|20190605 201530.000 fil|en.wikipedia.org-file-ns.000-db.069.xowa|1291468800|20190605 201600.000 fil|en.wikipedia.org-file-ns.000-db.070.xowa|1170239488|20190605 201642.000 fil|en.wikipedia.org-file-ns.000-db.071.xowa|1269862400|20190605 201722.000 fil|en.wikipedia.org-file-ns.000-db.072.xowa|1406136320|20190605 201806.000 fil|en.wikipedia.org-file-ns.000-db.073.xowa|1290366976|20190605 201856.000 fil|en.wikipedia.org-file-ns.000-db.074.xowa|1230934016|20190605 201938.000 fil|en.wikipedia.org-file-ns.000-db.075.xowa|1189560320|20190605 202020.000 fil|en.wikipedia.org-file-ns.000-db.076.xowa|1182552064|20190605 202100.000 fil|en.wikipedia.org-file-ns.000-db.077.xowa|1243246592|20190605 202140.000 fil|en.wikipedia.org-file-ns.000-db.078.xowa|1472786432|20190605 202224.000 fil|en.wikipedia.org-file-ns.000-db.079.xowa|1473703936|20190605 202314.000 fil|en.wikipedia.org-file-ns.000-db.080.xowa|1419816960|20190605 202404.000 fil|en.wikipedia.org-file-ns.000-db.081.xowa|1632481280|20190605 202452.000 fil|en.wikipedia.org-file-ns.000-db.082.xowa|1631260672|20190605 202548.000 fil|en.wikipedia.org-file-ns.000-db.083.xowa|1639182336|20190605 202644.000 fil|en.wikipedia.org-file-ns.000-db.084.xowa|1638658048|20190605 202740.000 fil|en.wikipedia.org-file-ns.000-db.085.xowa|1635405824|20190605 202836.000 fil|en.wikipedia.org-file-ns.000-db.086.xowa|1577435136|20190605 202932.000 fil|en.wikipedia.org-file-ns.000-db.087.xowa|1630367744|20190605 203026.000 fil|en.wikipedia.org-file-ns.000-db.088.xowa|1630990336|20190605 203122.000 fil|en.wikipedia.org-file-ns.000-db.089.xowa|1633583104|20190605 203218.000 fil|en.wikipedia.org-file-ns.000-db.090.xowa|1633136640|20190605 203314.000 fil|en.wikipedia.org-file-ns.000-db.091.xowa|1633337344|20190605 203410.000 fil|en.wikipedia.org-file-ns.000-db.092.xowa|1635237888|20190605 203506.000 fil|en.wikipedia.org-file-ns.000-db.093.xowa|1631375360|20190605 203602.000 fil|en.wikipedia.org-file-ns.000-db.094.xowa|1632960512|20190605 203658.000 fil|en.wikipedia.org-file-ns.000-db.095.xowa|951586816|20190605 203752.000 fil|en.wikipedia.org-file-ns.004-db.001.xowa|1397968896|20190605 195948.000 fil|en.wikipedia.org-file-ns.004-db.002.xowa|1460211712|20190605 200908.000 fil|en.wikipedia.org-file-ns.008-db.001.xowa|40960|20190605 201454.000 fil|en.wikipedia.org-file-ns.014-db.001.xowa|1548111872|20190605 200034.000 fil|en.wikipedia.org-file-ns.014-db.002.xowa|624410624|20190605 200122.000 fil|en.wikipedia.org-file-ns.100-db.001.xowa|417030144|20190605 200142.000 fil|en.wikipedia.org-file-user.xowa|57344|20190728 031639.412 fil|en.wikipedia.org-html-ns.000-db.002.xowa|1673682944|20190605 185024.000 fil|en.wikipedia.org-html-ns.000-db.003.xowa|1704902656|20190605 185118.000 fil|en.wikipedia.org-html-ns.000-db.004.xowa|1727160320|20190605 185212.000 fil|en.wikipedia.org-html-ns.000-db.005.xowa|1749565440|20190605 185306.000 fil|en.wikipedia.org-html-ns.000-db.006.xowa|1765638144|20190605 185400.000 fil|en.wikipedia.org-html-ns.000-db.007.xowa|1784786944|20190605 185454.000 fil|en.wikipedia.org-html-ns.000-db.008.xowa|1792643072|20190605 185548.000 fil|en.wikipedia.org-html-ns.000-db.009.xowa|1820286976|20190605 185642.000 fil|en.wikipedia.org-html-ns.000-db.010.xowa|1789857792|20190605 185738.000 fil|en.wikipedia.org-html-ns.000-db.011.xowa|1833316352|20190605 185832.000 fil|en.wikipedia.org-html-ns.000-db.012.xowa|1835347968|20190605 185926.000 fil|en.wikipedia.org-html-ns.000-db.013.xowa|1846693888|20190605 190020.000 fil|en.wikipedia.org-html-ns.000-db.014.xowa|1827201024|20190605 190116.000 fil|en.wikipedia.org-html-ns.000-db.015.xowa|1836941312|20190605 190210.000 fil|en.wikipedia.org-html-ns.000-db.016.xowa|1814478848|20190605 190304.000 fil|en.wikipedia.org-html-ns.000-db.017.xowa|1827848192|20190605 190400.000 fil|en.wikipedia.org-html-ns.000-db.018.xowa|1840189440|20190605 190454.000 fil|en.wikipedia.org-html-ns.000-db.019.xowa|1842585600|20190605 190548.000 fil|en.wikipedia.org-html-ns.000-db.020.xowa|1845903360|20190605 190644.000 fil|en.wikipedia.org-html-ns.000-db.021.xowa|1346199552|20190605 190736.000 fil|en.wikipedia.org-html-ns.000.xowa|1647775744|20190605 184932.000 fil|en.wikipedia.org-html-ns.004-db.002.xowa|1805766656|20190605 190910.000 fil|en.wikipedia.org-html-ns.004-db.003.xowa|1822384128|20190605 191004.000 fil|en.wikipedia.org-html-ns.004-db.004.xowa|962916352|20190605 191056.000 fil|en.wikipedia.org-html-ns.004.xowa|1749217280|20190605 190818.000 fil|en.wikipedia.org-html-ns.008.xowa|2011136|20190605 191158.000 fil|en.wikipedia.org-html-ns.014.xowa|976404480|20190605 191132.000 fil|en.wikipedia.org-html-ns.100.xowa|233406464|20190605 191122.000 fil|en.wikipedia.org-xtn.category.core.xowa|27738112|20190605 183812.000 fil|en.wikipedia.org-xtn.category.link-db.001.xowa|1667010560|20190605 183818.000 fil|en.wikipedia.org-xtn.category.link-db.002.xowa|1662660608|20190605 183918.000 fil|en.wikipedia.org-xtn.category.link-db.003.xowa|1681801216|20190605 184016.000 fil|en.wikipedia.org-xtn.category.link-db.004.xowa|1689272320|20190605 184114.000 fil|en.wikipedia.org-xtn.category.link-db.005.xowa|1687150592|20190605 184214.000 fil|en.wikipedia.org-xtn.category.link-db.006.xowa|1687420928|20190605 184314.000 fil|en.wikipedia.org-xtn.category.link-db.007.xowa|1687031808|20190605 184412.000 fil|en.wikipedia.org-xtn.category.link-db.008.xowa|1688354816|20190605 184512.000 fil|en.wikipedia.org-xtn.category.link-db.009.xowa|1688825856|20190605 184610.000 fil|en.wikipedia.org-xtn.category.link-db.010.xowa|1686618112|20190605 184710.000 fil|en.wikipedia.org-xtn.category.link-db.011.xowa|1687658496|20190605 184810.000 fil|en.wikipedia.org-xtn.category.link-db.012.xowa|622157824|20190605 184908.000 fil|en.wikipedia.org-xtn.search.core.xowa|440864768|20190605 183404.000 fil|en.wikipedia.org-xtn.search.link-title-ns.000-db.001.xowa|1675399168|20190605 183620.000 fil|en.wikipedia.org-xtn.search.link-title-ns.999-db.001.xowa|786956288|20190605 183736.000

scanning: F:\Xowa__4.6.2.1907__Windows__Portable__64\file\en.wikipedia.org\ dir|thumb\

scanning: F:\Xowa__4.6.2.1907__Windows__Portable__64\file\en.wikipedia.org\fsdb.main\

scanning: F:\Xowa__4.6.2.1907__Windows__Portable__64\file\en.wikipedia.org\fsdb.user\

scanning: F:\Xowa__4.6.2.1907__Windows__Portable__64\file\en.wikipedia.org\fsdb.update_00\
err.txt
session.txt

@gnosygnu
Copy link
Owner

Thanks for the logs.

I missed this bit from your earlier post:

namely the 'en.wikipedia.org Articles (2019-05)' and 'en.wikipedia.org Images (2019-05)',

I've been testing with 'en.wikipedia.org Wikitext (2019-05)' which does work. Unfortunately, Articles does not. This became obvious as I went through the logs

Let me go through the error logs a little more. I'll push out a fix sometime this week.

Sorry for the bug, and thanks for following up.

@gnosygnu gnosygnu added this to In progress in A <div> in D Jul 29, 2019
@desb42
Copy link
Collaborator

desb42 commented Aug 1, 2019

Having just built my own html for enwiki (2019-06-01), I have been examining that source and can see where @lawnowner is coming from

the file Http_server_wrk.java is supposed to cope with the "/wiki/" -> "/en.wikipedia.org/wiki/" translation in the Convert_page function.

The problem is that it is too strict, instead of

page_html = String_.Replace(page_html, "<a href=\"/wiki/"	, "<a href=\"/" + wiki_domain + "/wiki/");

relaxing this slightly

page_html = String_.Replace(page_html, " href=\"/wiki/"	, " href=\"/" + wiki_domain + "/wiki/");

now transposes the html page properly

However ...

There is a second (and I think more serious bug)
In page en.wikipedia.org/wiki/Wild_Goose_Pass_Tree, I have an image link of the form

<img id="xoimg_0" src="file:///g:/xowa/file/en.wikipedia.org/thumb/f/d/c/3/Portal-puzzle.svg/16px.png" width="16" height="14" alt="Portal">

Note inside the 'src' attribute 'g:/xowa'
This is the directory I used to generate this dump - it should not be hard coded
I have made a further modification to Http_server_wrk.java to eliminate this (by use of a regex)
Please see my mod http_server

gnosygnu added a commit that referenced this issue Aug 3, 2019
@gnosygnu gnosygnu moved this from In progress to Done in A <div> in D Aug 5, 2019
@gnosygnu
Copy link
Owner

gnosygnu commented Aug 5, 2019

Hi!

I fixed this in tonight's release. See https://github.com/gnosygnu/xowa/releases/tag/v4.6.3.1908

Also, I like to give credit to users for finding bugs. Right now, there's a line in the Change Log like this: HTTP Server: Fix links and images not working for 2019-05 enwiki HTML databases. {detected by anonymous} Let me know if you would like to be referred to as lawnowner or some other nick or would rather remain anonymous.

Thanks!

@lawnowner
Copy link
Author

@gnosygnu That's kind of you. I think you can credit ctd for that. Thanks for the fix!

@gnosygnu
Copy link
Owner

gnosygnu commented Aug 5, 2019

@lawnowner: No problem. ctd it is. I updated it now: http://xowa.org/home/wiki/Change_log . Thanks again!

@desb42: Oops! My email is sometimes unreliabe. I missed your last comment above.

I took a look at https://github.com/desb42/xowa/commits/http_server and it matches my commit except for the regex on file://.*/file/. I'll move this issue back into TODO and make another commit to incorporate it.

As for the root cause, I'm still trying to figure out what it is. I had hoped I had fixed it in an earlier version, but since you're getting it now, it must still be occurring. I'll open up a separate issue for it later. Thanks!

@gnosygnu
Copy link
Owner

Added more general logic to handle /home/lnxusr as well as G:/xowa with the commit above.

Thanks

@desb42
Copy link
Collaborator

desb42 commented Aug 10, 2019

Interesting implementation of the regex

Just a small query.

Would it be an idea to place the call to Replace_fsys_hack in Http_server_mgr.java before the final conversion to a String
Then the conversion from a String to byte[] and back again within Replace_fsys_hack could be avoided

@gnosygnu
Copy link
Owner

Interesting implementation of the regex

Yeah, I'm not a fan of regex, and there were some rules that are hard to express (> 300).

That said, I'll probably use regex more as I port over the MediaWiki parser

Would it be an idea to place the call to Replace_fsys_hack in Http_server_mgr.java before the final conversion to a String
Then the conversion from a String to byte[] and back again within Replace_fsys_hack could be avoided

Honestly, there's a lot going on there which can be optimized (all the String replaces)

That said, the suggestion is pretty straightforward so I added it per the above commit.

Thanks!

@desb42
Copy link
Collaborator

desb42 commented Aug 12, 2019

Unfortunately, now that I have merged the commits into my sources a problem has arisen

There are two 'regex' conversions going on

String_.Replace(page_html, root_dir_http , "/fsys/")

in my case root_dir_http is g:/xowa/
and

page_html.replaceAll("\"file\\:.*?/file/", "\"/fsys/file/")

Replace_fsys_hack does not cope well with the likes of

file:///g:/xowa/bin/any/xowa/file/app.window/app_icon.png

Also there is another line

url(file:///g:/xowa/user/anonymous/wiki/www.wikidata.org/html/logo.png)

which does not use quotes

My version of this routine is

public static byte[] Replace_fsys_hack(byte[] html_bry) {
    // init
    Bry_bfr bfr = Bry_bfr_.New();
    int len = html_bry.length;
    int pos = 0;
    writeFile(String_.new_u8(html_bry), "d:/des/xowa_x/xxhtml.htm");

    // loop while finding "file:///.*/file/"
    // or root_dir_fsys
    while (true) {
        // find "file:"
        int lhs_bgn = Bry_find_.Find_fwd(html_bry, Bry__file_lhs, pos);

        // exit if nothing found
        if (lhs_bgn == Bry_find_.Not_found) 
            break;

        // set lhs_end (after "file:///")
        int lhs_end = lhs_bgn + Bry__file_lhs.length;

        // skip if page literally starts with "file:///"
        if (lhs_bgn == 0) {
            bfr.Add_mid(html_bry, pos, lhs_end);
            pos = lhs_end;
            continue;
        }

        int mid_bgn;
        // is this the local root
        if (Bry_.Has_at_bgn(html_bry, root_dir_fsys, lhs_end, len)) {
            mid_bgn = lhs_end + root_dir_fsys.length - 1;
        } else {
            // find "/file/"
            mid_bgn = Bry_find_.Find_fwd(html_bry, Bry__file_mid, lhs_bgn, len);
            
            // skip if no "/file/"
            if (mid_bgn == Bry_find_.Not_found) {
                mid_bgn = -2;
            }
            // skip if "'file:/// ... '" is too long. should be no more than 300
            if (mid_bgn - lhs_end > 300) {
                mid_bgn = -2;
            }
        }

        // add everything up to "file:"
        bfr.Add_mid(html_bry, pos, lhs_bgn);

        // have we a valid mid_bgn
        if (mid_bgn == -2) {
            bfr.Add_mid(html_bry, lhs_bgn, lhs_end);
            pos = lhs_end;
        } else {
            // add "/fsys/"
            bfr.Add(Bry__file_fsys);
            
            // move pos forward
            pos = mid_bgn;
        }
    }

    // add rest
    bfr.Add_mid(html_bry, pos, len);
    return bfr.To_bry_and_clear();
}

the definition of root_dir_fsys needs to be changed too

	private static   byte[] root_dir_fsys;

@gnosygnu
Copy link
Owner

Oops. Missed all these edge cases. Let me add this back to to-do and have a fix for the weekend. Thanks

@gnosygnu gnosygnu removed this from Done in A <div> in D Aug 13, 2019
@gnosygnu gnosygnu added this to In progress in A <div> in D Aug 13, 2019
@gnosygnu gnosygnu changed the title Image and link problem in http_server mode HTTP_Server: Handle edge cases for /fsys (Image and link problem in http_server mode) Aug 13, 2019
@gnosygnu gnosygnu moved this from In progress to Done in A <div> in D Aug 14, 2019
@gnosygnu gnosygnu moved this from Done to In progress in A <div> in D Aug 14, 2019
@gnosygnu
Copy link
Owner

Okay. So I ended up generalizing the logic even more and just doing something like src = Replace(src, "file:///g:/xowa/", "/fsys/") Hopefully the third time is a charm. Thanks

@desb42
Copy link
Collaborator

desb42 commented Aug 15, 2019

Unfortunately, there are two replacements going on
The current fix only replaces one scenario

As an example, I build the world in G:\xowa
I think the html build is hard coding this

I then copy the whole wiki to G:\xowa_dev

I then need to convert both file:///g:/xowa and file:///g:/xowa_dev

I think the hard coding in the html build should be avoided (somehow)

@gnosygnu
Copy link
Owner

I then need to convert both file:///g:/xowa and file:///g:/xowa_dev

Ugh. Should have realized that's why I didn't go with the simple fix before. Slip of mind at late hour last night.

Speaking of late hour, I went back to the previous version and tried a more "general" version that handles both issues. In brief, I look for /file/, /bin/any/, and /user/anonymous/ after a file:///. Hoping the third time is the charm

I think the hard coding in the html build should be avoided (somehow)

Yup, created a new issue at #553

After I get through the current issues in the TODO board, I'll spend some time going through all the HTML Databases ones since they're blockers to creating any new HTML dumps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
A <div> in D
  
Done
Development

No branches or pull requests

3 participants