Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI generates gz versions of the json files #7

Closed
ewized opened this issue Jan 14, 2020 · 11 comments
Closed

CLI generates gz versions of the json files #7

ewized opened this issue Jan 14, 2020 · 11 comments
Assignees

Comments

@ewized
Copy link
Contributor

ewized commented Jan 14, 2020

On mac/linux I had to extract all the generated files by hand to use with a self hosted webserver. I notice the CLI is WIP so not sure if you are aware of this issue. Felt like posting this so anyone who run across this can run the command bellow to fix it with out re rending an entire world (since mine toke over 3 hours to complete)

The command was:

find data/ --name "*.gz" | xargs gunzip

@TBlueF
Copy link
Member

TBlueF commented Jan 14, 2020

Oh god no! This is intended! :D
The files are saved as .gz to save space (a lot!!) and are sent by the webserver directly using the Content-Encoding: gzip header, so the browser extracts them on the fly..

You can setup your webserver so it does that:
with nginx there is the setting gzip_static always; you can use for the .gz files
and apache can do that as well im sure.

Please don't extract them, they will be up to 20x as big xD

@TBlueF TBlueF closed this as completed Jan 14, 2020
@ewized
Copy link
Contributor Author

ewized commented Jan 14, 2020

At least there should be an option to generate the non gz versions of the files

@TBlueF
Copy link
Member

TBlueF commented Jan 15, 2020

I will add that to the config

@TBlueF TBlueF reopened this Jan 15, 2020
@TBlueF TBlueF self-assigned this Jan 15, 2020
@TBlueF
Copy link
Member

TBlueF commented Jan 18, 2020

Added with 3b53932

@TBlueF TBlueF closed this as completed Jan 18, 2020
@Sciroccogti
Copy link

Sciroccogti commented Apr 1, 2020

Oh god no! This is intended! :D
The files are saved as .gz to save space (a lot!!) and are sent by the webserver directly using the Content-Encoding: gzip header, so the browser extracts them on the fly..

You can setup your webserver so it does that:
with nginx there is the setting gzip_static always; you can use for the .gz files
and apache can do that as well im sure.

Please don't extract them, they will be up to 20x as big xD

I think you should notice those who use self hosted webservers to turn the gzip_static option on 😄!

I 'm new to Apache, and I couldn't have my map hosted correctly until I found this issue. Maybe more guys will run into this problem.

Thanks! Anyway it's a great mapping tool!


Update: I still don't know how to configure the Apache to host the map correctly. Of course it's not your fault, but I'll appreciate it if there's any help.

@TBlueF
Copy link
Member

TBlueF commented Apr 1, 2020

@Sciroccogti Since i am not an apache expert myself, all i can offer you currently is this:
https://coderwall.com/p/sx-8xa/serving-pre-compressed-assets-through-apache

the difference would be that its .json.gz files in our case..

maybe you can adapt from that

@Sciroccogti
Copy link

@TBlueF It's very kind of you to offer the tutorial, but it's still not working.

I have now switched to Nginx, however it turns out that my website is not working yet.

I'm sorry to bother you, but could you PLZ have a look at my conf in nginx/sites-enabled/?

server {
        listen 80;
        listen [::]:80;

        server_name map.sciroccogti.top;

        root /path/to/BlueMap/web/;
        index index.html;

    gzip on;
    gzip_static on;
    gzip_http_version 1.1;
    gzip_disable "MSIE [1-6]\.";
    gzip_vary on;
    gzip_proxied expired no-cache no-store private auth;
    #gzip_types text/plain text/json  application/x-javascript text/xml application/xml application/xml+rss text/javascript application/json;

    #return 301 https://$server_name$request_uri;

        location / {
                try_files $uri $uri/ =404;
        }
}

BTW, I noticed that gzip over ssl is not safe, is that true?

@TBlueF
Copy link
Member

TBlueF commented Apr 2, 2020

@Sciroccogti Sorry, i can and will not help you here with bluemap unrelated issues.
Also, the "gzip over ssl is not safe" link you posted links to an issue with nginx from 2014 that has been resolved.

@Sciroccogti
Copy link

@TBlueF That's OK. Thanks anyway.

@Sciroccogti
Copy link

I figured it out. The lines about location make the server no working. The successful conf is list below:

server {
        listen 80;
        listen [::]:80;

        server_name map.sciroccogti.top;

        root /path/to/BlueMap/web/;
        index index.html;

        gzip off;
        gzip_static on;
        gzip_proxied expired no-cache no-store private auth;
}

@kencinder
Copy link

kencinder commented Apr 12, 2020

Just in case anyone else wants to know how to get this working on their Apache webserver, I figured it out.
Tweak the following to suit your setup.

DocumentRoot /var/www/
<Directory /var/www/>
allow from all
Options FollowSymLinks
Require all granted
SetEnv no-gzip

        RewriteEngine on

        # Make sure the browser supports gzip encoding before we send it
        # without it, Content-Type will be "application/x-gzip"

        RewriteCond %{HTTP:Accept-Encoding} \b(x-)?gzip\b
        RewriteCond %{REQUEST_FILENAME}.gz -s
        RewriteRule ^(.+) $1.gz [L]

    # Also add a content-encoding header to tell the browser to decompress

    <FilesMatch .gz$>
        ForceType application/json
        Header set Content-Encoding gzip
    </FilesMatch>
    
</Directory>

Edit - I forgot to note that this needs the HEADERS and REWRITE mods for Apache enabled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants