suggest add stripped binary file gz download #50397

l1t1 · 2023-05-31T15:42:24Z

l1t1
May 31, 2023

curl https://clickhouse.com/ | sh
for aarch64 Linux often aborted due to network error.
a link that supports resume after break and a smaller size file is needed.
thank you.

Felixoid · 2023-06-02T15:33:43Z

Felixoid
Jun 2, 2023
Maintainer

Hello. There are two questions here.

First, the size of the binary is already the smallest possible. Why? Because of the black magic, kudos to @yakov-olkhovskiy!

[felixoid@felix-carbon workflows]$ ls -l clickhouse
-rw-r--r-- 1 felixoid felixoid 551637250 Jun  2 17:35 clickhouse
[felixoid@felix-carbon workflows]$ gzip clickhouse
[felixoid@felix-carbon workflows]$ ls -l clickhouse.gz
-rw-r--r-- 1 felixoid felixoid 542730067 Jun  2 17:35 clickhouse.gz
[felixoid@felix-carbon workflows]$ gzip -d clickhouse.gz
[felixoid@felix-carbon workflows]$ chmod +x clickhouse
[felixoid@felix-carbon workflows]$ ./clickhouse
Use one of the following commands:
clickhouse local [args]
clickhouse client [args]
clickhouse benchmark [args]
clickhouse server [args]
clickhouse extract-from-config [args]
clickhouse compressor [args]
clickhouse format [args]
clickhouse copier [args]
clickhouse obfuscator [args]
clickhouse git-import [args]
clickhouse keeper [args]
clickhouse keeper-converter [args]
clickhouse install [args]
clickhouse start [args]
clickhouse stop [args]
clickhouse status [args]
clickhouse restart [args]
clickhouse static-files-disk-uploader [args]
clickhouse su [args]
clickhouse hash-binary [args]
clickhouse disks [args]
[felixoid@felix-carbon workflows]$ ls -l clickhouse
-rwxr-xr-x 1 felixoid felixoid 2525451040 Jun  2 17:36 clickhouse

The binary is a self-extracting archive (zstd, AFAIR).

The second part. The link supports downloading continue:

[felixoid@felix-carbon workflows]$ wget https://builds.clickhouse.com/master/aarch64/clickhouse --continue
--2023-06-02 17:23:43--  https://builds.clickhouse.com/master/aarch64/clickhouse
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving builds.clickhouse.com (builds.clickhouse.com)... 2606:4700:3108::ac42:2b07, 2606:4700:3108::ac42:28f9, 172.66.40.249, ...
Connecting to builds.clickhouse.com (builds.clickhouse.com)|2606:4700:3108::ac42:2b07|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 489207053 (467M), 317239290 (303M) remaining [binary/octet-stream]
Saving to: ‘clickhouse’

clickhouse                                            42%[+++++++++++++++++++++++++++++++++++++++++========>                                                                    ] 198,87M  12,4MB/s               ^C
[felixoid@felix-carbon workflows]$ wget https://builds.clickhouse.com/master/aarch64/clickhouse --continue
--2023-06-02 17:23:47--  https://builds.clickhouse.com/master/aarch64/clickhouse
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving builds.clickhouse.com (builds.clickhouse.com)... 2606:4700:3108::ac42:28f9, 2606:4700:3108::ac42:2b07, 172.66.43.7, ...
Connecting to builds.clickhouse.com (builds.clickhouse.com)|2606:4700:3108::ac42:28f9|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 489207053 (467M), 280331969 (267M) remaining [binary/octet-stream]
Saving to: ‘clickhouse’

clickhouse                                            51%[++++++++++++++++++++++++++++++++++++++++++++++++++=========>                                                          ] 239,24M  12,5MB/s    eta 18s    ^C
[felixoid@felix-carbon workflows]$ wget https://builds.clickhouse.com/master/aarch64/clickhouse --continue
--2023-06-02 17:23:51--  https://builds.clickhouse.com/master/aarch64/clickhouse
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving builds.clickhouse.com (builds.clickhouse.com)... 2606:4700:3108::ac42:2b07, 2606:4700:3108::ac42:28f9, 172.66.40.249, ...
Connecting to builds.clickhouse.com (builds.clickhouse.com)|2606:4700:3108::ac42:2b07|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 489207053 (467M), 236912008 (226M) remaining [binary/octet-stream]
Saving to: ‘clickhouse’

clickhouse                                            60%[++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=========>                                                ] 280,69M  12,5MB/s    eta 15s    ^C
[felixoid@felix-carbon workflows]$ type -f terminate-and-exit ^C
[felixoid@felix-carbon workflows]$ curl -C https://builds.clickhouse.com/master/aarch64/clickhouse -O
curl: option -C: expected a proper numerical parameter
curl: try 'curl --help' for more information
[felixoid@felix-carbon workflows]$ curl -C - https://builds.clickhouse.com/master/aarch64/clickhouse -O
** Resuming transfer from byte position 296665278
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 22  183M   22 41.9M    0     0  11.2M      0  0:00:16  0:00:03  0:00:13 11.2M^C
[felixoid@felix-carbon workflows]$ curl -C - https://builds.clickhouse.com/master/aarch64/clickhouse -O
** Resuming transfer from byte position 345465022
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 28  137M   28 38.5M    0     0  12.1M      0  0:00:11  0:00:03  0:00:08 12.1M^C
[felixoid@felix-carbon workflows]$ curl -C - https://builds.clickhouse.com/master/aarch64/clickhouse -O
** Resuming transfer from byte position 395997374
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 88.8M  100 88.8M    0     0  12.3M      0  0:00:07  0:00:07 --:--:-- 12.4M
[felixoid@felix-carbon workflows]$

Should we support continuing in the script?.. Well, I don't know. The best person to ask here is @alexey-milovidov. Now the idea is to download the binary into ./clickhouse.

You can continue the download manually though:

> curl https://clickhouse.com | sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2739    0  2739    0     0  32276      0 --:--:-- --:--:-- --:--:-- 32607

Will download https://builds.clickhouse.com/master/amd64/clickhouse into clickhouse

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  526M  100  526M    0     0  11.9M      0  0:00:44  0:00:44 --:--:-- 11.5M

Successfully downloaded the ClickHouse binary, you can run it as:
    ./clickhouse

You can also install it:
    sudo ./clickhouse install
> curl -O -C - https://builds.clickhouse.com/master/amd64/clickhouse
** Resuming transfer from byte position 551637250
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0  526M    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

3 replies

alexey-milovidov Jun 2, 2023
Maintainer

Yes, the universal download script is the main ClickHouse installation channel.

alexey-milovidov Jun 2, 2023
Maintainer

Please edit the script to add retries and continuation.

Felixoid Jun 4, 2023
Maintainer

I don't think continuing with an existing file is a good idea. We have a self-extracting binary, which means we should at least ask if it should be overwritten. But it's better to just override it on a script launch.

Another thing, I didn't quickly find, is how curl will manage retries on changed content. The content is updated twice an hour, and it's possible that the file behind the cache will be changed. Besides that, retries look like a good improvement.

A quick search doesn't give anything particular on how curl behaves on retries if the content-length is changed

Felixoid · 2023-06-02T15:59:27Z

Felixoid
Jun 2, 2023
Maintainer

Oh well. There's another part regarding stripped binaries.

For stripped, please use https://packages.clickhouse.com/tgz/stable/

0 replies

l1t1 · 2023-06-03T00:21:49Z

l1t1
Jun 3, 2023
Author

@Felixoid thank you a lot.
a bit more questions.

the stripped version at https://packages.clickhouse.com/tgz/stable/ only has one version of arm, and from curl https://clickhouse.com/ script, there are two versions. namely, aarch64v80compat and aarch64, in one of my machine(my PC and an old android phone), only aarch64v80compat works.
I know the site supports resume, but the curl often return wrong information and refuse to resume, I tried many times, it happened again and again. following log shows I tried 4 times, only once resumes.

root@localhost:~/ck2# curl -o clickhouse https://builds.clickhouse.com/master/aarch64v80compat/clickhouse -C -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 34  521M   34  181M    0     0   115k      0  1:17:08  0:26:47  0:50:21  126k^C
root@localhost:~/ck2# curl -o clickhouse https://builds.clickhouse.com/master/aarch64v80compat/clickhouse -C -
** Resuming transfer from byte position 189956096
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0  521M    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.
root@localhost:~/ck2# curl -o clickhouse https://builds.clickhouse.com/master/aarch64v80compat/clickhouse -C -
** Resuming transfer from byte position 189956096
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0  521M    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.
root@localhost:~/ck2# curl -o clickhouse https://builds.clickhouse.com/master/aarch64v80compat/clickhouse -C -
** Resuming transfer from byte position 189956096
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0  521M    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.
root@localhost:~/ck2# curl -o clickhouse https://builds.clickhouse.com/master/aarch64v80compat/clickhouse -C -
** Resuming transfer from byte position 189956096
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  4  340M    4 15.6M    0     0  15252      0  6:29:56  0:17:53  6:12:03     0
curl: (18) HTTP/2 stream 1 was not closed cleanly before end of the underlying connection

the black magic works, but some time bring me an interesting problem.
I make a copy of the download file(don't know if it is fine or not), when run it at the first time, it raise Error (ZSTD), and left a temp file , when rename the temp file and try to run it, it RUNS without any warning, but when visit a non-exist parquet file, it raise an (CANNOT_PARSE_ELF)error, if the parquet file exists, it return the right answer as normal. when insert a parquet file exists, it raise an (CANNOT_PARSE_ELF)error again.

root@localhost:~/ck2# cp clickhouse ck2
root@localhost:~/ck2# chmod +x ck2
root@localhost:~/ck2# time ./ck2 local
Unable to obtain inode.

real    0m0.062s
user    0m0.000s
sys     0m0.010s
root@localhost:~/ck2# pwd
/root/ck2

root@localhost:~/ck2# mv ck2 /data/data/com.termux/files/home/
root@localhost:~/ck2# time /data/data/com.termux/files/home/ck2 local
Error (ZSTD): -10 Unknown frame descriptor
Error (ZSTD):18446744073709551596 Data corruption detected
Error happened during decompression.

real    0m2.624s
user    0m4.350s
sys     0m1.820s
root@localhost:/data/data/com.termux/files/home# ll
total 0
-rwxr-xr-x  1 root root  546797860 May 31 14:48 ck2*
-rw-------  1 root root 2928498448 May 31 14:52 ck2.decompressed.f5b7A7

root@localhost:/data/data/com.termux/files/home# mv ck2.decompressed.f5b7A7 ck22
root@localhost:/data/data/com.termux/files/home# chmod +x ck22

root@localhost:/data/data/com.termux/files/home# ./ck22 local
<jemalloc>: Number of CPUs detected is not deterministic. Per-CPU arena disabled.
ClickHouse local version 23.5.1.2378 (official build).

root@localhost:/data/data/com.termux/files/home# ll *par*
-rw-r--r-- 1 root root 4674606386 May 31 15:55 t.parquet
-rw-r--r-- 1 root root  458508518 May 31 21:49 t1.parquet
root@localhost:/data/data/com.termux/files/home# ll /data/*par*
-rw-r--r-- 1 root root 458508521 May 31 21:47 /data/t2.parquet
root@localhost:/data/data/com.termux/files/home# ll /data/*par*
-rw-r--r-- 1 root root 458508521 May 31 21:47 /data/t2.parquet
root@localhost:/data/data/com.termux/files/home# ./ck22 local
<jemalloc>: Number of CPUs detected is not deterministic. Per-CPU arena disabled.
ClickHouse local version 23.5.1.2378 (official build).

localhost.localdomain :) SELECT avg(i) FROM file('/data/t.parquet') group by round(log10(i));

SELECT avg(i)
FROM file('/data/t.parquet')
GROUP BY round(log10(i))

Query id: ce4e2b14-e9b8-4169-9f69-b85004bc50f5


0 rows in set. Elapsed: 1.107 sec.

Received exception:
Code: 464. DB::Exception: The ELF '/data/data/com.termux/files/home/ck22' doesn't have string table with section names. (CANNOT_PARSE_ELF)

localhost.localdomain :) SELECT avg(i) FROM file('/data/t2.parquet') group by round(log10(i));

SELECT avg(i)
FROM file('/data/t2.parquet')
GROUP BY round(log10(i))

Query id: 572e58f8-456b-4f5d-9a03-13372ce826e9

┌─────avg(i)─┐
│          2 │
│   17392527 │
│    17392.5 │
│     1739.5 │
│     173925 │
│       17.5 │
│        174 │
│ 65811388.5 │
│  1739252.5 │
└────────────┘

9 rows in set. Elapsed: 11.048 sec. Processed 100.00 million rows, 500.00 MB (9.05 million rows/s., 45.26 MB/s.)

localhost.localdomain :) insert into function file('t1.parquet') select number::int i FROM numbers_mt(1,10000000);

INSERT INTO FUNCTION file('t1.parquet') SELECT CAST(number, 'int') AS i
FROM numbers_mt(1, 10000000)

Query id: 89a69d82-e3fb-4b4f-b469-d82e20c90ae7


0 rows in set. Elapsed: 0.417 sec.

Received exception:
Code: 464. DB::Exception: The ELF '/data/data/com.termux/files/home/ck22' doesn't have string table with section names. (CANNOT_PARSE_ELF)

the upx tools also make an executable compressed bianry, see clickhouse after upx chksum error in termux upx/upx#673, but it only works on PC, not a phone.

1 reply

Felixoid Jun 4, 2023
Maintainer

For self-extracting details we need @yakov-olkhovskiy. I don't know the implementation details

l1t1 · 2023-06-03T00:47:21Z

l1t1
Jun 3, 2023
Author

TLDR,
I prefer a upx compressd stripped binary, it works on PC, and I can use upx -d to decompressed it to run on phone.

version	size
black magic	521M
not stripped	2900M
normal stripped	416M
upx stripped	92M

2 replies

Felixoid Jun 4, 2023
Maintainer

We never thought of providing stripped binaries AFAIK.

And we strip them only in Debian packages and their repacked versions (rpm, tgz)

Felixoid Jun 4, 2023
Maintainer

If you need the binaries for some particular architect, you can download it as described in documentation. Just open some of the recent commits status ClickHouse special build check

l1t1 · 2023-06-04T14:07:29Z

l1t1
Jun 4, 2023
Author

@Felixoid thank you

0 replies

yakov-olkhovskiy · 2023-06-06T16:33:48Z

yakov-olkhovskiy
Jun 6, 2023
Collaborator

The message Unable to obtain inode says that something wrong with extraction. In order to prevent race condition for extracting process we create lock file in /tmp based on the inode of compressed executable - inode we retrieve by parsing /proc/self/maps file.
@l1t1 could you please post results of the next command on your system:
cat /proc/self/maps

0 replies

l1t1 · 2023-06-06T22:19:13Z

l1t1
Jun 6, 2023
Author

root@localhost:/data/data/com.termux/files/home# cat /proc/self/maps
00200000-00201000 r--p 00000000 fc:01 1972635                            /data/data/com.termux/files/usr/libexec/proot/loader
2000000000-2000001000 r-xp 00001000 fc:01 1972635                        /data/data/com.termux/files/usr/libexec/proot/loader
2000001000-2000002000 r--p 00001000 fc:01 1972635                        /data/data/com.termux/files/usr/libexec/proot/loader
3000000000-3000006000 r-xp 00000000 fc:01 1974005                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/bin/cat
3000015000-3000016000 r--p 00005000 fc:01 1974005                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/bin/cat
3000016000-3000017000 rw-p 00006000 fc:01 1974005                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/bin/cat
3000017000-3000039000 rw-p 00000000 00:00 0
3f00000000-3f0002b000 r-xp 00000000 fc:01 2097421                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1
3f0003a000-3f0003c000 r--p 0002a000 fc:01 2097421                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1
3f0003c000-3f0003e000 rw-p 0002c000 fc:01 2097421                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1
7a1e7b9000-7a1e7db000 rw-p 00000000 00:00 0
7a1e7db000-7a1e832000 r--p 00000000 fc:01 2098957                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/usr/lib/locale/C.utf8/LC_CTYPE
7a1e832000-7a1e833000 r--p 00000000 fc:01 2098964                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/usr/lib/locale/C.utf8/LC_NUMERIC
7a1e833000-7a1e834000 r--p 00000000 fc:01 2098967                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/usr/lib/locale/C.utf8/LC_TIME
7a1e834000-7a1e835000 r--p 00000000 fc:01 2098956                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/usr/lib/locale/C.utf8/LC_COLLATE
7a1e835000-7a1e836000 r--p 00000000 fc:01 2098962                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/usr/lib/locale/C.utf8/LC_MONETARY
7a1e836000-7a1e837000 r--p 00000000 fc:01 2098961                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/usr/lib/locale/C.utf8/LC_MESSAGES/SYS_LC_MESSAGES
7a1e837000-7a1e838000 r--p 00000000 fc:01 2098965                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/usr/lib/locale/C.utf8/LC_PAPER
7a1e838000-7a1e839000 r--p 00000000 fc:01 2098963                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/usr/lib/locale/C.utf8/LC_NAME
7a1e839000-7a1e840000 r--s 00000000 fc:01 2098119                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/usr/lib/aarch64-linux-gnu/gconv/gconv-modules.cache
7a1e840000-7a1e9c9000 r-xp 00000000 fc:01 2097429                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/lib/aarch64-linux-gnu/libc.so.6
7a1e9c9000-7a1e9d8000 ---p 00189000 fc:01 2097429                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/lib/aarch64-linux-gnu/libc.so.6
7a1e9d8000-7a1e9dc000 r--p 00188000 fc:01 2097429                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/lib/aarch64-linux-gnu/libc.so.6
7a1e9dc000-7a1e9de000 rw-p 0018c000 fc:01 2097429                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/lib/aarch64-linux-gnu/libc.so.6
7a1e9de000-7a1e9ea000 rw-p 00000000 00:00 0
7a1e9ea000-7a1e9eb000 r--p 00000000 fc:01 2098955                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/usr/lib/locale/C.utf8/LC_ADDRESS
7a1e9eb000-7a1e9ec000 r--p 00000000 fc:01 2098966                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/usr/lib/locale/C.utf8/LC_TELEPHONE
7a1e9ec000-7a1e9ee000 rw-p 00000000 00:00 0
7a1e9ee000-7a1e9ef000 r--p 00000000 fc:01 2098959                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/usr/lib/locale/C.utf8/LC_MEASUREMENT
7a1e9ef000-7a1e9f0000 r--p 00000000 fc:01 2098958                        /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/usr/lib/locale/C.utf8/LC_IDENTIFICATION
7a1e9f0000-7a1e9f1000 r--p 00000000 00:00 0                              [vvar]
7a1e9f1000-7a1e9f2000 r-xp 00000000 00:00 0                              [vdso]
7fe7956000-7fe7977000 rw-p 00000000 00:00 0                              [stack]

0 replies

yakov-olkhovskiy · 2023-06-06T23:20:58Z

yakov-olkhovskiy
Jun 6, 2023
Collaborator

@l1t1 well... apparently you are using proot, and it doesn't mask chrooted path in the maps, as a result function which is trying to identify inode through executable name fails. So, right now we can only state that extraction doesn't work under chroot, sorry ¯\(ツ)/¯
You can create issue for this I think.

6 replies

Felixoid Jun 7, 2023
Maintainer

I guess, the case here is not the place, but the process. Can you try it without proot?

yakov-olkhovskiy Jun 7, 2023
Collaborator

let me clarify
the problem, I think, is that when you are running it under chroot, path of executable, retrieved by whatever means we use, will be relative - i.e. if chroot was on directory /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu, then retrieved path for cat will be /bin/cat. Then we want to find inode for executable /bin/cat - we read file /proc/self/maps and trying to find corresponding /bin/cat there, which provides us inode. However /proc/self/maps still contain the real path /data/data/com.termux/files/usr/var/lib/proot-distro/installed-rootfs/ubuntu/bin/cat - so, the search fails.

l1t1 Jun 8, 2023
Author

I cannot run clickhouse without proot, because I run Android + termux, it missing some .so file that clickhouse required.

~ $ ldd clickhouse
        librt.so.1 => not found
        libpthread.so.0 => not found
        libc.so.6 => not found
        libm.so.6 => not found
        libdl.so.2 => not found

~ $ pwd
/data/data/com.termux/files/home
 
~ $ proot-distro login ubuntu
root@localhost:~# cd /data/data/com.termux/files/home/

root@localhost:/data/data/com.termux/files/home# ldd ./clickhouse
        librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000007cac940000)
        libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007cac920000)
        libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007cac770000)
        libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007cac6d0000)
        libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007cac6b0000)
        /lib/ld-linux-aarch64.so.1 (0x0000003000000000)

l1t1 Jun 8, 2023
Author

and under proot, only clickhouse local can run, try clickhouse server faild

root@localhost:/data/data/com.termux/files/home# ./clickhouse local
ClickHouse local version 23.4.1.1073 (official build).

localhost.localdomain :) exit
Bye.
root@localhost:/data/data/com.termux/files/home# ./clickhouse server
Processing configuration file 'config.xml'.
There is no file 'config.xml', will use embedded config.
2023.06.08 01:42:55.618589 [ 7298 ] {} <Trace> Pipe: Pipe capacity is 1.00 MiB
2023.06.08 01:42:55.710845 [ 7298 ] {} <Information> Application: Starting ClickHouse 23.4.1.1073 (revision: 54473, git hash: 9eee7c72fe63ffee07b8bb18359633876d5effa1, build id: 36CF40AE1B3167B28AE9D49AC2B983BF43434ED4), PID 7298
2023.06.08 01:42:55.710919 [ 7298 ] {} <Information> Application: starting up
2023.06.08 01:42:55.711904 [ 7298 ] {} <Information> Application: OS name: Linux, version: 6.2.1-PRoot-Distro, architecture: aarch64
2023.06.08 01:42:55.717472 [ 7298 ] {} <Information> Application: Available RAM: 7.56 GiB; physical cores: 8; logical cores: 8.
2023.06.08 01:42:55.872217 [ 7298 ] {} <Trace> AsynchronousMetrics: Scanning /sys/class/thermal
2023.06.08 01:42:55.872448 [ 7298 ] {} <Trace> AsynchronousMetrics: Scanning /sys/block
2023.06.08 01:42:55.898250 [ 7298 ] {} <Error> Application: std::exception. Code: 1001, type: std::__1::__fs::filesystem::filesystem_error, e.what() = filesystem error: in directory_iterator::directory_iterator(...): Permission denied ["/sys/block"], Stack trace (when copying this message, always include the lines below):

0. std::system_error::system_error(std::error_code, String const&) @ 0x1413e100 in /data/data/com.termux/files/home/clickhouse
1. ? @ 0xbc15cd0 in /data/data/com.termux/files/home/clickhouse
2. ? @ 0x140e55cc in /data/data/com.termux/files/home/clickhouse
3. ? @ 0x140e38c0 in /data/data/com.termux/files/home/clickhouse
4. std::__fs::filesystem::directory_iterator::directory_iterator(std::__fs::filesystem::path const&, std::error_code*, std::__fs::filesystem::directory_options) @ 0x140e3718 in /data/data/com.termux/files/home/clickhouse
5. DB::AsynchronousMetrics::openBlockDevices() @ 0xbcbe298 in /data/data/com.termux/files/home/clickhouse
6. DB::AsynchronousMetrics::AsynchronousMetrics(int, std::function<std::vector<DB::ProtocolServerMetrics, std::allocator<DB::ProtocolServerMetrics>> ()> const&) @ 0xbcbd094 in /data/data/com.termux/files/home/clickhouse
7. DB::ServerAsynchronousMetrics::ServerAsynchronousMetrics(std::shared_ptr<DB::Context const>, int, int, std::function<std::vector<DB::ProtocolServerMetrics, std::allocator<DB::ProtocolServerMetrics>> ()> const&) @ 0x1068c8d4 in /data/data/com.termux/files/home/clickhouse
8. DB::Server::main(std::vector<String, std::allocator<String>> const&) @ 0xbc56928 in /data/data/com.termux/files/home/clickhouse
9. Poco::Util::Application::run() @ 0x1206bdbc in /data/data/com.termux/files/home/clickhouse
10. DB::Server::run() @ 0xbc530e0 in /data/data/com.termux/files/home/clickhouse
11. Poco::Util::ServerApplication::run(int, char**) @ 0x12081864 in /data/data/com.termux/files/home/clickhouse
12. mainEntryClickHouseServer(int, char**) @ 0xbc50400 in /data/data/com.termux/files/home/clickhouse
13. main @ 0x7df1e40 in /data/data/com.termux/files/home/clickhouse
14. ? @ 0x273fc in /lib/aarch64-linux-gnu/libc.so.6
15. __libc_start_main @ 0x274cc in /lib/aarch64-linux-gnu/libc.so.6

Cannot print extra info for Poco::Exception (version 23.4.1.1073 (official build))
2023.06.08 01:42:55.898712 [ 7298 ] {} <Error> Application: filesystem error: in directory_iterator::directory_iterator(...): Permission denied ["/sys/block"]
2023.06.08 01:42:55.898735 [ 7298 ] {} <Information> Application: shutting down
2023.06.08 01:42:55.898750 [ 7298 ] {} <Debug> Application: Uninitializing subsystem: Logging Subsystem
2023.06.08 01:42:55.898920 [ 7299 ] {} <Trace> BaseDaemon: Received signal -2
2023.06.08 01:42:55.898967 [ 7299 ] {} <Information> BaseDaemon: Stop SignalListener thread

yakov-olkhovskiy Jun 8, 2023
Collaborator

this is strange

2023.06.08 01:42:55.898250 [ 7298 ] {} Application: std::exception. Code: 1001, type: std::__1::__fs::filesystem::filesystem_error, e.what() = filesystem error: in directory_iterator::directory_iterator(...): Permission denied ["/sys/block"], Stack trace (when copying this message, always include the lines below):

could you please run
ls -al /sys
ls -al /sys/block

l1t1 · 2023-06-08T21:47:52Z

l1t1
Jun 8, 2023
Author

root@localhost:# ls -al /sys
ls: cannot open directory '/sys': Permission denied
root@localhost:# ls -al /sys/block
ls: cannot open directory '/sys/block': Permission denied
root@localhost:~#

1 reply

yakov-olkhovskiy Jun 8, 2023
Collaborator

server requires access to these directories
I don't have much knowledge about termux - probably you need to enable this access somwhere...

l1t1 · 2023-06-08T22:03:40Z

l1t1
Jun 8, 2023
Author

root@localhost:~# ls -al  /sys/class/thermal
ls: cannot open directory '/sys/class/thermal': Permission denied
root@localhost:~# ls -al /
total 0
drwxr-xr-x  27 root root          3488 Apr 17 07:19 .
drwxr-xr-x  27 root root          3488 Apr 17 07:19 ..
drwx------   2 root root          3488 Apr 15 09:53 .l2s
drwxr-xr-x  16 root root           320 Jan 30 10:37 apex
drwxr-xr-x   2 root root          3488 Mar 20 15:43 bin
drwxr-xr-x   2 root root          3488 Feb 16 16:02 boot
drwx------   5 root root          3488 May 23 00:16 data
drwxr-xr-x  24 root root          5900 Jan 30 10:37 dev
drwxr-xr-x  29 root root          3488 Apr 15 09:53 etc
drwxr-xr-x   2 root root          3488 Feb 16 16:02 home
drwxr-xr-x   7 root root          3488 Mar 20 15:43 lib
drwxr-xr-x   2 root root          3488 Mar 20 15:43 media
drwxr-xr-x   2 root root          3488 Mar 20 15:43 mnt
drwxr-xr-x   2 root root          3488 Mar 20 15:43 opt
dr-xr-xr-x 761 root root             0 Jan  1  1970 proc
drwx------  10 root root          3488 Jun  8 01:41 root
drwxr-xr-x   3 root root          3488 Mar 20 15:43 run
drwxr-xr-x   2 root root          3488 Mar 20 15:43 sbin
drwxrwx---  84 root aid_everybody 4096 Jun  8 12:59 sdcard
drwxr-xr-x   2 root root          3488 Mar 20 15:43 srv
drwxr-xr-x   4 root root            80 Jan 30 10:37 storage
dr-xr-xr-x  15 root root             0 Jan 30 10:37 sys
drwxr-xr-x   1 root root           423 Aug  7  2018 system
drwxrwxrwt   4 root root          3488 Jun  8 01:41 tmp
drwxr-xr-x  11 root root          3488 Mar 20 15:43 usr
drwxr-xr-x  11 root root          3488 Mar 20 15:43 var
drwxr-xr-x   1 root          2000  287 Aug  7  2018 vendor

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

suggest add stripped binary file gz download #50397

{{title}}

Replies: 10 comments 13 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

suggest add stripped binary file gz download #50397

l1t1 May 31, 2023

Replies: 10 comments · 13 replies

Felixoid Jun 2, 2023 Maintainer

alexey-milovidov Jun 2, 2023 Maintainer

alexey-milovidov Jun 2, 2023 Maintainer

Felixoid Jun 4, 2023 Maintainer

Felixoid Jun 2, 2023 Maintainer

l1t1 Jun 3, 2023 Author

Felixoid Jun 4, 2023 Maintainer

l1t1 Jun 3, 2023 Author

Felixoid Jun 4, 2023 Maintainer

Felixoid Jun 4, 2023 Maintainer

l1t1 Jun 4, 2023 Author

yakov-olkhovskiy Jun 6, 2023 Collaborator

l1t1 Jun 6, 2023 Author

yakov-olkhovskiy Jun 6, 2023 Collaborator

Felixoid Jun 7, 2023 Maintainer

yakov-olkhovskiy Jun 7, 2023 Collaborator

l1t1 Jun 8, 2023 Author

l1t1 Jun 8, 2023 Author

yakov-olkhovskiy Jun 8, 2023 Collaborator

l1t1 Jun 8, 2023 Author

yakov-olkhovskiy Jun 8, 2023 Collaborator

l1t1 Jun 8, 2023 Author

l1t1
May 31, 2023

Replies: 10 comments 13 replies

Felixoid
Jun 2, 2023
Maintainer

alexey-milovidov Jun 2, 2023
Maintainer

alexey-milovidov Jun 2, 2023
Maintainer

Felixoid Jun 4, 2023
Maintainer

Felixoid
Jun 2, 2023
Maintainer

l1t1
Jun 3, 2023
Author

Felixoid Jun 4, 2023
Maintainer

l1t1
Jun 3, 2023
Author

Felixoid Jun 4, 2023
Maintainer

Felixoid Jun 4, 2023
Maintainer

l1t1
Jun 4, 2023
Author

yakov-olkhovskiy
Jun 6, 2023
Collaborator

l1t1
Jun 6, 2023
Author

yakov-olkhovskiy
Jun 6, 2023
Collaborator

Felixoid Jun 7, 2023
Maintainer

yakov-olkhovskiy Jun 7, 2023
Collaborator

l1t1 Jun 8, 2023
Author

l1t1 Jun 8, 2023
Author

yakov-olkhovskiy Jun 8, 2023
Collaborator

l1t1
Jun 8, 2023
Author

yakov-olkhovskiy Jun 8, 2023
Collaborator

l1t1
Jun 8, 2023
Author