Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crawling 18 sites to depth of 5 stops yacy and logging info. Minor work in progress to remove hazelcast lib #504

Closed
smokingwheels opened this issue Sep 28, 2022 · 2 comments

Comments

@smokingwheels
Copy link

`I 2022/09/28 19:07:34 com.hazelcast.internal.diagnostics.HealthMonitor [192.168.1.20]:5701 [YaCyP2P] [4.2] processors=8, physical.memory.total=16.0G, physical.memory.free=8.1G, swap.space.total=0, swap.space.free=0, heap.memory.used=518.6M, heap.memory.free=81.4M, heap.memory.total=600.0M, heap.memory.max=600.0M, heap.memory.used/total=86.44%, heap.memory.used/max=86.44%, minor.gc.count=8289, minor.gc.time=32418ms, major.gc.count=1321, major.gc.time=126135ms, load.process=0.00%, load.system=9.51%, load.systemAverage=n/a thread.count=339, thread.peakCount=656, cluster.timeDiff=1989, event.q.size=0, executor.q.async.size=0, executor.q.client.size=0, executor.q.client.query.size=0, executor.q.client.blocking.size=0, executor.q.query.size=0, executor.q.scheduled.size=0, executor.q.io.size=0, executor.q.system.size=0, executor.q.operations.size=0, executor.q.priorityOperation.size=0, operations.completed.count=30679, executor.q.mapLoad.size=0, executor.q.mapLoadAllKeys.size=0, executor.q.cluster.size=0, executor.q.response.size=0, operations.running.count=0, operations.pending.invocations.percentage=0.00%, operations.pending.invocations.count=0, proxy.count=1, clientEndpoint.count=0, connection.active.count=0, client.connection.count=0, connection.count=0

I 2022/09/28 19:07:33 YACY * publish: handshaked senior peer 'agent-gafumaf-ufe-45' at [217.83.192.73]`

Was crawling 18 sites to depth of 5 with 600 mb of memory for a little while as a test.

Windows 11 old I7
Java 19
HDD
When crawling my wifi was pulling approx 16 mega bits a second of data.

@smokingwheels
Copy link
Author

smokingwheels commented Sep 28, 2022

Crawler is paused with green button on the pause. Click it and yacy starts to crawl again.
Yes it ran out of memory again. Got to 92% used.

@Orbiter
Copy link
Member

Orbiter commented Sep 28, 2022

hazelcast is removed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants