Configuration documentation?

[juju4]

Hello,

I gave a try to AIL with docker. It starts correctly. after I filled some terms in Terms Management but I don't see anywhere where I can configure where AIL should fetch data, which websites (pastebin...) and at which frequency.

Where are those options?
Didn't find any official documentation (No wiki, AIL-framework/doc/ seems more about code/architecture)

Thanks

[adulau]

@juju4 This is separated from AIL.

If you want to some raw data via ZMQ, you can get in touch directly with us.

Alternatively you can setup pystemon with the Redis connector:

https://github.com/CIRCL/pystemon/blob/master/pystemon.yaml

and use the feeder built-in in AIL:

https://github.com/CIRCL/AIL-framework/blob/master/bin/feeder/pystemon-feeder.py

I hope this helps.

[juju4]

Thanks @adulau ! will review that!

[AverageS]

Hello, @adulau! You said one can get in touch directly with you for raw ZMQ data. What should i do for data and instructions on how to use it? :)

[SteveClement]

@AverageS is this still of interest?

[ag-michael]

Hi,

Pystemon does not work (will file an issue on the project's repo), there is this page https://github.com/CIRCL/AIL-framework/blob/04b9d9fc1d65d1758200e2a9ac1a52db9ee88569/HOWTO.md which has information about crawlers but it has all been striken out. When I "send to spider" a crawl task, I get an error: ERROR: Script: http://127.0.0.1:8052 SPASH DOWN