Scripts to get data for various datasources.
To be able toget data from any IRC channel we need to:
- Install supybot on that channel, which requires:
- Dockerizing supybot to be able to run it on Kubernetes.
- Run
supybot-wizard
somehow to generate bot config file, that will be used by the doc ker image. - Configure non-root user to run the bot (docker image must run as non-root, supybot forbids root).
- Store supybot logs into some PV to make it available for the Perceval IRC backend pod
- Create Perceval IRC datasource pod that will have access to bot logs to analyze them
- List all IRC channels we need and subscribe supybot to all of them (as a part of creating config files)
- Finally figure out how to analyze logs using
perceval supybod
command.
You neeed to:
- Go to Hyperledger groups.io or Zephyr groups.io or any other groups.io page.
- Create account, remember
<email>
and<password>
. - Log in to that account, subscribe to
hyperledger
orzephyrproject
or any other groups.io page. - Created token directory:
mkdir /etc/groupsio
. - Run attached
groupsio.sh
script to get groups.io data for the project:./groupsio.sh projectname
.projectname
can behyperledger
orzephyrproject
or any other groups.io page. - Go to all given groups.io pages and subscribe you
<email>
account to all of them. - It will ask for email and password, use
<email>
and<password>
you have used to register to groups.io. It will generate/etc/groupsio/token
file. - Subsequent runs will use that file and will not ask for
email/password
pair, it will use saved token. - You can always run
./groupsio_subscriptions.py < /etc/groupsio/token
to get a list of subscriptions for your groups.io account.
Caveats:
- The final approach should have all this data stored in GrimoireLab config (whatever it is) - probably in Kubernetes secrets.
To get dockerhub
data for the Hyperledger project run: ./hyperledger_dockerhub.sh
.
To get data for Hyperledger Jenkins:
To get Jenkins data run: ./jenkins.sh
.
To get data from meetups:
- Register in meetup, for example go here and sing up.
- Create an OAuth token TODO.
- For now we can try accessing without token but this will hit raate limits, I've requested more informatiuons here.
Caveats:
- New meetup users cannot generate API tokens anymore, they're deprecated.
- If you have an old meetup account you can still use old API key, just put it in
/etc/meetup/token
file.
Yocto project uses bugzilla 4.4.X. there are two perceval backends for Bugzilla: bugzilla
(old one) and bugzillarest
using a lot faster RES API, but it requires Bugzilla 5+. So we can only use bugzilla
for the Yocto project.
You need to provide /etc/bugzilla/login
and /etc/bugzilla/pwd
files (user and password) for bugzilla backed. Go to Youct Bugzilla and create an account there.
Run: ./bugzilla.sh
to get data.
Run ./git.sh
script to get git data, note that this script doesn't have all possible repos listed - this is just an example.
There is no Rocket.Chat support in Perceval, if we need this, we have to implement this and upstream, see here.
Run ./pipermail.sh
script, please note that this script is only an example with a single source. The final approach must have all pipermail sources from LF Jira/Confluence.
Go to EdgeX Disclosure, sign up.
Create API token and save it in /etc/discourse/token
[TODO] how.
Run: ./discourse.sh
.
Other historical scripts are in perceval
and elk
folders, more perceval documentation is here.
New DevStats Hyperledger page is here, please not that this is a bare Grafana without SSL, meant to be embedded in some LFDA panel in the future.