Facebook Private Group Crawler

This project target goal is help me to crawl down all the information that I want from Facebook private group. Filter out the post that contain specific target keyword , save data into Firebase Realtime Database and notify user by LINE.

Environment

• Python 3.8.3 or higher
• Ubuntu 20.04.2 LTS
• LINE Notify account
• Firebase

Build

1.Install requirements.txt

pip3 install -r requirements.txt

2.Prepare LINE-notify token

How to get LINE-notify token

3.Prepare Firebase serviceAccount.json

How to get your Firebase serviceAccount.json and certificate

4.Run

python3 crawler.py -a [facebook username/email address] -p [facebook password]

Result

Notes

1. You can set your specific keyword in the post that you are looking for in the crawler.py.

2. This script can work succesfully with once a time, and terminate. But cannot execute continuously because Facebook will block you down, and ask you to verify and relogin again. So after some consider, I'm not going to make extra features or functions to solve this contidion. There have some better way to accomplish such goal on other website, but not in Facebook. API is recommended for these requires, but you have to create your own functions API in special require.

3. Atleast in this script, I had learned most of the crawl and scrapy skill so it is not meaningless to me.

4. Feel free to contact me if you have any problem or idea!

References

• Crawler tutorial
• Firebase serviceAccount.json
• Firebase firebase_admin_db module document
• Facebook API