We implement our Machine Learning in status or 'about me' in Profile User at android application. Our machine learning model classify status as:
- Toxic status
- Non- Toxic status
For the process, you can look at colab notebook
- Mobile send information (register,login, comment, or post the job vacancy)
- For saving entity (profile data, posting data,status), it will be saved at Realtime Database, but email and password will be saved in Firebase authentication for account authentication
- Event (update,delete,creat of comment) in Realtime Database will trigger Cloud Function to run source code
- Cloud Function download ML model in Cloud Storage, then ML model classify the comment as hate and abusive or toxic status or not
- File in Cloud Storage loaded to Cloud Function
- If there is abusive and toxic comment, it will update data in Realtime Database and stream status to Bigquery
- Mobile app request to get data from Realtime Database
- Firebase Console
- Realtime Database
- Cloud Storage (extend into GCP automatically)
- GCP Console
- GCP Project
- Service Account
- Cloud Function
- Bigquery
- Open https://console.firebase.google.com
- Click Add Project
- Click bar in Enter Your Project Name. For integrated with GCP, you should enter GCP Project name
- Slide toggle to not use Firebase Analytic and click project
- In right side of Firebase console, you can click the service, then click get started for realtime database and cloud storage,ensure you keep realtime database and cloud storage link
For more information for Firebase, you can see the documentation
- Open your GCP Project
- Click Navigation > IAM & Admin > service accounts
- Click Create Service Account
- Enter name, then click create
- For the testing,we set role owner for service account,select role > Basic > owner
- Click Done
- Create a new key as JSON for service account
- Open Home to GCP Dashboard > Navigate > Storage
- Create Bucket
- Enter name for bucket, it must unique, you can enter
[project-id]-bucket
for the name of bucket - Leave other default setting,Click Create
- Open powershell in GCP,ensure that cloud shell connect into your account and project
- Clone this repository
git clone https://github.com/raihanarfi12-lab/gcp-babgkit-b21-cap0233.git
- To copy the ML file, you can copy this code
cd babgkit-b21-cap0233
gsutil cp tokenizer.pickle gs://[project-id]-bucket
gsutil cp variables.data-00000-of-00001 gs://[project-id]-bucket
gsutil cp variables.index gs://[project-id]-bucket
- Open Home to GCP Dashboard > Navigate > Bigquery
- Click Done for Welcoming in Bigquery UI
- Click three-dot beside of your project ID,then create dataset
- Enter 'jobstify' for dataset ID, and click create
- Open dataset and Click Plus in the box symbol to create table
- For table name, you can enter 'toxic_status'
- For schema you can clik add field 3 times, then enter 'uid','full_name' and 'status' for the field
- Click create table
- Open Home to GCP Dashboard > Navigate > Cloud Function
- Create function
- Choose Firebase Realtime Database as trigger
- Input Firebase Realtime Database name
- For databse path you can enter /Profile User/{push_id} and choose write for event type
- Click Runtime, Build, and Connection setting
- For Runtime service account, you can choose service account that you make
- For Memory allocated, choose 4 GiB and choose 30 seconds for timeout, then click next
- Open powershell editor, open gcp-babgkit-b21-cap0233 > main.py, you can edit name of [BUCKET-NAME] to bucket that have you made, then copy to Cloud Function
- For runtime, you can choose Python 3.8
- For entrypoint, keep default name of entrypoint, hello_rtdb. If you want to change, then change function name in main.py same with entrypoint
- For requirements.txt, you can copy the the file from gcp-babgkit-b21-cap0233 > requirements.txt
- Actually, key from service account downloaded automatically to your computer. Open the file with notepad,then copy the content of file. In Cloud Function, add file to deploy, give the filename "function.json"
- Click Deploy to deploy Cloud Function
NB: Sometimes, for deploy Cloud function, there is problem to load tensorflow. For that, you can try to deploy cloud function again
- Input with key name and status, then you can give random name and hate,abusive, and toxic status in Indonesia Language in Firebase Realtime Database
- Check the logs in function, there will be output similiar like [0.56787], if prediction[0][0]>0.5, it will be toxic comment. Under of that, there will be information about status. For non toxic comment, it will be viewed only in logs.
- After that, go to Bigquery, and in query editor type this query to get the table
SELECT * FROM `[project-id].jobstify.toxic_status`
NB: Table need time to update content, if you directly check the table through project-id > jobstify > toxic_status, so we recommend to use query editor to look quickly the content,if you need simulation for testing, you can checkout link
- Open Home to GCP Dashboard > Navigate > Monitoring > Dashboard
- Create Dashboard
- Choose resource type and metrics for dashboard and choose stacked bar for the shape of graph In our project we choose:
- Resource type : Firebase Realtime Database, Metric : Database Load
- Resource type : Cloud Function, Metric : Cloud Function
- Resource type : GCS Bucket , Metric : Received Bytes
- Resource type : Bigquery, Metric : Uploaded rows