on YouTube

why? instagram is the biggest vangav backend template designed for big scale and covering all the core features/utilities: service oriented architecture, worker service, multi-entry-point api, multi-keysapce database, oauth 2, facebook authentication, facebook graph api, rest jobs, periodic jobs, vangav mighty, push notifications, logging, analytics, client generator, test service, bots service, ...

instagram

• instagram, instagram dispense, instagram jobs, instagram worker and instagram dash board services work together and are generated using vangav backend

• instagram test is used to test the services above and instagram bots is used to simulate normal users' behavior; both services are generated using vangav backend

prerequisite

• vangav backend tutorials

functionality

instagram

• signup/login using email/facebook
• request authentication using oauth 2
• fetching/updating users' information
• follow/unfollow and get followers/following lists for users
• posting photos
• get posts by feed and by user
• interacting with posts (like, comment, ...)
• get top users/posts
• searching for users
• get server time

instagram dispense

• is a helper service for instagram; responsible for distributing posted photos to followers (because on photo can be posted to tens of millions of followers; such an expensive operation should be distributable and shouldn't overload the main service)

instagram jobs

• crawls for failed jobs in the database to retry executing them (e.g.: a dispense job, ...)
• uses vangav mighty to update top users (per geo grid, per country and world wide) every week
• uses vangav mighty to update top posts (per geo grid, per country and world wide) every day

instagram worker

• writes logs and analytics in the database
• sends push notifications (ios and android)

instagram dash board

• handles fetching logs and analytics for system admin

instagram test

• has a full-coverage test for all the services above

instagram bots

• simulates incremental usage behavior from many cities around the world with some celebrity users

overview

• this service is based on vangav backend's instagram template
• this service has the 90+% of the vangav backend's generated code + the 10-% of the logic code needed to complete the service

try this service

1. for first timers - follow the steps in the system requirements tutorial
2. for first timers - follow the steps in the workspace initialization tutorial
3. download instagram, instagram dispense, instagram jobs, instagram worker and instagram dash board projects (from the green clone or download button) inside the workspace directory created previously (my_services) and unzip them
4. rename unzipped directories, remove the -master from their names
5. in the terminal cd to vos_instagram/cassandra/cql/
6. execute ./_start_cassandra.sh to start cassandra
7. cd to vos_instagram/cassandra/cql/drop_and_create/
8. execute the commands ./_execute_cql.sh ig_....cql to initialize the services' database tables; repeat this step for all the .cql files
9. cd to vos_instagram_worker and execute ./_run.sh to start the instagram worker service on port 8000
10. cd to vos_instagram_dispense and execute ./_run.sh 3000 to start the instagram dispense service on port 3000
11. cd to vos_instagram_jobs and execute ./_run.sh 4000 to start the instagram jobs service on port 4000
12. cd to vos_instagram_dash_board and execute ./_run.sh 5000 to start the instagram dash board service on port 5000
13. cd to vos_instagram and execute ./_run.sh to start the instagram service on port 9000
14. make a call to the start_jobs controller from the vos_instagram_jobs backend service in order to start the periodic jobs
15. from your prefered client (we recommned postman) start trying the service; refer to the features and service references sections for reference

• at the end to stop the services: press control + d in the terminal session where each service was started in (9, 10, 11, 12 and 13)
• to stop cassandra: execute ps auwx | grep cassandra to get cassandra's (pid) then kill -9 (pid) to stop cassandra

eclipse

• follow the following steps to import the downloaded backend service in eclipse

1. cd to the service's directory and execute the ./_eclipsify.sh script
2. file > import > general > existing projects into workspace > next > set "select root directory" to my_services > under projects make sure that vos_calculate_sum is selected > finish
3. double check the java version used for compiling the project: right click the project > properties > java compiler > enable project specific settings > compiler compliance level > 1.7 or 1.8

covered topics

• generate multiple services (main + dispense + jobs + worker + dash board) to work together in a service oriented architecture
• generate a multi-keyspace database
• oauth 2 and facebook authentication
• facebook graph api
• rest jobs
• periodic jobs
• vangav mighty
• push notifications
• logging
• analytics
• client generator
• test and bots services

features

instagram

controller(s)	feature
signup email, login email, login facebook, get access token, refresh access token and logout	handle user signup, loging and requests' authentication
update facebook info, update last active time, update location, update profile picture, get user info and get profile picture	handle setting/updating users' info and getting it
follow, unfollow, get followers and get following	handle building the social graph and exploring it
post photo	handles posting a new photo to a user's followers
get feed posts, get user posts, get post info, get post photo id, get photo, get photo thumbnail, get post likes and get post comments	handle getting feed/users' posts and posts' info/content/likes/comments
like post, unlike post, comment and delete comment	handle interacting with posts
get top users and get top posts	handle exploring top users/posts
search by email	handles searching for users by their e-mail
get server time	getting server's time is important for various reasons like enabling the client to set how old a post is (2h, 1w, ...)

instagram dispense

controller	feature
post photo to followers	handles distributing new posts to followers; used by instagram: post photo

instagram jobs

controller	feature
start jobs	stars the periodic job for retrying failed rest jobs and the periodic jobs for ranking users and posts

instagram dash board

controller(s)	feature
get daily users error logs, get daily users logs and get hourly controllers error logs	handle getting logs per user/controller
get annual regional counters, get daily regional counters, get daily requests counters and get hourly requests counters	handle getting backend-wide analytics
get top users and get top posts	gets in-depth information about top users/posts on specified dates/locations

instagram test

controller	feature
test all controllers	runs a full-coverage test for the instagram services above and returns a full report (success, failure, request-to-response time, ...)

instagram bots

controller	feature
start bots	starts simulating incremental usage behavior from many cities around the world with some celebrity users

service references

instagram

reference	explanation
routes	api routes
prop	in addition to vangav backend's properties files, this directory contains instagram properties files: constants and dispense
reverse_geo_coding	reverse geo coding data
controllers.json	api request/response's elements
ig_auth.keyspace	ig_auth is the keyspace used for all authentication-related tables
ig_app_data.keyspace	ig_app_data is the keyspace used for all user-info-related tables which are directly relevant to the app's functionality (e.g.: posts, followers, ...)
ig_blobs.keyspace	ig_blobs is the keyspace used for all blob-related tables (photos, thumbnails, ...)
ig_jobs.keyspace	ig_jobs is the keyspace used for all jobs-related tables -- a job is an operation that's either executed at a later time or issued by one service and executed by another where a jobs table keeps track of unsuccessful jobs which need to be reexecuted
ig_logging.keyspace	ig_logging is the keyspace used for all logging-related tables
ig_analytics.keyspace	ig_analytics is the keyspace used for all analytics-related tables
Global.java	initializes reverse geo coding data
common	handles common operations like authentication, cassandra's frequent-operations, omitting blobs from requests/responses for logging, ...
controllers	api implementation
ig_auth	ig_auth cassandra's keyspace client
ig_app_data	ig_app_data cassandra's keyspace client
ig_blobs	ig_blobs cassandra's keyspace client
ig_jobs	ig_jobs cassandra's keyspace client
ig_logging	ig_logging cassandra's keyspace client
ig_analytics	ig_analytics cassandra's keyspace client

instagram dispense

reference	explanation
routes	api routes
controllers.json	api request/response's elements
ig_auth.keyspace	ig_auth is the keyspace used for all authentication-related tables
ig_app_data.keyspace	ig_app_data is the keyspace used for all user-info-related tables which are directly relevant to the app's functionality (e.g.: posts, followers, ...)
ig_jobs.keyspace	ig_jobs is the keyspace used for all jobs-related tables -- a job is an operation that's either executed at a later time or issued by one service and executed by another where a jobs table keeps track of unsuccessful jobs which need to be reexecuted
ig_logging.keyspace	ig_logging is the keyspace used for all logging-related tables
ig_analytics.keyspace	ig_analytics is the keyspace used for all analytics-related tables
controllers	api implementation
ig_auth	ig_auth cassandra's keyspace client
ig_app_data	ig_app_data cassandra's keyspace client
ig_jobs	ig_jobs cassandra's keyspace client
ig_logging	ig_logging cassandra's keyspace client
ig_analytics	ig_analytics cassandra's keyspace client

instagram jobs

reference	explanation
routes	api routes
prop	in addition to vangav backend's properties files, this directory contains instagram jobs properties files: constants, users rank and posts rank
reverse_geo_coding	reverse geo coding data
controllers.json	api request/response's elements
ig_app_data.keyspace	ig_app_data is the keyspace used for all user-info-related tables which are directly relevant to the app's functionality (e.g.: posts, followers, ...)
ig_jobs.keyspace	ig_jobs is the keyspace used for all jobs-related tables -- a job is an operation that's either executed at a later time or issued by one service and executed by another where a jobs table keeps track of unsuccessful jobs which need to be reexecuted
ig_logging.keyspace	ig_logging is the keyspace used for all logging-related tables
Global.java	initializes reverse geo coding data and periodic jobs
common	handles common operations like instagram-jobs-service's properties
periodic_jobs	handles retrying failed jobs as well as ranking users and posts
controllers	api implementation
ig_app_data	ig_app_data cassandra's keyspace client
ig_jobs	ig_jobs cassandra's keyspace client
ig_logging	ig_logging cassandra's keyspace client

instagram dash board

reference	explanation
routes	api routes
controllers.json	api request/response's elements
ig_auth.keyspace	ig_auth is the keyspace used for all authentication-related tables
ig_app_data.keyspace	ig_app_data is the keyspace used for all user-info-related tables which are directly relevant to the app's functionality (e.g.: posts, followers, ...)
ig_logging.keyspace	ig_logging is the keyspace used for all logging-related tables
ig_analytics.keyspace	ig_analytics is the keyspace used for all analytics-related tables
common	handles common operations like service-wide constants
controllers	api implementation
ig_auth	ig_auth cassandra's keyspace client
ig_app_data	ig_app_data cassandra's keyspace client
ig_logging	ig_logging cassandra's keyspace client
ig_analytics	ig_analytics cassandra's keyspace client

instagram test

reference	explanation
routes	api routes
prop	in addition to vangav backend's properties files, this directory contains the clients properties file clients
controllers.json	api request/response's elements
vos_instagram.client_java	instagram client's api request/response's elements
vos_instagram_dash_board.client_java	instagram dash board client's api request/response's elements
common	handles common operations like representing instagram users
clients	contains the generated clients for instagram and instagram dash board services
controllers	api implementation

instagram bots

reference	explanation
routes	api routes
setup_data	contains a one-time setup data with all the cities (id, location, population, ...) where the bots are created
prop	in addition to vangav backend's properties files, this directory contains the clients properties file clients
controllers.json	api request/response's elements
vos_instagram.client_java	instagram client's api request/response's elements
ig_bots.keyspace	ig_bots is the keyspace used for all bots-related vos_instagream tables
Global.java	initializes cities and bots
cities	has the cities' json representation and loader
bots	has the bots' periodic jobs and the bots creator
clients	contains the generated clients for instagram service
controllers	api implementation
ig_bots	ig_bots cassandra's keyspace client

change log

• this section lists the 10-% code added after vangav backend generated 90+% of the code

instagram

file/dir	change
prop	added instagram properties files: constants and dispense
reverse_geo_coding	added reverse geo coding data
Global.java	added the initialization of reverse geo coding data
common	added common operations like authentication, cassandra's frequent-operations, omitting blobs from requests/responses for logging, ...
controllers	added the implementation of request processing logic under controller_name/HandlerControllerName.java classes and nested response json structures under controller_name/response_json packages

instagram dispense

file/dir	change
controllers	added the implementation of request processing logic under controller_name/HandlerControllerName.java classes and nested response json structures under controller_name/response_json packages

instagram jobs

file/dir	change
prop	added instagram jobs properties files: constants, users rank and posts rank
reverse_geo_coding	added reverse geo coding data
Global.java	added the initialization of reverse geo coding data and periodic jobs
common	added common operations like instagram-jobs-service's properties
periodic_jobs	added the periodic jobs for retrying failed jobs as well as ranking users and posts
controllers	added the implementation of request processing logic under controller_name/HandlerControllerName.java classes and nested response json structures under controller_name/response_json packages

instagram dash board

file/dir	change
common	added common operations like service-wide constants
controllers	added the implementation of request processing logic under controller_name/HandlerControllerName.java classes and nested response json structures under controller_name/response_json packages

instagram test

file/dir	change
common	added common operations like representing instagram users
controllers	added the implementation of request processing logic under controller_name/HandlerControllerName.java classes and nested response json structures under controller_name/response_json packages

instagram bots

file/dir	change
setup_data	added a one-time setup data with all the cities (id, location, population, ...) where the bots are created
Global.java	added the initialization of cities and bots
cities	added the cities' json representation and loader
bots	added the bots' periodic jobs and the bots creator
controllers	added the implementation of request processing logic under controller_name/HandlerControllerName.java classes and nested response json structures under controller_name/response_json packages

error codes

• following are the error codes of instagram services

instagram

class	code : sub_code	explanation
Constants
	300 : 1	couldn't initialize kAuthCodeLifeTime
	300 : 2	couldn't initialize kAccessTokenLifeTime
	300 : 3	couldn't initialize kFacebookAppId
	300 : 4	couldn't initialize kDefaultRegion
	300 : 5	couldn't initialize kWorldRegion
	300 : 6	couldn't initialize kDefaultGridId
	300 : 7	couldn't initialize kGeoGridDimensionMetres
	300 : 8	couldn't initialize kGeoGridsConfig
	300 : 9	couldn't initialize kThumbnailDimension
CommonPlayHandler
	400 : 1	no access token for user id and device token
	400 : 2	wrong access token
	400 : 3	already expired access token
	400 : 4	just expired access token
HandlerComment
	401 : 1	can't comment on a post that doesn't exist
	401 : 2	non-fatal: invalid device type
DeleteComment
	402 : 1	post/comment doesn't exist
HandlerFollow
	403 : 1	can't follow self
	403 : 2	can't follow an non-existing user
	403 : 3	already following user
	403 : 4	non-fatal: invalid device type
HandlerGetAccessToken
	404 : 1	auth code doesn't exist
	404 : 2	wrong auth code
HandlerGetFollowers
	406 : 1	can't get followers for non-existing user
HandlerGetFollowing
	407 : 1	can't get following for non-existing user
HandlerGetPhoto
	408 : 1	photo doesn't exist
HandlerGetPhotoThumbnail
	409 : 1	photo doesn't exist
HandlerGetPostComments
	410 : 1	post doesn't exist
HandlerGetPostInfo
	411 : 1	post doesn't exist
HandlerGetPostLikes
	412 : 1	post doesn't exist
HandlerGetPostPhotoId
	413 : 1	post doesn't exist
HandlerGetProfilePicture
	414 : 1	user doesn't exist
HandlerGetUserInfo
	418 : 1	user doesn't exist
HandlerGetUserPosts
	419 : 1	user doesn't exist
HandlerLikePost
	420 : 1	post doesn't exist
	420 : 2	already liked
	420 : 3	non-fatal: invalid device type
HandlerLoginEmail
	421 : 1	e-mail isn't signed up
	421 : 2	wrong password
	421 : 3	e-mail isn't signed up - triggers in case authentication is disabled
HandlerLoginFacebook
	422 : 1	couldn't get user's name from facebook graph api
HandlerLogout
	423 : 1	no refresh token for this user/device_token
	423 : 2	wrong refresh token
HandlerRefreshAccessToken
	425 : 1	no access token for user
	425 : 2	wrong access token
	425 : 3	no refresh token
	425 : 4	wrong refresh token
HandlerSignupEmail
	427 : 1	e-mail already signed up
HandlerUnfollow
	428 : 1	can't unfollow a user that doesn't exist
	428 : 2	a user can't unfollow another unless she/he is following that user already
HandlerUnlikePost
	429 : 1	can't unlike a post that doesn't exist
	429 : 2	user didn't like this post before to unlike it

instagram dash board

class	code : sub_code	explanation
HandlerGetDailyUsersErrorLogs
	404 : 1	e-mail isn't registered
HandlerGetDailyUsersLogs
	405 : 1	e-mail isn't registered

instagram test

class	code : sub_code	explanation
Constants
	1000 : 1	couldn't initialize kVosInstagramUrl
	1000 : 2	couldn't initialize kVosInstagramDashBoardUrl

instagram bots

class	code : sub_code	explanation
Constants
	1000 : 1	couldn't initialize kVosInstagramUrl

notes

• instagram, instagram dispense, instagram jobs, instagram worker, instagram dash board, instagram test and instagram bots services has no connection with Instagram Inc.; these services are just a vangav backend template to demonstrate using some features/utilities in the context of photo sharing apps
• the "instagram" name is used for users to easier understand the services' design, features and source code

share

free consulting