The [new] Social Feed Manager user interface application.
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
docker Fix for Apache. Jul 24, 2018
docs Bumped version to 2.0.2. Aug 13, 2018
requirements Bumped version to 2.0.2. Aug 13, 2018
sfm Bumped version to 2.0.2. Aug 13, 2018
.bumpversion.cfg Bumped version to 2.0.2. Aug 13, 2018
.dockerignore
.gitignore
.travis.yml refs #920. Upgrade to python3, django2, and other libraries. Removes … Jul 24, 2018
Dockerfile Fix for Apache. Jul 24, 2018
Dockerfile-consumer refs #920. Upgrade to python3, django2, and other libraries. Removes … Jul 24, 2018
Dockerfile-runserver refs #920. Upgrade to python3, django2, and other libraries. Removes … Jul 24, 2018
LICENSE
README.md refs #918. Deprecates web harvesting. Jun 6, 2018

README.md

sfm-ui / Social Feed Manager

DOI Build Status

Social Feed Manager (SFM) harvests social media data from multiple platforms' public APIs to help archivists, librarians, and researchers to build social media collections. More information about the project itself.

Getting started

This is a re-architected version of an earlier Social Feed Manager which had been in use at GWU Libraries since 2012.

Overview

Social Feed Manager allows users to:

  • define collections comprising sets of targeted accounts, keywords, and other search strategies appropriate to different platforms.
  • authorize SFM to harvest data from platforms on the user's behalf.
  • view collection information and metadata about harvests.
  • extract, filter, and export the datasets to formats appropriate to the user's work.

Repositories

The full Social Feed Manager is made up of several component repositories:

sfm-ui is a Django app which:

  • Provides a user interface to set up Collection Sets, Collections, and Seeds
  • Provides Django admin views to administer Credentials, Groups, and other model entities.
  • Publishes harvest.start messages for flickr collections. The app schedules harvest.start messages for publication when the user updates an existing, active Collection.
  • Includes a scheduler which uses apscheduler to schedule publication of harvest.start messages.
  • Binds to harvest.status.*(.*) messages and creates a Harvest object (visible in the admin views) for each harvest status message received. The message consumer is started via the startconsumer management command.

Behind the scenes, SFM uses a set of carefully managed processes to harvest and and store this data, recording its actions in detail.

Tickets / pull requests / discussion are welcome.

Unit tests

cd sfm

./manage.py test --settings=sfm.settings.test_settings