Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Planning Issue: Expand, Coordinate, and Test API #1449

Closed
david-days opened this issue Jun 3, 2017 · 13 comments

Comments

@david-days
Copy link
Collaborator

commented Jun 3, 2017

Goals

GSoC 2016 introduced RESTful services to the plots2 system. Since then, there have been several fundamental changes to the site, as well as new projects and goals for GSoC 2017. This issue is intended to cover cleanup of work from 2016, as well as add new capabilities for further development.

Additionally, the existence and usage of the RESTful services (current and future) is rather opaque, at the moment. Developers do not have a centralized point of reference in which to become familiar with existing code. Because the RESTful subsystem is based on a particular technology, consumers and developers of the services also have to surmount a rather steep learning curve before any practical usage.


Phase 1: Documentation and Testing UI

Addition of a test and documentation interface for developers and consumers of existing RESTful services.

  • Documentation of the web services
  • Testing interface for potential consumers of RESTful services, as well as developer testing before deployment/upgrade

Phase 2: Full Coverage of Current Services

Full coverage of the Jasmine Tests for existing services and JS scripts, such as Issue #812.

  • Full test coverage of existing services
  • Supporting documentation, design patterns, and helper classes for future tests

Phase 3: User Authentication

This phase covers integration of RESTful services with plots2 user authentication.

The current implementations are read-only services that are deemed "safe", in that they do not require various levels of user authentication to separate anonymous calls from standard users from administrators. Before the development of more powerful RESTful services can occur, an authentication system must be designed and implemented. While there are standard patterns available, actual implementation of any service requires tight integration and testing with the specific authentication/authorization implementation.

Once this phase is complete, future developers should have a standardized approach (via helper classes, system configurations, or design patterns) to apply restrictions to unsafe web service calls that will developed in the future.

  • Standardized login and user role mechanism to be applied to protected services
  • Role-level access controls for RESTful service calls
  • Testing for this system
  • Standardized test patterns and test classes for additional services.

Phase 4: Additional RESTful Services

This phase is more generalized. As various projects move forward, the opportunity to add new services will occur, and decisions need to be made for both the location and capabilities of those services. As each of the various projects are evaluated, the following checklist of items should be considered:

  • List potentially useful RESTful services that can be created, with a focus on aiding current and future usage of the project's data and capabilities
    • What data should be made available?
    • What project-specific computations and logic should be made available?
    • With respect to the previous items, what supporting data structures should be created or modified?
  • Evaluate locations and access controls for the services
    • Which services are "safe" (read-only and publicly available, not requiring access controls for a logged in user?)
    • Which services require access controls?
    • Should the services be coded within the sub-project, or within the plots2 umbrella project
    • Required testing and evaluation for the same
  • Documentation and Testing interface for non-RESTful consumers. Options are below:
    • Use the primary plots2 system from Phase 1
    • Use a similiar system via clone or helper classes
    • Use a unique UI designed around the specific project's logic and goals.

As I said, this is a generalized issue to try to bolster the current RESTful API system, as well as roll out new capabilities in a standardized manner. Phase 4 (Additional RESTful Services) may occur with no new services from GSoC 2017. However, the patterns and work from the previous phases should establish a baseline for any future work.

@david-days

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 3, 2017

@jywarren, I'm already working on the Phase 1 part, and should be submitting that soon. Fortunately, the work that @ujithaperera and I did last year already laid the foundation. Once that part is finished, I'll be able to move fairly quickly through the next 2 phases. My overall goal is to work with @StlMaris123 in particular on the email subsystem, while keeping an eye on the rest of the work from this year and last year to apply for Phase 4.

Comments and criticism are always welcome.

@ujithaperera

This comment has been minimized.

Copy link
Contributor

commented Jun 5, 2017

hi @david-days , thank you very much for re organizing to make this RESTful services to bring valuable end. I really appreciate it. give me some time to catch up the things you are planing ahead. I am still finding a place to begin my contribution. Thank you.

@ananyo2012

This comment has been minimized.

Copy link
Member

commented Jun 6, 2017

Hey guys I am in for help too! Since the Bot Project would need the API I would really want to be a part of this. I will look in how the API is set up and Swagger documentation.

@jywarren

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2017

@david-days

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 6, 2017

I think it's feasible and desirable. I've got a plan (subject to better ideas, changes, or just "huh...never thought of that") that I can talk through.

I already have a PR (Pull Request #1450 ) for adding the Swagger front end. It already shows the documentation from the work that Ujitha and I did last year. This would go a long way towards getting everyone on the same page.

For some reason, it's failing Travis CI on the DB setup--i didn't change anything there, but I did a rebase/upstream-merge and might have screwed that up somehow.

@jywarren

This comment has been minimized.

Copy link
Contributor

commented Jun 7, 2017

Great, i left some comments over there and look forward to this!

@jywarren

This comment has been minimized.

Copy link
Contributor

commented Jun 7, 2017

Also be aware that Jasmine cannot test the Rails responses, the way it's set up (and the convention) is that we decide what the API boundary between the front and backends are, and we use static or mocked responses to test each of the two, rather than testing both in an integrated way. This makes sense because it holds both front and backend to the strict promised API by testing each against a static version. It also ensures 2 things aren't changing at the same time and we're missing something between.

@jywarren

This comment has been minimized.

Copy link
Contributor

commented Jun 15, 2017

Hi, David! After #1450, I'd like to propose starting with unit tests for SearchService and TypeaheadService:

Not every sub-method, but the public ones, like:

  • textSearch_all
  • textSearch_tags
  • textSearch_notes
  • etc

Since these make use of the other methods (like find_notes and notes), that should give us full coverage. Actually the remainder could become private and not need to be tested.

Later, we can set up functional tests for:

How does that sound? Once unit tests are in place for SearchService and TypeaheadService, we can work on refining them as described in the TODOs in the comments on each one, without worrying about messing up their functionality (the tests will ensure this for us).

@david-days

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 15, 2017

@jywarren ,

Sounds like exactly what we need. I'm going to try to make the changes to #1450 tonight, and then work on the test coverage (also updating comments that will appear in the Swagger UI, add references to the implemented JS clients, etc).

Sounds like we have a plan moving forward.

@jywarren

This comment has been minimized.

Copy link
Contributor

commented Jun 15, 2017

@jywarren

This comment has been minimized.

Copy link
Contributor

commented Jan 16, 2018

OK, - this is still a great plan for the API work, though we'll need to update and adapt some sections due to recent work. I also put more links to existing API methods here: https://github.com/publiclab/plots2/pull/2017/files

@jywarren jywarren changed the title Planning Issue: Expand and Coordinate RESTful API services (GSoC 2017) Planning Issue: Expand, Coordinate, and Test API Jan 16, 2018
@jywarren

This comment has been minimized.

Copy link
Contributor

commented Jan 16, 2018

Also need updates regarding the token system #1513 for the write-API

@stefannibrasil

This comment has been minimized.

Copy link
Collaborator

commented Sep 27, 2018

the API has changed a lot, more updated details at #3520

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.