Tech spec: Pagination

[DevinCLane]

Summary:
This tech spec outlines the technical requirements for implementing pagination on Dev Directory

Scope:
Define the scope of the project, including what it will and will not include. List the features, functions, and requirements of the project.

• Potential Design
• How is Frontend is getting the information
• What options do we offer the user in regards to pagination
  • Where is that located?
  • What filter buttons are offered
• Layout of developer data within the page
  • Name, discord name, social links, interests
    • Tech stacks/specialties
  • What is available on the page and what should be only stored in the dev profile
• Scrolling UX

Data model:
Define the data model of the project, including the database schema and relationships between data entities. Describe how the data will be stored, accessed, and updated.

• Users
  • individual users
  • profile picture
  • bio
  • mentor or mentee status
  • coffee chat?
  • mock interviews
  • pair programming
  • tech stack
  • social links
  • Discord
• Current pages
• Total number of pages

No changes needed to the data model. This tech spec is only responsible for reading from the DB.

User Stories:
List the ways the user will interact with the project. e.g. "User will be able to edit their profile."

1. User lands on pagination page with a random assortment of developers within Dev directory, most likely 8.
2. As the user filters, profiles are removed/added to the page, results should be randomized to provide diversity.
3. Results should be randomized such that multiple requests for results with same filters produce different results. However, if there are more results than viewable slots, it should not be randomized on every addition of a filter. For example, if half of the currently rendered profiles fit with the next applied filter they should not be removed because of randomness.

User has the option to load more, and can either click "load more" and scroll, or infinite scroll is triggered by scrolling down.

Implementation:
Describe how the features are implememented

• Loading of user data into view

  • Load 8 random profiles into view from the DB

  -- If this doesn't work blame Caleb
  SELECT * FROM users ORDER BY random () LIMIT 8
  
  import { Client } from 'pg';
  
  const client = new Client({
    user: 'your_database_user',
    host: 'your_database_host',
    database: 'your_database_name',
    password: 'your_database_password',
    port: 5432,
  });
  
  async function getRandomUsers(): Promise<User[]> {
    await client.connect();
    const res = await client.query('SELECT * FROM users ORDER BY random() LIMIT 8');
    const randomUsers = res.rows;
    await client.end();
    return randomUsers;
  }
  
  interface User {
    id: number;
    name: string;
    email: string;
    // add other properties as needed
  }

• Infinite scroll

  • use IntersectionObserver to detect when the user has scrolled to the bottom of the page
    • Using intersection observer
  • Have a button that will initialize infinite scroll/Intersection observer
  • Re-render on filter decisions. For example:

  useEffect(() => {
    if (!cardRef?.current) return;
  
    const observer = new IntersectionObserver(([entry]) => {
      if (isLast && entry.isIntersecting) {
        newLimit();
        observer.unobserve(entry.target);
      }
    });
  
    observer.observe(cardRef.current);
  }, [isLast]);

• "Load more" button

  <button>Load more</button>

• More information on pagination approaches

• We will paginate based on Keyset API Pagination. For example:

Select *
FROM users
WHERE ID > <since_id_param>
ORDER BY ID ASC
LIMIT 100

APIs:
Define the APIs of the project, including the endpoints, methods, and request/response formats. Describe how the APIs will be documented, tested, and secured.

The directory API will be used to retrieve the dataset to be paginated and to load each page of items when a user navigates to a new page. The API will also be used to calculate the total number of pages based on the number of items per page and the total number of items in the dataset. (adapted from chatGPT)

• A new allUsers route will be implemented that will display the first 8 users.
  • GET /api/allUsers/:id - gets all user's profile details and socials
• A new searchUsers route will be implemented that will display users that match certain filtered criteria
  • GET /api/searchUser/:id - returns users that match the filtered criteria

Security:
Describe the security features and requirements of the project, including authentication, authorization, encryption, and data protection.

• Are we restricting the viewing of this page only to users who are authenticated?
• GET /api/users/:id should not return the discord_name if the caller is not authed

Testing:
Describe the testing approach and strategy for the project, including unit tests, integration tests, and end-to-end tests. Describe how the tests will be automated and integrated into the development process.

• Upon initial load, are the correct number of user profiles shown? We want 8 as a starting point.
• 'load more' button is tested to verify that it loads more users when clicked
• Are the number of users loaded correct?
• Different devices and screen sizes: does this behavior work across screen sizes and devices such as mobile, tablets, etc. Responsive design.
• Pagination feature is tested with different amounts of data to ensure it performs well with large and small data sets. Ensure the correct profiles are shown for the relevant results.

Open Questions:
Use this space to ask any questions of which you're not sure of the answer, whether it be a technical question or an implementation detail.

• Is the Search feature a part of this pagination feature? Search might need a separate tech spec.
• Is Pagination accessible to users who are not authenticated? Will profiles be displayed automatically? What restrictions are in place for users who are not auth'ed?

Deployment:
Describe any special considerations needed when deploying the project. For example, if there are breaking schema changes, how do we migrate existing data?

• Test features thoroughly before deployment

[MainlyColors]

For the random implementation (line below) what guarantees on page 2 that a user from page 1 isn't selected again when randomly selecting from the DB?

const res = await client.query('SELECT * FROM users ORDER BY random() LIMIT 8');

[Caleb-Cohen]

• GET /api/users/:id should not return the discord_name if the caller is not authed

Good question. This logic should only apply to initial page loads. I don't want the same 8 profiles being shown to everyone the first time the page is loaded.

There will need to be more complex logic for the infinity scroll such that profiles are not randomly selected again in the scrolling process.

[timmyichen]

@DevinCLane can you repost this to Discussions? Realizing we should be using that for discussions and once finalized, move to here for referencing.

https://github.com/timmyichen/dev-directory/discussions/new?category=technical-specification-review

[Caleb-Cohen]

@DevinCLane I want to walk back the randomness. I think this will add complexity and solve a problem we won't even have with MVP due to limited users.

I'd recommend lowering the complexity and setting pagination to just a sort of some default criteria.

[DevinCLane]

Okay moving this over to Discussions.

[timmyichen]

Closed via #17