2022-01-24 weekly planning

[kgodey]

About

At the start of each week, we start a discussion covering Mathesar's progress, team member check ins, and general updates. Everyone is welcome to add a new topic to discuss or to comment on existing topics.

Discussion topics

• Last Week in Mathesar
• Frontend priorities
• Backend priorities
• Design priorities
• Product priorities
• Comms Assignee schedule
• Brent's check in
• Pavish's check in
• Sean's check in
• Dominykas' check in
• Ghislaine's check in
• Kriti's check in
• Mukesh's check in

Weekly check-in

@centerofci/mathesar-core please post your weekly check-in as a top-level response and edit the discussion topics above to add your check in as a topic. Others are also welcome to respond.

Prompts:

• How did your most recent work go?
• What are you planning to work on soon?
• Do you anticipate any dependencies or blockers that someone else on the team can help you with?
• Any thoughts on the new Last Week in Mathesar section?
• If you attended the Code for Japan hackathon, how did it go? Any takeaways or action items?

If you'd like to add anything else to your check-in that's not related to the prompts, feel free to do so.

[kgodey]

Last Week in Mathesar

Feedback Requested

• Discussions in progress:
  • Code Styling guidelines for the backend #1007
  • Switch frontend to new filtering API #1001
• Specs in review:
  • Product spec for Views mathesar-wiki#22
• PRs needing review:
  • Please see frontend and backend priorities sections.

Complete

• Designs finalized:
  • Left Sidebar Improvements mathesar-wiki#19
  • Split Date/Time into Date, Time and DateTime types mathesar-wiki#21
• Major wiki updates:
  • Add more front end code standards mathesar-wiki#26
  • Update Review Guidelines mathesar-wiki#25
• PRs merged:
  • Boolean DB type casting extension #980
  • Move front end coding standards to wiki #977
  • First steps towards enabling TypeScript strict mode #826
  • Set up Prettier for automated front end code formatting #861
  • Tell Prettier to ignore storybook-static #998
  • Update CODEOWNERS to add templates to frontend list and shared code within mathesar directory to maintainers list #1000
  • Polymorphic serializer fix to change its class during runtime #990
  • Error structure core components #986
  • Bump ipython from 7.21.0 to 7.31.1 #1010
• We had a productive "State of Mathesar" meeting, notes are here.
• @Joi, @mathemancer, @pavish, and @silentninja attended our first event: Code for Japan's monthly Social Hack Day on Jan 22.

[kgodey]

Frontend priorities

1. Finalize the filtering API spec: Switch frontend to new filtering API #1001
2. Get Type configuration for Text, Number, Boolean, and all mathesar types with single associated db type #926 ready for review and merged.
3. Review and merge PRs ready for review.
   1. Display null while editing a null cell #961
   2. Keyboard Accessibility Pagination Component #963
   3. Implement renaming a column #976
   4. Improve line-height CSS #999
   5. UI for adding multi-column unique constraints #1005
   6. Use more strict export for proceed function within CancelOrProceedButtonPair component #1009
4. Continue to make progress on the Working with Tables milestone.
   1. Implement setting a default value for a column #500
   2. Improve UX when adding new records to a table with a NOT NULL column #775 should hopefully be unblocked by mid-next week
   3. Add UI to filter on a value being NULL #789 should be unblocked by Type configuration for Text, Number, Boolean, and all mathesar types with single associated db type #926
5. Work on some of the unassigned frontend issues in the 2022-01 improvements milestone while we're waiting for the type-related frontend PRs to be merged and more backend work to be done on data types. Feel free to also move some into later milestones if they're not critical.

@pavish @seancolsen Please comment below with thoughts/concerns.

[pavish]

Sounds good to me.

I'd moved Add UI to filter on a value being NULL #789 to the Data types milestone since this is now also blocked by Switch frontend to new filtering API #1001

Moved a bunch of unassigned good first issues in '2022-01 improvements milestone' issues to the 2022-03 improvements milestone.

[seancolsen]

Reactions

• Priorities look good, @kgodey.
• Those milestone adjustments look good, @pavish.

My plans

• For point 1., I still need to give feedback on ticket Switch frontend to new filtering API. I plan to work on that today.
• (I don't see anything I can do in 2., 3., and 4. right now.)
• Within point 5., I'm planning to work on:
  • Create components for building in-app documentation and help content #914 - I'm prioritizing this because I'm hoping it will be pretty quick and I think that having a modicum of in-app help content will help new users evaluate the alpha.
  • Add CI pipeline workflow to check for TypeScript errors #877 - I'm prioritizing this because I think it will be very quick.
  • Write integration tests for some important user scenarios #640 - I don't expect this to be quick, but I'd like to take a stab at working on it. If I hit some road blocks, I might switch course. But if I can make some progress I might end up spending most of the week on it. Pavish and I discussed this during our last call and decided it would be a good task for me.

[kgodey]

Backend priorities

1. API error handling issues: @silentninja, the plan for resolving this is currently unclear. There are a lot of issues and PRs in various states. Please prioritize organizing them and completing this work.
   1. Error 500 from records API when in violation of a not-null constraint #859 is the top priority since it's blocking the frontend.
2. Get Backend filtering refactor #921 merged.
3. Set up style guide and tooling for consistency in backend code #1008
4. Continue to make progress on other Initial Data Types issues. Some ideas, in no particular order:
   1. Data Types:
      1. Handle INTERVAL type in the backend #430
      2. Handle MONEY data type in the backend. #420
      3. Backend work for Mathesar money custom type  #417
   2. Date & Time parsing: Spec for human readable date and time parsing #845
   3. Grouping:
      1. Implement grouping for Text types. #408
      2. Implement grouping options for the email type #411
      3. Implement grouping options for the URI type #414
      4. Implement range based grouping for numeric types. #384
      5. Implement grouping options for date & time types #429
   4. Getting these draft filtering PRs ready to merge:
      1. Backend text filtering #879
      2. Backend email filtering #893

@dmos62 @mathemancer @silentninja Please comment below with thoughts/concerns.

[dmos62]

@kgodey Preparing a nuke-free merge would take some work, since it's pretty nukey at the moment. Is the goal to split into discrete PRs the introduction of new API and the replacement of previous API with the new API?

[kgodey]

@dmos62 Yes, exactly. That way it's easier to review and some of @mathemancer's grouping work might also be unblocked sooner.

[dmos62]

Sure. That would definitely simplify reviewing.

[silentninja]

@kgodey I am planning to work on the following issues

• Modify constraints API to use column id instead of column name #898
• Modify records API to use column id instead of column name #896
• Spec for human readable date and time parsing #845

Can you add these to the list

[kgodey]

@silentninja I moved #898 and #896 to Initial Data Types so they should be covered by list item 4.

[kgodey]

Design priorities

1. Design spec for Support for opening and interacting with a View #466 (please make sure the issue has been updated with latest requirements from the product spec by @kgodey before publishing the spec for review)
2. Design spec for Design for List data type #978
3. Continue exploratory work for Views in the design prototype.

@ghislaineguerin @ppii775 Please comment below with thoughts/concerns

[ghislaineguerin]

@kgodey this looks good.
@ppii775 as always, let me know if you have any questions.

[kgodey]

Product priorities

1. Merge Product spec for Views mathesar-wiki#22
2. Figure out remaining Views product requirements based on spec feedback and known holes (e.g. JOINs)
3. Finalize API support for storing "record preview" and "record selector" settings #944
4. Think through representing filtering based on CTEs to users (and when that would make sense in our roadmap), see:
   1. Duplicates filter should work with other filters #846
   2. More general mechanics for row grouping (and filtering)? #853
5. Create tickets to track implementation work: Listing Views Spec mathesar-wiki#16

[kgodey]

Comms Assignee schedule

Week	Person
2022-01-24 to 2022-01-28	Pavish
2022-01-31 to 2022-02-04	Mukesh
2022-02-07 to 2022-02-11	Sean
2022-02-14 to 2022-02-18	Dominykas
2022-02-21 to 2022-02-25	Ghislaine
2022-02-28 to 2022-03-04	Brent
2022-03-07 to 2022-03-11	Kriti

Instructions

You can also find the schedule and instructions for the Comms Assignee on the wiki page. Please feel free to update the wiki page if you think of anything useful during your shift.

[mathemancer]

Brent's check in

How did your most recent work go?

Last week went okay, but I spent more time on logistics, PR review, issues etc. than coding. I did manage to get some SQL-level grouping work done.

What are you planning to work on soon?

I'm going to get into the interval and money types first, and remain available to review any filtering PRs. Depending on how and when we decide to integrate grouping functions into the function framework associated with @dmos62 's filtering work, I'll either start on complete PRs for grouping or continue doing the SQL layer work for grouping. If the low-level stuff gets too far, I'll submit some PRs that will at least partially solve the issues so the code gets merged.

Do you anticipate any dependencies or blockers that someone else on the team can help you with?

There's still a dependency between the grouping and filtering work, since they use some of the same files, and the grouping should probably use the function framework set up by the filtering PRs.

Any thoughts on the new Last Week in Mathesar section?

I recognize the value of putting the review in one section, but it would be nice to still be able to see area-specific PRs (this was possible in the previous iteration). I guess it's a balance between clutter and specificity.

If you attended the Code for Japan hackathon, how did it go? Any takeaways or action items?

It went pretty well. @pavish noted the main action items in the Matrix channel. Some takeaways on my mind:

Mathesar has applicability in "sometimes available DBA" situations. The example use case was tracking disaster supplies. During non-disaster times, it's likely a DBA could manage the DB and set things up. However, in a disaster it might become necessary for a non-expert to both work with the data and potentially modify the data model. We've previously been mostly considering scenarios where the DB management would be through Mathesar except when some functionality wasn't available, using psql as a fallback. I thought the scenario of a "real" DBA managing the database how they want (e.g., through psql) under normal conditions, but the users being empowered to take control of the DB through Mathesar in an emergency quite compelling.

We came across a scenario where the ability to import a database scheme that enforces some data model would be quite useful. I.e., you would be able to distribute a data model for individual admins to import into their installation and use. It might be worth setting up a user flow for that.

One specific action item for @kgodey (if you want / think it makes sense) is to set up a bridge between the new #proj-mathesar channel in the Code for Japan Slack and our Matrix server.

[pavish]

Pavish's check in

How did your most recent work go?

It went alright. I wasn't focused much on writing code last week, most of my work went towards PR reviews, discussions, some meetings, the hackathon preparation etc.,

What are you planning to work on soon?

• Get the type config PR merged
• Finalize the filter API format
• Work on some issues in the 2022-01 improvements milestone
• Comms assignee work

Do you anticipate any dependencies or blockers that someone else on the team can help you with?

None

Any thoughts on the new Last Week in Mathesar section?

I think it's pretty valuable to have this section. I like that it gives a consise view of discussions requiring feedback along with being able to note down the non-development related work.

I assume anyone outside the team checking out our weekly updates would only be interested in this section.

If you attended the Code for Japan hackathon, how did it go? Any takeaways or action items?

It went well. Most of the event was discussions and a number of interesting points came up as noted down on the matrix channel and @mathemancer has given a gist of the usecases. A short summary:

Feature requests:

• Being able to export and import schemas
• Being able to create users that own a specific set of rows, essentially using Mathesar as a user login & data access portal for higher level applications

Real life usecases of participants which Mathesar can help solve:

• Mapping disater recovery stockpiles
• Conducting public surveys and sharing datasets

Some workflow related action items:

• New section in our wiki for notes on events
• A team email id for contacting us
• Bridge between CodeForJapan Mathesar slack channel: #proj-mathesar & matrix

Additional:

• People found Mathesar really hard to pronounce
• We had new contributors who helped & expressed interest in translating our wiki

[seancolsen]

People found Mathesar really hard to pronounce

@kgodey I want to gently highlight this. I also didn't know how to pronounce it initially. Anecdotally, most people's reaction when I tell them I'm working on a software project called Mathesar is to look at me quizzically and giggle to themselves a bit. Honestly I think "Mathesar" is somewhat of an awkward name. I've become hesitant to even share the name with friends, which is not a good sign. A good name is easy to pronounce and easy to spell. "Mathesar" falls short here. Is is, however, a very unique name which is also important! But I think it might be worth taking a brief moment to consider whether it's best to stick with "Mathesar" or to pick a new name more intentionally. Re-branding now won't be easy. But it will be near-impossible after the alpha. And a good name is crucially important in my opinion.

I spent years working with "CiviCRM", which I think is a terrible name. The pronunciation and spelling are difficult to derive from one another, much like "Mathesar". I'd say the general consensus among CiviCRM devs is that the name is hinders adoption, but that it's too hard to change. I observed perineal debates on re-branding which never progressed beyond a phase of disagreement over a suitable replacement. These debates demonstrated a specific challenge in re-branding which becomes amplified in open source and community-driven projects: the more stakeholders become involved, the more inertia the project acquires and the harder it is to break the escape velocity of the status quo.

I know "Mathesar" was only meant to be a provisional name, but provisional names always "stick" in my experience. There's still time to change it, but the window is closing fast. If you've already given thought to re-branding and you have intentionally chosen Mathesar at this point, then great. I'm on board. And I promise not to harp on this topic. But if you've been considering picking a new name at some point, then I'd suggest we prioritize this task and spend some time soliciting ideas from the team before the alpha release.

[kgodey]

@seancolsen I updated #286 to include figuring out if we want to rename Mathesar.

[seancolsen]

@kgodey Sounds good 👍

[kgodey]

@pavish I've added those action items to my to do list, thanks for the clear summary.

[seancolsen]

Sean's check in

How did your most recent work go?

• Pretty well. I feel like I'm picking up the pace again a bit, mostly because my wife and I are settling into a more functional routine with the kids still home from school because of the covid surge.
• I'm really happy to finally have Set up Prettier for automated front end code formatting #861 merged because I love coding with my editor set to auto-format my code. This is a breath of fresh air and a small boon to productivity.

What are you planning to work on soon?

• See my comment in the "Front end priorities" section.

Do you anticipate any dependencies or blockers that someone else on the team can help you with?

• I might need help from some back-end engineers while implementing integration tests. I'll reach out to people as-needed.

Any thoughts on the new Last Week in Mathesar section?

• I find the "Feedback Requested" section useful. The "Complete" section is not useful to me personally.

If you attended the Code for Japan hackathon, how did it go? Any takeaways or action items?

• N/A

[dmos62]

How did your most recent work go?

As I mentioned earlier, discussions with frontend team concerning new function API (#1001) are advancing steadily, but slowly.

What are you planning to work on soon?

Continue function API discussions; implement named parameters (function API); possibly implement generics; implement whatever else we decide on in the discussion (#1001); write the tests for filtering (and possibly dedicated tests for the function API).

Do you anticipate any dependencies or blockers that someone else on the team can help you with?

I could use more input on the latest function API discussion. @mathemancer's input would be valuable, I think.

Any thoughts on the new Last Week in Mathesar section?

I like it. The "Feedback requested" section is useful. Overall, the weekly discussion format seems to be constantly improving.

[ghislaineguerin]

Ghislaine's Check In

How did your most recent work go?

Prototype exploration went well. I had time to think through different scenarios, experiment and I feel I'll move much faster with the rest of the design of the views functionality because of that.

What are you planning to work on soon?

I'm planning on working on issue #466. My goal is to start a review this week

Do you anticipate any dependencies or blockers that someone else on the team can help you with?

None for now.

Any thoughts on the new Last Week in Mathesar section?

It's helpful as a summary. No specific thoughts about it.

[kgodey]

How did your most recent work go?

Good, making progress on the Views spec and should get it merged this week.

What are you planning to work on soon?

• Updates to the Views specs based on stuff pointed out by @mathemancer and @silentninja
• Update remaining Views design tickets to match the new spec
• Set up style guide and tooling for consistency in backend code #1008
• Think through representing filtering based on CTEs to users (and when that would make sense in our roadmap), see:
  • Duplicates filter should work with other filters Duplicates filter should work with other filters #846
  • More general mechanics for row grouping (and filtering)? More general mechanics for row grouping (and filtering)? #853

Do you anticipate any dependencies or blockers that someone else on the team can help you with?

Nope

Any thoughts on the new Last Week in Mathesar section?

I think it's nicer to have it all in one place and see how much progress we're making. I'll split up "PRs merged" into backend/frontend PRs from next week.

[silentninja]

Mukesh's check in

How did your most recent work go?

It went well, was able to finally get the error handling PR's closer to completion.

What are you planning to work on soon?

• Spec for human readable date and time parsing #845
• Modify constraints API to use column id instead of column name #898
• Modify records API to use column id instead of column name #896
• Look into grouping and filtering issues and start working on it

Do you anticipate any dependencies or blockers that someone else on the team can help you with?

None

Any thoughts on the new Last Week in Mathesar section?

I like it, makes it easier to understand the progress, I also like the fact that it is not split into backend, frontend as this gives a perspective on the whole.

If you attended the Code for Japan hackathon, how did it go? Any takeaways or action items?

It went well. Most of the event was discussions and a number of interesting issues and use cases came up

Feature requests:
@pavish and @mathemancer have summed up really well. There were a few use cases that came up in addition to what @pavish had listed.

Feature request:

• Being able to detect and maintain consistent Japanese language encoding
• Maintain data format
• Granular permission

Real life usecases of participants which Mathesar can help solve:

• Listing public restrooms
• Listing Sake related places, they have their own storage based out of MongoDB

[kgodey]

@silentninja what does "Maintain data format" mean?

[silentninja]

@kgodey We didn't go into the details, but they were aiming to maintain consistency of the data such as encoding, date format, missing values.

[kgodey]

@silentninja Thanks! Can you edit the place where I put these notes in the wiki page to clarify?