How can we make Shopify Ruby ecosystem better? #1181
Comments
|
Amen. Thank you for taking the time to write this up. I've basically stopped taking on new Shopify projects because things are such a mess. I would love to return to the ecosystem though, so I'm in full support of these requests. We won't let stale bot get its hands on this one. |
|
All good suggestions! Also, I wonder if Shopify uses the same gems for their own apps and internal projects. Maybe there's a slight disconnect here. I once reported what seemed like a major problem with the old GraphQL, and someone from Shopify wrote they used a different client internally, or at least had a different way to set it up, which meant they never caught the issue. One of the best things about the Rails ecosystem is knowing you're using the very same tool that some pretty large companies rely on, so breaking changes are not taken lightly and important bugs are addressed quickly. I'm not sure if Shopify is using this gem in particular - correct me if I'm wrong. I personally much prefer the new API library and the direction they chose, but leaving those issues without response impacts the developer experience for sure. Would be great to see more resources dedicated to these gems, and make the development of the dev tooling a bit more even. New features get launched all the time, but some parts are left behind. I'm sure the developers involved in the tooling are doing their best, but the leadership is perhaps not focused on these issues enough and not providing the time or financial resources. |
That's great to hear! What's missing is UPGRADING.md / MIGRATION.md guide.
I feel like this has been an issue at Shopify since forever. |
|
I totally agree and am quite frustrated. We have a long standing Shopify shop with a custom external Admin app using the API. I have spent hours now trying to figure out how do I authenticate it using access token and using the gem. As I see it, I have no choice but to rewrite my app using standard API calls instead of relying on this gem with the documentation hell. |
|
This needed to be said. We've been tripped up by this gem many times throughout the years. Building a Shopify app in some respect has gotten more difficult, especially when an app is embedded. It would be great to see more dedication toward keeping it well maintained. |
|
Thank you for taking the time to write this. I 100% agree and I'm also frustrated. I have one Shopify client and even that's pushing my limits. I'm definitely not keen on taking on any other ones. Building apps here is a giant PITA. As I just mentioned in another issue, I'm still stuck at <V9 because the move from 9-10 literally broke everything in my app and it would need an entire rewrite to upgrade. Are there any forks of a pre-v10 gem with improvements? Or has anyone built their own Shopify REST API wrapper that they are willing to share? Getting feedback from Shopify here is pretty much nil it seems, I'll be curious to see if they respond to this - highly doubtful. Everything you stated though is true - docs suck, upgrading sucks, changelogs suck. At times I think it's just a few greaseballs in a garage somewhere sadistically making breaking changes every month. |
|
We really appreciate the thoughtful comprehensive feedback on what the ruby community expects. Feedback is a gift and we treat this as such. I've made some progress and will continue to iterate on the low hanging documentation improvements that you've pointed out. Stay tuned for more updates on how we can improve here; we are working on it! |
|
Hey @nelsonwittwer - thank you so much for responding to us. In case it makes for an easier sell, here's the Rails commit that removed their auto-close bot. The commit message itself, penned by the venerable rafaelfranca:
And a huge thank you to @vfonic for this masterpiece. Any chance you'd be willing to finish the Song of Ice and Fire novels next, since GRRM cant? |
|
Oh geez @panckreous, I almost believed you when you said "masterpiece", until you asked me to finish George Martin's novels. 😅 @nelsonwittwer I cannot hide the disappointment with the response from Shopify/you. After 3 weeks of silence, the reply you sent (as a Shopify representative) is: "thank you for your feedback, we're working on it". You could have a feedback answering machine compose that reply. I'm sorry for the sarcasm and if that sounds too harsh, but it's a reality. Zero information except "we merged some minor documentation PRs that have been opened for months". So, it's been 6 weeks since I initially opened this issue and 3 weeks since you replied. When can we get some concrete information? What's Shopify's plan with Ruby? What's Shopify's plan with ShopifyApp and ShopifyAPI gems? Was there nothing of value in my initial issue message? I've proposed several suggestions and pointed out several low-hanging fruit and I see that two documentation PRs were merged. So, ok, we've made some progress. Thank you! How do we ensure that these issues don't happen again?
session = ShopifyAPI::Auth::Session.new(shop: shop.shopify_domain, access_token: shop.shopify_token)
ShopifyAPI::Context.activate_session(session)
I know I've been rather negative in my comments, and I'm sorry. I really want to make something positive come out of this. I have huge respect for Shopify engineers and I know this is a lot of work. ❤️ I'm not saying anyone is slacking, I'm saying we need more resources. |
vfonic
changed the title
oh no @vfonic that was an honest compliment. which, as often in real life too, I seemingly ruined via my terrible sense of humor. youve seen how many of us have tried over and over to no avail. you put the work and time in that finally got at least a response from one of the devs. I didnt think it was possible. I figured if you could accomplish that where no one else could... I feel so bad now! |
|
You are right @vfonic , we owed you a better update(s) than what we have provided. Your tone and feedback is completely justified. Thank you again for you candor and your patience. This discussion has spurred many discussions on priorities internally and we couldn't have done so without your effort and feedback. The voice of the community has given us so much help in arguing our case for our root problem as you pointed out -- we need more resources. In all sincerity, I thank you. The short update is this: I hope to share a vetted roadmap and change of philosophy/systems by the end of this week after we get final approvals on our internal changes we need to make to better serve the community. I really wish I could have closed this loop sooner before you had respond; sadly getting approvals for more resources, etc take time. |
|
Based on feedback from the Ruby community and within Shopify, we will be making some changes to better encourage participation from external developers in how we move forward with our Ruby open sources packages. We have revisited our core open source principles and have a roadmap that we are excited to share and work on. Build in publicShopify will be adopting the mantra of building in public with all of you. This will include:
RoadmapMoving forward our roadmap will be defined in the
We’d like to extend our thanks to the community for your patience and engagement. We recognize that we can’t do this alone and deeply value your feedback to build the best possible experience that meets the expectations of our developer community. We look forward to your contributions and suggestions! |
|
@nelsonwittwer can you please disable the stale-bot? #1103 has not been addressed and just got hit |
|
We have been brainstorming the stale bot @panckreous, but we can't commit to anything yet. The root problem of issue management is something we definitely still need to address. We plan on exploring that as we update Changes to issue management will be proposed in a PR. We look forward to discussing alternatives then :) |
Isn't prioritizing and solving or dismissing issues a good alternative? 😜 |
Which is why it is our top priority of our roadmap. |
|
@nelsonwittwer I also want to voice support for disabling stale bot as soon as possible. It seems that things mostly get marked as stale due to inaction from the Shopify team, and it's super frustrating to have to constantly comment "not stale" on issues that have been sitting around for months. That bot is one of the biggest contributors to the feeling that not only does Shopify not seem to care about these repos, but makes it hard not to wonder "are they actively trying to drive us away?" It feels like the equivalent of saying "thank you for your feedback, we haven't bothered to look at it, and will now throw it in the trash." |
|
@nelsonwittwer thank you so much! Responding timely to issues/PRs makes stalebot unnecessary and keeps you accountable, which can help you make sure to keep things in order. :) Looking forward to seeing more activity in Shopify Ruby ecosystem and seeing how we can help and this community coming alive! 🚀 PS here's a preview link of the ROADMAP.md (easier to read). |
|
Definitely +1 for removing Stalebot, It feels like a massive slap in the face for people who have taken the time to report an issue. |
|
I'm watching this thread and the repo closely to finally decide whether to abandon the gem in favor of a home-rolled API wrapper. I'm right on the edge of using enough features of the gem where this will be worth the effort. Its a lot of work, but feels like it might be worth it long term. From my perspective, it looks like the API is the product, the gem is treated as a resource sink. I can reach someone in support that will verify things in the API, but nothing here with the gem. Does anyone at shopify use the gem? I have a hard time believing these kinds of bugs (#1133) would have gone unnoticed if so. My personal wants are:
We're 3 weeks into the 6 week time period and I don't see anything on that roadmap publicly available. Not sure if I'm missing something. |
|
hey @matthewlein - while every case is different, switching over to dealing with the api directly was the best decision we could have made. I watch this space for the same hopeful reasons as you, but moreso because it was this same recommendation [in one of the many unanswered and auto-closed threads about this very same topic] that made my life that much better and I hope to pay that kindness forward. |
|
There are some definite benefits to rolling your own wrapper, for sure. It's my hope that our roadmap will enable more flexibility with this type of thing. I'd hate for people who want to go out on their own to have to implement our flavor of OAuth; it's our hope to offer a stable foundational gem that provides auth and base clients for REST / GQL APIs. From there if you wanted to use our wrappers to make requests, or some other wrapper that takes an auth token and talks to the APIs you'd be free to do so. Your frustrations are valid and we are working to fix them. If you care not to wait for that, rolling your own wrapper would is a totally viable option. |
|
Thanks, @vfonic for summarizing all the frustrations regarding this project. My owns are summarized here: #969 (comment) and I've moved on since then #969 (comment) I am still going to keep an eye on the Shopify team's goodwill about investing in that ruby gem. It was very disappointed with their lack of involvement in the last years, especially for a pre-v1 Ruby on Rails Canadian dev who has always been rooting for Shopify as a great representative of the community. |
|
can anyone confirm if this is a safe way to access the session? I have a table for sessions / shop data and currently I do it like so: Existing way: session = ShopifyAPI::Utils::SessionUtils.load_offline_session(shop:)
client = ShopifyAPI::Clients::Graphql::Admin.new(session:)Proposed new way (after the great "eject"): session = ShopifyAPI::Auth::Session.new(shop:, access_token:)
ShopifyAPI::Context.activate_session(session)
client = ShopifyAPI::Clients::Graphql::Admin.new(session:) |
Worked like a champion! |
Hey @mattsrobot, Yup that's a valid way to use it! However, you don't need to set so both of the following is valid! session = ShopifyAPI::Auth::Session.new(shop:, access_token:)
ShopifyAPI::Context.activate_session(session)
client = ShopifyAPI::Clients::Graphql::Admin.newor session = ShopifyAPI::Auth::Session.new(shop:, access_token:)
client = ShopifyAPI::Clients::Graphql::Admin.new(session:)We recently updated our docs so that these are better explained! |
|
Hey all! ShopifyAPI
ShopifyAppIf you're using our
We're doing our best to improve the development experience and really appreciate y'all's patience with our response. Please let us know if there are specific topics you'd like to see more documentation on. |
|
@zzooeeyy re: your most recent update Thanks for the effort, but while going through the documentation to start upgrading from v9 to v13 I tried one of the examples in the document you mentioned: Making REST API calls This has been reported: #1233 How is something basic like this not working?
If this were an open source project by volunteers I wouldn't be complaining, but my company is forking over a tremendous amount of money to Shopify every year. Encountering this issue is the final straw for us: Goodbye & good luck to the rest of you |
|
Hey @tdons, we're really sorry that this is causing you frustrations. I added the incorrect documentation mistakenly after seeing it in the base class without checking if it worked in the children classes. We do have a PR open to address this issue so that it will work as documented: Unfortunately we are a very small team at Shopify maintaining this, I also wish we had unlimited resource to make this the best library ever, but that's not the reality. :( Since this repo's code is public, we would really appreciate the community in helping us bug fix. See Contributing |
Shopify is a $100B company, not a scrappy open source project. If you want partners to continue working within your ecosystem, your bosses need to decide to invest in these tools. It's not up to us to do it for free. |
Yeah what in the world, is she serious? My client is paying thousands of dollars per month to Shopify and they are spouting this bs? |
|
To clarify out of the gate, Shopify is very invested in maintaining open source tooling like this repo. In fact, we are happy to introduce three new team members to our team to help to that end. We have enjoyed the growth we have made since this issue was initially opened up and we look forward to our continued partnership with the community. I get the frustration of development, especially when documentation / functionality gaps occur like this one. As Zoey pointed out, we had the PR ready to merge before the holidays but with everything in flight we didn’t get it merged before this discussion started. We have since merged in the fix upstream and are in the process of generating the REST resources for the new |
|
Ludicrous that some of these issues were not handled in any sort of meaningful way. Countless breaking changes have been introduced. I used to use Shopify to manage some local businesses that were looking to dip their toes into E-Commerce. These aren't companies that are large enough to receive direct support from the Spotify team or hire a full-time developer to just keep their stores up and running. The incurred costs and downtime were ridiculous and I've since decided to pull many clients out of the Shopify ecosystem with no cost to them. Maybe if I had some valid reason to build with GraphQL, maybe my experience would've been wildly different, but these were services running with low throughput (~$20/mo of store management services running on Lambda). Unfortunately Shopify Plus is the priority, it's just not sensible for these mom & pops who are just looking to break into the market. |
|
@jacksonSingleton trust me, having Shopify Plus doesn't make a lick of difference when it comes to support, sadly. I still get responses like I did in #1270. |
Thanks for the heads up. I'm sure Shopify is a better product for folks not hanging around in GH issues, but from a developer's perspective it was just ridiculous the hoops that had to be jumped through. For such a popular platform, the amount of roadblocks every step of the way was maddening. Wishing all current and future shopify devs the best lol. |
|
@skillmatic-co most likely your issue is related to #1203 (Aug 23, 2023), which came from fixing #1037 Oct 12, 2022. These 2 major bugs have been affecting the gem for at least 1.5 years. No response on asking about 1203 progress almost 2 months ago. @nelsonwittwer I'm happy that there are new resources for the gem, but if we can't count on major bugs being fixed within months or years, or even to see any progress on them in months, I can't have any faith in the future of the it. I have one idea that could really help: Could Shopify use this gem in at least 1 critical customer-facing feature? If so, I'd imagine that you'd get bugs discovered and fixed at a much faster pace. With the forced api version migrations that come every 3 months, and the api versions available being tied to specific gem versions, it makes for a really difficult situation. For me, v12's bugs are better than v13's, so I'm still using it, but the api version cutoff is only a few months away. We have to migrate off the gem now. |
|
Hi everyone, thanks for all the feedback here. I'm gonna close this issue because we have made a bunch of fixes on things that are under our direct control based on what you all have written here, such as:
I know that there are still things that you would like us to do, and to help us with that I would ask you to please open separate issues for things that are still outstanding, so that we can triage, prioritize and assign resources more effectively. |
Issue summary
This project has seen better days. So did ShopifyApp gem.
1. Docs site markdown is broken
https://shopify.github.io/shopify-api-ruby/usage/oauth.html
2. Docs are outdated
There's one week old PR about updating docs that still hasn't been merged:
#1172
Does it really take a week to get this merged?
BTW this has been reported more than a month ago:
#1163
Does it take 40 days (and counting) for a documentation fix?
2 commits, 4 documentation files changed, 19 lines of docs added and 81 lines of docs removed.
This takes 40+ days.
Shopify has truly abandoned their Ruby developers community.
In v13 of this gem there's no
ShopifyAPI::Auth::SessionStorage.This is being referenced in several places in documentation, even in README:
There are also several issues about this opened.
3. Authentication documentation
I'm trying to figure out a simple thing of authenticating/starting an offline session in my Rails app.
It was done like this before:
This no longer works.
I went to the documentation website:
https://shopify.github.io/shopify-api-ruby/getting_started.html
...and found nothing.
I found some info here on how to do online session:
https://github.com/Shopify/shopify-api-ruby/blob/main/docs/usage/rest.md
I found this piece of code here:
https://shopify.dev/docs/api/admin-rest
I don't know how I can convert this for offline use?
I finally figured out how to achieve this by reading ShopifyApp and Shopify API gems source code:
Seems like with all the unnecessary breaking changes, you also managed to rename
domain:keyword argument toshop:, which is first unclear, but also wrong, because in many Rails appsshoprepresents an instance of aShopActiveRecordmodel.4. Docs don't mention ShopifyApp gem
It's sad to see that on this page there's no reference to ShopifyApp gem, only to Shopify API gem:
https://shopify.dev/docs/api/admin-rest
I understand I'm on a part of documentation about REST API, but I could barely find any reference to ShopifyApp gem anywhere on the shopify.dev:
https://shopify.dev/docs/apps/auth/oauth/update/ruby#step-1-update-the-shopify_app-gem
Is this the best way for a new developer to find out that there's a ShopifyApp gem?
Here's another good candidate where you could mention ShopifyApp:
https://shopify.dev/docs/apps/tools/api-libraries
5. There are hundreds of issues never answered and automatically closed by a bot
This is inhuman and a slap in the face to anyone who took their time to report an issue and/or create a PR. If you cannot keep track of open issues, at least disable the stale bot.
https://github.com/Shopify/shopify-api-ruby/issues?q=is%3Aissue+is%3Aclosed
This tragedy has turned into a comedy:
Why Shopify is not actively taking more care of these issues? #986
People are fighting with stale bot. It is sad and hilarious at the same time.
#969 (comment)
😂
6. There are bugs being reported that seem quite critical to Ruby/Rails fundamental functionality
This seems rather critical:
ShopifyAPI::Rest::Base responds_to everything #1171
ShopifyAPI::Order.new.present?raises an error.Or how about this one:
Updating assets is broken in v13.0.0 #1164
7. Newer versions are more concerned about rewrites than keeping the existing functionality, newer versions have less functionality than older versions
GraphQL Support? #1168
Updating assets is broken in v13.0.0 #1164
@nelsonwittwer Can you elaborate a bit more on how session storage is used for a pure api client? If session storage is removed from Context.setup, how would it get defined without shopify_app? What role does it play in this scenario?
Breaking change notice for version 10.0.0 GREATLY underrepresented #1103
8. New Sorbet type system makes error messages useless
ShopifyAPI::Webhooks::Registry.add_registration occurred TypeError in fields parameter #1147
REST API Shop resource definitions TypeError #1145
Upgrading from 9.5.1 to 10+ adds a sorbet dependency that affects unit tests unrelated to Shopify #1109 - this issue talks about how upgrading Shopify API gem breaks unrelated tests. This is the worst kind of behavior that can happen for a new version of gem that you're being forced to upgrade to. And the suggested fix is to install more tooling. Which is not a fix.
Accessing the taxes_included property on a ShopifyAPI::Shop object returns a Sorbet TypeError #1170
Accessing the arguments property on a ShopifyAPI::Event object returns a Sorbet TypeError #1152
There are two pages of issues if you search for
typeerror:https://github.com/Shopify/shopify-api-ruby/issues?q=typeerror
9. Shopify forces developers to constantly upgrade to their latest buggy, undocumented, and with-many-breaking-changes gem/libraries versions
Remember the App Bridge upgrading fiasco? Oh, I 'member: [Docs] Documentation still refers to app bridge v1.x #71
We were asked to upgrade to App Bridge 2.x because our apps will be delisted, but the documentation on how to upgrade/switch a Rails app to App Bridge was non-existent.
Shopify API gem v10 introduced many breaking changes and we're constantly forced to keep upgrading to the latest gem versions or our apps will be delisted from the App Store because we're missing the functionality X.
It would be great to release breaking changes as a major version bump, instead of minor Shopify/shopify_app#875
#1103 (comment)
What can be done?
Please help me on this one, I'd love to hear more opinions/suggestions how we can make this better.
Here are some of my suggestions:
Dedicate more resources internally to Ruby gems
Don't make so many breaking changes in: a) Shopify REST/GraphQL APIs, authentication/session management, etc. and b) in ShopifyApp and Shopify API gems. Don't do things like removing session storage from this gem. Just leave it. There are tons of other functionalities you could add instead.
Announce a cash bounty and pay developers willing to work on Shopify's open source projects
This goes well with the previous point: make a frequent contributor a maintainer - a lot of open-source code and reliance on such code is based on trust. You could show some trust to active developers by making them maintainers. If it makes you feel better, you could ask them to sign some document so you could have some more confidence that they won't do anything malicious.
Support your community - promote unofficial libraries on your site. Here's how Airtable does it: https://airtable.com/developers/web/api/introduction
Here are some Ruby projects that could be good candidates for adding to your website:
- https://github.com/baoagency/polaris_view_components
- https://github.com/travishaynes/shopify-mock/tree/httpmock
- https://github.com/kirillplatonov/shopify-hotwire-sample
- https://github.com/forsbergplustwo/partner-metrics-for-shopify
- https://github.com/davefreiman/shopify-sample-data
- https://github.com/kevinhughes27/shopify-tax-receipts
- https://github.com/kirillplatonov/shopify_graphql
- https://github.com/igrigorik/shopify-core-web-vitals
- https://github.com/ScreenStaring/recharge-api
- https://github.com/ScreenStaring/shopify_api_retry
- https://github.com/lucidnz/lib-ruby-lucid-shopify
- https://github.com/colinsoleim/shopify-metafieldsync
- https://github.com/bravetheskies/polaris-html
- https://github.com/ripenecommerce/shopify-ruby-sha256
- https://github.com/remy727/rails-shopify-app-starter-kit
- https://github.com/pcantarelli/app-ruby
- https://github.com/jhnwr/shopify-scripts
Fix your release/changes workflow - a lot has been done in this area, but there's a lot to be desired. Where's the UPGRADING guide? Can you please add: "Do not release any breaking changes without upgrade steps" to your release workflow? Related issue from 4 years ago: Would be good to have MIGRATION_GUIDE.md or similar Would be good to have
MIGRATION_GUIDE.mdor similar shopify_app#774Ask for requests for features - instead of guessing what your users (us, developers) need/want to see as new functionality, you could...just ask.
EDIT: 8. Remove stale bot.
Apologies if it sounds like I'm a bit frustrated, that's because I am. I've got several Shopify projects that I handle by myself and it's a pain to keep all of the projects up to date. Not to mention that every time I start a new Shopify-related project, the workflow and generated project file structure is completely different.
Adding in CC people whose issues I've mentioned. I'd love to get more input what can be done on getting ShopifyApp and Shopify API projects the much needed love they deserve.
@nelsonwittwer @matthewlein @guillelopezgines @Fladdermuz @jacksonSingleton @benichu @acflint @fmichaut-diff @ClaytonPassmore @garethson @nickmealey @flavio-b @jagthedrummer @marclennox @panckreous @srgoldman @LovroM @raymondweidner @raphaelfeitoza @amoughnie @andyw8
I keep seeing on this and other Shopify projects (Dawn theme, theme-check, CLI, etc.) the lack of good leadership and oversight. Seems like Shopify employees add features the way they want and no one is looking at the big picture. Stuff breaks, no one is talking to community, but the KPIs are met, because the tool X is now unified with CLI v3 or the gem now supports session auth, but we forgot the docs, sorry.
Please can someone from Shopify reply what are your plans with these projects? Do you have a roadmap? How many developer resources are dedicated to these projects? What is the plan on addressing these above-mentioned issues?
Don't leave me at "seen", I put more than 1 hour creating this issue. /bracing for stale bot 😄
The text was updated successfully, but these errors were encountered: