Summer release ☀️ (Clockwork 5.0)

[itsgoingd]

This issue tracks the progress on the next major release (Clockwork 5.0), scheduled for late summer / fall 2020.

Please do share your feedback and feature requests for this release.

Must have

The most important changes most likely to make it into the release.

• explore an event-based metadata format
  • currently we use a json-based metadata format where each request is represented by a single object with pre-defined schema
    • pros: very simple and straight-forward implementation, plain-text readable
    • cons: requires the whole object to be loaded in memory before writing out the json (very memory intensive), updating partial metadata would require re-parsing the whole json
  • a different idea would be to have an array of events like "database-query" or "log-message" with respective payloads
    • pros: events can be written as they are processed (probably in batches, very low memory usage), updating partial metadata is free
    • cons: more complex implementation when both writing and parsing, less plain-text readable
  • Clockwork App would stay compatible with the old format forever
• explore client-side metrics (Client-side metrics #422, Client-side metrics underground-works/clockwork-app#55, https://github.com/underground-works/clockwork-browser)
  • collect client-side metrics like http overhead (dns time, connect time, etc.) and web vitals (first contentful paint, time ti interactive, etc.)
  • this would require a separate npm distributed js library (and/or installed clockwork extension)
• explore optional alternative ways to use Clockwork (Toolbar #423, https://github.com/underground-works/clockwork-browser)
  • adding a debugbar-style toolbar would provide immediate access to basic performance metrics, details would still link to full Clockwork web app (this would be injected and rendered client-side)
    • this would require a separate npm distributed js library (and/or installed clockwork extension)
  • adding an errors/warnings count to the browser toolbar would provide immediate feedback about performance issues in the current request
    • this would be part of the browser extensions
• consider changing the default web UI URL to /clockwork (Change web UI path #412)
  • make the url configurable, keep /__clockwork for compatibility
• add "models" tab (Added new "models" tab underground-works/clockwork-app#41, Added support for collecting model actions and counts #397)
  • collect model actions like retrieved, created, updated, delete and provide per-model stats
• add "notifications" tab (Notifications tab underground-works/clockwork-app#52, Collect notifications #413)
  • collect emails, sms and other notifications using modern Laravel notifications apis
  • replaces "emails" tab, which has subpar UX and implementation tied to Swift Mailer
• add sharing (Sharing underground-works/clockwork-app#43, https://github.com/underground-works/clockwork-share)
  • share full or part of a request to get a unique public url with a fully working clockwork app and embeddable image
  • official share service provided by us (the share-server will also be open-source)
  • local requests should also be linkable (eg. http://your.app/__clockwork/1592755870-0119-323983132)
• add tagging and starring requests
  • requests can be tagged both server-side and client-side additionally
  • requests should be searchable by tag
• general Clockwork App UI improvements (Design tweaks 2020 underground-works/clockwork-app#49)
  • rework counters and tabs UI, reconsider adding tab counts
  • switch to feather icons or a custom icon set, add tab icons
• improve timeline (Timeline rework underground-works/clockwork-app#47, Timeline improvements #409)
  • timeline events should support parent-children relationships, tagging, color-per-tag and showing events on the same line, the server-side api should also be streamlined
  • add threshold filter Timeline: Threshold filter underground-works/clockwork-app#46
• on-demand mode (On-demand mode #410, On-demand mode underground-works/clockwork-app#51)
  • an optional mode where the request is profiled only when Clockwork browser extension is open (or a GET parameter provided)
• explore adding credits to the Clockwork App (Credits underground-works/clockwork-app#60)
• add basic request information to the request sidebar (Design tweaks 2020 underground-works/clockwork-app#49)
  • method, URL, contorller etc.
• improve Clockwork App polling (Simple dynamic polling underground-works/clockwork-app#58)
  • make the polling interval dynamic
  • apply standalone platform polling improvements to extension
• add ability to collect only "interesting" requests and requests sampling (https://github.com/itsgoingd/clockwork/pull/411/files)
  • interesting requests like errors or slow requests
  • ability to sample collected requests (eg. only collect 1 in 100 requests)
  • ability to filter collected requests by closure with arbitrary logic
• explore moving Clockwork processing at the end of the request to a terminate callback (Move recording to a terminate callback #426, Handle delayed metadata underground-works/clockwork-app#57, Handle delayed metadata underground-works/clockwork-browser#1)
• add ability to (temporarily) disable Clockwork on runtime
• add ability to show only "my requests" in Clockwork App
  • this is implicit in the extension, but the web-app version shows all requests
  • implementation would most likely be based on client ip
• convert links in log messages to html links (Linkify log urls underground-works/clockwork-app#56)
• fix collecting file:line when stack traces are disabled (https://github.com/itsgoingd/clockwork/issues)
• open-source the website
• consider requiring PHP 7.x
  • currently we still support PHP 5.5 as our lowest PHP version
  • previously we updated PHP requirements only when we actually needed some features from the newer version

Nice to have

The less important changes and changes unlikely to make it due to time constraints or implementation concerns.

• improve handling of large metadata
  • we should in general avoid running out of memory at all costs and prefer collecting partial or no data
• improve assets handling
  • add ability to publish web UI assets as an alternative to serve them through php
  • feature request The WEB UI cannot loading its JS,CSS files #340
• add api docs
• add ability to reorder tabs
  • first available tab in order would be the default tab
  • lots of annoying UI code to write (animations, touch support)
• show duplicate queries in the performance tab
• add Laravel authentication support
• logging binary data
  • if we find a way to detect binary data without a large performance impact
• request replay
  • this is very tricky to implement for non-GET requests in cross-platform way
• collect (simple) response
  • main concern here is the response size
• better symfony support
• monolog 2 support
• xhprof support
  • possibly by converting the xhprof profiles to the xdebug format server-side (Convert XHProf to Cachegrind for PHPStorm usage tideways/php-xhprof-extension#92)

Release schedule

Clockwork follows a loose yearly release schedule with one major release in late summer / fall and multiple minor releases in-between.

• Clockwork 5.0 - summer / fall 2020
• Clockwork 5.1 - winter 2020
• Clockwork 5.2 - spring 2021

[mikeerickson]

@itsgoingd wow, that is a full list of changes, should make to be an awesome release.

[fgilio]

Looks like v5 it's gonna be a major step up 💥

Right now in the big project I work on constantly I've installed and regularly use Clockwork, Telescope with Telescope Toolbar and Laravel Debugbar. But, by far, the one I use the most is Clockwork. And my team actually does the same. We really love Clockwork, thanks for all the hard work.

I have a couple of things to share, stuff I run into constantly:

• IMO, the Preserve log and Clear buttons are misplaced. I think putting them closer to the requests list would feel much more natural. Until I get used to it, I used to click it to close the Request panel 😅 and I've seen at least 2 more people doing the same.
  • Same with the magnifying glass icon to search through the requests list, I'd put it in the requests sidebar. Also, right now it's visible even when the sidebar is hidden which may be a little confusing and it actually does nothing there.
• I'd very much appreciate a badge that signals that a particular request in the list has something in the Log.
• Always visible Database filter UI. Right now you have to hove the table header for the magnifying glass icon to appear and then click it. I don't exactly since when this feature is there, but it found it by pure luck and it's fantastic.
• This one is the most opinionated of my suggestions. I'd consider treating the Request sidebar as just one more tab.
• And one wish. I'd find very very useful if Clockwork could automatically detect that something logged is an Exception and provide a more conveniente UI to navigate the trace. It'd be amazing.

---

explore optional alternative ways to use Clockwork
adding a debugbar-style toolbar...

🎉🎉🎉

[itsgoingd]

@fgilio Hey, I think you will like v5, design tweaks for this release address a lot of your points. Check out underground-works/clockwork-app#49 for a little preview.

• In an effort to remove as many buttons from the tab-bar as possible - Clear button is now at the bottom of the requests list. Requests search no longer needs a button, simply scroll down the requests list to reveal the search input. Preserve log was moved to the settings modal, as I don't think this is something that people change too often.
• We already show the number of warnings and errors in the requests list, this includes warning or error level log messages. This was a bit simplified, in the new UI we show only a simple error or warning icon.
• The search fields are now always visible. A new help text will also be added to each search field describing available parameters (did you know you can search for queries over 500ms with duration:>500?).
• We originally used to have a request tab. The idea behind the sidebar is to move the request details (which are not immediately useful) to the side and expose the performance tab as the first thing you see. You can also easily cross-reference the request data with another tab, eg. GET parameters with executed queries. If you don't care about the request details at all, you can always hide the sidebar. The Clockwork UI is always evolving though, so this might change again in the future.
• We already do some special handling for exceptions. In the log tab you should see the full exception trace instead of the short trace for other log messages. You should also get an option to show previous exception if available. For HTTP 500 responses we also show the exceptions in the sidebar.

Thanks for the feedback, if you get any more ideas in the future please do share.

[fgilio]

Preserve log was moved to the settings modal, as I don't think this is something that people change too often.

Agree 👍🏼

did you know you can search for queries over 500ms with duration:>500?

I didn't 😮

We originally used to have a request tab

Lol I was sure that was the case

Wow the redesign indeed looks very nice! 🎉

[fgilio]

Been enjoying the redesign, much more user friendly. Thanks a lot!

[itsgoingd]

Thanks! I can't wait for the next design iteration I will be making for Clockwork 6 though. :)