Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support production monitoring use case, no git required #5

Closed
patrickhulce opened this issue Jun 26, 2019 · 14 comments · Fixed by #342
Closed

Support production monitoring use case, no git required #5

patrickhulce opened this issue Jun 26, 2019 · 14 comments · Fixed by #342
Labels

Comments

@patrickhulce
Copy link
Collaborator

Currently every build needs a .hash but we don't necessarily need one. We could switch to a .projectVersion and support timestamps instead?

@patrickhulce
Copy link
Collaborator Author

New better idea: report command just accepts a --hash-method flag of either git or timestamp which would do Date.now().toString(16) to generate a timestamp hash instead

@patrickhulce patrickhulce removed the MVP label Oct 1, 2019
@patrickhulce
Copy link
Collaborator Author

we're punting on non-git cases for MVP

@thasmo
Copy link

thasmo commented Dec 11, 2019

@patrickhulce Does that mean in general, that lighthouse-ci will work without a project's git repository when using a timestamp based versioning? It would be nice to just run lhci autorun with a config and send the results to the server; without the need for a local git repository at all. (e.g. it would make it possible to create regular snapshot reports for specific URLs without the need to run it in a git repo or ci environment)

@patrickhulce patrickhulce changed the title Support time-based versions in addition to hashes Support production monitoring use case, no git required Dec 11, 2019
@patrickhulce
Copy link
Collaborator Author

it would make it possible to create regular snapshot reports for specific URLs without the need to run it in a git repo or ci environment

This broader effort we refer to as "production monitoring" which is on our roadmap but requires many subtle but important changes to the types of UI and assumptions made in the CI server currently. We're discussing internally exactly how we want to handle this.

@AaronLayton
Copy link

Just placing my support behind this Issue

@Zizzamia
Copy link

@patrickhulce would this be just a new Project under the current UI?
Something like https://lhci-server.lovelhci.com/app/projects/my-app-prod?
One point I have been thinking, is also how we could adapt the UI for running different experiments, I will open a different issue for that conversation.

@patrickhulce
Copy link
Collaborator Author

patrickhulce commented Feb 26, 2020

@Zizzamia yeah you can basically do a new project and fake the build context to get this to work today.

Something like the below (copied from #225)

export LHCI_BUILD_CONTEXT__CURRENT_HASH=$(node -p "new Date().getTime().toString(16).split('').reverse().join('')")
export LHCI_BUILD_CONTEXT__COMMIT_TIME=$(date --iso-8601=seconds)
export LHCI_BUILD_CONTEXT__CURRENT_BRANCH=master
export LHCI_BUILD_CONTEXT__COMMIT_MESSAGE="Periodic run "
export LHCI_BUILD_CONTEXT__AUTHOR="No one <noreply@example.com>"
export LHCI_BUILD_CONTEXT__AVATAR_URL="https://picsum.photos/id/237/200/200"
lhci ...

@kunalnagar
Copy link

kunalnagar commented Oct 9, 2020

@patrickhulce I have a quick question on tracking production URLs. Before that, let me explain about the setup we have currently.

We have a repository with a lighthouserc.js file and a GitHub action that runs lhci autorun. The config file has the following structure:

module.exports = {
  ci: {
    collect: // localhost urls and server is started using `npm start`
    assert: ...
    upload: // to custom lhci server using build token, username and password
  },
}

After reading the documentation and this comment, I see we can add a server key. I'm curious about two things:

  1. Where are these results uploaded to if you don't specify a storage key
  2. How is the CRON set up exactly if multiple people keep pushing to the repo and the GitHub action keeps running with this config?

...or am I being completely off my rocker here 😅 Thanks!

@patrickhulce
Copy link
Collaborator Author

It seems you've mixed the two use cases together @kunalnagar.

#342 was about monitoring production URLs without a git repository at all. You just configure server.psiCollectCron without any of the other collect/assert/upload pieces. It runs on your server not in the CI process.

Where are these results uploaded to if you don't specify a storage key

I'm not sure what you mean by storage key in the context of the config you shared. If you're talking about the server's configuration then you must specify a storage key or it will simply fail. There's no such thing as the server without any storage mechanism.

If you're talking about where does data go if you don't configure an upload step, then the answer is nowhere, it will just stay on in your local CI.

How is the CRON set up exactly if multiple people keep pushing to the repo and the GitHub action keeps running with this config?

Cron doesn't run from the CI process CLI, it runs on the server.

@kunalnagar
Copy link

@patrickhulce thanks a lot for the clarification!

  1. If the psiCollectCron step runs on the server and it stores the results in a SQL database, can those results be viewed using the lighthouse UI like the ci steps? Or do we have to configure something custom to view them?

  2. I think I found an issue with the export stuff you mentioned in this comment. This is only to do with the LHCI_BUILD_CONTEXT__CURRENT_HASH variable:

    • If you define the variables as mentioned in your comment, sometimes the CURRENT_HASH value is incorrect and it takes the .git hash.

    • I tried moving it to using the env: key in GitHub actions like so:

      env:
         LHCI_BUILD_CONTEXT__CURRENT_HASH: $(node -e "new Date().getTime().toString(16).split('').reverse().join('')")
      

      ...and it sends the command instead of the calculated string. Screenshot below:

      image

Here's a simplified github action script that i am using as per your comment:

name: Check lighthouse (production build)

on:
  schedule:
    - cron: '0 */1 * * *' # every hour

jobs:
  lighthouseprd:
    runs-on: ubuntu-18.04
    steps:
      - uses: actions/checkout@v2
      - name: Export env variables
        run: |
          export LHCI_BUILD_CONTEXT__CURRENT_HASH=$(node -e "new Date().getTime().toString(16).split('').reverse().join('')")
          export LHCI_BUILD_CONTEXT__COMMIT_TIME=$(date --iso-8601=seconds)
          export LHCI_BUILD_CONTEXT__CURRENT_BRANCH=master
          export LHCI_BUILD_CONTEXT__COMMIT_MESSAGE="CRON Periodic run"
          export LHCI_BUILD_CONTEXT__AUTHOR="Kunal Nagar <me@email.com>"
          export LHCI_BUILD_CONTEXT__AVATAR_URL="image_link_here"
      - name: Run lighthouse
        run: |
          sudo npm install -g @lhci/cli@0.4.x
          lhci autorun // more config here

@patrickhulce
Copy link
Collaborator Author

This has ventured a little too far into "build my setup" territory sorry @kunalnagar 😅

@kunalnagar
Copy link

@patrickhulce no worries thanks for all the help! 🙂

I was just trying to understand if there's an issue with how Github Actions handles environment variables OR how the LHCI_BUILD_CONTEXT__CURRENT_HASH is implemented on lighthouse-ci - because the time (which is also calculated via bash) seems to go out correctly to the server, it's just the hash that is being taken from git.

@riginoommen
Copy link
Contributor

@Zizzamia yeah you can basically do a new project and fake the build context to get this to work today.

Something like the below (copied from #225)

export LHCI_BUILD_CONTEXT__CURRENT_HASH=$(node -e "new Date().getTime().toString(16).split('').reverse().join('')")
export LHCI_BUILD_CONTEXT__COMMIT_TIME=$(date --iso-8601=seconds)
export LHCI_BUILD_CONTEXT__CURRENT_BRANCH=master
export LHCI_BUILD_CONTEXT__COMMIT_MESSAGE="Periodic run "
export LHCI_BUILD_CONTEXT__AUTHOR="No one <noreply@example.com>"
export LHCI_BUILD_CONTEXT__AVATAR_URL="https://picsum.photos/id/237/200/200"
lhci ...

this tweak is awesome

@duveborg
Copy link

duveborg commented Oct 7, 2022

is there a better way to do this in 2022? and how to do the env var hack with treosh/lighthouse-ci-action ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants