Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deployment run fails when configuring prefect.yaml file pull section with Gitlab credential block #10282

Closed
4 tasks done
tc-chrisbui opened this issue Jul 21, 2023 · 2 comments
Labels
bug Something isn't working

Comments

@tc-chrisbui
Copy link

First check

  • I added a descriptive title to this issue.
  • I used the GitHub search to find a similar issue and didn't find it.
  • I searched the Prefect documentation for this issue.
  • I checked that this issue is related to Prefect and not one of its dependencies.

Bug summary

I'm trying to deploy my flows using the prefect.yaml file. I followed these instructions to set up the pull section to pull code from my Gitlab repo using my credentials block.

I don't run into problems creating the deployment on the server, but when i try to run it, I get the error described below.

Reproduction

# create gitlab credentials block

from prefect_gitlab.credentials import GitLabCredentials
import os
from dotenv import load_dotenv

load_dotenv()
# Gitlab Credentials Block
gitlab_credentials_block = GitLabCredentials(
    token=os.environ.get("ACCESS_TOKEN")
)
gitlab_credentials_block.save("gitlab-credential",overwrite=True)

create prefect.yaml

# File for configuring project / deployment build, push and pull steps

# Generic metadata about this project
name: prefect
prefect-version: 2.11.0

# build section allows you to manage and build docker images
build: null

# push section allows you to manage if and how this project is uploaded to remote locations
push: null

# pull section allows you to provide instructions for cloning this project in remote locations
pull:
- prefect.projects.steps.git_clone_project:
    repository: https://gitlab.xxx.com/xxx.git
    branch: main
    credentials: "{{ prefect.blocks.gitlab-credentials.gitlab-credential }}"

deployments:
- name: sample_print_flow
  entrypoint: flows/sample_flows/sample_flow_a.py:print_something
  parameters: {
    "value": "value1"
  }
  work_pool: local

deploy

prefect deploy --all

Run deployment

At this point, I am running an agent locally.



### Error

```python3
Traceback (most recent call last):
  File "REDACTED/python3.8/site-packages/prefect/deployments/steps/core.py", line 124, in run_steps
    step_output = await run_step(step, upstream_outputs)
  File "REDACTED/python3.8/site-packages/prefect/deployments/steps/core.py", line 95, in run_step
    result = await from_async.call_soon_in_new_thread(
  File "REDACTED/python3.8/site-packages/prefect/_internal/concurrency/calls.py", line 292, in aresult
    return await asyncio.wrap_future(self.future)
  File "REDACTED/python3.8/site-packages/prefect/_internal/concurrency/calls.py", line 316, in _run_sync
    result = self.fn(*self.args, **self.kwargs)
  File "REDACTED/python3.8/site-packages/prefect/_internal/compatibility/deprecated.py", line 97, in wrapper
    return fn(*args, **kwargs)
TypeError: git_clone_project() got an unexpected keyword argument 'credentials'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "REDACTED/python3.8/site-packages/prefect/engine.py", line 395, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "REDACTED/python3.8/site-packages/prefect/client/utilities.py", line 51, in with_injected_client
    return await fn(*args, **kwargs)
  File "REDACTED/python3.8/site-packages/prefect/deployments/deployments.py", line 203, in load_flow_from_flow_run
    output = await run_steps(deployment.pull_steps)
  File "REDACTED/python3.8/site-packages/prefect/deployments/steps/core.py", line 152, in run_steps
    raise StepExecutionError(f"Encountered error while running {fqn}") from exc
prefect.deployments.steps.core.StepExecutionError: Encountered error while running prefect.projects.steps.git_clone_project

Versions

13:44:40.409 | DEBUG   | prefect.profiles - Using profile 'dev'
Version:             2.11.0
API version:         0.8.4
Python version:      3.8.10
Git commit:          eeb9e219
Built:               Thu, Jul 20, 2023 4:34 PM
OS/Arch:             darwin/arm64
Profile:             dev
Server type:         cloud

Additional context

No response

@prefectcboyd
Copy link
Contributor

Hi Chris,

git_clone_project is a deprecated function, which has been replaced by git_clone in 2.10.14 (transitioning from projects -> deployments). Looking at this line here -
https://github.com/PrefectHQ/prefect/blob/main/src/prefect/deployments/steps/pull.py#L155
git_clone_project is returning a wrapper to git_clone, but missing the credentials.
Can you just swap these lines?

pull:
- prefect.projects.steps.git_clone_project:

to this:

pull:
- prefect.projects.steps.git_clone:

See here for implementation:
https://github.com/PrefectHQ/prefect/blob/main/src/prefect/deployments/steps/pull.py#L32

@tc-chrisbui
Copy link
Author

Thanks! This resolved my issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants