Skip to content

Cannot Deploy Endpoint #2617

@ujjawalmadan

Description

@ujjawalmadan

I am not able to deploy the endpoint and am met with this error message. Have tried diagnosing but not sure what the issue is:

import sagemaker

sess = sagemaker.Session()
role = sagemaker.get_execution_role()

from sagemaker.huggingface.model import HuggingFaceModel

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
   model_data="s3://qfn-transcription/ujjawal_files/model.tar.gz",  # path to your trained sagemaker model
   role=role, # iam role with permissions to create an Endpoint
   transformers_version="4.6", # transformers version used
   pytorch_version="1.7", # pytorch version used
   py_version='py36'
)

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
   initial_instance_count=1, 
   instance_type="ml.m5.xlarge"
)

# example request, you always need to define "inputs"
long_text= """
Hugging Face, the winner of VentureBeat’s Innovation in Natural Language Process/Understanding Award for 2021, 
is looking to level the playing field. The team, launched by Clément Delangue and Julien Chaumond in 2016,
was recognized for its work in democratizing NLP, the global market value for which is expected to 
hit $35.1 billion by 2026. This week, Google’s former head of Ethical AI Margaret Mitchell joined the team.
"""

parameters = {'repetition_penalty':4.,'length_penalty': 1.5}

predictor.predict({"inputs":long_text,"parameters":parameters})

# request
predictor.predict(data)

image

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions