Get started quickly

Here PyTorch ResNet50 take as an example. Introduce the process of model deployment. All the code can be found in HyperAI-serving-examples/pytorch/image-classifier-resnet50/ obtain.

Prepare the model

import torch
from torchvision import models

# Load pre trained model
model = models.resnet50(pretrained=True)

# Save to local
torch.save(model.state_dict(), "resnet50.pt")

Write model deployment scripts predict.py

predictor.py The file needs to contain a file named Predictor The class. Its structure is as follows:

import HyperAI_serving as serv


class Predictor:
    def __init__(self):
        """
        Responsible for loading the corresponding model and initializing metadata
        """
        ...

    def predict(self, json):
        """
        accept HTTP The requested content. Predict the results after necessary processing. Finally return the result to the caller
        """
        ...


if __name__ == '__main__':
    serv.run(Predictor)

Specific parameters and introductions can be found in Serving Predictor find.

And for PyTorch ResNet50 his predictor.py The content is as follows:

import torch
import cv2
import numpy as np
import json
import requests
from torchvision import models, transforms

import HyperAI_serving as serv


def get_url_image(url_image):
    resp = requests.get(url_image, stream=True).raw
    image = np.asarray(bytearray(resp.read()), dtype="uint8")
    image = cv2.imdecode(image, cv2.IMREAD_COLOR)
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    return image


class Predictor:
    def __init__(self):
        # Load classification metadata
        classes = json.load(open('classes.json'))
        self.idx2label = [classes [str(k)][1] for k in range(len(classes))]

        # Specify the model file name
        model_name = 'resnet50.pt'

        # Load model
        self.device = "cuda" if torch.cuda.is_available() else "cpu"
        self.model = models.resnet50()
        self.model.load_state_dict(torch.load(model_name))
        self.model.eval()
        self.model = self.model.to(self.device)

        # Image preprocessing, include normalization and resize
        normalize = transforms. Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
        self.transform = transforms. Compose(
            [
                transforms. ToPILImage(),
                transforms. Resize([224, 224]),
                transforms. ToTensor(),
                normalize,
            ]
        )

    def predict(self, json):
        # from json.url Obtaining images url. The requirement to transmit HTTP The request content is {"url": "xxx"}
        imageurl = json["url"]
        # Retrieve image content
        image = get_url_image(imageurl)
        # Image preprocessing
        image = self.transform(image)
        image = torch.tensor(image.numpy()[np.newaxis, ...])

        # reasoning
        results = self.model(image.to(self.device))

        # Get the top five results
        top5_idx = results[0].sort()[1][-5:]

        # Obtain the names of the top five categories
        top5_labels = [self.idx2label [idx] for idx in top5_idx]
        top5_labels = top5_labels[::-1]

        return top5_labels


if __name__ == '__main__':
    serv.run(Predictor)

Upload to data warehouse

Prepare what has already been prepared .pt Documents and predictor.py Place the files and other necessary files in the same directory. The files that must be included in the directory are as follows:

.
├── classes.json
├── predictor.py
└── resnet50.pt

among

• resnet50.pt For model files.
• predictor.py at present HyperAI Model deployment requires its file name to be fixed as predictor.py Using a different file name will result in deployment failure.
• classes.json contain resnet Classification metadata. When displaying the names of categories, use.

Create a model repository:

Upload the above three files:

View upload results:

Establish Serving

Left navigation bar "Computing power container" -> "Model deployment" Create a new one Serving:

Bind model directory. Select the model version directory that was just uploaded:

Click Deploy and wait for deployment to be marked as "Running" :

Click on the latest deployment version currently available (It is currently the only deployed version)view log:

Test

In the model display "Running" Later, you can see "overview" The page displays the currently deployed http Access address.

adopt curl The tool can test the currently deployed service:

curl -X POST \
    <Service Address> \
    -H "Content-Type: application/json" \
    --data-raw '{ "url": "http://hyperai-public.cn-bj.ufileos.com/cat.jpg" }'

"Model deployment" The default request used is JSON format, among url Fields and our previous text predictor.py stay json Obtained from url Correspondingly.

Can obtain corresponding results:

["tabby", "Egyptian_cat", "tiger_cat", "lynx", "tiger"]