Starting from scratch super-resolution service

stay Dependency managementPart of the dependency installation will be introduced in "Model deployment" Execute at startup. To accelerate service startup, you can use custom images. Pre install all dependencies in custom images.

MMEditing It is based on PyTorch Implemented open-source image and video editing libraries, belong to OpenMMLab Part of the project. MMEditing Can do image completion. Cutout image. Super resolution. Generate work. This tutorial uses super-resolution as an example to build a Serving service.

Go through the process in the development environment

first. Create a new one "model training" Computing power container, choice PyTorch image. Name it MMEditing.

wait until Jupyter After the workspace is launched. Open one Terminal, take MMEditing The warehouse clone come down, stay MMEditing of README Found in fileLink and install according to the documentation.

The container has already been provided CUDA. PyTorch and torchvision assembly. So from c Just start the steps.

implement pip install --user -r requirements.txt. One of the packages needs to be compiled, so the time may be a bit long.

Final execution pip install --user -v . Installed properly mmediting The library itself. Don't add here -e parameter (Development mode)But it's a normal installation. Because it is also needed after the model is launched.

Then we went into the document Model Zoo/Restration Models Found pre trained ones EDSR Model. Download the configuration and model files and upload them to the computing power container home Under the directory (Can be used Jupyter Upload button for workspace. You can also drag it directly to the file list).

After uploading is completed, stay home Create a new directory Notebook, implement

import mmedit

If there are no errors. It means our environment has been set up.

next. Upload a low resolution image. Used for testing purposes. Here we call it low-resolution.png.

stay mmediting Inside the warehouse demo/restoration_demo.py Sample code for super-resolution can be found

Let's rewrite and put it down Notebook Mid Experiment.

import mmedit, mmcv, torch
from mmedit.apis import init_model, restoration_inference
from mmedit.core import tensor2img

model = init_model(
    "edsr_x2c64b16_g1_300k_div2k.py",
    "edsr_x2c64b16_1x16_300k_div2k_20200604-19fe95ea.pth",
    device=torch.device('cuda', 0)
)

output = restoration_inference(model, "low-resolution.png")
output = tensor2img(output)
mmcv.imwrite(output, "high-resolution.png")

import PIL
low = PIL.Image.open('low-resolution.png')
high = PIL.Image.open('high-resolution.png')
display(low, high)

At this point, an image has been successfully processed with super-resolution.

To write Serving Service usage predictor.py

stay Serving Service WritingThere is an introduction in it predictor.py The specific writing method. No further explanation will be provided here.

import HyperAI_serving as serv

import mmedit, mmcv, torch
import cv2
from mmedit.apis import init_model, restoration_inference
from mmedit.core import tensor2img

import tempfile


class Predictor:
    def __init__(self):
        self.model = init_model(
            "edsr_x2c64b16_g1_300k_div2k.py",
            "edsr_x2c64b16_1x16_300k_div2k_20200604-19fe95ea.pth",
            device=torch.device('cuda', 0)
        )

    def predict(self, data):
        # The data we agree to input here is directly POST The picture that came up
        f = tempfile. NamedTemporaryFile()
        f.write(data)
        f.seek(0)
        output = restoration_inference(self.model, f.name)
        _, img = cv2.imencode('.png', output)
        return img

if __name__ == '__main__':
    serv.run(Predictor)

You can see that it's basically what we did before Notebook The code written in the middle. Filled into a fixed framework. Save the file as predictor.py, Open a new one Terminal Run it.

And then Notebook Test it in the middle, POST Just go to the local access address

import requests

with open('low-resolution.png', 'rb') as f:
    img = f.read()

resp = requests.post('http://0.0.0.0:8080', data=img)

But returning to predictor.py of Terminal window. Discovered a very strange error reported.

cv2.imencode Will fail. It's likely because output The parameters are not quite normal. We follow the prompts to open the debugging link. Observe it output The state of variables.

find output It's a PyTorch Tensor, however OpenCV I don't know Tensor. I only know numpy array.

Recheck the code. Discovered a missing sentence output = tensor2img(output) (Refer to the previous code). Let's first experiment in the debugging window.

Found that it can already run successfully. So let's add this sentence later. Run again python predictor.py, Test again.

Still reporting errors. But this time it was a very obvious mistake. We need to present the results numpy array Convert objects to bytes.

final return img Change to return img.tobytes() That's enough. Test again. Successfully returned the result.

Last, go online!

The development has been completed. Can you clean up the things that are not needed. The final result is like this:

Stop computing power container. After waiting for synchronization to be completed. On the left sidebar "data warehouse " Find here "Model" , click "Create a new model" .

Then return to the closed computing power container, click "Copy the current directory to the data warehouse" .

The processed model should look like this:

Then find it "Computing power container" Below "Model deployment" , click "Create a new deployment" , Choose the same image as during development. And choose one GPU Computing power.

Bind the model that has just been prepared, click "deploy"

Waiting for the model to start. Observing the image below indicates success

Then deploy the model "overview" Obtain from here API address,

Try it somewhere else

At this point, the super-resolution service has been successfully launched online.

Also, let's test the performance briefly

Note that this is not intended to showcase performance case. The tutorial uses the simplest way to reduce cognitive burden. The performance of the service is not optimal. This chapter is mainly used to demonstrate "Request statistics" The usage.

We will make the final decision requests.post Package in while True: in. Continuously requesting services. And then "Request statistics" Observation in the middle:

Observing an average (avg)The processing of a request requires 109ms. This data was observed in the backend. Excluding the creation of connections for sending requests, send data. Return data process. Maximum value of request time (max)With 50% Quantile (median)The difference is not significant. So the actual processing time for each request is very stable.

The interval between each data point is 10s, Capture each data point 61 A request, therefore QPS namely 6.1. The average processing time for each request is 10s/61 = 164ms, This is the actual perceived request time by the client.

Then open a few more Notebook. Send requests to services in parallel. Observe again.

And then discovered QPS It has become 20.5, But the high request time has already been 50% The percentile has widened the gap. This is a phenomenon that occurs when there is high service pressure. These indicators will help you evaluate the specific quality of service.