Overview


Computer Vision enables you to find meaning in visual content! Analyze images for scenes, objects, faces, and other content. Choose a default model off the shelf, or create your own custom classifier. Develop smart applications that analyze the visual content of images or video frames to understand what is happening in a scene.


Beginner's Guide


Computer vision is not just a way to convert pictures to pixels, and it can’t make sense of a picture just from its pixels. It’s the ability of a machine to take a step back and interpret the big picture that those pixels represent. And that’s much harder than we think it is.

For instance, when we see a picture of a model wearing a dress, we automatically identify which part of the body we’re looking at and from which angle. We can figure out lighting conditions. We may even be able to judge the color and texture of the clothes based on shadows, highlights and color temperature.


Quickstart


Following : 5 services/APIs are provided by Computer Vision as part of Release 2.0.0.

  • Face Detection : Face Detection Service either in form of image or json response.

  • Object Detection : Object Detection Service either in form of image or json response.

  • Image Classification : Image Classification Service predicts top 3 classes for an image.

  • Custom Image Classification : Custom Image Classification Service is used to train on custom images and predict either in the form of image or test images zip file.

  • Custom Object Detection :Custom Object Detection Service is used to train on custom images and predict either in the form of image or test images zip file.


APIs


All URIs below are relative to https://studio.spotflock.com

Face Detection Image POST   /api/v1/vision-service/phoenix-vision/face-detection/image
Face Detection JSON POST   /api/v1/vision-service/phoenix-vision/face-detection/json
Object Detection Image POST   /api/v1/vision-service/phoenix-vision/object-detection/image
Object Detection JSON POST   /api/v1/vision-service/phoenix-vision/object-detection/json
Image Classification POST   /api/v1/vision-service/phoenix-vision/image-classification
Custom Image Classification Train POST   /api/v1/vision-service/phoenix-vision/custom-image-classification/train
Custom Image Classification Predict on test images POST   /api/v1/vision-service/phoenix-vision/custom-image-classification/predict
Custom Image Classification Predict on single image POST   /api/v1/vision-service/phoenix-vision/custom-image-classification/predict/image
Custom Object Detection Train POST   /api/v1/vision-service/phoenix-vision/custom-object_detection/train
Custom Object Detection Predict on test images POST   /api/v1/vision-service/phoenix-vision/custom-object_detection/predict
Custom Object Detection Predict Image POST   /api/v1/vision-service/phoenix-vision/custom-object_detection/predict/image
Custom Object Detection Predict Json POST   /api/v1/vision-service/phoenix-vision/custom-object_detection/predict/json
Job Status GET   /api/v1/vision-service/phoenix-vision/status/{id}

Face Detection Image

Description

This API would enable you to detect multiple faces from a image and retrieve the response in terms of an image.

URI

POST  /api/v1/vision-service/phoenix-vision/face-detection/image

Headers

api-key Your App's API Key

Attributes

file MultiPart File (Form Data)

Request Example

Response

Face Detection Json

Description

This API would enable you to detect multiple faces from a image and retrieve the response in terms of a json.

URI

POST  /api/v1/vision-service/phoenix-vision/face-detection/image

Headers

api-key Your App's API Key

Attributes

file MultiPart File (Form Data)

Request Example

Response

{
    "faces": [
        {
            "x": 9,
            "y": 21,
            "width": 61,
            "height": 61
        }
    ]
}

Object Detection Image

Description

This API would enable you to detect objects from a image and retrieve the response in terms of a image.

URI

POST  /api/v1/vision-service/phoenix-vision/object-detection/image

Headers

api-key Your App's API Key

Attributes

file MultiPart File (Form Data)

Request Example

Response

Object Detection Json

Description

This API would enable you to detect objects from a image and retrieve the response in terms of a json.

URI

POST  /api/v1/vision-service/phoenix-vision/object-detection/json

Headers

api-key Your App's API Key

Attributes

file MultiPart File (Form Data)

Request Example

Response

{
    "object_0": {
        "name": "person",
        "accuracy": " 86%",
        "coordinates": {
            "point_0": [
                431.42370879650116,
                158.40198159217834
            ],
            "point_1": [
                507.74650275707245,
                347.192097902298
            ]
        }
    }
}

Image Classification

Description

This API would enable you to classify a image from model trained on 1000 classes and retrieve the response in terms of a json.

URI

POST  /api/v1/vision-service/phoenix-vision/image-classification

Headers

api-key Your App's API Key

Attributes

file MultiPart File (Form Data)

Request Example

Response

{
    "Great_Pyrenees": "78.4%",
    "kuvasz": "6.3%",
    "Tibetan_mastiff": "1.8%"
}

Custom Image Classification (CIC) Train

Description

This API would enable you to train custom image datastets. This API creates job and can get the status of the job using job-status API

URI

POST  /api/v1/vision-service/phoenix-vision/custom-image-classification/train

Headers

api-key Your App's API Key

Attributes

service "CLASSIFICATION" for custom image classification
name name of the model
zipFolderUrl url after uploading zipped images to spotflock storage.
imagesColumn Column name of images in csv file
labelsColumn Column name of classes in csv file
trainPercentage split percentage for splitting into train and validation
label label is "vision" in computer-vision
epochs number of iterations on over all data
stepsPerEpoch Total number of steps (batches of samples) before declaring one epoch finished and starting the next epoch

Request Example

{
    "service":"CLASSIFICATION",
    "name": "custom_image_clf",
    "zipFolderUrl": "/spotflock-storage/url/of/train/images/zip/folder/",
    "imagesColumn": "images",
    "labelsColumn": "labels",
    "trainPercentage": 80,
    "label": "vision",
    "epochs": 1,
    "stepsPerEpoch": 1
}

Response

{
    "job_id": 101
}

Custom Image Classification (CIC) Predict

Description

This API would enable you to predict on image datasets. This API creates job and can get the status of the job using job-status API

URI

POST  /api/v1/vision-service/phoenix-vision/custom-image-classification/predict

Headers

api-key Your App's API Key

Attributes

service "CLASSIFICATION" for custom image classification
name name of the model
zipFolderUrl url after uploading zipped images to spotflock storage.
modelUrl Url of the model which has been trained
label label is "vision" in computer-vision

Request Example

{
    "service": "CLASSIFICATION",
    "name": "custom_image_clf",
    "zipFolderUrl": "/spotflock-storage/url/of/train/images/zip/folder/",
    "modelUrl": "/spotflock-storage/url/of/model/",
}

Response

{
    "job_id": 102
}

Custom Image Classification (CIC) Predict Image

Description

This API would enable you to predict on image by using custom trained model.

URI

POST  /api/v1/vision-service/phoenix-vision/custom-image-classification/predict/image

Headers

api-key Your App's API Key

Attributes

image MultiPart File (Form Data)
modelUrl Url of the model which has been trained

Response

{
    'class_one': '0.14482878',
    'class_three': '0.34379542',
    'class_two': '0.5113758'
}

Custom Object Detection (COD) Train

Description

This API would enable you to train custom image datastets. This API creates job and can get the status of the job using job-status API

URI

POST  /api/v1/vision-service/phoenix-vision/custom-object-detection/train

Headers

api-key Your App's API Key

Attributes

name name of the model
zipFolderUrl url after uploading zipped images to spotflock storage.
imagesColumn Column name of images in csv file
labelsColumn Column name of classes in csv file
trainPercentage split percentage for splitting into train and validation
epochs number of iterations on over all data
stepsPerEpoch Total number of steps (batches of samples) before declaring one epoch finished and starting the next epoch

Request Example

{
    "service":"obj_detection"
    "name": "custom_image_clf",
    "zipFolderUrl": "/spotflock-studio-dev/603983621873866/vision/1559787665616_custom_clf.zip",
    "imagesColumn": "images",
    "labelsColumn": "labels",
    "trainPercentage": 80,
    "epochs": 1,
    "stepsPerEpoch": 1
}

Response

{
    "job_id": 103
}

Custom Object Detection (COD) Predict

Description

This API would enable you to predict on image datasets. This API creates job and can get the status of the job using job-status API

URI

POST  /api/v1/vision-service/phoenix-vision/custom-object-detection/predict

Headers

api-key Your App's API Key

Attributes

service "classification" for custom image classification
name name of the model
zipFolderUrl url after uploading zipped images to spotflock storage.
modelUrl Url of the model which has been trained

Request Example

{
    "service": "obj_detection",
    "name": "custom_image_clf",
    "zipFolderUrl": "zip/file/url",
    "modelUrl": "your/model/url",
}

Response

{
    "job_id": 104
}

Custom Object Detection (COD) Predict Image

Description

This API would enable you to predict on image. The response is base64 encoded string of prediction

URI

POST  /api/v1/vision-service/phoenix-vision/custom-object_detection/predict/image

Headers

api-key Your App's API Key

Attributes

image MultiPart File (Form Data)
modelUrl Url of the model which has been trained

Request Example

Response

Custom Object Detection (COD) Predict Json

Description

This API would enable you to predict on image. The response is a json.

URI

POST  /api/v1/vision-service/phoenix-vision/custom-object_detection/predict/json

Headers

api-key Your App's API Key

Attributes

image MultiPart File (Form Data)
modelUrl Url of the model which has been trained

Request Example

Response

{
    "object_0": {
        "name": "dog",
        "accuracy": " 94%",
        "coordinates": {
            "point_0": [
                19.67620849609375,
                24.85745394229889
            ],
            "point_1": [
                323.35205078125,
                554.6577959060669
            ]
        }
    }
}

Job Status

Description

This Api gives the Job Status.

URI

GET  /api/v1/vision-service/phoenix-vision/status/{id}

Headers

api-key Your App's API Key

Response

{
    "task": "TRAIN",
    "status": "FINISH",
    "output": {
        "job_id": 173,
        "loss": 1.4729795455932617,
        "fileId": 23259969267838,
        "user_id": "603983621873866",
        "accuracy": 0,
        "modelUrl": "/spotflock-studio-dev/603983621873866/vision/1559819519310_saved_model_7.zip",
        "modelName": "custom_image_clf"
    }
}


Release Notes


Following are the release notes as part of Release 2.0.0

  • Face Detection gives an accuracy of 93.5%.

  • Face Detection can be used to detect multiple faces at once.

  • Custom Image Classification and Custom Object Detection supports zip folder containing images and a csv file which has a column containing image name along with image extension and a label column which specifies class of the image.Image names and classes are case-sensitive.

  • Custom Object Detection takes image width and height as boundaries of object. So, well cropped images with single object gives good results.

  • Make sure the folder containing images and .csv file should be compressed but not images directly.

Helpdesk