Overview
Computer Vision enables you to find meaning in visual content! Analyze images for scenes, objects, faces, and other content. Choose a default model off the shelf, or create your own custom classifier. Develop smart applications that analyze the visual content of images or video frames to understand what is happening in a scene.
Beginner's Guide
Computer vision is not just a way to convert pictures to pixels, and it can’t make sense of a picture just from its pixels. It’s the ability of a machine to take a step back and interpret the big picture that those pixels represent. And that’s much harder than we think it is.
For instance, when we see a picture of a model wearing a dress, we automatically identify which part of the body we’re looking at and from which angle. We can figure out lighting conditions. We may even be able to judge the color and texture of the clothes based on shadows, highlights and color temperature.
Quickstart
Following : 5 services/APIs are provided by Computer Vision as part of Release 2.0.0.
Face Detection : Face Detection Service either in form of image or json response.
Object Detection : Object Detection Service either in form of image or json response.
Image Classification : Image Classification Service predicts top 3 classes for an image.
Custom Image Classification : Custom Image Classification Service is used to train on custom images and predict either in the form of image or test images zip file.
Custom Object Detection :Custom Object Detection Service is used to train on custom images and predict either in the form of image or test images zip file.
APIs
All URIs below are relative to https://studio.spotflock.com
| Face Detection Image | POST /api/v1/vision-service/phoenix-vision/face-detection/image |
| Face Detection JSON | POST /api/v1/vision-service/phoenix-vision/face-detection/json |
| Object Detection Image | POST /api/v1/vision-service/phoenix-vision/object-detection/image |
| Object Detection JSON | POST /api/v1/vision-service/phoenix-vision/object-detection/json |
| Image Classification | POST /api/v1/vision-service/phoenix-vision/image-classification |
| Custom Image Classification Train | POST /api/v1/vision-service/phoenix-vision/custom-image-classification/train |
| Custom Image Classification Predict on test images | POST /api/v1/vision-service/phoenix-vision/custom-image-classification/predict |
| Custom Image Classification Predict on single image | POST /api/v1/vision-service/phoenix-vision/custom-image-classification/predict/image |
| Custom Object Detection Train | POST /api/v1/vision-service/phoenix-vision/custom-object_detection/train |
| Custom Object Detection Predict on test images | POST /api/v1/vision-service/phoenix-vision/custom-object_detection/predict |
| Custom Object Detection Predict Image | POST /api/v1/vision-service/phoenix-vision/custom-object_detection/predict/image |
| Custom Object Detection Predict Json | POST /api/v1/vision-service/phoenix-vision/custom-object_detection/predict/json |
| Job Status | GET /api/v1/vision-service/phoenix-vision/status/{id} |
Face Detection Image
Description
This API would enable you to detect multiple faces from a image and retrieve the response in terms of an image.
URI
POST /api/v1/vision-service/phoenix-vision/face-detection/image
Headers
| api-key | Your App's API Key |
Attributes
| file | MultiPart File (Form Data) |
Request Example
Response
Face Detection Json
Description
This API would enable you to detect multiple faces from a image and retrieve the response in terms of a json.
URI
POST /api/v1/vision-service/phoenix-vision/face-detection/image
Headers
| api-key | Your App's API Key |
Attributes
| file | MultiPart File (Form Data) |
Request Example
Response
{
"faces": [
{
"x": 9,
"y": 21,
"width": 61,
"height": 61
}
]
}Object Detection Image
Description
This API would enable you to detect objects from a image and retrieve the response in terms of a image.
URI
POST /api/v1/vision-service/phoenix-vision/object-detection/image
Headers
| api-key | Your App's API Key |
Attributes
| file | MultiPart File (Form Data) |
Request Example
Response
Object Detection Json
Description
This API would enable you to detect objects from a image and retrieve the response in terms of a json.
URI
POST /api/v1/vision-service/phoenix-vision/object-detection/json
Headers
| api-key | Your App's API Key |
Attributes
| file | MultiPart File (Form Data) |
Request Example
Response
{
"object_0": {
"name": "person",
"accuracy": " 86%",
"coordinates": {
"point_0": [
431.42370879650116,
158.40198159217834
],
"point_1": [
507.74650275707245,
347.192097902298
]
}
}
}Image Classification
Description
This API would enable you to classify a image from model trained on 1000 classes and retrieve the response in terms of a json.
URI
POST /api/v1/vision-service/phoenix-vision/image-classification
Headers
| api-key | Your App's API Key |
Attributes
| file | MultiPart File (Form Data) |
Request Example
Response
{
"Great_Pyrenees": "78.4%",
"kuvasz": "6.3%",
"Tibetan_mastiff": "1.8%"
}Custom Image Classification (CIC) Train
Description
This API would enable you to train custom image datastets. This API creates job and can get the status of the job using job-status API
URI
POST /api/v1/vision-service/phoenix-vision/custom-image-classification/train
Headers
| api-key | Your App's API Key |
Attributes
| service | "CLASSIFICATION" for custom image classification |
| name | name of the model |
| zipFolderUrl | url after uploading zipped images to spotflock storage. |
| imagesColumn | Column name of images in csv file |
| labelsColumn | Column name of classes in csv file |
| trainPercentage | split percentage for splitting into train and validation |
| label | label is "vision" in computer-vision |
| epochs | number of iterations on over all data |
| stepsPerEpoch | Total number of steps (batches of samples) before declaring one epoch finished and starting the next epoch |
Request Example
{
"service":"CLASSIFICATION",
"name": "custom_image_clf",
"zipFolderUrl": "/spotflock-storage/url/of/train/images/zip/folder/",
"imagesColumn": "images",
"labelsColumn": "labels",
"trainPercentage": 80,
"label": "vision",
"epochs": 1,
"stepsPerEpoch": 1
}Response
{
"job_id": 101
}Custom Image Classification (CIC) Predict
Description
This API would enable you to predict on image datasets. This API creates job and can get the status of the job using job-status API
URI
POST /api/v1/vision-service/phoenix-vision/custom-image-classification/predict
Headers
| api-key | Your App's API Key |
Attributes
| service | "CLASSIFICATION" for custom image classification |
| name | name of the model |
| zipFolderUrl | url after uploading zipped images to spotflock storage. |
| modelUrl | Url of the model which has been trained |
| label | label is "vision" in computer-vision |
Request Example
{
"service": "CLASSIFICATION",
"name": "custom_image_clf",
"zipFolderUrl": "/spotflock-storage/url/of/train/images/zip/folder/",
"modelUrl": "/spotflock-storage/url/of/model/",
}Response
{
"job_id": 102
}Custom Image Classification (CIC) Predict Image
Description
This API would enable you to predict on image by using custom trained model.
URI
POST /api/v1/vision-service/phoenix-vision/custom-image-classification/predict/image
Headers
| api-key | Your App's API Key |
Attributes
| image | MultiPart File (Form Data) |
| modelUrl | Url of the model which has been trained |
Response
{
'class_one': '0.14482878',
'class_three': '0.34379542',
'class_two': '0.5113758'
}Custom Object Detection (COD) Train
Description
This API would enable you to train custom image datastets. This API creates job and can get the status of the job using job-status API
URI
POST /api/v1/vision-service/phoenix-vision/custom-object-detection/train
Headers
| api-key | Your App's API Key |
Attributes
| name | name of the model |
| zipFolderUrl | url after uploading zipped images to spotflock storage. |
| imagesColumn | Column name of images in csv file |
| labelsColumn | Column name of classes in csv file |
| trainPercentage | split percentage for splitting into train and validation |
| epochs | number of iterations on over all data |
| stepsPerEpoch | Total number of steps (batches of samples) before declaring one epoch finished and starting the next epoch |
Request Example
{
"service":"obj_detection"
"name": "custom_image_clf",
"zipFolderUrl": "/spotflock-studio-dev/603983621873866/vision/1559787665616_custom_clf.zip",
"imagesColumn": "images",
"labelsColumn": "labels",
"trainPercentage": 80,
"epochs": 1,
"stepsPerEpoch": 1
}Response
{
"job_id": 103
}Custom Object Detection (COD) Predict
Description
This API would enable you to predict on image datasets. This API creates job and can get the status of the job using job-status API
URI
POST /api/v1/vision-service/phoenix-vision/custom-object-detection/predict
Headers
| api-key | Your App's API Key |
Attributes
| service | "classification" for custom image classification |
| name | name of the model |
| zipFolderUrl | url after uploading zipped images to spotflock storage. |
| modelUrl | Url of the model which has been trained |
Request Example
{
"service": "obj_detection",
"name": "custom_image_clf",
"zipFolderUrl": "zip/file/url",
"modelUrl": "your/model/url",
}Response
{
"job_id": 104
}Custom Object Detection (COD) Predict Image
Description
This API would enable you to predict on image. The response is base64 encoded string of prediction
URI
POST /api/v1/vision-service/phoenix-vision/custom-object_detection/predict/image
Headers
| api-key | Your App's API Key |
Attributes
| image | MultiPart File (Form Data) |
| modelUrl | Url of the model which has been trained |
Request Example
Response
Custom Object Detection (COD) Predict Json
Description
This API would enable you to predict on image. The response is a json.
URI
POST /api/v1/vision-service/phoenix-vision/custom-object_detection/predict/json
Headers
| api-key | Your App's API Key |
Attributes
| image | MultiPart File (Form Data) |
| modelUrl | Url of the model which has been trained |
Request Example
Response
{
"object_0": {
"name": "dog",
"accuracy": " 94%",
"coordinates": {
"point_0": [
19.67620849609375,
24.85745394229889
],
"point_1": [
323.35205078125,
554.6577959060669
]
}
}
}Job Status
Description
This Api gives the Job Status.
URI
GET /api/v1/vision-service/phoenix-vision/status/{id}
Headers
| api-key | Your App's API Key |
Response
{
"task": "TRAIN",
"status": "FINISH",
"output": {
"job_id": 173,
"loss": 1.4729795455932617,
"fileId": 23259969267838,
"user_id": "603983621873866",
"accuracy": 0,
"modelUrl": "/spotflock-studio-dev/603983621873866/vision/1559819519310_saved_model_7.zip",
"modelName": "custom_image_clf"
}
}Release Notes
Following are the release notes as part of Release 2.0.0
Face Detection gives an accuracy of 93.5%.
Face Detection can be used to detect multiple faces at once.
Custom Image Classification and Custom Object Detection supports zip folder containing images and a csv file which has a column containing image name along with image extension and a label column which specifies class of the image.Image names and classes are case-sensitive.
Custom Object Detection takes image width and height as boundaries of object. So, well cropped images with single object gives good results.
Make sure the folder containing images and .csv file should be compressed but not images directly.
