API Reference

All 14 request types, with the fields each one reads and the response it sends back. Taken from the server's source.

Protocol Overview

Open a TCP connection to the server (default 127.0.0.1:10210, loopback only: the protocol has no authentication) and write one JSON object per line. Each request has an id you choose, a type naming the operation, and a data object with its fields. Paths are paths on the server's own filesystem.

The reply is one JSON line: id, success, a human-readable message, and data. Most computer-vision results sit at data.result; image processing results are data itself. Output files are written on the server, usually under /tmp, and returned as paths.

On failure success is false, message says why, and data is null. A request the server could not parse, or an unknown type, comes back with an empty id.

Error json

1{
2  "id": "req-1",
3  "success": false,
4  "message": "Image rotate failed: Image file not found: /srv/photos/missing.jpg",
5  "data": null
6}

Operations

Computer Vision

image_object_detection Object Detection

YOLOv4 through OpenCV DNN, COCO's 80 classes. Boxes are [x, y, width, height] in source pixels, highest confidence first. Writes a JSON dump and a copy of the image with the boxes drawn.

Fields in `data`

Field	Type	Default	Meaning
inputFile	string	required	Path to the image on the server host.
model	string	"tiny"	"tiny" (YOLOv4-tiny, fast) or "full" (YOLOv4, slower, more accurate).

Request

1{
2  "id": "req-1",
3  "type": "image_object_detection",
4  "data": {"inputFile": "/srv/photos/street.jpg", "model": "tiny"}
5}

Response

1{
2  "id": "req-1",
3  "success": true,
4  "message": "Image object detection completed - found 2 objects",
5  "data": {
6    "outputFile": "/tmp/mediaprocessor_20260925_174527.json",
7    "result": {
8      "model": "tiny",
9      "objects": [
10        {
11          "class": "person",
12          "confidence": 0.91,
13          "bbox": [412, 180, 96, 288]
14        },
15        {
16          "class": "car",
17          "confidence": 0.84,
18          "bbox": [40, 260, 310, 170]
19        }
20      ],
21      "rendered_image_path": "/tmp/mediaprocessor_20260925_174527_objects.png"
22    }
23  }
24}

image_depth_estimation Depth Estimation

Monocular depth from edge gradients (Gaussian blur, Sobel X/Y, magnitude), rendered with the JET colormap. One PNG, the same size as the input.

Fields in `data`

Field	Type	Default	Meaning
inputFile	string	required	Path to the image on the server host.

Request

1{
2  "id": "req-1",
3  "type": "image_depth_estimation",
4  "data": {"inputFile": "/srv/photos/street.jpg"}
5}

Response

1{
2  "id": "req-1",
3  "success": true,
4  "message": "Depth estimation completed using monocular edge-based analysis",
5  "data": {
6    "result": {
7      "depth_image_path": "/tmp/mediaprocessor_20260925_174610_depth.png"
8    }
9  }
10}

image_segmentation Image Segmentation

K-means clustering in LAB colour space. Returns a composite first, then one mask image per segment.

Fields in `data`

Field	Type	Default	Meaning
inputFile	string	required	Path to the image on the server host.

Request

1{
2  "id": "req-1",
3  "type": "image_segmentation",
4  "data": {"inputFile": "/srv/photos/street.jpg"}
5}

Response

1{
2  "id": "req-1",
3  "success": true,
4  "message": "Image segmentation completed using K-means clustering - found 2 segments",
5  "data": {
6    "result": {
7      "masks": [
8        {
9          "class": "complete_segmentation",
10          "confidence": 1.0,
11          "mask_path": "/tmp/mediaprocessor_…_segmentation.png"
12        },
13        {
14          "class": "segment_0",
15          "confidence": 0.277,
16          "mask_path": "/tmp/mediaprocessor_…_segment_0.png"
17        }
18      ]
19    }
20  }
21}

image_keypoint_detection Keypoint Detection

Points of interest, strongest first: ORB (fast corners) or SIFT (blobs across scales). confidence is the detector response normalised so the strongest is 1; angle is -1 where the detector gives none. Also writes the image with the keypoints drawn.

Fields in `data`

Field	Type	Default	Meaning
inputFile	string	required	Path to the image on the server host.
method	string	"orb"	"orb" or "sift".
max_keypoints	number	500	How many to return, 1–5000.

Request

1{
2  "id": "req-1",
3  "type": "image_keypoint_detection",
4  "data": {
5    "inputFile": "/srv/photos/street.jpg",
6    "method": "orb",
7    "max_keypoints": 500
8  }
9}

Response

1{
2  "id": "req-1",
3  "success": true,
4  "message": "Keypoint detection completed - found 1 keypoints",
5  "data": {
6    "result": {
7      "method": "orb",
8      "keypoints": [
9        {
10          "x": 212.4,
11          "y": 88.0,
12          "confidence": 1.0,
13          "size": 31.0,
14          "angle": 97.3
15        }
16      ],
17      "rendered_image_path": "/tmp/mediaprocessor_…_keypoints.png"
18    }
19  }
20}

image_face_detection Face Detection

YuNet finds faces; SFace describes each one. Boxes are [x, y, width, height] in source pixels, surest first, with five landmarks (eyes, nose tip, mouth corners) and a 128-number embedding. Embeddings are L2-normalised: the dot product of two is their cosine similarity, and 0.363 is SFace's same-person threshold.

Fields in `data`

Field	Type	Default	Meaning
inputFile	string	required	Path to the image on the server host.
score_threshold	number	0.8	Minimum detection score, clamped to 0.3–0.99.

Request

1{
2  "id": "req-1",
3  "type": "image_face_detection",
4  "data": {"inputFile": "/srv/photos/street.jpg", "score_threshold": 0.8}
5}

Response

1{
2  "id": "req-1",
3  "success": true,
4  "message": "Face detection completed - found 1 faces",
5  "data": {
6    "result": {
7      "model": "yunet-2022mar+sface-2021dec",
8      "faces": [
9        {
10          "bbox": [512.0, 140.5, 88.0, 112.0],
11          "confidence": 0.94,
12          "landmarks": [
13            [535.1, 183.2],
14            [574.8, 182.6],
15            [556.0, 206.9],
16            [538.7, 229.4],
17            [571.2, 228.8]
18          ],
19          "embedding": [0.041, -0.087, 0.112, …]
20        }
21      ]
22    }
23  }
24}

Image Processing

image_resize Resize

ImageMagick resize. Keeps the aspect ratio inside the width × height box unless maintain_aspect_ratio is false.

Fields in `data`

Field	Type	Default	Meaning
image_path	string	required	Path to the image on the server host.
parameters.width	number	—	Target width in pixels.
parameters.height	number	—	Target height in pixels.
parameters.maintain_aspect_ratio	boolean	true	Fit inside the box rather than stretch to it.

Request

1{
2  "id": "req-1",
3  "type": "image_resize",
4  "data": {
5    "image_path": "/srv/photos/street.jpg",
6    "parameters": {"width": 800, "height": 600}
7  }
8}

Response

1{
2  "id": "req-1",
3  "success": true,
4  "message": "Image resize completed successfully",
5  "data": {
6    "transformed_image_path": "/tmp/mediaprocessor_…_resize.png",
7    "operation": "resize",
8    "original_dimensions": [2400, 1600],
9    "new_dimensions": [800, 533],
10    "format": "PNG",
11    "intensity": 1.0,
12    "background_color": "transparent"
13  }
14}

image_crop Crop

ImageMagick crop in source pixels, from the top-left corner.

Fields in `data`

Field	Type	Default	Meaning
image_path	string	required	Path to the image on the server host.
parameters.x / parameters.y	number	0	Top-left corner of the region.
parameters.width / parameters.height	number	the full image	Size of the region.

Request

1{
2  "id": "req-1",
3  "type": "image_crop",
4  "data": {
5    "image_path": "/srv/photos/street.jpg",
6    "parameters": {"x": 100, "y": 50, "width": 500, "height": 400}
7  }
8}

Response

1{
2  "id": "req-1",
3  "success": true,
4  "message": "Image crop completed successfully",
5  "data": {
6    "transformed_image_path": "/tmp/mediaprocessor_…_crop.png",
7    "operation": "crop",
8    "original_dimensions": [2400, 1600],
9    "new_dimensions": [500, 400],
10    "format": "PNG",
11    "intensity": 1.0,
12    "background_color": "transparent"
13  }
14}

image_rotate Rotate

ImageMagick rotate, in degrees clockwise. The canvas grows to hold the whole rotated image.

Fields in `data`

Field	Type	Default	Meaning
image_path	string	required	Path to the image on the server host.
parameters.angle	number	0	Degrees clockwise, -360 to 360.
parameters.background_color	string	"transparent"	Fill for the corners a rotation exposes.

Request

1{
2  "id": "req-1",
3  "type": "image_rotate",
4  "data": {
5    "image_path": "/srv/photos/street.jpg",
6    "parameters": {"angle": 90, "background_color": "transparent"}
7  }
8}

Response

1{
2  "id": "req-1",
3  "success": true,
4  "message": "Image rotate completed successfully",
5  "data": {
6    "transformed_image_path": "/tmp/mediaprocessor_…_rotate.png",
7    "operation": "rotate",
8    "original_dimensions": [2400, 1600],
9    "new_dimensions": [1600, 2400],
10    "format": "PNG",
11    "intensity": 1.0,
12    "background_color": "transparent"
13  }
14}

image_filter Filter

One ImageMagick filter: blur, sharpen, edge, emboss, enhance, equalize or normalize. radius and sigma shape the blur, sharpen and edge kernels.

Fields in `data`

Field	Type	Default	Meaning
image_path	string	required	Path to the image on the server host.
parameters.filter	string	"blur"	blur, sharpen, edge, emboss, enhance, equalize or normalize.
parameters.radius	number	1.0	Kernel radius.
parameters.sigma	number	0.5	Kernel standard deviation.

Request

1{
2  "id": "req-1",
3  "type": "image_filter",
4  "data": {
5    "image_path": "/srv/photos/street.jpg",
6    "parameters": {"filter": "edge", "radius": 1, "sigma": 0.5}
7  }
8}

Response

1{
2  "id": "req-1",
3  "success": true,
4  "message": "Image filter completed successfully",
5  "data": {
6    "transformed_image_path": "/tmp/mediaprocessor_…_filter.png",
7    "operation": "filter",
8    "original_dimensions": [2400, 1600],
9    "new_dimensions": [2400, 1600],
10    "format": "PNG",
11    "filter_type": "edge",
12    "intensity": 1.0,
13    "background_color": "transparent"
14  }
15}

image_composite Composite

Lays one image over another with ImageMagick: an offset, an opacity and a compose mode.

Fields in `data`

Field	Type	Default	Meaning
base_path	string	required	The bottom image (image_path is accepted too).
overlay_path	string	required	The image laid on top.
parameters.x / parameters.y	number	0	Where the overlay's top-left corner goes.
parameters.opacity	number	1.0	Overlay opacity, 0.0–1.0.
parameters.compose	string	"over"	over, multiply, screen or plus.

Request

1{
2  "id": "req-1",
3  "type": "image_composite",
4  "data": {
5    "base_path": "/srv/photos/street.jpg",
6    "overlay_path": "/srv/photos/logo.png",
7    "parameters": {"x": 40, "y": 40, "opacity": 0.8, "compose": "over"}
8  }
9}

Response

1{
2  "id": "req-1",
3  "success": true,
4  "message": "Image composite completed successfully",
5  "data": {
6    "transformed_image_path": "/tmp/mediaprocessor_…_composite.png",
7    "operation": "composite",
8    "original_dimensions": [2400, 1600],
9    "new_dimensions": [2400, 1600],
10    "format": "PNG",
11    "intensity": 1.0,
12    "background_color": "transparent"
13  }
14}

image_transform Transform (any of the above)

One entry point for resize, crop, rotate and filter: parameters.operation picks which, and the rest of parameters are that operation's.

Fields in `data`

Field	Type	Default	Meaning
image_path	string	required	Path to the image on the server host.
parameters.operation	string	"resize"	resize, crop, rotate or filter.
parameters.…	—	—	The chosen operation's parameters, as above.

Request

1{
2  "id": "req-1",
3  "type": "image_transform",
4  "data": {
5    "image_path": "/srv/photos/street.jpg",
6    "parameters": {"operation": "rotate", "angle": 45}
7  }
8}

Response

1{
2  "id": "req-1",
3  "success": true,
4  "message": "Image transform completed successfully",
5  "data": {
6    "transformed_image_path": "/tmp/mediaprocessor_…_rotate.png",
7    "operation": "rotate",
8    "original_dimensions": [2400, 1600],
9    "new_dimensions": [2828, 2828],
10    "format": "PNG",
11    "intensity": 1.0,
12    "background_color": "transparent"
13  }
14}

Video & Audio

video_effects Video Effect

One ffmpeg filter over the whole clip. Output is MP4.

Fields in `data`

Field	Type	Default	Meaning
inputFile	string	required	Path to the clip on the server host (file:// URLs are accepted).
effect	string	required	grayscale, sepia, negate, blur, sharpen, edge, vignette, hflip or vflip.
outputFile	string	a temp file	Where to write the result.

Request

1{
2  "id": "req-1",
3  "type": "video_effects",
4  "data": {"inputFile": "/srv/clips/walk.mp4", "effect": "grayscale"}
5}

Response

1{
2  "id": "req-1",
3  "success": true,
4  "message": "Video effect 'grayscale' applied",
5  "data": {
6    "outputFile": "/tmp/mediaprocessor_…_video_effects.mp4",
7    "operation": "video_effects",
8    "effect": "grayscale"
9  }
10}

video_stabilization Video Stabilization

ffmpeg deshake over the whole clip. Output is MP4.

Fields in `data`

Field	Type	Default	Meaning
inputFile	string	required	Path to the clip on the server host (file:// URLs are accepted).
outputFile	string	a temp file	Where to write the result.

Request

1{
2  "id": "req-1",
3  "type": "video_stabilization",
4  "data": {"inputFile": "/srv/clips/walk.mp4"}
5}

Response

1{
2  "id": "req-1",
3  "success": true,
4  "message": "Video stabilization completed",
5  "data": {
6    "outputFile": "/tmp/mediaprocessor_…_stabilized.mp4",
7    "operation": "video_stabilization"
8  }
9}

audio_extraction Audio Extraction

The clip's audio track, re-encoded. Fails on a clip with no audio track.

Fields in `data`

Field	Type	Default	Meaning
inputFile	string	required	Path to the clip on the server host (file:// URLs are accepted).
format	string	"mp3"	mp3, wav, m4a, flac or ogg.
outputFile	string	a temp file	Where to write the result.

Request

1{
2  "id": "req-1",
3  "type": "audio_extraction",
4  "data": {"inputFile": "/srv/clips/walk.mp4", "format": "mp3"}
5}

Response

1{
2  "id": "req-1",
3  "success": true,
4  "message": "Audio extraction completed",
5  "data": {
6    "outputFile": "/tmp/mediaprocessor_…_audio.mp3",
7    "operation": "audio_extraction",
8    "format": "mp3"
9  }
10}

Start Building with the API

Connect to the server and start processing media in any language that supports TCP sockets.

Getting Started Guide

API Reference

Protocol Overview

Operations

Computer Vision

Image Processing

Video & Audio

Computer Vision

Fields in data

Request

Response

Fields in data

Request

Response

Fields in data

Request

Response

Fields in data

Request

Response

Fields in data

Request

Response

Image Processing

Fields in data

Request

Response

Fields in data

Request

Response

Fields in data

Request

Response

Fields in data

Request

Response

Fields in data

Request

Response

Fields in data

Request

Response

Video & Audio

Fields in data

Request

Response

Fields in data

Request

Response

Fields in data

Request

Response

Start Building with the API

Fields in `data`

Fields in `data`

Fields in `data`

Fields in `data`

Fields in `data`

Fields in `data`

Fields in `data`

Fields in `data`

Fields in `data`

Fields in `data`

Fields in `data`

Fields in `data`

Fields in `data`

Fields in `data`