Application Protocol Interface (API) Reference
- Introduction
- Space operations
- Hub operations
- Sequence operations
- Instance operations
- Topics operation on data
Introduction
Scramjet Cloud Platform API supports the following methods:
- [ GET ] - all operations requesting data from the Scramjet Cloud Platform (SCP),
- [ POST ] - creating new Hub, Sequence, Sequence Instances or for messaging and streaming data to a Sequence,
- [ DELETE ] - removeing operations eg. Hub or Sequence.
API Base URL examples
# Self Hosted Hub
http://127.0.0.1:11000/api/v1/
# Scramjet Cloud Platform Space
https://api.beta.scramjet.cloud/api/v1/space/:id/api/v1
# Scramjet Cloud Platform Hub
https://api.beta.scramjet.cloud/api/v1/space/:id/api/v1/sth/:id/api/v1/
To communicate with the Cloud Platform API, please remember to provide authorization Bearer Token in Headers.
Example cURL
curl -s -k -X GET https://api.beta.scramjet.cloud/api/v1/space/:id/api/v1/list -H 'accept: */*' -H 'authorization: Bearer {Token}' -H 'content-type: application/json'
No token provided or wrong token response
{
"message": "Forbidden"
}
Space operations
API Base SCP Space URL
https://api.beta.scramjet.cloud/api/v1/space/:id/api/v1
[ GET ] {API Base}/version
- show space version
Parameters
No parameters
Responses
Status: 200
- Success
{
"service": "@scramjet/manager",
"apiVersion": "v1",
"version": "0.22.0",
"build": "8b39d26"
}
[ GET ] {API Base}/load
- show space load info
Parameters
No parameters
Responses
Status: 200
- Success
{
"avgLoad": 0.15,
"currentLoad": 10.34503968983302,
"memFree": 27063754752,
"memUsed": 16267390976,
"fsSize": [
{
"fs": "/dev/mapper/ubuntu--vg-ubuntu--lv",
"type": "ext4",
"size": 51716325376,
"used": 11687919616,
"available": 37864574976,
"use": 23.59,
"mount": "/etc/hosts"
}
]
}
[ GET ] {API Base}/list
- list space Hubs
Parameters
No parameters
Responses
Status: 200
- Success
[
{
"id": "sth-6cd78f999b-ntdtw",
"info": {
"created": "2022-04-07T11:42:59.185Z",
"lastConnected": "2022-04-07T11:42:59.187Z"
},
"healthy": true,
"isConnectionActive": true
}
]
[ GET ] {API Base}/sth/:id/info
- show the Hub info
Parameters
No parameters
Responses
Status: 200
- Success
{
"id": "sth-0",
"info": {
"created": "2022-07-08T08:54:53.394Z",
"lastConnected": "2022-07-18T15:10:37.567Z",
"lastDisconnected": "2022-07-18T15:10:37.323Z"
},
"healthy": true,
"isConnectionActive": true
}
Hub operations
API Base SCP Hub URL
https://api.beta.scramjet.cloud/api/v1/space/:id/api/v1/sth/:id/api/v1/
[ GET ] {API Base}/version
- show the Hub version
Parameters
No parameters
Responses
Status: 200
- Success
{
"service": "@scramjet/host",
"apiVersion": "v1",
"version": "0.24.2",
"build": "source"
}
[ GET ] {API Base}/config
- show the Hub config
Parameters
No parameters
Responses
Status: 200
- Success
{
"logLevel": "trace",
"logColors": true,
"cpmUrl": "mm-api.org-5fa96f7c-058h-9o59-8645-y7d4b0af2717.svc.cluster.local:11000",
"cpmId": "org-5fa96f7c-058h-9o59-8645-y7d4b0af2717-manager",
"docker": {
"prerunner": {
"image": "scramjetorg/pre-runner:0.24.2",
"maxMem": 128
},
"runner": {
"image": "",
"maxMem": 512,
"exposePortsRange": [30000, 32767],
"hostIp": "0.0.0.0"
},
"runnerImages": {
"python3": "scramjetorg/runner-py:0.24.2",
"node": "scramjetorg/runner:0.24.2"
}
},
"identifyExisting": false,
"host": {
"hostname": "0.0.0.0",
"port": 8000,
"apiBase": "/api/v1",
"instancesServerPort": 8001,
"infoFilePath": "/tmp/sth-id.json",
"id": "sth-0"
},
"instanceRequirements": {
"freeMem": 256,
"cpuLoad": 10,
"freeSpace": 128
},
"safeOperationLimit": 512,
"instanceAdapterExitDelay": 9000,
"runtimeAdapter": "kubernetes",
"startupConfig": "",
"exitWithLastInstance": false,
"heartBeatInterval": 10000,
"kubernetes": {
"namespace": "org-5fa96f7c-058h-9o59-8645-y7d4b0af2717",
"sthPodHost": "sth-0.msth.org-5fa96f7c-058h-9o59-8645-y7d4b0af2717.svc.cluster.local",
"runnerImages": {
"python3": "scramjetorg/runner-py:0.24.2",
"node": "scramjetorg/runner:0.24.2"
},
"timeout": "0",
"runnerResourcesRequestsCpu": "0.01",
"runnerResourcesRequestsMemory": "8M",
"runnerResourcesLimitsCpu": "0.565",
"runnerResourcesLimitsMemory": "1024MB"
}
}
[ GET ] {API Base}/load-check
- monitor CPU, memory and disk usage metrics on the Hub machine
Parameters
No parameters
Responses
Status: 200
- Success
{
"avgLoad": 0.04,
"currentLoad": 2.34338747099768,
"memFree": 26667257856,
"memUsed": 16138383360,
"fsSize": [
{
"fs": "/dev/mapper/ubuntu--vg-ubuntu--lv",
"type": "ext4",
"size": 51716325376,
"used": 11699834880,
"available": 37852659712,
"use": 23.61,
"mount": "/etc/hosts"
}
]
}
[ GET ] {API Base}/log
- show logs form Hub
Parameters
No parameters
Responses
Content-type: application/octet-stream
Status: 200
- Success
2021-11-19T16:04:47.094Z log (object:Hub) Hub main called.
2021-11-19T16:04:47.100Z info (object:SocketServer) Server on: /tmp/scramjet-socket-server-path
2021-11-19T16:04:47.104Z info (object:Hub) API listening on: 127.0.0.1:8000
2021-11-19T16:05:08.228Z info (object:Hub) New sequence incoming...
2021-11-19T16:05:08.229Z log (object:LifecycleDockerAdapterSequence) Docker sequence adapter init.
2021-11-19T16:05:08.229Z log (object:DockerodeDockerHelper) Checking image scramjetorg/pre-runner:0.12.2
2021-11-19T16:05:12.234Z info (object:LifecycleDockerAdapterSequence) Docker sequence adapter done.
2021-11-19T16:05:12.246Z log (object:LifecycleDockerAdapterSequence) Volume created. Id: c50fe4d3-89cc-4685-a82a-16cbc744733d
2021-11-19T16:05:12.246Z log (object:LifecycleDockerAdapterSequence) Starting PreRunner { image: 'scramjetorg/pre-runner:0.12.2', maxMem: 128 }
2021-11-19T16:05:13.536Z log (object:DockerodeDockerHelper) Checking image scramjetorg/runner:0.12.2
2021-11-19T16:05:16.670Z info (object:SequenceStore) New sequence added: c50fe4d3-89cc-4685-a82a-16cbc744733d
2021-11-19T16:05:16.672Z info (object:Hub) Sequence identified: {
container: {
image: 'scramjetorg/runner:0.12.2',
maxMem: 512,
exposePortsRange: [ 30000, 32767 ],
hostIp: '0.0.0.0'
},
name: '@scramjet/multi-outputs',
version: '0.12.2',
engines: {},
config: {},
sequencePath: 'index.js',
packageVolumeId: 'c50fe4d3-89cc-4685-a82a-16cbc744733d'
}
2021-11-19T16:05:16.691Z debug (object:Hub) Request date: 2021-11-19T16:05:08.239Z, method: POST, url: {API Base}/sequence, status: 202
Status 404 Not Found
Sequence operations
API Base SCP Space URL
https://api.beta.scramjet.cloud/api/v1/space/:id/api/v1
[ GET ] {API Base}/sequences
- show all Sequences on Space
Parameters
No parameters
Responses
Status: 200
- Success
["a54675bd-fed8-4b67-9c3e-47c3302083a7", "00539df4-d0e2-4190-8ea9-52a2a22c3122"]
API Base SCP Hub URL
https://api.beta.scramjet.cloud/api/v1/space/:id/api/v1/sth/:id/api/v1/
[ GET ] {API Base}/sequences
- show all Sequences saved on the Hub
Parameters
No parameters
Responses
Status: 200
- Success
[
{
"id": "a54675bd-fed8-4b67-9c3e-47c3302083a7",
"config": {
"type": "kubernetes",
"entrypointPath": "index.js",
"version": "0.13.1",
"name": "@scramjet/simple-counter-js",
"id": "a54675bd-fed8-4b67-9c3e-47c3302083a7",
"sequenceDir": "/root/.scramjet_k8s_sequences/a54675bd-fed8-4b67-9c3e-47c3302083a7",
"engines": {
"node": ">=14"
}
},
"instances": []
},
{
"id": "00539df4-d0e2-4190-8ea9-52a2a22c3122",
"config": {
"type": "kubernetes",
"entrypointPath": "./index",
"version": "0.13.1",
"name": "@scramjet/hello",
"id": "00539df4-d0e2-4190-8ea9-52a2a22c3122",
"sequenceDir": "/root/.scramjet_k8s_sequences/00539df4-d0e2-4190-8ea9-52a2a22c3122",
"engines": {}
},
"instances": ["bde7e8a1-0aa6-41ae-b922-f29eaf7dbe26"]
}
]
[ POST ] {API Base}/sequence
- add new Sequence
Parameters
Name | Description | Type | Required |
---|---|---|---|
file | compressed package in tar.gz format | binary | yes |
appConfig | additional package.json config file | json | no |
Responses
Status: 202
- accepted
{
"id": "2c3068e5-7c74-45bb-a017-1979c41fc6d0" // sequence id
}
[ POST ] {API Base}/sequence/:id/start
- start chosen Sequence
Parameters
Name | Description | Type | Required |
---|---|---|---|
appConfig | additional package.json config file | json | no |
args | additional arguments that instance should starts with | json | no |
Responses
Status: 200
- Success
{
"result": "success",
"opStatus": "OK",
"id": "c9803353-f44e-4e75-8b2e-11a8b07924c7"
}
[ DELETE ] {API Base}/sequence/:id
- delete a Sequence by id
Parameters
No parameters
Responses
Status: 200
- Success
{
"opStatus": "OK",
"id": "a54675bd-fed8-4b67-9c3e-47c3302083a7"
}
Status 409
- conflict - the instance is still running
{
"error": "Can't remove sequence in use."
}
Instance operations
API Base SCP Space URL
https://api.beta.scramjet.cloud/api/v1/space/:id/api/v1
[ GET ] {API Base}/instances
- show list of Instances on Space
Parameters
No parameters
Responses
Status: 200
- Success
[
[
["66e0e68f-7ad8-4c90-9233-23748c5f2445"],
["bde7e8a1-0aa6-41ae-b922-f29eaf7dbe26"]
]
]
API Base SCP Hub URL
https://api.beta.scramjet.cloud/api/v1/space/:id/api/v1/sth/:id/api/v1/
[ GET ] {API Base}/instances
- show list of Instances running on the Hub
Parameters
No parameters
Responses
Status: 200
- Success
[
{
"id": "bde7e8a1-0aa6-41ae-b922-f29eaf7dbe26",
"appConfig": {},
"sequence": "00539df4-d0e2-4190-8ea9-52a2a22c3122",
"created": "2022-07-19T21:07:11.043Z",
"started": "2022-07-19T21:07:13.668Z",
"status": "running"
},
{
"id": "217ff6c1-2054-46e4-8f69-d5151cd313a7",
"appConfig": {},
"sequence": "00539df4-d0e2-4190-8ea9-52a2a22c3122",
"created": "2022-07-19T22:08:46.577Z",
"started": "2022-07-19T22:08:49.121Z",
"status": "running"
},
{
"id": "fe314445-ccd7-41f5-bff4-37163cd21940",
"appConfig": {},
"sequence": "a54675bd-fed8-4b67-9c3e-47c3302083a7",
"created": "2022-07-19T22:09:36.852Z",
"started": "2022-07-19T22:09:39.444Z",
"status": "running"
}
]
[ GET ] {API Base}/instance/:id
- show data of chosen Instance
Parameters
No parameters
Responses
Status 200 Accepted
{
"id": "bde7e8a1-0aa6-41ae-b922-f29eaf7dbe26",
"appConfig": {},
"sequence": "00539df4-d0e2-4190-8ea9-52a2a22c3122",
"created": "2022-07-19T21:07:11.043Z",
"started": "2022-07-19T21:07:13.668Z",
"status": "running"
}
Status 404 Not Found - when the Instance is not found, for example: the Instance was already stopped.
[ POST ] {API Base}/instance/:id/_stop
- end instance gracefully and prolong operations or not for task completion
Parameters
Name | Description | Type | Required |
---|---|---|---|
timeout | The number of milliseconds before the Instance will be killed. Default: 7000ms. | number | no |
canCallKeepalive | If set to true, the instance will prolong the running. Default: false. | boolean | no |
Responses
Status: 200
- Success
{
"code": 0,
"type": "string",
"message": "string"
}
[ POST ] {API Base}/instance/:id/_kill
- end instance gracefully waiting for unfinished tasks
Parameters
No parameters
Responses
Status: 202
- accepted
{
"opStatus": "Accepted",
"id": "bde7e8a1-0aa6-41ae-b922-f29eaf7dbe26",
"appConfig": {},
"sequence": "00539df4-d0e2-4190-8ea9-52a2a22c3122",
"created": "2022-07-19T21:07:11.043Z",
"started": "2022-07-19T21:07:13.668Z",
"status": "killing"
}
[ GET ] {API Base}/instance/:id/health
- check status about Instance health
Parameters
No parameters
Responses
Status: 200
- Success
{
"healthy": true
}
Events
Cloud Platform allows interaction (communication) with an instance by sending events with or without any information defined in the object. Furthermore, it lets providing and receiving streams with all kinds of data files.
Event contains <eventName>
, <handler>
with optional <message>
of any type: string, num, json obj, array, etc..
[ POST ] {API Base}/instance/:id/_event
- send event to the Instance
Parameters
Name | Type | Description | Required |
---|---|---|---|
eventName |
string |
Name of an event | true |
message |
string |
JSON formatted event payload | false |
Responses
Content-type: application/octet-stream
[ GET ] {API Base}/instance/:id/event
- get the data stream with the events from the Instance
Parameters
No parameters
Responses
Content-type: application/octet-stream
[ GET ] {API Base}/instance/:id/once
- get the last event sent by the Instance
Parameters
No parameters
Responses
Content-type: application/octet-stream
Input/Output (I/O)
The data can be sent to the input stream of the Instance where it can be consumed. Everything the Instance writes to its output stream also can be consumed through the endpoint.
[ POST ] {API Base}/instance/:id/input
- send data to the Instance
Parameters
No parameters
Responses
Status: 200
- Success
Status: 406
- Not Acceptable - with an error message "Input provided in other way." This status code is returned when the Instance expects the input to be provided from the Topic API.
[ GET ] {API Base}/instance/:id/output
- get the output stream from the Instance
Parameters
No parameters
Responses
Content-type: application/octet-stream
[ POST ] {API Base}/instance/:id/stdin
- process.stdin
Parameters
No parameters
Responses
Status: 200
- Success
[ GET ] {API Base}/instance/:id/stdout
- process.stdout
Parameters
No parameters
Responses
Content-type: application/octet-stream
[ GET ] {API Base}/instance/:id/stderr
- process.stderr
Parameters
No parameters
Responses
Content-type: application/octet-stream
[ GET ] {API Base}/instance/:id/log
- stream all instance logs
Parameters
No parameters
Responses
Content-type: application/octet-stream
Status: 200
- Success
2021-11-19T16:12:22.948Z log (Sequence) 42
2021-11-19T16:12:23.949Z log (Sequence) 41
2021-11-19T16:12:24.950Z log (Sequence) 40
2021-11-19T16:12:25.951Z log (Sequence) 39
2021-11-19T16:12:26.952Z log (Sequence) 38
2021-11-19T16:12:27.952Z log (Sequence) 37
2021-11-19T16:12:28.953Z log (Sequence) 36
2021-11-19T16:12:29.953Z log (Sequence) 35
Status 404 Not Found
Topics operation on data
API Base SCP Hub URL
https://api.beta.scramjet.cloud/api/v1/space/:id/api/v1/sth/:id/api/v1/
Type of Headers
-
"x-end-stream" - close topic stream [optional, boolean]. If x-end-stream header value is true, the topic stream is closed after processing this request. The default value is false.
-
"content-type" - specify stream content type [optional, boolean]. The content-type header specifies data type of this topic. The recognized values are: text/x-ndjson, application/x-ndjson, application/x-ndjson, text/plain, application/octet-stream. The default value is application/x-ndjson.
[ POST ] {API Base}/topic/:name
- sends data to the Topic, if it does not exist, the Topic is created
Parameters
No parameters
Request Headers
Content-type: application/x-ndjson
Responses
Status: 200
- Success - when data to topic is sent with the header indicating the end of data
Status: 202
- accepted - when data to topic is sent without the header indicating the end of data (default)
[ GET ] {API Base}/topic/:name
- get data from the Topic
Parameters
No parameters
Responses
Topic data stream.
Status: 200
- Success
{
"source": "Twitter",
"id": "850006245121695778",
"content": "Natural wetlands make up ~30% of global total CH4 emissions",
"user": {
"id": 1234994945,
"name": "Climate Change Conference",
"screen_name": "Climate Change"
}
}
[ GET ] {API Base}/topis
- list Hub Topics
Parameters
No parameters
Responses
Status: 200
- Success
[
{
"contentType": "application/x-ndjson",
"stream": {
"_readableState": {
"objectMode": false,
"highWaterMark": 16384,
"buffer": {
"head": {
"data": {
"type": "Buffer",
"data": [
123,
10,
32,
32,
32
//..
]
},
"next": {
"data": {
"type": "Buffer",
"data": [10]
},
"next": null
}
},
"tail": {
"data": {
"type": "Buffer",
"data": [10]
},
"next": null
},
"length": 2
},
"length": 51,
"pipes": [],
"flowing": null,
"ended": false,
"endEmitted": false,
"reading": true,
"constructed": true,
"sync": false,
"needReadable": true,
"emittedReadable": false,
"readableListening": false,
"resumeScheduled": false,
"errorEmitted": false,
"emitClose": true,
"autoDestroy": true,
"destroyed": false,
"errored": null,
"closed": false,
"closeEmitted": false,
"defaultEncoding": "utf8",
"awaitDrainWriters": null,
"multiAwaitDrain": false,
"readingMore": false,
"dataEmitted": false,
"decoder": null,
"encoding": null
},
"_events": {},
"_eventsCount": 1,
"_writableState": {
"objectMode": false,
"highWaterMark": 16384,
"finalCalled": false,
"needDrain": false,
"ending": false,
"ended": false,
"finished": false,
"destroyed": false,
"decodeStrings": true,
"defaultEncoding": "utf8",
"length": 0,
"writing": false,
"corked": 0,
"sync": false,
"bufferProcessing": false,
"writecb": null,
"writelen": 0,
"afterWriteTickInfo": null,
"buffered": [],
"bufferedIndex": 0,
"allBuffers": true,
"allNoop": true,
"pendingcb": 0,
"constructed": true,
"prefinished": false,
"errorEmitted": false,
"emitClose": true,
"autoDestroy": true,
"errored": null,
"closed": false,
"closeEmitted": false
},
"allowHalfOpen": true
},
"localProvider": "api",
"topic": "currency"
}
]