Transform Hub

---

Environment preperation

Scramjet's Transform Hub installation requires the following 3 step:

1. Setup a Linux machine (local UNIX/Linux OS, Vitrual Machine)
2. Install Docker on Linux machine
3. Install npm on the machine. Currently it is recommended to install Node.js version 18.x LTS or higher.

---

Transform Hub installation

On a new terminal execute the below command lines for Scramjet's Transform Hub (STH) installation:

• Install Scramjet Transform Hub and STH CLI:

  npm i -g @scramjet/sth @scramjet/cli

• Run Scramjet's Transform Hub:

  scramjet-transform-hub # or sth

More detailed information on how to install Scramjet's Transform Hub can be found at STH GitHub repository.

---

The Sequence

Scramjet's Sequences are usually the user's source code with all its required dependencies packed and ready to be executed on Scramjet's Transform Hub. With this set of algorithmic steps, a Sequence enhancement through Scramjet's Framework provides the ability to produce, consume and transform data. Scramjet's Sequences allow the user's demand to iterate over data asynchronously. This section illustrates a series of steps that will ensure the execution of your Typescript/JavaScript Sequence successfully on the Scramjet Transform Hub.

For demonstrational purposes, Scramjet's audio2text-input TypeScript sample found on GitHub provides a sneak preview of the prerequisite files a Sequence should contain to be able to be run on STH and, demonstrates the integration of AssemblyAI code for speech recognition in a Scramjet Sequence which can be run by a user through the below command. A user must include their AssemblyAI token once executing the start Sequence command. As a reminder escaping the quotation marks is a must for this command. For more information on si command-lines check CLI reference section.
Prerequisite files of a Sequence written in JavaScript:

• index.js - The main file with the user's logic
• package.json - manifest file that describes this particular Sequence

The audio2text-input was written in TypeScript. Therefore, transpiling is required before being packed and sent to STH as a Sequence. However, it is possible to write code of a Sequence in plain JavaScript or Python. The Templates section illustrates the content of each prerequisite file. The Sequence Development Guide section provides in-depth how to create a Sequence.

---

Run a Sequence

Five steps to deploy a Sequence on STH:

1. Pack a Sequence into a *.tar.gz package

Each Sequence must be packaged as a *.tar.gz before sending it to the Transform Hub. A Sequence is a TAR file or an archive, as it is only a collection of multiple files put together inside a single file. And a GZ file which is a compressed file zipped using the gzip algorithm.

si sequence pack <path/to/folder>

2. Start Scramjet's Transfrom Hub

Prior to sending a Sequence to Scramjet's Transform Hub, the STH must run by executing the below command lines on terminal #1. Before running Scramjet's Transform Hub, check whether the environment configuration setting is assigned to development by si config print; If Environment Configuration is for production, use the first of the command lines below.

si c set env development # to switch between the Cloud Platform and the local Transform Hub environment

Note: Three types of adapters for STH to run with. Determine the adapter while starting STH through --runtime-adapter <adapter> whether docker as the default one if pre-installed , process or kubernetes.

DEVELOPMENT=true scramjet-transform-hub # or sth

3. Send the Sequence package

While STH is still running on terminal #1, send the Sequence audio2text.tar.gz or any other Sequence by executing the below command line on terminal #2. A return output with a globally unique identifier number (GUID) assigned to this Sequence shall be used for starting the Sequence as an Instance once or multiple times.
Note: For debugging the request, use the below config command line.

si config set log --debug true $ si sequence send <path/to/filename.tar.gz> Request ok: http://127.0.0.1:8000/api/v1/sequence status: 202 Accepted SequenceClient { _id: 'xl1200x0-883l-883n-883c-1200v0o1200n', host: HostClient { apiBase: 'http://127.0.0.1:8000/api/v1' }, sequenceURL: 'sequence/xl1200x0-883l-883n-883c-1200v0o1200n' }

4. Run the Sequence

Adding the Sequence-id to the below command line on terminal #2 starts a Sequence while STH is still running on terminal #1. A return output with a globally unique identifier number (GUID) identifies the Instance-id. The hyphen alias (-) assumes the last Sequence-id.

$ si sequence start <Sequence-id> --args [\"<AssemblyAI-token>\"] # or $ si seq start - Request ok: http://127.0.0.1:8000/api/v1/sequence/xl1200x0-883l-883n-883c-1200v0o1200n/start status: 200 OK InstanceClient { host: HostClient { apiBase: 'http://127.0.0.1:8000/api/v1' }, _id: '117fxlrs-fxdl-fxbr-vrsc-hd1130vrscdx', instanceURL: 'instance/117fxlrs-fxdl-fxbr-vrsc-hd1130vrscdx' }

5. Send input data to the Sequence

An Instance can receive data as input after the start. For the sample audio2text-input, use terminal #2 to send the audio file to the Instance while STH runs on terminal #1.

si instance input <Instance-id> /path/to/file -e -t application/octet-stream

6. Get the Instance output

The output command line returns a value from an Instance if assigned. In the case of the audio2text-input sample, once the Instance has completed its task, a return output text is retrieved through the following command line.
Note: Instance Output is a streaming endpoint where data are consumed when they are read.

si instance output <Instance-id> | tee filename.txt

---