Pack and Send

Prerequisites

1. Install Node.js version 18.x LTS
2. Install si Scramjet's Command Line Interface

---

For the Sequence deployment on Scramjet's Transform Hub (STH) or the Cloud Platform (SCP), the script and all its dependencies packed in a tar.gz format before being uploaded. One or multiple files might be required to be in the same directory of the Sequence folder. The below points are the essential structural components of a Sequence to run on STH.

Required Sequence files

JavaScript

• index.js - The main file with the user's logic

• Files - In case the user's Sequence requires any additional files it is mandatory to add those files to the same directory alongside the rest of requirements

• Dependencies - All dependencies that are needed to execute the Sequence. Dependencies must be in the same directory alongside the rest of required files which will be packed as a Sequence

• package.json - The manifest file that describes this particular Sequence

  To create a package.json file with the values that you supply, use the following command-line
  npm init - creates a new package.json file with elementary properties. Contains prompts about the project , such as the name, version, etc. for more information on package.json. A brief illustration is shown in the below section.

  {
  "name": "SequenceName",
  "version": "0.23.0",
  "main": "./index",
  "author": "USER",
  "license": "GPL-3.0",
  "description": "what is it about",
  "keywords": [
      "data transformation",
      "Data Transformer"
  ],
  "repository": {
      "type": "git",
      "url": "https://github.com/"
  },
  "engines": {
      "node": "^18.0.0"
  },
  "scripts": {
  "test": "echo \"Error: no test specified\" && exit 1",
  "build": "tsc -p tsconfig.json",
  "postbuild": "cp -r package.json dist/ && (cd dist && npm i --only=production) && cp src/utils/song.wav dist/utils && cp src/utils/config.json dist/utils ",
  "pack": "si seq pack ./dist/",
  "clean": "rm -rf ./dist ./*.tar.gz"
  },
  "assets": [
      "song.wav"
  ],
  "dependencies": {
          "@scramjet/utility": "^0.33.5",
          "@types/node": "^18.16.2",
          "axios": "^1.3.6",
          "date-and-time": "^2.4.3",
          "typescript": "^5.0.4"
      }
  }
  

TypeScript

• index.js - The main file with the user's logic

• Files - In case the user's Sequence requires any additional files it is mandatory to add those files to the same directory alongside the rest of requirements

• Dependencies - All dependencies that are needed to execute the Sequence. Dependencies must be in the same directory alongside the rest of required files which will be packed as a Sequence

• package.json - The manifest file that describes this particular Sequence

  To create a package.json file with the values that you supply, use the following command-line
  npm init - creates a new package.json file with elementary properties. Contains prompts about the project , such as the name, version, etc. for more information on package.json. A brief illustration is shown in the below section.

  {
  "name": "SequenceName",
  "version": "0.23.0",
  "main": "./index",
  "author": "USER",
  "license": "GPL-3.0",
  "description": "what is it about",
  "keywords": [
      "data transformation",
      "Data Transformer"
  ],
  "repository": {
      "type": "git",
      "url": "https://github.com/"
  },
  "engines": {
      "node": "^18.0.0"
  },
  "scripts": {
  "test": "echo \"Error: no test specified\" && exit 1",
  "build": "tsc -p tsconfig.json",
  "postbuild": "cp -r package.json dist/ && (cd dist && npm i --only=production) && cp src/utils/song.wav dist/utils && cp src/utils/config.json dist/utils ",
  "pack": "si seq pack ./dist/",
  "clean": "rm -rf ./dist ./*.tar.gz"
  },
  "assets": [
      "song.wav"
  ],
  "dependencies": {
          "@scramjet/utility": "^0.33.5",
          "@types/node": "^18.16.2",
          "axios": "^1.3.6",
          "date-and-time": "^2.4.3",
          "typescript": "^5.0.4"
      }
  }
  

• tsconfig.json - The indicator file for TypeScript.

  It is a crucial file for managing your project, it creates itself after running tsc --init command, which initializes the TypeScript project. This file is where you can set the compiler options. Basically this is an indicator for TypeScript, which says that the project in which this file lies and all the other sub folders should be managed by TypeScript and it also tells Typescript how to compile all the *.ts files in the project

  {
      "compilerOptions": {
      "lib": [ "ESNext" ],
      "target": "ESNext",
      "module": "CommonJS",
      "outDir": "./dist",
      "esModuleInterop": true,
      "skipLibCheck": true,
      "resolveJsonModule": true,
      }
  }
  
  note

  For Sequences using TypeScript language, running npm run Build is mandatory to transpile .ts files to .js before packing a Sequence.
  

Python

• main.py - The main file with the user's logic

• Files - In case the user's Sequence requires any additional files it is mandatory to add those files to the same directory alongside the rest of requirements

• requirements.txt - prerequisite Libraries and their versions needed for packaging a Python program

  scramjet-framework-py
  pyee

• package.json - The manifest file that describes this particular Sequence.

  To create a package.json file with the values that you supply, use the following command-line
  npm init - creates a new package.json file with elementary properties. Contains prompts about the project , such as the name, version, etc. for more information on package.json. A brief illustration is shown in the below section.

  {
  "name": "SequenceName",
  "version": "0.23.0",
  "main": "./main.py",
  "author": "USER",
  "license": "GPL-3.0",
  "description": "what is it about",
  "keywords": [
      "data transformation",
      "Data Transformer"
  ],
  "repository": {
      "type": "git",
      "url": "https://github.com/"
  },
  "engines": {
      "python3": "3.8.0"
  },
  "scripts": {
  "build": "mkdir -p dist/__pypackages__/ && pip3 install -t dist/__pypackages__/ -r requirements.txt && cp -t ./dist/ *.py *.json *.wav *.mp3",
  "pack": "si seq pack ./dist/",
  "clean": "rm -rf ./dist ./*.tar.gz"
  },
  "assets": [
      "song.wav"
  ],
  "dependencies": {
          "@scramjet/cli": "^0.31.2",
          "@scramjet/types": "^0.31.2"
      }
  }
  

• pypackages - all dependencies that are needed to execute the Sequence.

  • 1. create directory __pypackages__ in the same directory as main.py

  mkdir __pypackages__

  • 2. Installing dependencies in the __pypackages__ folder. If the user uses any packages that are written in C in order to run them on SCP a user needs to install the dependencies on a Linux Machine. Simply because Scramjet Cloud Platform run on Linux OS.

  pip3 install -t __pypackages__ -r requirements.txt

Pack a Sequence

1. A Sequence can be packed manually in the form of a tar.gz file before being sent to STH through the command-line:

   si sequence pack <path/to/Sequence/folder> 

2. To send the packed Sequence to Scramjet's Transform Hub or Cloud Platform:

   si sequence send <path/to/filename.tar.gz> --progress

Alternatively, the Sequence can be deploy (pack, send and run) on STH or SCP through the command-line:

si sequence deploy|run [options] <path/to/Sequence/folder>

info

In the case of TypeScript, after transpiling deploy command should point out to the ./dist folder path

si sequence deploy|run [options] <path/to/folder/dist>

​​Related resources

Check out Scramjet's Command lines Interface (CLI) section

Visit Scramjet's platform-samples GitHub repository for more examples of Sequences