• Handler
• AppContext
• Package
• Deploy
• Logs

Sequence Development Guide for Python

---

Scramjet's Sequences are usually the user's source code with all its required dependencies packed and ready to be uploaded, executed on Scramjet Cloud Platform. With this set of algorithmic steps a Sequence can produce, consume and transform data with the enhancement of its abilities through Scramjet Framework. Scramjet's Sequences allow the user to asynchronously iterate over data on demand. The following sections will insure a proper buildup of a Sequence with all its requirements ready to be executed on Scramjet's Cloud Platform. In this section You as a user will be guided through a series of steps which will insure the execution of your Python Sequence successfully on Scramjet Cloud Platform.

Handler

Scramjet Cloud Platform using the following method to process the context argument in order to return a response which can be also used by other sequences. keeping in mind that Events can be triggered while a Sequence is running and Topics can be produced and consumed as part of a Sequence.

def run(context, input): # your code here, you can also process an input stream return output # return stream value

AppContext

By running the asynchronous function on SCP, the object of this interface is passed through to the Application context which allows it to communicate with Scramjet Cloud Platform to ensure that it's in operation and should be kept alive without interruption. Methods of the class AppContext will be invoked only once the Sequence is uploaded and run on SCP. AppContext methods will not work properly if the Sequence is run on your local machine unless being run though STH. Available methods of this class can be found on Scramjet Github repository.

Packaging

For a Sequence to be deployed on Scramjet Cloud Platform (SCP), the script and all its dependencies must be packed in a tar.gz file before being uploaded to SCP. One or multiple files must be included in the same directory in order for a Sequence to be packed. the following points are the basic structural components required for a Sequence to be run on SCP. Running npm run Build is preferable before packing a Sequence. Samples and templates are found under the Example section.

• main.py - The main file with the user's logic
  

• Files - In case the user's Sequence requires any additional files it is mandatory to add those files to the same directory alongside the rest of requirements.
  

• requirements.txt - prerequisite Libraries and their versions needed for packaging a Python program.

  scramjet-framework-py pyee
• package.json - The manifest file that describes this particular Sequence.

  To create a package.json file with the values that you supply, use the following command-line
  npm init - creates a new package.json file with elementary properties. Contains prompts about the project , such as the name, version, etc. for more information on package.json. A brief illustration is shown in the below section.

  { "name": "SequenceName", "version": "0.23.0", "main": "./main.py", "author": "USER", "license": "GPL-3.0", "description": "what is it about", "keywords": [ "streaming", "data transformation", "Data Transformer" ], "repository": { "type": "git", "url": "https://github.com/" }, "engines": { "python3": "3.8.0" }, "scripts": { "build": "mkdir -p dist/__pypackages__/ && pip3 install -t dist/__pypackages__/ -r requirements.txt && cp -t ./dist/ *.py *.json *.wav *.mp3", "pack": "si seq pack ./dist/", "clean": "rm -rf ./dist ./*.tar.gz" }, "assets": [ "song.wav" ], "dependencies": { "@scramjet/cli": "^0.31.2", "@scramjet/types": "^0.31.2" } }
• __pypackages__ - all dependencies that are needed to execute the Sequence.

  • 1. create directory __pypackages__ in the same directory as main.py

  mkdir __pypackages__

  • 2. Installing dependencies in the __pypackages__ folder. If the user uses any packages that are written in C in order to run them on SCP a user needs to install the dependencies on a Linux Machine. Simply because Scramjet Cloud Platform run on Linux OS.

  pip3 install -t __pypackages__ -r requirements.txt

Deploy

A user can manually pack a Sequence after installing all dependencies needed in the form of a tar.gz file by using
si sequence pack <path/to/folder/dist> command and then send a Sequence to the Scramjet Cloud Platform by using
si sequence send <path/to/filename.tar.gz> --progress command. Or a user can deploy a Sequence on SCP by using the deploy command
si sequence deploy|run [options] <path/to/folder> which will pack, send and start the Sequence in one go and return both Sequence and Instance ids. More information is found under the CLI section.

Logs

Scramjet Cloud Platform is constantly monitoring the user's Sequence, and producing log output for a Sequence which can be accessible. Details of each invoked Sequence can be viewed through different methods. More information on logging library used by Scramjet.

• SCP Console Panel > Space > Show logs > Download
  

• Returns the log through calling the methods of the logging library inside the main.py file. In order to be retrieved use /instance/<Instance-id>/log API endpoint. The below example will provide a brief illustration.
  

  import asyncio async def run(context, input): # your code here, you can also process an input stream context.logger.debug('Debug log message') context.logger.info('Info log message') context.logger.warn('Warn log message') await asyncio.sleep(10) return output # return stream value

• Returns the log through calling the stderr method inside the main.py file. In order to be retrieved through /instance/<Instance-id>/stderr API endpoint, or through CLI as
  si instance stderr <Instance-id>. The below example will provide a brief illustration.
  

  import sys async def run(context, input): # your code here, you can also process an input stream sys.stderr.write("Error") return output # return stream value

• For displaying the content through print a user can use /instance/<Instance-id>/stdout API endpoint, or through CLI as
  si instance stdout <Instance-id>. The below example will provide a brief illustration.
  

  async def run(context, input): # your code here, you can also process an input stream print("Hello Alice") return output # return stream value

• For displaying the content through output a user can use /instance/<Instance-id>/output API endpoint, or through CLI as
  si instance ouput <Instance-id>. The below example will provide a brief illustration.
  

  from scramjet.streams import Stream async def run(context, input): # your code here, you can also process of an input stream return Stream.read_from("Hello Alice") # return stream value

---