Frequently Asked Questions

💡 Note: You can perform same actions directly from the API as you can with our CLI.

SCP - Scramjet Cloud Platform
STH - Scramjet Transform Hub
STH is the equivalent of Hub on the SCP.

1. Ways to debug locally

• Detailed debugging:

  To get the hands down most out of the debugging, you should run Transform Hub locally and deploy sequences there:
  Transform Hub installation.
  You run it like so:

  DEVELOPMENT=1 sth --runtime-adapter=process

  --runtime-adapter flag controls runtime mode. There are 3 options available: process, docker, kubernetes.

  Use 'process' to launch STH without docker container, making it significantly easier to track the execution of your sequences.

• Logs:

  You may want to view logs directly from the Hub because they contain loads of information on what's going on in the background.

  ❗ To communicate with the Cloud Platform API, you have to provide authorization Bearer Token in Headers - read more.

  • using our CLI:
  si hub logs
  • directly through API (e.g. using curl):
  curl https://api.beta.scramjet.cloud/api/v1/space/<spaceId>/api/v1/sth/<hubId>/api/v1/logs --header 'Authorization: Bearer <your API key>' # or `stdout` instead of `log`

  You can also view logs just from an Instance:

  • using our CLI:
  si inst log <instanceId> si inst stdout <instanceId> # captures console.log() and console.err()
  • directly through API:
  curl https://api.beta.scramjet.cloud/api/v1/space/<spaceId>/api/v1/sth/<hubId>/api/v1/instance/<instanceId>/log --header 'Authorization: Bearer <your API key>' # or `stdout` instead of `log`

  More about endpoints: https://docs.scramjet.org/platform/api-reference#inputoutput-io

  Example output of the inst log command:

  {"msg":"Hookup streams","level":"TRACE","ts":1670668850783,"from":"CSIController","CSIController":{"id":"fb9deb9b-bcbd-4788-a6e6-ef90e104987u"}} {"msg":"PING received","level":"DEBUG","ts":1670668850787,"from":"CSIController","CSIController":{"id":"fb9deb9b-bcbd-4788-a6e6-ef90e104987u"},"data":[[3000,{}]]} {"msg":"Received a PING message but didn't receive ports config","level":"TRACE","ts":1670668850788,"from":"CSIController","CSIController":{"id":"fb9deb9b-bcbd-4788-a6e6-ef90e104987u"}} {"msg":"Instance started","level":"INFO","ts":1670668850788,"from":"CSIController","CSIController":{"id":"fb9deb9b-bcbd-4788-a6e6-ef90e104987u"},"data":[{"created":"2022-01-01T10:40:50.690Z","started":"2022-01-01T10:40:50.788Z"}]} {"msg":"Connected to host","level":"DEBUG","ts":1670668850783,"from":"HostClient","HostClient":{"id":"fb9deb9b-bcbd-4788-a6e6-ef90e104987u"},"Runner":{"id":"fb9deb9b-bcbd-4788-a6e6-ef90e104987u"}}

  Logs can also be fetched from your Space as follows:

  curl https://api.beta.scramjet.cloud/api/v1/space/<spaceId>/api/v1/log --header 'Authorization: Bearer <your API key>'

• Built-in logger:

  You may also use our logger inside your code:

  this.logger.debug(`Creating GitClient`);

  Example usage: send-to-github

2. Exchanging data between Instances

• TOPICS:

  It is the recommended way of sharing data among 2 instances and most likely what you need. A topic can be consumed by numerous Instances and every one of them will receive the same data.
  > Read more about topics

• API Client:

  If you wish to use more than 1 data provider or consumer, you must use STH Api Client. It'll let your Instance access anything that API provides, including input/output of all other Instances and Topics.

  NOTE: As of now, this method is only possible while using STH hosted locally. We are working on implementing this functionality to the Runner out-of-the-box, so Instances could easily use as many Topics as needed.

3. Passing secrets to Instance

• as an argument while starting/deploying Sequence:

  si seq start <sequenceId> --args [\"SECRET\"] # if using 'deploy', you pass them the same way: si seq deploy <folderPath> --args [\"SECRET\"] Examples on github: #1 #2.

• from a config file:

  si seq start <sequenceId> -f config.json # or si seq deploy <folderPath> -f config.json sample config.json file: { "RAPID_API_KEY": "YOUR_RAPID_API_KEY_GOES_HERE" }

• through a request to API:

  curl -X POST -H 'authorization: Bearer YOUR-ACCESS-TOKEN-HERE' https://api.beta.scramjet.cloud/api/v1/space/<spaceId>/api/v1/sth/<hubId>/api/v1/sequence/<sequenceId>/start \ -H 'content-type: application/json' \ --data-raw '{ "appConfig": {}, "args": [] # arguments to pass }'

4. Webhooks and Instances

Unfortunately, we don't have a reasonable solution at the moment but stay around and look out for updates! 👀

5. Data storage in user program

Data can be stored in memory or saved to a local volume.
If the data should be retrievable via the API, it's best to have it saved to an endpoint stream that API exposes, e.g. /log or /stdout.
If you wish to store the data in a long term, then you should use a database. We do not have integration with any of databases out-of-the-box (as of now), so it would have to be configured on the user's program side.

6. Instance lifecycle management

❗ To communicate with the Cloud Platform API, you have to provide authorization Bearer Token in Headers - read more.

List all of the Instances (regardless of their status):

• using our CLI:

si inst ls

• directly through API (e.g. using curl):

curl https://api.beta.scramjet.cloud/api/v1/space/<spaceId>/api/v1/sth/<hubId>/api/v1/instances --header 'Authorization: Bearer <your API key>'

More information about Instance lifecycle and errors can be tracked with Hub logs (see #1.)