SkySpark Integration

The two integration surfaces

There are two ways the server talks to SkySpark:

1. Filesystem — reads config/ under the SkySpark home to enumerate projects and extract user-defined functions without needing the instance to be running.
2. HTTP + Haystack — executes queries, evaluates expressions, and commits functions against a running instance.

Filesystem discovery runs at server start-up. HTTP calls happen on demand when a client invokes queryHaystack, executeAxonCode, or commitAxonFunction.

Auto-discovery

With skyspark.autoDiscover: true in the config, the server walks the SkySpark home directory at start-up:

/path/to/skyspark/
├── config/            ← enumerated
│   ├── project-a/
│   ├── project-b/
│   └── sensorRead/
└── ...

For each project folder found, the server:

1. Reads the project's metadata file to confirm it's a valid Haystack project.
2. Fetches the list of user-defined functions (func records).
3. Indexes those functions into the local Axon index, tagged with project:<name>.

The result: searchAxonExamples can surface functions from real SkySpark projects alongside the static library examples.

Function sync

Discovery only enumerates. Sync fetches the actual function source code.

With autoSyncFunctions: true, sync runs automatically on start-up. Otherwise a client can trigger it via discoverProjectFunctions:

{
  "tool": "discoverProjectFunctions",
  "arguments": { "project": "sensorRead" }
}

Concurrency

syncConcurrency (default 10) controls how many functions are fetched in parallel. Raise it for instances on a fast LAN, lower it for slow or rate-limited instances.

Versioning

With functionVersioning: true and maxVersions: 4 (defaults), every commitAxonFunction call:

1. Reads the current version from SkySpark.
2. Writes the new version.
3. Archives the previous version in .data/versions/<project>/<function>-v<N>.axon.
4. Prunes revisions older than maxVersions.

Restoring an old version is currently a manual file-copy back into SkySpark — there's no restoreFunctionVersion tool yet.

The primary-project concept

Execution and commit tools need to pick a single project. The primary project is that choice:

"primaryProject": {
  "instance": "localhost8080",
  "project":  "sensorRead"
}

Clients can switch it at runtime:

{ "tool": "setPrimaryProject", "arguments": { "instance": "localhost8080", "project": "plantRoom" } }

Subsequent executeAxonCode and commitAxonFunction calls target plantRoom until the next switch.

Running queries

queryHaystack takes a raw Haystack query string and returns the resulting grid as JSON:

{
  "tool": "queryHaystack",
  "arguments": {
    "query": "read(site)"
  }
}

Result:

{
  "meta": { "ver": "3.0" },
  "cols": [ { "name": "id" }, { "name": "dis" }, { "name": "site" } ],
  "rows": [
    { "id": "@p:sensorRead:r:1a2b", "dis": "Main Campus", "site": true }
  ]
}

Queries are executed with the primary project's authentication context.

Fallback credentials

If auto-discovery fails (wrong path, SkySpark not installed on the same host, etc.), the server falls back to the explicit connection block:

"fallback": {
  "host":     "skyspark.example.com",
  "port":     443,
  "project":  "demo",
  "username": "readonly",
  "password": "<use-env-var>",
  "protocol": "https"
}

Prefer environment-variable substitution for the password (SKYSPARK_PASSWORD) rather than committing it to the config file.

Troubleshooting connectivity

• SKYSPARK_UNREACHABLE — the instance isn't accepting connections at the configured host/port. Check curl http://<host>:<port>/about.
• SKYSPARK_AUTH_FAILED — credentials are wrong, or the user lacks permission for the target project.
• PROJECT_NOT_FOUND — the primary project doesn't exist on the instance. Run listSkySparkProjects to see what's available.