Get started

This walkthrough creates one repo-owned DuckDB from one checked-in SQL Server query. You will validate the complete contract before Skiff reads and publishes data.

Before you begin

You need:

a Skiff source checkout and a Rust toolchain;
a SQL Server hostname, database, and login;
SELECT permission for the exact source data you intend to copy;
a repository where the manifest, queries, and checks will live.

Build and install the current checkout:

cargo install --path /path/to/skiff --locked
skiff --version

For a repeatable install, use a tagged checkout. These docs describe Skiff 0.6.0; tagged versions and release notes are available on GitHub.

Create the project files

From the repository that will own the DuckDB:

mkdir -p queries checks data

Create skiff.toml:

version = 1

[[sources]]
name = "reporting"
host = "sql.example.com"
database = "Reporting"
user = "skiff_reader"
password_env = "REPORTING_DB_PASS"

[output]
file = "data/reporting.duckdb"

[limits]
max_rows = 500000
timeout_secs = 60

[[queries]]
source = "reporting"
file = "queries/incidents.sql"
table = "incidents"

[[checks]]
name = "incident ids are unique"
file = "checks/incident_ids.sql"

Create queries/incidents.sql:

SELECT
    id,
    opened_at,
    priority
FROM dbo.incidents;

Create checks/incident_ids.sql:

SELECT id
FROM incidents
GROUP BY id
HAVING count(*) > 1;

A check passes when it returns zero rows. Checks run against the completed private DuckDB before publication.

Load the password

Skiff reads the variable named by password_env; it does not read .env files itself.

For one shell session:

export REPORTING_DB_PASS='your-password'

For a local .env workflow, keep the file out of source control and load it through the shell:

cp .env.example .env
set -a
source .env
set +a

Validate before loading data

Run check from the repository or any child directory:

skiff check

Skiff validates the manifest and file paths, parses each query, requires every password variable, and tests every source connection. A successful check names the source, query, and check counts plus the resolved output path.

Publish the DuckDB

skiff refresh

Progress is written to stderr as each table finishes. The final stdout line reports the number of tables, rows loaded during this run, total rows in the published file, and its path.

Open the destination read-only in downstream tools. The database also contains _skiff_refresh, a single metadata row for the current generation.

If the refresh fails

Read the Action: line in the error. Query, source, limit, or check failures occur before publication, so the prior DuckDB is left untouched. Correct the reported cause and run skiff check or skiff refresh again.

Next, learn how refresh publication works or configure incremental loads.