Importing JSON or CSV Data

This page shows how to load records exported from another system into tables on your Insites instance. You shape the data to match your tables, load it with the CLI, the GraphQL import mutations, or a migration, and then check the counts.

What You Need Before You Start

• An Insites instance you can deploy to, through Insites CloudShell or the Insites GitHub App.
• The exported data, and the number of records of each kind in it. You need these numbers at the end.
• For the CLI route, insites-cli installed and set up for the instance.
• A staging instance to try the import on first.

Step 1: Check What Already Stores It

An instance already has tables for contacts, companies, activities, tasks, opportunities, orders, products, events and locations. Put that kind of data into the module that owns it, not into a table of your own. You then get the module's admin screens, validation and API.

People who will sign in are users, not rows in a table. Load them with import_users, shown in Step 4.

Step 2: Shape the Data to Your Tables

Write one schema file for each kind of record. The name must match the file name, and it is also the table name you import into.

app/schema/member.yml

name: member
properties:
  - name: full_name
    type: string
  - name: email
    type: string
  - name: joined_on
    type: date
  - name: points
    type: integer
  - name: active
    type: boolean

• Use string for text of any length. The text type is deprecated.
• Deploy the schema files before you import. Deploying a schema file creates the table.
• Keep each record's id from the old system. The import stores it as the record's external_id, so you can find the record again later.

Step 3: Load It With the CLI

insites-cli data import staging --path data.json

• The file must be JSON. A file that is not valid JSON is refused, including a CSV file. Convert CSV rows to JSON first.
• The CLI also takes a .zip archive with --zip. insites-cli data export writes this kind of archive by default, so an export from one instance can be imported into another.
• The file is a JSON object with a models list for table rows and a users list for people.
• Each row in models has a type_name, which is the table name such as member, and a properties object of field names and values. Keep properties flat: one level of field names, each with a plain value.
• Ids from your file are remapped to new ids by default. --raw-ids true keeps them.

The CLI waits until the import job finishes. If the job fails, check the instance logs.

Step 4: Or Load It With GraphQL

Use the import mutations when a script sends the data, for example in batches. Call the admin GraphQL API at https://<instance>/api/graph with a Console CLI token. See the Logic Engine for how to get one.

mutation import_rows($models: [CustomizationImport!]!) {
  import_models(models: $models) {
    ids
    external_ids
  }
}

Each item in models takes type_name, id, properties, and, if you want to keep the old dates, created_at and updated_at. Use import_users the same way for people. Each user needs an email and a slug.

These behaviors were measured on Insites instances. Plan around them:

• Your id becomes the external_id. The instance creates a new id for each row. An external_id sent beside the id is ignored.
• Match ids to external_ids, not to the order you sent. The answer holds both lists. import_users has answered in a different order from the people sent.
• Sending the same batch twice makes duplicates. Each row is created a second time. If an answer is lost, for example on a timeout, do not send the batch again. Read the table, filtered by external_id with value_in, and check whether every id is already there.
• Old dates are kept. created_at and updated_at sent on each row are saved on the new row.
• A deleted row keeps its id. If you delete rows and import them again, they come back with new ids.

Step 5: Or Seed a Few Rows With a Migration

For a small fixed list, such as ten membership levels, a migration is simpler. For large imports, use the CLI or GraphQL instead. Create a migration with insites-cli migrations generate staging seed_membership_levels. A pending migration runs on deploy, and it runs only once.

app/graphql/records/create.graphql

mutation create($table: String!, $properties: [PropertyInputType]) {
  record_create(record: { table: $table, properties: $properties }) { id }
}

app/migrations/20261009120000_seed_membership_levels.liquid

{% liquid
  assign levels = 'Bronze,Silver,Gold' | split: ','
  for level in levels
    assign name_field = {}
    assign name_field.name = 'name'
    assign name_field.value = level
    assign props = []
    assign props = props | add_to_array: name_field
    graphql saved = 'records/create', table: 'membership_level', properties: props
  endfor
%}

Step 6: Check the Counts

Compare the number of records on the instance with the number in your export. Do this after every import, and before you run any import again.

app/graphql/records/count.graphql

query count($table: String!) {
  records(per_page: 1, filter: { table: { value: $table } }) {
    total_entries
  }
}

{% graphql result = 'records/count', table: 'member' %}
{{ result.records.total_entries }}

When you read records page by page to check them, always sort by a unique field such as id. Without a sort, a paged read can repeat some rows and miss others.

Next Steps

• Imported people can sign in without a password. See Passwordless Sign-In.
• Read how Insites works before you build, in the Insites Logic Engine.