ImportAPI.com — Import Notes & Guides

ImportAPI.com

Sun, 20 Sep 2026 12:00:00 -0700

Every format. More possibilities.

Big ideas startwith better imports.

Spreadsheets, websites, documents, images, and AI-ready data. Find a clearer path from the information you have to the application you’re building.

Explore import APIs ↗View documentation

7 import categoriesPractical examplesNo account needed

ONE WORKFLOW. MANY POSSIBILITIES.

CSV

XLSX

HTML

PDF

PNG/JPG

JSON

Your data. Ready for what’s next.Map it. Validate it. Make it useful.

A BETTER HANDOFF STARTS WITH A CLEAR PLAN.

Find your starting point

One big idea. Seven ways to bring it in.

Explore the decisions, formats, and workflows behind your next import.

Data Import API

Turn external records into application-ready data with explicit mappings and understandable validation.

JSONCSV

Explore Data import

Spreadsheet Import API

Move beyond fragile column matching. Plan consistent headers, field types, dates, and error reports.

CSVXLSX

Explore Spreadsheet import

Website Import API

Extract authorized website content while preserving headings, source links, and editorial context.

HTMLURLs

Explore Website import

Document Import API

Prepare readable document sections with source identity, revision context, and extraction review.

PDFDOCXTXT

Explore Document import

Image Import API

Organize originals, metadata, image variants, and publication review into a coherent asset library.

PNGJPEGWEBP

Explore Image import

AI & LLM Import API

Prepare evidence-rich content for retrieval with meaningful chunks, provenance, and permissions.

TEXTJSONRAG

Explore AI & LLM import

AI Prompt Import API

Keep prompt roles, variables, versions, and evaluation cases together as reviewable artifacts.

JSONMESSAGES

Explore AI prompt import

Not sure where to start?

Follow a simple learning path, from your first record to a more resilient workflow.

Find your guide

A little structure goes a long way

From “I have a file” to “I have a plan.”

Three useful steps for a better handoff. No black boxes required.

Choose your source

Know what you’re bringing in, where it came from, and what the destination actually needs.

Source identity · File formats · Scope

Map and validate

Preserve important meaning, make field conversions visible, and catch problems before the write.

Field mapping · Clear rules · Useful errors

Build your workflow

Plan the destination change, the retry, and the review. Make the result easy for the next person to understand.

Traceable results · Safe retries · Review

Made for curious builders

A clearer contract.
A better starting point.

Go beyond the happy path. Explore source records, field mappings, validation errors, and the small decisions that make imports easier to maintain.

Start with a fixed local example. See the transformation, inspect the result, and take the sample files with you.

Try the local example ↗

A destination starts with a clear record

{
  "source": "customer-export",
  "schema_version": "1.0",
  "record": {
    "external_id": "001",
    "name": "Ada",
    "country": "GB"
  }
}

Illustrative data shape. The quickstart runs locally with synthetic data.

Built around the work

Different sources. Real-world workflows.

Find an approach for the job in front of you.

All solutions

Customer Onboarding

Give customers a clear path from their first spreadsheet to useful application records.

Explore workflow

Data Migration

Plan mappings, rehearsals, cutover, and recovery before moving the production dataset.

Explore workflow

Content Workflows

Connect authorized web and document sources to a reviewable editorial destination.

Explore workflow

A few useful answers

Less guesswork.
More clarity.

What you can explore, what you can try, and where to go next.

Import API Types & Workflow Guides

Sun, 20 Sep 2026 12:00:00 -0700

The import collection

Meet your next starting point.

Seven ways to bring useful information into an application. Choose a source category, then explore its contract, transformation rules, and review path.

Data Import API

Turn external records into application-ready data with explicit mappings and understandable validation.

JSONCSV

Explore Data import

Spreadsheet Import API

Move beyond fragile column matching. Plan consistent headers, field types, dates, and error reports.

CSVXLSX

Explore Spreadsheet import

Website Import API

Extract authorized website content while preserving headings, source links, and editorial context.

HTMLURLs

Explore Website import

Document Import API

Prepare readable document sections with source identity, revision context, and extraction review.

PDFDOCXTXT

Explore Document import

Image Import API

Organize originals, metadata, image variants, and publication review into a coherent asset library.

PNGJPEGWEBP

Explore Image import

AI & LLM Import API

Prepare evidence-rich content for retrieval with meaningful chunks, provenance, and permissions.

TEXTJSONRAG

Explore AI & LLM import

AI Prompt Import API

Keep prompt roles, variables, versions, and evaluation cases together as reviewable artifacts.

JSONMESSAGES

Explore AI prompt import

Data Import API Guide

Sun, 20 Sep 2026 12:00:00 -0700

Data Import API

Give every record a clear destination.

Start with your destination schema, not a pile of files. Explore patterns for receiving structured records, preserving identifiers, catching invalid values, and reporting what changed.

JSONCSV

Start with a small, explicit contract

Define the fields your application accepts, their types, and what missing values mean. Keep external identifiers as strings when their exact representation matters. Separate source metadata from ownership and permissions: an imported field should never choose a tenant or grant access.

For a first catalog import, agree on a product identifier, display name, and status. Add other fields only when their intended use and correction rules are clear.

Make mapping and validation visible

Rename fields through a versioned mapping and validate the result against the destination requirements. Keep parsing errors, invalid records, and destination conflicts distinct. A preview should show the original value, the proposed value, and any rule that changed it.

Decide whether partially valid batches can proceed. A helpful report names the source record, affected field, and correction needed rather than simply saying the file failed.

Plan the write and the retry

Use stable source identity to distinguish inserts from updates. Document whether blank values clear a field or leave it unchanged. Keep a logical job separate from its processing attempts so a connection interruption does not automatically create another import.

Before committing real records, test a repeated request, an invalid identifier, an unexpected field, and a batch that is interrupted after partial progress.

Turn the idea into a test

Use a small, representative source and write down the expected result for one accepted record, one correction, and one repeated attempt. Keep the original values and the destination expectations visible throughout the review.

Use the format guides for source decisions and the local quickstart to see a simple mapping in action.

Spreadsheet Import API Guide

Sun, 20 Sep 2026 12:00:00 -0700

Spreadsheet Import API

From spreadsheet chaos to clear records.

Make the handoff between a spreadsheet and your application understandable. Explore CSV and XLSX parsing decisions, column mapping, and correction-friendly previews.

CSVXLSX

Agree on the table before importing it

Select the intended worksheet and header row explicitly. Check duplicate or blank column names before mapping. Keep instructional rows and totals outside the accepted data region. A recurring import should reuse an approved mapping rather than silently guessing on every new export.

Keep a representative input file that includes ordinary rows, an optional blank field, and an identifier with leading zeroes. That fixture makes the agreed handoff concrete.

Preserve the meaning of each cell

Do not treat a digit-only identifier as a number by default. Write down date conventions, decimal separators, currency rules, and how blank cells behave. For workbook formulas, test the chosen parser and define whether stored results or values-only exports are required.

CSV quoting is a parsing concern, not a field-mapping shortcut. The complete guide covers quoted commas and line breaks and links to the documented CSV format.

Make corrections easy to find

Return a source row and column reference with each rejected value. Distinguish an unsupported file layout from a record that violates a business rule. Show the intended destination field beside the original header so the operator can correct the right source.

Test updates as well as inserts. Existing destination records and duplicate rows within one file need different policies, particularly when a corrected batch is submitted again.

Turn the idea into a test

Use the format guides for source decisions and the local quickstart to see a simple mapping in action.

Website Import API Guide

Sun, 20 Sep 2026 12:00:00 -0700

Website Import API

Import the useful part of the page.

Plan website content imports around an authorized source inventory, a clear extraction rule, and a destination readers can actually use.

HTMLURLs

Choose a deliberate source inventory

Identify the pages, languages, and revisions that belong in your collection. Prefer an approved structured export when it preserves the required content better than rendered HTML. Keep retrieval credentials and source authorization outside imported records.

A link on a page is not an instruction to expand the migration. Decide which external references remain links and which assets need an authorized copy.

Keep structure and remove noise

Identify the main content area and preserve headings, lists, code, and essential table context. Review selectors on several layouts before applying them broadly. Parsing HTML is not a substitute for a sanitization policy when content will be displayed in an active page.

Keep a traceable source address and extraction version. A cleanup rule that removes a footer should not also remove a repeated warning in an instruction.

Review revisions and dependencies

Resolve relative links using the source context, then map them to the intended destination. Track missing images and attachments as review tasks. Separate a temporary retrieval failure from an intentional source deletion.

Use a reference set with a tutorial, a code-heavy page, a table, and an image-dependent article. Check meaningful content, not just the presence of a title and body.

Turn the idea into a test

Use the format guides for source decisions and the local quickstart to see a simple mapping in action.

Document Import API Guide

Sun, 20 Sep 2026 12:00:00 -0700

Document Import API

Give your documents a second life.

Create a useful path from authorized manuals and documents to structured content. Preserve meaning, flag uncertain extraction, and keep every result connected to its source.

PDFDOCXTXT

Understand the collection first

Inventory text documents, scans, tables, columns, and embedded material. Test representative files before choosing a parsing path. Different destination uses need different quality checks: search text and a structured procedure are not interchangeable outputs.

Use an approved access process for protected files. A parser returning no text should create an explicit review outcome, not a success status for an empty document.

Preserve source and section context

Give documents stable identifiers that do not depend only on filenames. Keep revisions, extraction configuration, and accurate locations where available. A page number should remain unknown when the extraction method cannot establish it.

Preserve headings and connect tables with labels. Inspect reading order on multi-column documents and flag uncertain sections rather than inventing missing transitions.

Build a review and reprocessing path

Separate machine extraction from editorial approval. Route scanned material through an explicit recognition and review process when needed. Decide which comments and embedded objects belong in the destination.

Keep original and derived records under appropriate access and retention rules. Reprocessing should produce a reviewable new result instead of silently overwriting an already approved version.

Turn the idea into a test

Use the format guides for source decisions and the local quickstart to see a simple mapping in action.

Image Import API Guide

Sun, 20 Sep 2026 12:00:00 -0700

Image Import API

More than pixels. A better media library.

Build an image import around the asset record, not just its filename. Explore format decisions, verified metadata, derivatives, and meaningful descriptions.

PNGJPEGWEBP

Define an asset, then accept the file

Separate the creative asset, its original file, and its generated variants. Use stable asset identifiers and keep product associations explicit. Choose a deliberate set of formats and inspect accepted files through an appropriately controlled processing path.

A supplied filename or content type is not proof of file content. Keep invalid files distinct from valid formats that your workflow does not accept.

Create derivatives with a clear recipe

Keep originals separate from thumbnails and editorial crops when your retention policy allows. Record output dimensions and the transformation version. Test orientation handling and inspect diagrams or labels before applying automated cropping.

A square product tile and a wide editorial image can be legitimate variants of the same asset. Store that relationship rather than relying on names such as final-v2.

Review description and publication

Decide what embedded metadata belongs in public output. Preserve useful provenance in a protected record when public derivatives exclude it. Write descriptions for their page context instead of publishing camera-generated filenames.

Separate inspection from publication approval. Test replacement, duplicate handling, and deletion so retiring one image does not unexpectedly break other references.

Turn the idea into a test

Use the format guides for source decisions and the local quickstart to see a simple mapping in action.

AI & LLM Import API Guide

Sun, 20 Sep 2026 12:00:00 -0700

AI & LLM Import API

Better context starts before the prompt.

Explore the ingestion decisions behind an AI knowledge collection: reviewed extraction, useful chunks, source lineage, access boundaries, and real evaluation questions.

TEXTJSONRAG

Start with questions and trusted sources

Identify the tasks your collection should support and the cases it should not answer. Assemble representative questions with expected source sections. Inspect extracted documents before turning them into retrieval units.

A search index cannot repair an instruction whose table labels or prerequisites disappeared during extraction. Make source quality visible before tuning retrieval.

Keep context with every chunk

Use section boundaries and complete ideas as candidates for chunking. Test size and overlap against the actual material rather than assuming one recipe fits every document. Attach source identity, revision, and necessary access attributes to each derived unit.

Keep embedding inputs distinct from stored application records. Record transformation and embedding configurations so changed processing can be evaluated and reproduced.

Evaluate, refresh, and remove deliberately

Inspect retrieved passages before judging generated answers. Check whether the expected source is present, current, and accessible to the requesting user. Imported text remains evidence, not authorization to change application instructions or tool permissions.

Plan how source edits, revoked access, and deletions propagate to derived records and caches. A complete lifecycle includes removing obsolete chunks, not only adding new ones.

Turn the idea into a test

Use the format guides for source decisions and the local quickstart to see a simple mapping in action.

AI Prompt Import API Guide

Sun, 20 Sep 2026 12:00:00 -0700

AI Prompt Import API

Your prompts deserve a proper workflow.

Move beyond scattered snippets. Explore a prompt registry that preserves intent, makes changes reviewable, and separates reusable definitions from live executions.

JSONMESSAGES

Import definitions, not private executions

Give each prompt a purpose, stable identifier, declared variables, and message structure. Keep actual customer conversations separate from reusable instructions. Use reviewed synthetic examples when examples belong in the production definition.

A prompt named Ticket summary is not a unique identity. Define an owning collection or namespace so separate teams can maintain related artifacts without collisions.

Preserve roles and version behavior

Store the logical messages separately from model-specific rendering. Declare input variables and reject unresolved references. Keep substitution rules simple; imported prompt text should not execute arbitrary expressions or grant tool access.

Version output expectations, examples, and renderer configuration alongside wording when they affect behavior. An execution should identify the complete reviewed definition it used.

Review before activating

Separate received, validated, reviewed, and active states. Test resolved cases, ambiguous cases, and cases with missing information against explicit acceptance criteria. Valid formatting is not evidence of a faithful result.

Keep a known reviewed version for rollback and test the promotion path. Moving a prompt to a different model requires evaluation rather than an assumption of identical behavior.

Turn the idea into a test

Use the format guides for source decisions and the local quickstart to see a simple mapping in action.

Import Solutions & Practical Workflows

Sun, 20 Sep 2026 12:00:00 -0700

Solutions

A workflow for what’s next.

Start with the outcome you need, then connect source data, validation, review, and destination changes into one understandable plan.

Customer Onboarding

Give customers a clear path from their first spreadsheet to useful application records.

Explore workflow

Data Migration

Plan mappings, rehearsals, cutover, and recovery before moving the production dataset.

Explore workflow

Content Workflows

Connect authorized web and document sources to a reviewable editorial destination.

Explore workflow

AI Knowledge Imports

Build a retrieval collection around evidence, source lineage, and enforceable access.

Explore workflow

Media Libraries

Organize asset identity, original files, derivatives, descriptions, and publishing decisions.

Explore workflow

Customer Onboarding Workflow

Sun, 20 Sep 2026 12:00:00 -0700

Import solutions

Customer Onboarding

Give customers a clear path from their first spreadsheet to useful application records.

Design the first successful handoff

Start with a sample file that mirrors the customer’s actual destination. Explain the required identifier, accepted header names, and treatment of blank values. Keep the initial contract small enough to review without an implementation meeting. A preview should connect familiar source columns to the fields in your application.

Do not hide invalid values behind an import-failed message. Identify the record and field that need attention. Distinguish a correctable source issue from a permission or processing problem, and route each to the right person.

Reduce correction cycles

Keep a versioned mapping for repeat exports from the same source. Show transformations explicitly, particularly dates and identifiers. Let the customer review a representative preview, then validate the full collection before treating it as ready.

Define whether valid records can proceed when other rows fail. Explain how the customer should resubmit corrections and whether already accepted rows will be updated or skipped. The correction path is part of onboarding, not an afterthought.

Agree on acceptance

Use a small dataset with a blank required value, a duplicate identifier, and a correctly formatted update. Check that source totals reconcile with accepted and rejected records. Ask someone unfamiliar with the import to repair the example using only its report.

A useful handoff ends with understandable records and an explanation of anything left behind. Track correction effort and unresolved imports rather than treating the number of uploaded files as the only measure of progress.

Define your first acceptance check

Choose a representative source, an owner for the destination contract, and a small set of results that a reviewer can inspect. Include a correction case and an interrupted attempt before expanding the workflow.

The developer guides connect these planning decisions to record shapes and local examples. Keep your implementation narrow enough that its outcome is easy to explain.

Data Migration Workflow

Sun, 20 Sep 2026 12:00:00 -0700

Import solutions

Data Migration

Plan mappings, rehearsals, cutover, and recovery before moving the production dataset.

Inventory before moving

Define the source collection, ownership, destination schema, and relationships between record types. Separate active data from archives and intentionally excluded fields. Identify which external identifiers remain stable and where the destination will generate its own keys.

Keep an explicit mapping for each source type. A field with a similar name may have a different meaning in the new application. Document conversions and the behavior of absent, empty, and null values.

Rehearse the difficult cases

Run a representative dry run and reconcile counts by record type. Include duplicate identifiers, unsupported values, missing relationships, and a partial processing failure. Keep parsing and business-rule errors separate so correction work has a clear owner.

Decide how changes made during the migration window are captured. A one-time export is a snapshot; a destination that continues changing needs a plan for conflicts and final reconciliation.

Make cutover recoverable

Record the source revision, mapping version, and outcome of each logical import. Distinguish retries from new requests, and verify behavior when the result of an attempt is uncertain. Keep rollback or compensating changes within a defined boundary.

Before declaring the migration complete, compare agreed totals and representative records, verify permissions, and test the destination workflows. Archive reports under an appropriate retention policy so later corrections can reference what actually happened.

Define your first acceptance check

The developer guides connect these planning decisions to record shapes and local examples. Keep your implementation narrow enough that its outcome is easy to explain.

Content Workflows Workflow

Sun, 20 Sep 2026 12:00:00 -0700

Import solutions

Content Workflows

Connect authorized web and document sources to a reviewable editorial destination.

Start with an editorial inventory

Choose which pages, manuals, and revisions belong in the collection. Identify source ownership and the fields needed by the destination. Preserve the distinction between a publication date, a source modification observation, and a processing timestamp.

Map headings, code, tables, and attachments intentionally. Not every visible page element belongs in the content record. Review navigation removal on different layouts so cleanup does not accidentally discard meaningful warnings.

Separate extraction from approval

Keep received, extracted, reviewed, and published states distinct. A nonempty title and body are not sufficient evidence that an article survived migration. Provide a side-by-side source reference and a clear list of unresolved links or missing assets.

Assign correction work to the right owner. A broken extraction selector needs engineering attention; an obsolete instruction needs editorial review. The same generic failure status should not conceal those differences.

Maintain the collection after launch

Track source revisions and derived content relationships. A revised document should create a reviewable change rather than automatically inherit approval from an older extraction. Define whether missing source material is archived, deleted, or queued for review.

Test a code-heavy page, a document with tables, and an article that relies on an image. A useful content pipeline preserves meaning across these variations and leaves enough provenance for future maintenance.

Define your first acceptance check

The developer guides connect these planning decisions to record shapes and local examples. Keep your implementation narrow enough that its outcome is easy to explain.

AI Knowledge Imports Workflow

Sun, 20 Sep 2026 12:00:00 -0700

Import solutions

AI Knowledge Imports

Build a retrieval collection around evidence, source lineage, and enforceable access.

Define useful retrieval

Write representative questions and identify the source passages expected to support each answer. Include questions outside the collection’s scope and questions with conflicting or obsolete material. These cases make evaluation more meaningful than a single impressive demonstration.

Review extracted content before chunking it. Missing table labels or detached prerequisites can make a passage misleading even when retrieval finds it successfully.

Keep context and permissions together

Create chunks around useful units of meaning. Attach document identity, revision, section path, and the access attributes the serving layer actually enforces. Test whether a restricted source remains restricted in results, citations, previews, and caches.

Record the configuration used for transformations and embeddings. Changes to either can require reprocessing even when the source file remains unchanged. Compare a candidate index against a reviewed reference set before switching the application.

Close the lifecycle

Plan how edits, deletions, and revoked permissions propagate through derived records. Distinguish current instructions from historical revisions. Keep imported text as evidence rather than authority over the application’s tool permissions or operating instructions.

Measure failed jobs, correction work, and retrieval quality alongside throughput. A smaller collection with explainable results is a useful starting point for expanding the system responsibly.

Define your first acceptance check

The developer guides connect these planning decisions to record shapes and local examples. Keep your implementation narrow enough that its outcome is easy to explain.

Media Libraries Workflow

Sun, 20 Sep 2026 12:00:00 -0700

Import solutions

Media Libraries

Organize asset identity, original files, derivatives, descriptions, and publishing decisions.

Give assets a stable identity

Separate the asset from its source file and generated variants. Connect a product or article to an asset record rather than assuming a filename uniquely identifies the image. Retain the original filename as descriptive metadata.

Choose accepted formats and inspect files through a controlled processing path. Differentiate invalid content from a valid format that the library does not support. Route both outcomes to an understandable correction process.

Make variants explainable

Keep authorized originals separate from web derivatives when retention permits. Store dimensions, format, crop intention, and transformation version for each variant. Test orientation and visual detail on photographs and diagrams rather than assuming one crop rule fits everything.

Review embedded metadata and public descriptions separately. Information useful for internal provenance may not belong in a downloadable image. A description should fit the image’s actual context on the destination page.

Publish and retire deliberately

Separate a successfully inspected file from an approved public asset. Require the appropriate product association, descriptive text, and content review before publication. Preserve a relationship between variants and their parent so replacement remains predictable.

Test exact duplicates, alternate crops, missing descriptions, and deletion. A maintenance operation should reveal which references depend on an asset before those references become broken images.

Define your first acceptance check

The developer guides connect these planning decisions to record shapes and local examples. Keep your implementation narrow enough that its outcome is easy to explain.

Import Notes: Data, Content & AI Import Blog

Sun, 20 Sep 2026 12:00:00 -0700

The ImportAPI.com blog

Import Notes.
Ideas worth bringing in.

Practical guides for the messy, meaningful work of importing data. From your first CSV to a maintainable AI knowledge collection.

Reliability & securityAugust 13, 2026

Safer imports start at the boundary.

Review format policies, authorization, isolated parsing, resource limits, inert previews, and cleanup across the complete import lifecycle.

Read the guide

Reliability & securityJune 27, 2026

Retry without the uncertainty.

Separate logical jobs from processing attempts and make duplicate handling, partial outcomes, and uncertain commits understandable.

Read the guide

AI workflowsMay 4, 2026

Prompts deserve a proper workflow.

Import prompts as maintained artifacts with explicit roles, declared variables, review history, repeatable tests, and a rollback path.

Read the guide

AI workflowsAugust 20, 2025

Better context starts before the prompt.

Prepare a knowledge collection with meaningful chunks, source lineage, permission boundaries, and evaluation grounded in real questions.

Read the guide

Content & mediaJuly 29, 2025

More than pixels. Better media records.

Connect original images, descriptive metadata, derivatives, and publication review in a maintainable product-image library.

Read the guide

Content & mediaMay 24, 2025

Give your documents a second life.

Build a document import that preserves headings, handles uncertain extraction, and connects useful text to its original source and revision.

Read the guide

Content & mediaMarch 7, 2025

Import the content. Leave the clutter.

Plan an authorized website migration with clean extraction, source provenance, meaningful structure, and a reviewable destination collection.

Read the guide

Data engineeringDecember 29, 2024

Map it. Validate it. Understand it.

Separate transformations from validation, make every conversion explainable, and design useful errors around an explicit destination contract.

Read the guide

SpreadsheetsNovember 19, 2024

CSV or XLSX? Make every field count.

Navigate CSV quoting, workbook sheets, leading zeroes, date ambiguity, and correction-friendly previews without losing the meaning of your data.

Read the guide

Data engineeringOctober 27, 2024

What is an import API?

Turn external records into clear, validated application data. A practical introduction to contracts, transformations, errors, and job lifecycles.

Read the guide

Import Integration Patterns & Destination Workflows

Sun, 20 Sep 2026 12:00:00 -0700

Integration patterns

Make the handoff make sense.

Connect the format you have to the destination you need. These are implementation patterns—not preconnected accounts or one-click third-party integrations.

Spreadsheets → application records

Choose the data region, preserve string identifiers, and map columns to a destination contract. Keep parsing and business validation separate.

Explore spreadsheet imports

Website content → editorial records

Use an authorized source, preserve headings and links, and keep extraction separate from publication review.

Explore content workflows

Documents → a knowledge collection

Extract reviewable sections, attach source revisions, and evaluate content quality before indexing.

Explore knowledge imports

Images → a media library

Connect originals, derivatives, descriptive metadata, and publishing status through stable asset identity.

Explore media workflows

Knowledge records → retrieval

Carry section context and permissions into derived chunks. Evaluate retrieval with representative questions and expected evidence.

Explore LLM imports

Prompt definitions → an application

Preserve message roles and variables, then use a reviewed adapter and evaluate behavior before activation.

Explore prompt imports

Review the boundary,
not just the connector.

A source system and a destination can agree on JSON while disagreeing on identity, permissions, dates, or update behavior. Capture those decisions before moving real records.

Compare source formats ↗

Before connecting a destination

Identify the owner of the destination schema. Decide which fields can be updated, how missing values behave, and whether existing records are matched by a stable external identifier.

Before repeating the workflow

Document source revisions, credential handling in your own implementation, and a recovery path for uncertain outcomes. Test the exact configuration and permissions rather than inferring compatibility from a brand name.

Developer Documentation & Local Examples

Sun, 20 Sep 2026 12:00:00 -0700

Developer documentation

Good imports start
with clear contracts.

Small examples, explicit transformations, and understandable results. Start locally, then adapt the concepts to your own destination.

Run the quickstart ↗Read the example reference

Quickstart

Map three synthetic customer records and inspect one useful validation error.

Open guide

Example reference

Inspect the source shape, mapped fields, validation rules, and outcome record.

Open guide

Example workflows

Connect record mapping, content review, and prompt versioning to a practical first test.

Open guide

Format guides

Compare source formats and the decisions a parser cannot make for you.

Open guide

See the change.
Understand the result.

The quickstart uses a fixed source dataset and runs mapping and validation in the browser. It does not make a network request, upload a file, or write into an application.

The downloadable examples give you the same source and outcome to inspect without running JavaScript.

Browse example files

An error worth understanding

{
  "source_record": 3,
  "field": "external_id",
  "code": "required",
  "message": "Supply a stable customer identifier."
}

Local Import Quickstart: Map & Validate JSON

Sun, 20 Sep 2026 12:00:00 -0700

Developer quickstart

Your first import,
minus the mystery.

A tiny dataset. A visible transformation. One useful error. Run the fixed customer example locally and follow the source all the way to its validation result.

Meet the source

The example contains three synthetic customer records. Customer 001 has whitespace around the name; record 3 has no identifier. The workflow renames fields, trims whitespace, preserves identifiers as strings, and checks three deliberately small validation rules.

Three synthetic source records. One identifier is missing; one name has surrounding spaces.

[
  {
    "Customer ID": "001",
    "Name": "  Ada  ",
    "Country": "GB"
  },
  {
    "Customer ID": "002",
    "Name": "Milo",
    "Country": "US"
  },
  {
    "Customer ID": "",
    "Name": "Sam",
    "Country": "FR"
  }
]

Download result

What the mapping changes

Customer ID becomes external_id, Name becomes name, and Country becomes country. Surrounding whitespace is removed and country labels are uppercased. The value 001 remains a string; the mapping does not turn it into the number 1.

What validation checks

The identifier and name must be nonempty. Country must be GB, US, or FR for this specific exercise. Those three allowed values are a teaching choice, not a complete country standard or a rule for your application. Each result also carries a one-based source record number.

Read the result

The complete output contains evaluated: 3, two accepted records, and one rejected record. The error points to source record 3 and the missing external_id. No records are saved. A production write, authentication, full CSV parsing, and arbitrary file processing are outside this local example.

Turn it into your own test

Download the source, result, and example schema. In your own project, agree on identifier behavior, accepted values, and missing-field semantics before expanding the example. Add duplicate checks and destination authorization where your workflow requires them.

Continue with the example reference for exact shapes or the mapping and validation guide for the broader design decisions.

Example Record Reference & Validation Results

Sun, 20 Sep 2026 12:00:00 -0700

Example reference

A record shape you can actually inspect.

Exact fields and outcomes for the local customer example, with clear boundaries between source parsing, mapping, validation, and a future application write.

Scope of this reference

This reference documents the fixed customer example on ImportAPI.com. It describes local JavaScript behavior and downloadable JSON fixtures. There is no HTTP endpoint to call, no authentication header, and no live record-storage operation. Use these shapes as a starting point for designing your own application contract.

Source records

The source is an array containing three objects. Each object has the string fields Customer ID, Name, and Country. The example intentionally includes one missing identifier and one name with surrounding spaces. The source data is fixed in the page script, so the demonstration accepts no user-entered or uploaded records.

Mapped records

The mapping creates external_id, name, and country. It trims surrounding whitespace, uppercases the country string, and adds source_record, a one-based position in the original array. Identifiers stay strings. This mapping does not merge records, infer missing values, or consult a destination.

Validation rules

Both external_id and name must be nonempty after trimming. The local exercise accepts the country labels GB, US, and FR. Each failure contains a field, a stable code, and a readable message. A record with at least one failure is rejected; otherwise it is accepted.

Outcome envelope

evaluated counts source records. accepted contains the mapped records that passed these checks. rejected contains the source record number and errors for each unsuccessful record. For this fixture, accepted plus rejected equals three. Accepted here means validated by the example, not committed to an application.

Complete local result · JSON

{
  "evaluated": 3,
  "accepted": [
    {
      "source_record": 1,
      "external_id": "001",
      "name": "Ada",
      "country": "GB"
    },
    {
      "source_record": 2,
      "external_id": "002",
      "name": "Milo",
      "country": "US"
    }
  ],
  "rejected": [
    {
      "source_record": 3,
      "errors": [
        {
          "field": "external_id",
          "code": "required",
          "message": "Supply a stable customer identifier."
        }
      ]
    }
  ]
}

Extending the contract

A production workflow needs additional decisions: authorization, identity conflicts, schema evolution, source retention, and commit behavior. Keep these decisions separate from the example’s narrow validation rules. A schema can describe expected structure without proving that a record belongs to a particular customer or tenant.

Read the reliable batch import guide before designing retries. Use the schema mapping guide to make transformations and business checks independently testable.

Import Workflow Examples & Downloadable Fixtures

Sun, 20 Sep 2026 12:00:00 -0700

Developer workflows

A small example.
A useful next step.

Start with an inspectable source and an expected outcome. Adapt the workflow in your own implementation rather than treating a successful sample as a complete integration.

Customer record mapping

Rename columns, trim whitespace, preserve string identity, and report one missing required value. The working quickstart runs this fixed fixture locally.

Explore this workflow

Document-to-knowledge review

Choose a manual, preserve headings and source identity, and define questions with expected source sections before splitting content into retrieval units.

Explore this workflow

Prompt registry handoff

Keep the definition separate from actual executions. Preserve roles, variables, a version, and review status before promoting an application instruction.

Explore this workflow

Downloadable fixtures

Small enough to understand at a glance.

Synthetic source records, explicit output, and a prompt record you can inspect.

Customer source · JSON

Three original records before mapping. One required identifier is deliberately missing.

Download example ↗

Customer source · CSV

The same source values in a header-based CSV. Use a proper CSV parser in your own implementation.

Download example ↗

Validation result · JSON

The exact accepted and rejected output from the working quickstart.

Download example ↗

Customer contract · JSON Schema

The example record shape, nonempty string checks, and three allowed country labels.

Download example ↗

Prompt definition · JSON

A synthetic support-summary definition with messages, declared variables, and a version.

Download example ↗

Keep an expected result beside every fixture

A useful example includes its failure behavior. When expanding these fixtures, write down whether a new case should be accepted, rejected, or require review. Test missing values separately from explicit empty values and test identifiers that must retain leading zeroes.

For document and prompt workflows, an acceptable result involves meaning as well as shape. Use reviewed source sections and task-specific evaluation cases. No document extraction or model inference is performed by these downloads.

CSV, XLSX, JSON & Content Format Guides

Sun, 20 Sep 2026 12:00:00 -0700

Format guides

Different formats.
One clear destination.

Compare CSV, XLSX, JSON, HTML, documents, images, and prompt records—and identify the decisions that a file format cannot make for you.

Start with the source’s meaning

A filename extension identifies a likely representation, not a complete contract. Choose the parser and settings for the formats you deliberately accept, then define what the destination expects. Preserve identifiers, source references, and useful structure before applying business validation.

Import format decisions
Source	Useful for	Decide explicitly
CSV	One rectangular set of records	Delimiter, quoting, headers, encoding, and field types.
XLSX	An agreed worksheet in a workbook	Sheet selection, the data region, formulas, and date interpretation.
JSON	Structured records and nested values	Object shape, required fields, identifiers, and unexpected properties.
HTML	Authorized web content	Main content boundaries, links, assets, sanitization, and revision identity.
PDF / DOCX	Document content and metadata	Reading order, headings, tables, extraction uncertainty, and review.
PNG / JPEG / WebP	Image assets and variants	Accepted types, inspection limits, original identity, and derivative recipes.
Prompt records	Reusable application instructions	Message roles, declared variables, version, ownership, and activation status.

CSV: agree on a dialect

The CSV description in RFC 4180 documents a commonly used quoting convention. Test embedded commas, doubled quotes, and quoted newlines. Do not assume that splitting every line on a comma is an adequate parser. Explain whether a header is required in your own workflow.

JSON: define the record contract

The JSON specification provides the interchange structure; the application still chooses its fields and semantics. Keep identifiers as strings when their exact representation matters. Separate absent values, explicit nulls, and empty strings according to the destination’s needs.

Documents and web pages: retain context

Text extraction and content approval are separate stages. Preserve useful heading relationships and source revisions, and flag layout that the extractor cannot interpret reliably. A successful parse does not prove that the resulting content is complete or ready for publication.

Images and prompts: model the lifecycle

An original image, its web derivative, and its page description are related but distinct artifacts. A prompt definition, its rendered representation, and an actual execution are similarly different. Keep the relationships explicit so replacement, evaluation, and rollback remain understandable.

Next step

Choose the relevant import category and use a small representative file to establish a destination contract. The local quickstart illustrates mapping and validation with fixed JSON records; it is not a general-purpose file parser.

Import Guides & Curated Learning Paths

Sun, 20 Sep 2026 12:00:00 -0700

Import guides

Your next step,
a little clearer.

Choose a reading path that fits the work in front of you. Each path connects an explanation, a practical workflow, and a useful first test.

PATH 01

Start with your first record

Understand the boundary between external data and destination changes. Then try the local walkthrough and inspect a deliberately imperfect source.

PATH 02

Move spreadsheet data with care

Agree on headers and preserve meaningful values. Design a correction report that helps the source owner repair a file without reading your code.

PATH 03

Prepare content and media

Keep meaningful structure, asset relationships, and source identity. Separate extraction from editorial approval and maintenance.

PATH 04

Build an AI knowledge collection

Start with real questions and reviewed sources. Carry permissions and provenance through derived records, and evaluate prompts as maintained artifacts.

PATH 05

Make failure understandable

Plan repeated attempts, uncertain outcomes, and the complete source lifecycle. Test recovery and authorization before expanding volume or format support.

Import API FAQ: Formats, Examples & Workflows

Sun, 20 Sep 2026 12:00:00 -0700

Frequently asked questions

Good questions.
Useful answers.

Understand the resources, the local examples, and the decisions behind a dependable import workflow.

What is ImportAPI.com?

Can I try an import example?

Which import types are covered?

Do I need an account or an API key?

No account or API key is needed to read the guides or run the local example. This site does not provide a hosted import endpoint, account dashboard, or paid API subscription.

Will a preview tell me everything about an import?

A preview helps explain mappings and validation results. A production workflow must also validate the complete dataset and handle changing destination state. The guides distinguish a readable file, a validated record, and a committed change.

How do I get help or suggest a topic?

Email info@importapi.com with the page address and a description of your question. Use a small, synthetic example when describing a data problem, and leave out credentials or private records.

Does CSV or XLSX always work better?

Neither format chooses your business rules for you. CSV is useful for an agreed rectangular table; XLSX may fit an existing workbook handoff. Decide on headers, sheets, quoting, identifiers, and date interpretation for the actual source.

Are an accepted record and a committed record the same?

No. A record may pass validation before the destination is changed. A production workflow needs a commit policy and a way to handle destination changes or interruptions. The local example validates only and never commits records.

Does importing documents train an AI model?

The LLM guides focus on preparing a retrieval collection and supplying source context to an application. That workflow is distinct from model training. It also needs evaluation and permission enforcement; an imported document is not a guarantee of a correct answer.

Can I subscribe to new content without an email form?

Yes. Add the RSS feed to your preferred feed reader. It contains the substantive guide pages and full article text. No email address or site account is required.

Can I use the downloadable examples offline?

Yes. The JSON and CSV fixtures are ordinary local files. The website assets also have no required third-party fonts or scripts. To navigate the full site locally with its clean URLs, serve the extracted site root through a local static web server.

Where do I find sources for technical claims?

Each long-form article includes one relevant editorial reference to an authoritative specification or technical source. The format guide also links to the CSV and JSON specifications. Implementation patterns are presented as design choices, not service guarantees.

Free Import Resources & Implementation Cost Planning

Sun, 20 Sep 2026 12:00:00 -0700

Pricing & planning

Start with ideas.
Not a subscription.

Read the guides, try the local example, and download the fixtures without an account. Plan the cost of your own implementation separately.

THE RESOURCE COLLECTION

Explore freely.

For the public guides and local examples on this site.

Seven import-category guides
Ten long-form Import Notes articles
Five practical solution workflows
Local mapping and validation example
Downloadable JSON and CSV fixtures

Try the free example ↗

YOUR IMPLEMENTATION

Plan the whole workflow.

Your scope. Your costs.

Estimate costs from your sources, destination, volume, and review needs—not a made-up per-record promise.

Parsing and transformation work
Storage, retention, and transfers
Document extraction or model usage
Monitoring, retries, and support
Human review and source correction

Explore implementation patterns

What this site does—and does not—charge for

There is no paid plan, checkout, hosted API subscription, or usage meter on ImportAPI.com. The examples run locally and do not consume a hosted import quota. Reading the resources does not require payment details or registration.

Build an estimate from measured work

Start with a representative batch and track each stage: parsing, transformations, extraction, review, and the destination write. Account for repeated attempts and retained originals. For an AI knowledge workflow, distinguish source processing, embedding, indexing, and ongoing retrieval.

Keep vendor-specific rates in your own cost model and verify them with the provider when planning a real implementation. No current provider prices or implied savings percentages are presented here.

Include the correction path

An inexpensive processing run can still be costly when its result requires extensive cleanup. Track the source errors your team encounters, the time needed to repair them, and the reliability of repeated jobs. Clear contracts and useful reports give that work a more predictable shape.

About ImportAPI.com

Sun, 20 Sep 2026 12:00:00 -0700

About ImportAPI.com

Big ideas deserve
better handoffs.

ImportAPI.com helps curious builders think clearly about moving data, content, and context into the applications they want to create.

A home for practical import thinking

An import touches more than a file format. It connects the person producing the source, the person fixing the errors, and the application that needs a trustworthy result. ImportAPI.com brings those perspectives together through approachable technical guides and small examples.

The collection covers structured data, spreadsheets, authorized website content, documents, images, AI knowledge records, and reusable prompts. Each category connects source-specific decisions to a destination contract, review path, and implementation test.

Our editorial approach

We favor explicit assumptions over invisible guesses. The guides explain transformations, distinguish validation from committed changes, and discuss failures alongside successful workflows. Each long-form Import Notes article includes a directly relevant technical reference and links to related reading.

Examples use synthetic data and are identified as examples. Workflow patterns are suggestions to evaluate in your own application, not claims that a particular library, provider, or file type guarantees a result. We do not publish customer testimonials, performance benchmarks, or partnerships that have not been established.

Start with something you can inspect

The local quickstart takes three fixed records through mapping and validation. One record deliberately fails a requirement. That small result makes it possible to see the source, the transformation, and a useful correction message without setting up an account or making a network request.

Use the reading paths to explore a new topic, or open the developer quickstart and follow the example from beginning to end. A narrower, understandable workflow is often the best starting point for a larger project.

Keep the conversation useful

Questions and corrections help make a technical resource more useful. Send the relevant page address, the specific point you want to discuss, and a small synthetic example where appropriate. Avoid sending credentials, private records, or complete production exports.

Reach ImportAPI.com at info@importapi.com.

Contact ImportAPI.com

Sun, 20 Sep 2026 12:00:00 -0700

Contact

Let’s make the next
handoff clearer.

Questions, corrections, and useful ideas are welcome. Send a note with enough context to point the conversation in the right direction.

Say hello.

info@importapi.com

Your email application will open when you use this address. There is no contact form or account required.

Write an email ↗

Make your message easy to follow

Include the page address, a short description of your question, and the result you expected. For a technical issue, a small synthetic example is more useful than an entire dataset.

Keep private information private

Do not email API keys, passwords, confidential records, or production exports. Remove sensitive values from screenshots and sample errors before sharing them.

Support & Local Example Troubleshooting

Sun, 20 Sep 2026 12:00:00 -0700

Support

Find the next
useful step.

Get help navigating the guides, understanding the local example, or reporting a problem with this website.

The local example does not change

Use a browser with JavaScript enabled to switch between source, mapping, and validation. The example always starts from the same three synthetic records. It does not accept uploaded files or connect to your accounts. A single rejected record is intentional: source record 3 is missing an identifier.

When JavaScript is unavailable, the source is still readable and the complete result can be downloaded. The accompanying CSV is a source fixture, not a file processed by the live demonstration.

Copying or downloading an example

The Copy JSON button uses the browser’s clipboard capability when permitted. When that is unavailable, it selects the example text and explains how to copy it through your browser. Download links provide ordinary JSON or CSV files; how they are saved can depend on the browser’s settings.

Choosing the right guide

Start with import categories when you know your source format. Use solution workflows when you know the business outcome. The reading paths connect introductory articles to more detailed implementation decisions.

Reporting a site issue

Include the page address, browser, device type, and the action that did not behave as expected. Describe the observed result and what you expected instead. A cropped screenshot can help, but remove private information and unrelated browser content before sharing it.

For an accessibility issue, describe the navigation method or assistive technology involved when you are comfortable sharing that information. The accessibility page explains the interaction features included in the site.

Scope of support

Questions about the published resources and local examples are welcome. This site does not operate a hosted import service, manage your third-party accounts, or process production datasets. For a problem in your own implementation, include a minimal synthetic example and the specific design question rather than credentials or live customer data.

Email info@importapi.com. No response-time guarantee or service-level agreement is offered on this site.

Security & Import Boundary Guidance

Sun, 20 Sep 2026 12:00:00 -0700

Security

A smaller boundary.
A clearer review.

Understand how the local example works and where security belongs in your own import implementation.

How the local example handles data

The quickstart works with three synthetic records defined in its page script. It does not read your files, access your accounts, upload content, or send the sample output to an import service. The copy action writes the visible example to your clipboard only when you activate it. Downloads are static files included with the website.

The site does not require login credentials or an API key. It does not embed third-party analytics, advertising scripts, remote font requests, or account widgets. Ordinary web requests are still made to the host serving each page and asset; the privacy page explains that distinction.

What a production import needs

Define accepted formats, resource limits, and the destination authorization boundary before accepting real files. Run untrusted processing with the capabilities it actually needs. Keep source fields separate from permissions and prevent imported values from choosing their own tenant or administrative role.

Treat previews and reports as another boundary. Display untrusted strings as text unless a separate, reviewed rendering policy applies. Restrict access to error details, original files, and derived outputs rather than protecting only the submission action.

Review the failure lifecycle

Rejected and interrupted jobs can leave temporary files, partial results, and useful diagnostic records. Define retention and cleanup for each. Keep enough context to understand the outcome without copying entire source datasets or credentials into application logs.

Use a small collection of controlled fixtures to test file limits, unauthorized access, missing required values, and repeated attempts. Document what should happen to the destination in each case. A successful happy-path import does not establish the behavior of these boundaries.

Report a concern with this website

Email info@importapi.com with the affected page, observed behavior, and a non-sensitive description. Do not send credentials, private records, or harmful payloads. A reproducible, minimal explanation is the most useful starting point for investigating a site issue.

Read the secure file import checklist for a broader implementation review. It includes an authoritative reference and separates design principles from claims about any particular deployment.

Privacy Policy: Browsing, Examples & Email

Sun, 20 Sep 2026 12:00:00 -0700

Your privacy.
A clear explanation.

How this website’s pages, local examples, downloads, and email contact interact with information.

Browsing the resource collection

ImportAPI.com serves public pages and static assets. The site code does not include analytics trackers, advertising scripts, third-party embeds, remote fonts, or a site account system. It does not set cookies or use browser local storage to remember you. Motion and example-stage choices apply only while the current page is open.

The hosting provider necessarily receives ordinary requests for pages and assets. Those requests may include connection information such as an IP address, browser details, requested address, and time. Hosting-level logs and their retention depend on the provider and deployment settings; this page does not claim that those logs are absent or set a retention period that has not been established.

The local example

The developer quickstart uses fixed, synthetic customer records. It does not accept uploaded files or user-entered records and does not transmit the example data to an import endpoint. Clicking a copy button may write the visible example text to your device’s clipboard. Downloading a fixture saves a static file according to your browser’s behavior.

Contacting ImportAPI.com

An email link opens your chosen email application. Sending a message shares the information you include with the recipient and the email services involved in delivery. Include only information relevant to your question, and do not send passwords, API keys, private customer records, or confidential exports.

Questions about a message you have sent, including requests concerning that correspondence, can be directed to info@importapi.com. Do not include additional sensitive information merely to make an initial request.

External references

Some guides link to technical specifications or documentation on other websites. Following those links takes you to a separate site with its own information practices. This website does not embed those services into the local examples or give them access to an uploaded dataset.

Changes to the site

This explanation describes the features included in the current website. Adding analytics, forms, hosted processing, account features, or third-party services would require reviewing the information practices and updating this page. No such features are included in the supplied local examples.

Terms of Use for Guides & Examples

Sun, 20 Sep 2026 12:00:00 -0700

Useful resources.
Clear expectations.

The scope of the ImportAPI.com guides, examples, and public resources.

Using the resource collection

ImportAPI.com provides technical articles, workflow suggestions, and local examples about importing data and content. The resources are available without a site account. This website does not offer a hosted import endpoint, manage third-party credentials, store imported production records, or sell an API subscription.

Evaluate examples in your own context

The examples use synthetic data and deliberately limited rules. They are not a complete production parser, authorization system, or destination write path. Test any adapted approach with your actual requirements, approved source material, and appropriate access controls before using it with real data.

The guides distinguish design suggestions from format specifications and link to relevant technical sources. Software behavior and external documentation can change. Verify the specific library, configuration, and service contract used by your implementation rather than assuming an illustrative pattern guarantees a result.

Source rights and responsible use

Work with files and website content that you are authorized to process. Preserve source attribution, permissions, and relevant restrictions in your own workflow. Public visibility of a page or asset is not a substitute for checking whether your intended use is permitted.

Do not use this site to submit confidential material, credentials, or private datasets. The email contact is intended for relevant questions, feedback, and corrections, not as a production file-ingestion channel.

Site content and example fixtures

You may download and adapt the synthetic example fixtures for learning and implementation testing. This permission for the fixtures does not assert rights over third-party documentation linked from the site. ImportAPI.com branding and original editorial content remain associated with their source; contact the site regarding republication or broader reuse.

Availability and external resources

No uptime commitment, response-time commitment, or production service-level agreement is offered through this resource site. External references are provided for context; their inclusion does not establish a partnership, endorsement, or integration with their publishers.

Questions and corrections

Send questions about these resources or their use to info@importapi.com. Include the relevant page address and a clear description of the issue. The privacy policy explains the site’s local example behavior and email contact pathway.

Accessibility Features & Feedback

Sun, 20 Sep 2026 12:00:00 -0700

Accessibility

More ways in.
Fewer barriers.

Navigation, reading, and interaction features designed to make the ImportAPI.com collection easier to use.

Keyboard navigation

A skip link moves directly to the main content. Links, buttons, and disclosure controls have visible focus indicators. The main navigation uses ordinary links and a native disclosure for import categories rather than requiring a mouse hover. On small screens, the menu button exposes its expanded state and Escape closes an open menu.

The mobile menu keeps keyboard navigation within its visible controls while open. After dismissal through Escape or the backdrop, focus returns to the menu button. Browser zoom and device orientation can change the layout without requiring a separate mobile site.

Reading and structure

Pages use a clear heading hierarchy, descriptive link text, semantic navigation, and a main content region. Article pages include linked contents and source references. Image dimensions are declared to help keep layouts stable, and featured images have descriptive text alternatives. The article itself remains readable text rather than being embedded in an image.

Motion preferences

The website respects a system preference for reduced motion. The homepage also includes a Pause animations control for its decorative animation. The setting affects the current page; it is not stored as a personal preference. Important information does not depend on movement or hover effects.

Local examples and downloads

The quickstart’s stage controls use buttons with an announced pressed state. Result changes are described through a status region. No time limit applies to completing the example. When JavaScript is off, the source text and downloadable results remain available, although the interactive stage controls are not shown.

Ongoing feedback

These implementation features are not a claim of independent certification or universal compatibility with every assistive technology. Please report a barrier at info@importapi.com. Include the page address, the action you were taking, and your browser or assistive technology when relevant and comfortable to share.

A clear description of the issue helps identify a useful correction. The HTML site map offers another way to reach the collection without using the main navigation menu.

Secure File Imports: A Practical Defense-in-Depth Checklist

Thu, 13 Aug 2026 12:00:00 -0700

An import boundary accepts material created outside the destination application's control. That material may be ordinary but malformed, unexpectedly large, mislabeled, or deliberately harmful. Security therefore belongs throughout the workflow, not only in a file-extension check at the beginning. A dependable design limits what each processing stage can do and keeps unreviewed material away from trusted application state.

This guide proposes a defensive review for a business application accepting a small set of approved data files. It focuses on architectural questions and acceptance tests rather than exploit instructions. The goal is to identify what you allow, where you enforce it, and how an operator can understand a rejection without exposing unnecessary source information.

Begin with a narrow acceptance policy

List the formats and operations the business actually needs. An importer for customer CSV files does not need to accept arbitrary archives, executable content, or every document type supported by a parsing library. A smaller accepted set makes validation, error reporting, and testing more tractable. Expand that set only when a concrete use case justifies the additional processing path.

Define limits alongside formats. Consider source size, record count, field length, nesting depth where relevant, and processing duration. Make the limits part of the public handoff instructions. A clear unsupported-format or limit-exceeded message is more useful than an unexplained failure after a large amount of work has already occurred.

Validate in layers rather than trusting one signal

A filename extension and a supplied content type are useful clues, not proof of safety. Use a combination of format checks, controlled parsing, resource limits, and content-specific validation appropriate to the accepted material. Each layer answers a different question about whether the input belongs in the workflow. Keep these defenses independent enough that a misleading filename cannot bypass all of them.

Use defense in depth

The OWASP File Upload Cheat Sheet recommends defense in depth, including allowlists, generated storage names, limits, authorization, safe storage, and appropriate scanning. Use those principles to guide a review of your implementation. No single successful check proves that a file or every later transformation is safe.

Keep authorization separate from source claims

A field inside an uploaded file should not decide which tenant or account owns the resulting records. Derive the permitted destination from the authenticated application context and enforce it when changes are written. Treat source-provided ownership labels as data to validate, not as authority. This distinction matters even for files submitted by legitimate users.

Review permissions for more than submission. Status pages, error reports, original-file downloads, and derived records can expose sensitive information too. A user permitted to start one import should not automatically be able to inspect another team's rejected rows. Test the access boundary with separate accounts and deliberately restricted fixtures.

Isolate parsing from sensitive application capabilities

Parsing untrusted material should not run with broad access to application secrets or unrelated storage. Give the processing component only the resources needed for its task. Separate source handling from final database writes where the architecture allows it. A narrow worker role reduces the consequences of a parser failure or unexpected behavior.

Review library configuration as well as dependency versions. Disable unnecessary features and external lookups appropriate to the format and parser. Do not assume that installing an updated package automatically selects a safe configuration for every use case. Keep the chosen settings documented so future maintenance does not unknowingly expand the processing boundary.

Control resource consumption before expensive work

A small source representation can sometimes require much more memory or processing than its apparent size suggests. Set limits on the work after decoding or expansion, not only on the initial transfer. Use execution timeouts and cancellation behavior that leave a clear job outcome. Do not let one troublesome file indefinitely occupy shared processing capacity.

Test the limits with controlled fixtures near the intended thresholds. Confirm that rejection does not leave partial destination changes or orphaned temporary material. Provide an operator message that distinguishes a supported file exceeding policy from a damaged file that cannot be parsed. These are different correction paths and should not look identical.

Store untrusted material outside active content paths

Separate received files from application code and publicly served assets. Use generated storage identifiers, and retain the original filename only as metadata after appropriate handling. Do not construct storage paths directly from a user-controlled filename. Keep quarantine and approved-output locations distinct so later publishing steps cannot accidentally expose unreviewed files.

Define who can retrieve originals and for how long. Temporary files, rejected material, and derived previews need retention rules as well as successful imports. A cleanup job should report failures so sensitive source data does not remain indefinitely because one deletion attempt silently failed. Test cleanup as part of the import lifecycle.

Make previews inert and reports selective

An import preview should show data, not execute it. Render source strings as text unless the application has an explicit, reviewed sanitization and rendering policy. This matters for HTML-like fields, filenames, and error messages as well as document content. A parser returning a string does not make that string safe for every display context.

Show only the information needed to repair a rejected record. A missing required field does not justify displaying unrelated personal data from the same row. Use stable record references and field-level explanations. Keep detailed source access behind the same authorization boundary as the original import rather than copying full rows into broad operational dashboards.

Review exported reports as another boundary

An error report may later be opened in a spreadsheet or another application. Consider how that destination interprets cell contents and choose an output policy that avoids turning untrusted values into active instructions. The correct handling depends on the report format and the application consuming it; simply surrounding a value with quotation marks may not address every interpretation risk.

Make report generation an explicit, tested transformation. Include source values only when they are necessary for correction, and label changes made for safe display. An operator should be able to distinguish the original value from a presentation-safe representation. Security improvements should not make the underlying correction task ambiguous.

Restrict remote-source retrieval

Some import workflows accept a source address instead of an uploaded file. That adds an outbound network boundary. Restrict allowed destinations, schemes, redirects, and network access according to the application's purpose. Do not treat every address supplied by a user as an acceptable place for a server to connect.

For recurring imports, prefer a documented, authorized source configuration over arbitrary per-request destinations when that fits the use case. Keep retrieval credentials outside imported content and logs. The website import guide describes extraction and provenance after a source has been approved; source approval is a separate prerequisite, not a side effect of successful retrieval.

Build a reviewable operational trail

Record the job identifier, actor, destination, configuration version, stage outcomes, and stable error categories. Avoid logging entire files, access tokens, or sensitive fields by default. The trail should answer what happened and which policy applied without becoming an uncontrolled duplicate of the source dataset.

Test incident handling with a deliberately rejected fixture. Confirm that the operator can quarantine the result, identify related attempts, and verify cleanup without publishing the source. Review recurring rejection patterns for both accidental producer mistakes and signs that the boundary is being misused. Operational clarity is part of the defensive design.

Conclusion: secure the complete lifecycle

A safer import workflow combines narrow acceptance, appropriate authorization, isolated processing, controlled resources, careful display, and deliberate retention. Apply those controls to rejected material and generated reports as well as accepted records. The weakest boundary may be a preview or temporary file rather than the main upload handler.

Use the security overview to organize an implementation review and the developer documentation to inspect the site's example contracts. Before broadening format support, verify that one supported workflow has understandable limits, tested access controls, and a complete cleanup path. Predictable boundaries are more valuable than a long list of accepted file extensions.

Reliable Batch Imports: Idempotency, Retries, and Recoverable Jobs

Sat, 27 Jun 2026 12:00:00 -0700

The hardest import failures are often uncertain outcomes. A client sends a batch, the connection closes, and nobody knows whether the destination committed the records. Repeating the request may be necessary, but repeating it blindly can create duplicates or apply an update twice. A reliable batch workflow makes that uncertainty visible and gives both clients and operators a safe way to resolve it.

This guide proposes a catalog import with stable product identifiers and an asynchronous processing job. The design examples are patterns to evaluate in your own application, not a description of a hosted ImportAPI.com service. Start by defining what a repeated request means, then build retries and progress reporting around that agreement.

Distinguish a logical operation from an attempt

A logical import is the change the caller intends to make. An attempt is one effort to deliver or process that change. Network interruptions and worker restarts can create several attempts for one logical operation. Keep their identifiers distinct so the user sees one import with a history rather than several unrelated jobs.

For a catalog handoff, the logical operation might mean apply this source revision to this destination under this mapping. An attempt records when processing began, which stage it reached, and why it stopped. This separation gives support teams a coherent story and prevents infrastructure-level retries from automatically appearing as new business actions.

Define idempotency at the boundary

An idempotent operation is designed so repeating the same intended request does not repeatedly apply its effect. For a batch API, a caller-supplied request identifier can help the service recognize a retry. The identifier needs a defined scope, retention policy, and relationship to the request content. A token alone is not a complete implementation.

The AWS Builders' Library discussion of idempotent APIs explains the importance of expressing caller intent and handling repeated requests. Apply that idea deliberately: a repeated identifier with different content should not silently be treated as the original request. Define whether it produces a conflict, and preserve enough information to recognize the mismatch.

Keep request deduplication and record identity separate

Request deduplication asks whether the caller is repeating the same operation. Record identity asks whether a source product already exists in the destination. Both matter. A brand-new request may intentionally update existing products, while a retry of an earlier request should not create another set of those products.

Use stable external identifiers for records where the source can provide them. Decide how the destination handles a missing or conflicting identifier. Do not substitute a random identifier on every attempt if repeated processing should refer to the same record. When a source lacks stable identity, make that limitation explicit before promising safe automatic retries.

Make the job record a source of truth

A job record should report a meaningful stage and reconciled outcomes. Suggested fields include source revision, mapping version, total records evaluated, accepted records, rejected records, committed changes, and the most recent error category. Define these fields precisely. Accepted during validation is not necessarily the same as committed to the destination.

Avoid progress labels that imply more certainty than the system has. A completed upload means the bytes arrived, not that the destination has changed. A worker finishing its loop does not prove every requested write succeeded. Separate transport, validation, and commit states so an operator can determine which part of the workflow needs attention.

Choose atomic or partial behavior deliberately

An atomic batch aims to apply the intended set of changes together or not at all within the defined transaction boundary. A partial batch can accept some records while rejecting others. The right policy depends on the destination and its consistency requirements. A set of interdependent configuration records may need different handling from an independent product catalog.

For partial processing, retain record-level outcomes and a clear continuation strategy. A correction batch should identify which rejected records it replaces or retries. Do not force the caller to guess whether previously successful rows will be skipped, updated, or duplicated. The mapping guide explains how stable contracts make those decisions easier to express.

Retry only when the failure justifies it

Classify failures before retrying. A temporary service interruption may justify another attempt. A missing required product identifier needs correction, not repeated processing. An authorization failure should prompt investigation rather than an aggressive loop. Stable error categories help client software choose an appropriate response without parsing human-readable sentences.

Bound the retry policy

Use bounded retries and a delay policy appropriate to the system. Avoid synchronized retry bursts that add load during a recovery. Respect documented retry guidance from any downstream service. After the retry limit, leave a clear unresolved state and an operator path. Endless background attempts can hide persistent errors while consuming resources and delaying useful feedback.

Handle timeouts as unknown outcomes

A timeout describes what the caller observed, not necessarily what the destination did. After a connection closes, a worker might still finish a commit. Provide a status lookup by logical job or request identifier so the caller can reconcile the result before starting a new operation. This is particularly important when destination changes are not easily reversible.

Test the case explicitly. Interrupt the response after the server has accepted the request, then repeat the request with the same identifier. Confirm that the system returns or resolves the existing operation according to its contract. A happy-path demonstration cannot reveal whether the uncertain-outcome path creates duplicates.

Make commits and bookkeeping consistent

Consider what happens if records are written but the job's completion status is not. Conversely, consider a completion status being recorded before all intended changes are durable. Design the transaction boundary and recovery procedure so these disagreements can be detected and resolved. The exact mechanism depends on the destination and processing architecture.

Keep a durable relationship between the logical import and committed records or effects where that is appropriate. A reconciliation process can then inspect the actual destination instead of relying solely on a worker's last message. Do not claim exactly-once behavior unless the complete boundary and its failure assumptions have been established and tested.

Plan cancellation and correction

Cancellation is simple only before any effects occur. Once a partial batch has committed some records, stopping further work is different from undoing completed changes. Name those actions distinctly. An operator should know whether cancel means stop processing, roll back, or create a compensating operation.

For a catalog, a correction may be safer and clearer than attempting to reconstruct every prior state automatically. Preserve the source revision and outcome report so the correction can target the right records. Where rollback is required, establish what prior state must be retained and how concurrent updates affect that process before offering the action to users.

Test a realistic failure matrix

Use a small fixture batch and inject failures before validation, during processing, after a partial commit, and after completion but before the response reaches the caller. Add a repeated request with changed content and a corrected request with a new identifier. For each scenario, write down the expected job count and destination state.

Test the operator view too. Someone should be able to distinguish waiting, retrying, requiring correction, and completed without reading internal logs. Redact sensitive source values while preserving useful record references. A reliable workflow includes understandable recovery, not merely code that eventually succeeds in a controlled test.

Conclusion: make repeats and uncertainty explicit

Reliable batch imports depend on well-defined intent, stable identity, visible job states, and a tested recovery path. Separate the logical operation from its attempts, choose partial or atomic behavior deliberately, and treat timeouts as potentially uncertain outcomes. Retries become safer when they follow a contract rather than a guess.

Use the data migration workflow to connect these ideas to a complete project, and the developer reference to inspect a sample outcome shape. A small import that recovers predictably from an interrupted response is a better foundation than a fast pipeline whose repeated effects are difficult to explain.

AI Prompt Imports: Version Messages, Variables, and Evaluation Cases

Mon, 04 May 2026 12:00:00 -0700

A prompt collection often begins as a folder of useful text snippets. Over time, those snippets accumulate variables, role messages, model assumptions, output expectations, and evaluation notes. Importing them as undifferentiated strings loses the context that makes them reusable. A prompt import should create a reviewable application artifact, not simply move text into another storage location.

This guide proposes a small registry for an internal support-summary workflow. Each prompt needs a purpose, a version, declared inputs, a message structure, and representative tests. The approach is intentionally provider-neutral. Model-specific rendering remains an adapter responsibility rather than an assumption hidden inside the imported text.

Separate a prompt from a prompt execution

A prompt definition describes a reusable instruction and its expected inputs. A prompt execution combines a particular definition with actual data and a model configuration. Store those concepts separately. An imported definition should not contain yesterday's customer message merely because somebody copied a complete test conversation into a shared document.

For the support-summary example, the definition might ask for a concise issue summary and unresolved questions. The execution supplies an individual support conversation. Keeping the separation explicit helps prevent accidental reuse of confidential examples and makes it possible to compare multiple executions against the same reviewed definition.

Define a practical import contract

A useful record includes prompt_id, version, purpose, messages, variables, and an expected output description. Add review status and ownership appropriate to your team. The exact field names are a design choice, but their meanings should be documented. Avoid accepting arbitrary additional properties as operational settings without reviewing what authority they would grant.

Require stable identifiers so repeated imports can be interpreted as updates, new versions, or duplicates. A display title is not a sufficient identity; two teams can both call a prompt Ticket summary. Namespace the identifier or include an owning collection when that matches your application's organizational model.

Preserve roles and message boundaries

Many chat-oriented workflows distinguish instructions, user messages, and assistant examples. Flattening them into one large text string can change their intended relationship. Preserve the structured messages in the imported record and let a reviewed adapter prepare the representation expected by the chosen model or serving interface.

The Hugging Face chat-template documentation explains how role-and-content messages are converted into model-specific token sequences. Different models may use different control tokens. That supports a practical design rule: preserve the logical conversation separately from its final serialization, and verify the renderer instead of assuming one formatted prompt string works identically everywhere.

Declare variables rather than discovering them accidentally

List the variables a prompt expects and the kind of data each contains. In the support-summary example, conversation_text is required while preferred_language may be optional. Define what happens when an optional input is absent. A missing variable should produce a clear validation result rather than appearing as an unresolved template marker in a live request.

Validate that declared variables and actual references agree. A misspelled reference can silently omit important context. Keep interpolation rules simple enough to inspect. Do not evaluate arbitrary expressions or imported code just to substitute values into text. A prompt file should not acquire execution privileges merely because it uses a template-like syntax.

Keep examples free of unnecessary sensitive data

Examples are useful for explaining the intended output, but they can carry private information into every execution. Prefer synthetic or appropriately reviewed examples that preserve the relevant structure without retaining real customer details. Check attachments and hidden fields in the source export as well as the visible prompt text.

Store evaluation fixtures separately when they do not belong in the production prompt. A long collection of test cases can make an imported artifact hard to review and easy to deploy incorrectly. The registry should show which examples are included in the rendered prompt and which are used only by the evaluation process.

Treat imported instructions as reviewable changes

A prompt may influence what an application says, what data it requests, or how it uses tools. Importing a new version should therefore be a change-management event, not an automatic activation step. Separate received, validated, reviewed, and active states. Define who can promote a version and what checks are required.

Pay particular attention to instructions that request access to other systems or attempt to redefine the application's safety boundaries. A text file is not authorization. The application's tool permissions and data-access controls should remain outside the imported prompt's authority. Reviewers need to see meaningful differences between versions rather than only a new filename.

Version the whole behavior contract

A wording change is only one possible change. Variables, output structure, included examples, and rendering configuration can also affect behavior. Record these together in an immutable version or in explicitly linked versioned artifacts. An execution record should be able to identify the exact combination it used.

Use human-readable release notes to explain intent. For example, a revision may ask the summary to distinguish confirmed facts from unresolved questions. That note helps reviewers choose evaluation cases and interpret regressions. Avoid replacing a production version in place while leaving its identifier unchanged; later results would be difficult to reproduce or compare.

Evaluate the task, not just the formatting

For the support-summary workflow, create examples with a resolved issue, an unresolved issue, conflicting statements, and insufficient information. Specify what a useful result should contain and what it should avoid. Valid JSON alone is not proof that the summary faithfully represents the source conversation.

Use stable acceptance cases

Evaluate changes against the same reference cases before promotion. Keep observations concrete: a missing escalation reason, an invented resolution, or an omitted uncertainty is more actionable than a vague quality score. Include cases where the correct output explicitly acknowledges that the conversation does not establish an answer. This makes restraint part of the intended behavior.

Design rollback before activation

Keep a known reviewed version available when promoting a replacement. If the new version produces unacceptable results, the application should be able to return to an earlier definition without reconstructing it from old messages. Document which components roll back together, including the output parser and renderer when those changed.

Test the rollback procedure using a nonproduction environment and representative inputs. Confirm that execution records still identify their original versions after the active pointer changes. Historical results should not appear to have been generated by whichever prompt happens to be active today. That distinction matters for both debugging and honest evaluation.

Keep portability expectations realistic

A provider-neutral record format can make storage and review more consistent, but it does not guarantee identical behavior across models. Differences in message handling, context limits, tokenization, and output behavior may require adapter-specific configuration and fresh evaluation. Preserve those requirements as visible metadata or deployment settings rather than silently rewriting the imported source.

Treat migration as a testable project. Select representative cases, render the same logical prompt for each target, and compare the results against the task's acceptance criteria. Do not infer compatibility from a single successful example. The integrations guide describes destination boundaries without claiming that all systems share one universal contract.

Conclusion: import prompts as maintained artifacts

A dependable prompt import preserves purpose, structure, variables, and review history. It separates definitions from executions, keeps activation explicit, and evaluates behavior against meaningful cases. This gives a team a way to improve prompts without losing track of what changed or which version produced a result.

Start with the AI prompt import overview and inspect the example workflows. One small registry with clear ownership and repeatable tests is a stronger foundation than a large collection of anonymous snippets. The goal is reusable, accountable instructions that remain understandable as the application evolves.

LLM Data Imports and RAG Ingestion: Build an Evidence-First Pipeline

Wed, 20 Aug 2025 12:00:00 -0700

An LLM import workflow prepares material so an application can find relevant information and present it to a language model with useful context. It is not the same as training a model, and importing a document does not guarantee that every future answer will be correct. The practical challenge is to preserve reliable content, control access, and evaluate whether retrieval finds the right evidence for the questions people actually ask.

This guide proposes an ingestion workflow for an internal equipment knowledge base. Manuals, troubleshooting notes, and approved procedures must become searchable records without losing source identity or permission boundaries. The emphasis is on decisions and tests that make the collection maintainable, not on a particular model provider or a promised accuracy score.

Define the questions before processing the collection

Start with representative information needs. An equipment technician might ask for installation prerequisites, a specific error-code explanation, or the approved replacement procedure for a component. These questions require different kinds of context. A broad summary and a precise instruction may not benefit from the same retrieval unit.

Create a small evaluation set with expected source sections. Include questions that the collection should not answer and questions involving obsolete equipment. This makes it possible to test more than whether a system returns some text. The goal is to find relevant, current, authorized evidence and recognize cases where the available material is insufficient.

Preserve provenance during extraction

Give each document and revision a stable identity before splitting its text. Keep the source title, section path, and an accurate location reference where the extraction supports one. Derived chunks should connect back to the source record. Without that relationship, a plausible answer may be difficult to verify or correct.

Keep extraction quality separate from retrieval quality. A well-tuned search index cannot restore a missing table header or a paragraph extracted in the wrong order. Review representative documents before embedding them. The document import guide covers the earlier boundary between files and coherent text records.

Choose chunks that preserve meaning

A chunk should be useful when encountered outside the full document. For the equipment example, a troubleshooting step may need its error-code heading and the warning that precedes it. Splitting at an arbitrary point can leave a technically grammatical passage whose conditions are missing. Treat heading boundaries and complete procedures as candidate units.

The Microsoft Learn guide to document chunking discusses approaches such as fixed-size and content-aware splitting, along with overlap. Use those concepts as options to evaluate rather than a universal recipe. The right size and overlap depend on your material, retrieval approach, model constraints, and the context needed to answer the intended questions.

Keep metadata with the retrieval unit

Attach source identity, revision, section path, and access attributes to each chunk. A label stored only on the parent document is not useful if your retrieval layer cannot enforce it when returning individual chunks. Design the derived record so its necessary context survives movement between processing stages.

For equipment material, useful metadata may include product family, manual revision, and content status. Keep user-facing titles distinct from technical identifiers. Avoid filling missing metadata with guesses that later appear authoritative. An unknown revision is a review issue, not an invitation to assign the current year or copy the timestamp of an unrelated file operation.

Separate embedding inputs from application records

An embedding input is a representation selected for retrieval; it is not necessarily the entire stored record. Decide which text enters that representation and which metadata remains in structured fields. Including repeated navigation or boilerplate can obscure the distinctive content you are trying to retrieve. Removing too much context can make the chunk ambiguous.

Record the embedding configuration and the transformation version used to prepare each input. When either changes, identify which records need reprocessing. Keep enough lineage to compare results before switching the application to a new index. Treating all embeddings as interchangeable makes later evaluation and maintenance unnecessarily difficult.

Enforce access at retrieval time

An internal knowledge base may contain material for different teams or customers. Access controls should apply to retrieved content, not only to the source upload. A user who cannot read a document should not receive its paragraphs because they happen to be semantically relevant to a question.

Test the actual end-to-end permission behavior with a deliberately restricted document. Include derived chunks, cached results, citations, and preview views in that test. If the source permission changes, determine how the change reaches the index and any downstream copies. Permission metadata is useful only when the serving path enforces it consistently.

Treat imported text as evidence, not instructions

Retrieved material can contain instructions written for a human reader, quoted commands, or content that attempts to influence an automated assistant. Your application should preserve a distinction between its governing instructions and the external material supplied as context. Do not give a document authority to change access rules or execute tools simply because it was successfully imported.

Include a benign adversarial fixture in evaluation: a source passage that tells the assistant to ignore its normal task. Check that the application continues to treat the passage as quoted data. This is one part of a broader safety design, not a guarantee that delimiters or a single prompt sentence solve every injection risk.

Evaluate retrieval before judging generated answers

Inspect what the retriever returns for each reference question. Is the expected section present? Is it the correct revision? Does the result include enough context to understand the instruction? A fluent answer can conceal a retrieval failure, while an awkward answer may still rest on good evidence. Separate those observations.

Compare rules on the same questions

Compare candidate chunking rules on the same source set and questions. Record cases where overlap helps and cases where it produces repetitive results. Do not choose a configuration solely because one demonstration looks impressive. Keep the evaluation set small enough to review manually at first, then expand it with real failures encountered during use.

Design freshness and deletion explicitly

A revised manual should not leave obsolete chunks silently competing with current instructions. Decide whether old revisions remain searchable, become clearly labeled historical material, or are removed from the serving index. Maintain a relationship between source revisions and all of their derived records so updates can be applied completely.

Deletion also needs a propagation plan. Removing an original file is not sufficient if extracted text and index entries remain available elsewhere. Identify the storage locations and serving layers under your control, then test the deletion workflow. Record partial failures so an operator can see whether cleanup is complete rather than assuming one successful action covered every copy.

Keep operational costs observable

Track the work that ingestion actually performs: extraction, normalization, chunk creation, embedding, indexing, and review. An unchanged document may not need every stage repeated. A configuration change may require reprocessing even when the source bytes are identical. Distinguish these cases in job records before adding optimization rules.

Use measured workload information rather than invented savings percentages. Monitor failed jobs and reprocessing volume as well as successful throughput. A cheap ingestion path that repeatedly loses structure can create more review work later. The pricing and cost-planning guide provides a framework for identifying these costs without assuming a particular vendor's current rates.

Conclusion: ingestion is an evidence pipeline

A useful LLM import preserves source meaning, controls visibility, and makes retrieval behavior testable. Start with real questions and reviewed documents. Carry provenance and permissions through each transformation, then evaluate whether the resulting chunks support the intended task before relying on generated answers.

The AI and LLM import overview and AI knowledge workflow connect these decisions to a broader implementation plan. The strongest first milestone is a small collection with explainable retrieval results, not a large index whose contents and access boundaries are difficult to inspect.

Image Import Workflows: Metadata, Originals, and Useful Media Libraries

Tue, 29 Jul 2025 12:00:00 -0700

An image import is not just a transfer from one folder to another. A useful media record connects the original asset to its identity, dimensions, intended use, descriptive text, permissions, and generated variants. Without that structure, a growing library becomes a collection of duplicated filenames and thumbnails whose relationship to the original is unclear.

This guide proposes a workflow for a product-image library. The team receives authorized photographs and diagrams from several internal sources, then prepares consistent assets for a website. The examples focus on decisions you can make and test rather than assuming every file is safe, every caption is accurate, or every image should be published immediately.

Define what a media record represents

Decide whether a record represents a specific file, a creative work, or an image associated with a particular product. These concepts overlap but are not identical. The same photograph can exist as an original file, a cropped thumbnail, and a compressed web variant. A product can have several distinct photographs. Model those relationships explicitly.

A useful starting record has an asset identifier, a source reference, an original filename, a verified media type, dimensions, and a publication state. Add product associations separately. Avoid making the original filename the permanent primary key; two different sources may both provide an image called front.jpg without referring to the same product or the same photograph.

Accept a deliberate set of formats

Choose accepted formats according to the destination and the tools used to inspect them. A product library might initially accept a narrow set of raster images while treating vector artwork as a separate workflow. Narrow scope reduces the number of decoding, transformation, and display behaviors you need to understand and test.

The MDN image format guide describes common image formats and their characteristics, including differences in compression and transparency support. Use that reference to inform format choices, then test the actual source material and target environment. A format's general capabilities do not guarantee that every uploaded file is valid or that a particular conversion preserves the details important to your use case.

Inspect the file rather than trusting its label

A filename and a supplied media-type label are useful clues, not proof of content. Use appropriate validation and decoding within a controlled processing environment. Set limits before expensive work begins, and handle unsupported or damaged files as explicit outcomes. Keep the reason understandable to the person who supplied the asset.

Distinguish an invalid file from a valid but disallowed format. A product team can respond differently to those two reports. The first may require obtaining an uncorrupted original; the second may require an approved conversion or a policy change. The secure import guide discusses the broader boundary around untrusted files.

Preserve originals and name derivatives clearly

Keep the authorized original separate from generated variants when your retention policy allows it. A thumbnail is a derived artifact, not a replacement for the source. Record the transformation recipe so a future design change can produce another size or crop without guessing how an older variant was made.

Choose stable identifiers for derivatives and connect them to the parent asset. For example, distinguish a square catalog crop from a wide editorial crop. A filename containing the word final does not explain those relationships. Include output dimensions, format, and recipe version in metadata rather than relying entirely on naming conventions that people may interpret differently.

Check orientation and visual consistency

An image that appears correct in one viewer may need normalization before it is used consistently in another workflow. Inspect orientation handling with representative source files and ensure the stored dimensions describe the intended displayed result. Test portraits, landscapes, and images whose orientation metadata affects presentation.

Visual consistency also depends on cropping rules. A centered crop may work for a simple product photograph but remove essential labels from a diagram. Let the asset's purpose influence the rule. Avoid applying an aggressive automated crop to every image merely because the website uses square cards. A preview and review step can catch obvious losses of meaning.

Treat embedded metadata as a policy decision

Files may contain descriptive and technical metadata beyond the pixels. Decide what to preserve, what to expose, and what to remove from public derivatives. Information useful for internal provenance may be inappropriate for a public asset. The policy should reflect the collection and its intended audience rather than a blanket assumption that more metadata is always better.

Keep useful provenance in a protected media record even when a public derivative excludes embedded metadata. This separates operational traceability from public disclosure. Ask the content owner which fields are authoritative; an embedded description may be outdated or copied from an earlier file and should not automatically become a trusted caption.

Write descriptions for the destination context

Alternative text and captions serve different purposes. A caption can add editorial context, while alternative text should help convey the image's relevant meaning in the surrounding page. A file import can carry existing descriptions, but it should not assume that the same wording fits every placement of the asset.

For a product image, identify what the reader needs to learn from that view. For a decorative texture, the appropriate treatment may be different. Keep a review field for imported descriptions and avoid substituting filenames such as IMG_2048 into user-facing text. Descriptive quality is a content responsibility that file validation alone cannot solve.

Identify duplicates without confusing similarity

An exact file fingerprint can identify repeated bytes, but two visually similar images may be different files. Conversely, the same photograph can have several legitimate crops or compression settings. Decide whether your duplicate policy concerns identical originals, redundant derivatives, or duplicate product associations. These are different cleanup tasks.

Review similarity before removal

When similarity tools are part of the workflow, treat their output as a review signal rather than automatic permission to delete assets. Keep the comparison context visible. A small label change on product packaging may matter to the catalog even when the rest of the image looks nearly identical. Preserve a reversible decision path for uncertain cases.

Publish only after the record is complete

Separate received, inspected, reviewed, and published states. A successful file decode should not automatically make an asset public. The media record may still lack a product association, an approved description, or evidence that it belongs in the collection. Explicit states help route work to the right owner.

Validate the selected derivative before linking it from a page. Confirm that it exists, has the expected dimensions, and is associated with the intended original. Broken image paths and incorrect variants are operational problems that deserve tests. The media libraries solution shows how these stages can fit together without mixing upload status and publication approval.

Test the whole asset lifecycle

Build a fixture set containing a transparent image, a portrait, a diagram with small text, a duplicate file, and an unsupported format. Include an asset missing required descriptive metadata. Define the expected result for each stage, including which files should remain unpublished and what correction message the operator should see.

Test deletion and replacement too. Replacing a product photograph should not accidentally break another page that references the original asset. Decide how old derivatives are retired and how cached references are handled in your deployment. Record relationships clearly enough that maintenance work does not depend on somebody remembering why a particular file exists.

Conclusion: organize meaning around the pixels

A reliable image import creates a maintainable media record, not merely a successful copy operation. Keep original and derived assets related, validate the accepted formats, review descriptions, and distinguish file processing from publication. These choices make later resizing, replacement, and cleanup much less ambiguous.

Use the image import API overview to plan your first implementation. Begin with a small, representative collection and a clear destination use. An asset library earns its value when the team can find the right image, understand its history, and publish an appropriate version without reconstructing its story from filenames.

PDF and DOCX Imports: Preserve Document Structure and Source Context

Sat, 24 May 2025 12:00:00 -0700

A document import is successful only when the extracted information remains understandable outside its original file. A PDF may look orderly while storing text in an awkward reading sequence. A word-processing document may contain useful headings alongside comments, tracked edits, and embedded objects. Extracting every available character does not automatically produce a coherent record for search, review, or an AI knowledge system.

This guide proposes a workflow for importing an authorized collection of internal operating manuals. The destination needs readable sections, source references, revision information, and a way to flag uncertain extraction. The goal is not to promise perfect conversion. It is to make the quality of each result visible enough that a reviewer can decide whether it is ready to use.

Inventory the documents before selecting a parser

Begin with the formats and content patterns you actually have. Separate ordinary text documents from scanned pages, forms, heavily illustrated manuals, and table-dense references. Identify encrypted or password-protected files through an approved access process. A parser demonstration using one simple file is not evidence that it handles the whole collection.

Create a small sample set that represents the difficult cases as well as the common ones. Include a document with two columns, one with repeated headers, and one whose meaning depends on a table. Record the intended destination use. A full-text search index and a structured maintenance checklist may require very different extraction quality from the same source.

Separate detection, extraction, and interpretation

Format detection identifies what kind of file is being processed. Extraction obtains text and metadata. Interpretation decides how that material maps to the destination's fields or sections. Keep the stages distinct so a file with little extractable text does not get reported as a perfectly processed empty manual.

The Apache Tika parser interface documentation describes a parsing interface that emits document content and metadata. That is a useful architectural reference for separating extracted material from application-specific interpretation. A common interface does not eliminate differences among file formats or guarantee that every layout will produce the exact reading order your workflow needs.

Preserve source identity and revision context

Give each source document a stable identifier. Store its original filename as descriptive metadata rather than relying on the filename as the only identity. Filenames can be duplicated, renamed, or reused for revised documents. A content fingerprint can help recognize identical bytes, while a separate document identifier can connect different revisions of the same manual.

Record the revision supplied by the document owner when one exists. A file modification timestamp is an observation, not necessarily an authoritative publication date. Keep those concepts separate. When a section later appears in a search result or review queue, its provenance should identify the exact source representation used during extraction.

Inspect reading order rather than assuming it

A visually clear page can contain sidebars, captions, footnotes, and columns. Flattening these elements into a single text stream may interleave unrelated sentences. Test whether the extraction order preserves complete instructions. For procedures, a reordered warning or a detached prerequisite can change how a reader interprets the material.

Use a review strategy appropriate to the risk and complexity of the collection. Sample simple documents and give more attention to layout-heavy ones. Record uncertain sections instead of silently smoothing them into plausible prose. The importer should preserve evidence; it should not invent connecting text to conceal an extraction problem.

Treat scanned pages as a separate path

A scanned page may require optical character recognition before its words become available as text. That introduces a different kind of uncertainty from parsing a document that already contains text. Route these files through an explicit workflow and preserve a connection to the original page image. Do not imply that every recognized character is verified.

Check names, identifiers, units, and numeric values especially carefully in your acceptance examples. Where recognition quality is inadequate, use a review queue or request a better source. The decision to accept, correct, or reject an extraction should be documented. A nonempty result is not a substitute for evidence that the text can support its intended use.

Keep headings and tables connected

Headings establish scope. A paragraph under Installation requirements means something different from the same paragraph under Troubleshooting. Preserve section paths when they can be identified reliably. For a structured destination, keep the heading hierarchy in explicit fields instead of placing all content into one undifferentiated body string.

Keep table labels attached

Tables need their own attention. A row of numbers without column labels is rarely useful. Preserve a table caption and headers where the extraction supports them, or retain a page reference for review when it does not. Do not automatically reinterpret a complicated layout as a simple table merely because the text happens to align in the source view.

Decide what to do with document extras

Comments, tracked changes, attachments, and embedded media can be meaningful, but they may not belong in the published destination. Set a policy with the document owner. For example, an internal review comment may be relevant to an editorial workflow but inappropriate in a public knowledge base. This is a content decision, not just a parser option.

Make the chosen treatment visible in the import configuration. A reviewer should know whether comments were excluded and whether embedded objects were retained, linked, or flagged. Test the actual parser behavior with a representative file. Avoid assuming that the behavior of a desktop document viewer matches what the extraction library returns.

Produce reviewable sections

A useful section record can include document_id, revision_id, section_path, text, and a source location when available. Add extraction status and the configuration version. Be careful about claiming exact page locations for formats or extraction methods that cannot provide them reliably. A missing location should remain missing rather than becoming an invented page number.

For lengthy manuals, choose section boundaries that preserve complete ideas. Avoid splitting an instruction from its conditions solely to hit a fixed character target. When the next destination is a retrieval system, the LLM import guide explores chunking and evaluation. Keep the document extraction stage understandable before adding retrieval-specific transformations.

Validate more than the presence of text

Define quality checks that match the collection. A manual expected to contain several sections should not pass unnoticed with one short paragraph. A document containing an important table should not be marked complete when all table content disappears. Such checks can flag suspicious results without pretending to prove semantic accuracy automatically.

Maintain a reference set with expected headings, important phrases, and known difficult layouts. Compare new extraction versions against it. Track both unwanted additions and missing information. A cleanup rule that removes repeated headers may help one document and accidentally remove meaningful repeated warnings from another, so test its effect on different content patterns.

Plan retention and reprocessing

Decide how long to keep original files, extracted text, and rejected results. These materials can have different access needs. An operating manual may be widely readable while attached review notes are restricted. Propagate those distinctions to derived records rather than assuming that everything created during an import has the same visibility.

Store enough configuration history to reprocess a source after an extraction improvement. Keep the new result separate until it has passed the relevant checks. Reprocessing should not automatically overwrite a reviewed version with a different, unreviewed interpretation. The document import overview connects this lifecycle to a complete handoff.

Conclusion: extraction needs an accountable review path

Document imports work best when they preserve structure, expose uncertainty, and connect every useful result to its source. Begin with representative files and a destination contract, then separate parsing from interpretation and review. A clean-looking output should earn trust through checks, not through presentation alone.

Use the AI knowledge imports workflow when the destination is a retrieval system, or the content workflow guide for editorial migration. In either case, the strongest foundation is a document record whose source, revision, extraction method, and review status are easy to understand.

Website Content Imports: Extract Useful Content, Not Page Noise

Fri, 07 Mar 2025 12:00:00 -0700

A website content import should produce more than a pile of HTML. A useful result separates the article or product information you need from navigation, repeated footers, advertising, and presentation details. It also preserves where each record came from. Without that context, a content migration can replace one messy collection of pages with an equally confusing collection of database rows.

This guide proposes an import for a documentation site your organization controls. The destination needs a title, a canonical source address, a hierarchy of headings, body text, and a last-seen source revision. Working with content you are authorized to process keeps the project focused on a clear handoff rather than assuming that public visibility grants unrestricted reuse.

Define the content boundary first

List the pages and sections that belong in the import. A documentation migration may need tutorials and reference pages but not account settings, search results, or archived marketing campaigns. Decide how to handle translated versions and old revisions. An explicit inventory makes it possible to reconcile the destination against the intended source collection.

Set a policy for links to pages outside that inventory. They might remain external references, become flagged migration tasks, or be omitted from a particular export. Do not let a crawler expand the scope simply because another link exists. A bounded source collection is easier to review, test, and update than an open-ended attempt to import an entire domain.

Prefer structured exports when they exist

Before extracting rendered pages, check whether the content system can provide an authorized structured export or a documented content interface. Such an export may preserve fields that are difficult to recover from a page, including editorial identifiers, explicit hierarchy, and publication state. HTML remains useful when it is the available source, but it should not automatically be the first choice.

Compare one exported record with the visible page and the destination requirements. A structured source can still omit important context or contain fields whose meaning needs clarification. Choose the source that best preserves the required information, not simply the source that appears easiest to fetch during an initial experiment.

Separate retrieval from extraction

Retrieval obtains an approved source representation. Extraction identifies the relevant content within that representation. Keep these responsibilities separate so a transient retrieval problem does not become a silent empty article. Save enough job context to distinguish unavailable content from content that was successfully fetched but not understood by the extractor.

For browser-based implementations, verify the source's access arrangement rather than assuming arbitrary pages can be fetched from another origin. In a server-side implementation, restrict outbound destinations and redirect behavior. The examples on ImportAPI.com illustrate record design and transformation concepts; the local walkthrough does not fetch or import live websites.

Parse HTML into a tree

When your source is HTML, use a parser that understands document structure. Identify the main content container, then inspect headings, paragraphs, lists, and relevant attributes. Avoid treating markup as a single string with a few text substitutions. Nested elements, character references, and repeated navigation can make simplistic extraction misleading.

The MDN documentation for DOMParser.parseFromString describes parsing strings into a separate document. It also warns that parsing is not the same as sanitizing content for reinsertion into an active page. Keep imported HTML away from active rendering unless an appropriate sanitization and security policy has been applied. Plain-text display is often a simpler choice for extraction previews.

Preserve structure that carries meaning

A heading is not just a larger font. In documentation, headings organize instructions and identify the scope of examples. Preserve heading relationships where the destination supports them. Keep a table connected to its caption and column labels rather than flattening it into an unexplained sequence of values. Maintain code blocks separately from ordinary prose.

Remove only the right repetition

Remove presentation noise only through explicit rules. A repeated footer is usually different from a repeated warning inside a procedure. If the extractor removes every repeated sentence, it may discard meaningful cautions. Review representative pages with different layouts before applying cleanup across the entire collection. Record the extraction rule version with each result.

Normalize links without losing provenance

Relative links need the correct base address to become meaningful outside the source page. Resolve them using the actual source context, then decide whether they should point to a migrated destination or remain references to the original site. Track redirects and canonical metadata as distinct observations rather than blindly treating every address as the same document.

Use a stable source identifier in addition to a display URL when available. Page addresses can change during a migration. An editorial identifier or controlled mapping table makes it easier to recognize a moved article without importing it as a duplicate. Preserve the original address in provenance so reviewers can compare the destination with the source representation used for that job.

Handle images and attachments deliberately

An article may depend on a diagram, downloadable example, or screenshot. Decide whether the migration copies authorized assets, retains source references, or marks missing dependencies for review. A text-only export that silently removes a crucial figure may be technically well-formed but no longer useful to its reader.

Keep asset metadata separate from extracted body text. Record the source address, file type when verified, attribution information where relevant, and the destination reference. Review alternative text in context rather than copying filenames into descriptions. The image import guide covers asset records and derivative handling in more detail.

Detect meaningful changes between imports

A periodic import needs a definition of change. Comparing an entire HTML response can flag differences caused only by navigation or a rotating banner. Comparing normalized main content can be more useful, provided the normalization rules are stable and documented. Retain the extractor version so a rule change can be distinguished from an editorial change.

Choose what happens when a source page disappears. Immediate deletion, archival, and manual review have different consequences. For a documentation migration, a review queue may be safer than automatically removing destination content after one failed retrieval. A missing response and an intentional removal are not necessarily the same event.

Evaluate quality with a small reference set

Select representative pages: a short article, a long guide, a table-heavy reference, a code example, and a page with an essential image. Write down what the imported record should contain. Compare extraction results against that reference set whenever selectors or cleanup rules change. Include at least one page where navigation resembles ordinary body text.

Measure both omissions and unwanted additions. An importer that captures all the correct paragraphs but also includes a large sidebar can degrade search and downstream AI use. Human review should focus on the meaning and usability of the result, not merely whether the output has a nonempty title and body.

Plan the destination review

Provide a review view that shows the source reference, extracted structure, and unresolved dependencies. Let the reviewer distinguish content problems from migration-policy choices. A missing attachment may require another retrieval step, while an obsolete instruction may require editorial revision. Combining both into a generic failed-import status makes ownership unclear.

Keep review decisions attached to the relevant record and revision. If the source changes later, a prior approval should not automatically certify the new material. The content workflows solution offers a practical way to organize this handoff between engineering and editorial teams.

Conclusion: import usable content, not page noise

A successful website import preserves the material a reader actually needs and the context required to maintain it. Start with an authorized inventory, choose a suitable source representation, and test extraction against real layout variations. Keep links, assets, revisions, and review decisions traceable instead of treating them as incidental details.

Use the website import overview to plan the broader workflow. The most useful result is not the largest number of pages collected. It is a destination collection whose records are coherent, attributable, and straightforward to correct when the source or the extraction rules change.

Schema Mapping and Data Validation: From Source Fields to Trusted Records

Sun, 29 Dec 2024 12:00:00 -0700

Field mapping and validation solve different problems. Mapping decides how a source value becomes a destination value. Validation decides whether the resulting value satisfies the destination's requirements. Combining both into a single opaque transformation makes failures difficult to explain. Keeping them distinct gives operators a clearer preview and gives developers smaller, more reliable tests.

Imagine importing a membership directory into an application that expects member_id, full_name, country, and active. The source instead contains Member Number, Name, Country Name, and Membership Status. A good mapping does more than rename those columns. It defines which conversions are permitted, which assumptions require review, and which source values cannot be accepted. This guide proposes a practical way to make that agreement explicit.

Start with a destination contract

Write the destination fields before looking for automatic matches. For each field, specify its type, whether it is required, and any relevant limits. Clarify whether absent, empty, and null values have different meanings. In the membership example, an absent active field might mean no change during an update, while false explicitly disables a membership. Treating both states as the same would erase an important distinction.

Include the fields you intentionally do not accept. An import boundary should not expose every property on an internal database record. Administrative flags and ownership identifiers need their own authorization rules. A small, deliberate destination contract also gives teams a shared vocabulary when source exports use inconsistent labels.

Build a mapping that can be reviewed

Represent each mapping as a source field, a destination field, and a named transformation. For example, Membership Status might map to active through an explicit lookup that accepts Current and Paused. Avoid a vague truthiness conversion where every nonempty string becomes true. Such a rule would make the word Inactive behave very differently from the operator's expectation.

Show the mapping alongside representative source values. Automatic suggestions can save time, but make them proposals rather than invisible decisions. Name similarity is not sufficient evidence that two fields have the same meaning. A field called Account may refer to an organization, a customer number, or a financial category depending on the source system.

Normalize carefully and keep evidence

Normalization is useful when it removes accidental variation without changing intended meaning. Trimming surrounding whitespace from a country code may be appropriate. Rewriting a person's name, changing punctuation in an identifier, or discarding accents is a different kind of decision. Do not apply broad cleanup rules to every string just because they make comparison easier.

Retain a source reference and the mapping version for each processing result. Where privacy requirements permit, keep a restricted original-value record for disputed transformations. The application log does not need to contain the entire directory. A job identifier and a row reference can connect an error to protected source material without spreading personal information across operational systems.

Use schema validation for structural expectations

The JSON Schema object reference explains properties, required fields, and the treatment of additional properties. In particular, listing a property does not by itself make it required. An explicit required list expresses that requirement. Those distinctions are useful when reviewing a machine-readable contract for mapped records.

Apply the contract to an example

For the membership example, define member_id as a nonempty string and active as a boolean. Decide whether unexpected fields should be rejected or ignored, and expose that decision to the producer. Schema validation helps describe record structure; it does not independently prove that a member exists, a country choice is appropriate for your business, or the caller has permission to change the record.

Keep business rules outside basic shape checks

A structurally valid record can still violate business policy. A member identifier may already belong to another organization. A new expiration date may precede a contractual start date. A category may be valid in the application generally but unavailable to this particular tenant. These checks require context that a basic field-type definition does not contain.

Organize validations by the data they need. Field-level checks can run against one normalized value. Record-level checks compare values within a row. Batch-level checks detect conflicts across source rows. Destination checks consult current application state. This organization helps explain why some errors appear immediately while others require a complete batch or a final precommit review.

Make transformations deterministic

A transformation is easier to test when the same input and configuration produce the same output. Avoid conversions that silently depend on the machine's locale, current date, or a changing external lookup. When such context is necessary, capture it as part of the job configuration. A replay should not unexpectedly reinterpret last month's source file.

Consider a country mapping table maintained by the importing team. Record the table version used for the job. If a new alias is added later, that change should not retroactively alter the explanation of earlier imports. Determinism is not only a developer convenience; it makes support conversations and audit trails far more understandable.

Design errors for the person fixing them

Return a stable code, source location, destination field, and a helpful explanation. A message such as country_not_supported is useful to software, while a sentence explaining the accepted country representation is useful to a person. Supply both. Avoid a generic record_invalid response that forces the operator to guess which of several transformations failed.

Show only the information needed to correct the issue. A report about a missing member identifier does not need to repeat private notes from another column. Where the same cause affects many records, provide a summary and a way to identify affected rows. Hundreds of identical alerts can conceal the single mapping mistake responsible for them.

Version mappings and schemas together

A source field can change meaning without changing its name. A destination can introduce a new required field. Store a version for both the mapping and the destination contract so a completed job has an interpretable history. The version identifiers need not be complicated, but they must refer to configurations that can actually be retrieved or reconstructed.

Review compatibility deliberately. Adding an optional destination field may preserve old imports, while changing the interpretation of blank values can alter results significantly. Keep fixtures for older mappings when they remain supported. A successful test of the latest mapping does not prove that recurring jobs using a previous configuration still behave correctly.

Test the preview and the final write

A preview should exercise the same mapping and validation rules used by the final processing path. Otherwise, it can reassure the user about a result that will never actually occur. Differences caused by changing destination state still need handling: another process might create a member after preview but before commit.

Write tests for an unchanged repeat, a corrected rejected row, and a conflicting update. For each case, specify the expected record count and destination state. Include a test in which an optional field is omitted and another in which it is explicitly empty. The batch reliability guide connects those outcomes to safe retry behavior.

Review one example from beginning to end

Take a source row with a zero-prefixed member number, surrounding whitespace in the country field, and an unfamiliar membership status. Trace it through every stage. The identifier should retain its intended representation. The approved whitespace rule should be visible. The unknown status should produce a correction request rather than an invented interpretation.

Ask a person outside the implementation team to explain the result using only the preview and error report. Any point requiring an undocumented explanation is a candidate for clearer guidance. This small walkthrough is often more revealing than a demonstration containing only perfect source records. It tests whether the contract is understandable, not just whether the code executes.

Conclusion: make every conversion explainable

A robust import workflow treats mapping as a visible agreement and validation as a set of specific checks. Preserve meaning, keep important context, and describe failures at the level where they occurred. Automation is most useful when it handles known cases predictably and asks for help on genuinely ambiguous ones.

Use the developer reference to explore an example record envelope, and the data import page to connect these rules to a complete workflow. Before adding more transformations, make sure an operator can explain one accepted record and one rejected record without reading the implementation.

CSV vs. XLSX Imports: Build a Spreadsheet Workflow That Preserves Meaning

Tue, 19 Nov 2024 12:00:00 -0700

A spreadsheet import becomes difficult at the point where a neat table stops being a reliable data contract. A workbook may contain instructions above the header, identifiers with leading zeroes, formulas, multiple tabs, and dates that depend on regional conventions. CSV removes some workbook complexity but introduces its own questions about separators, quoting, and encoding. Treat both formats as source material that needs interpretation, not as a ready-made database.

This guide walks through a proposed customer-directory import. The destination needs an external identifier, a display name, and a country code. The objective is to preserve meaning, explain corrections, and make repeated imports predictable. The spreadsheet import API overview provides the broader workflow and related resources.

Choose CSV or XLSX for the actual handoff

CSV is often a convenient choice when the handoff is one rectangular table. An XLSX workbook is useful when a team already works with multiple worksheets or needs richer authoring context. That convenience does not mean every workbook feature belongs in the importer. Decide whether your workflow accepts one named sheet, asks an operator to choose a sheet, or treats several sheets as related datasets.

Write a small delivery guide for the producer. Include the expected header, one ordinary example, and one example containing an optional blank value. State which format variants you accept. A clear agreement with the exporting team is usually more valuable than adding complicated guesses for every possible spreadsheet layout.

Parse CSV records, not individual lines

A tempting shortcut is to split text on newline characters and then split each line on commas. That approach confuses separators with characters inside quoted fields. A customer name or note can legitimately contain a comma, and a quoted field may contain a line break. Use an appropriate parser and test it with those cases rather than relying on a small, clean sample.

The CSV format description in RFC 4180 documents quoting conventions, including doubled quotation marks within quoted fields. It also describes an optional header. Treat the RFC as a reference for a documented format, not proof that every file called CSV follows it. Your importer should state its accepted dialect and return a useful parsing error when a file falls outside that agreement.

Make header selection explicit

Consider a workbook whose first row says Customer export and whose actual field names appear in row four. A detector might guess correctly in a preview, yet fail when someone adds another instruction row. For recurring imports, store the selected worksheet and header rule with the mapping configuration. For one-off imports, show the operator what was selected before proceeding.

Check for duplicate headers, blank headers, and invisible whitespace around names. Two columns called Name cannot safely map to different destination fields without additional identification. Preserve column positions in the source reference so an error can point to the correct cell even when the displayed header is ambiguous. Do not silently discard an unexpected extra column.

Preserve identifiers as identifiers

A customer identifier such as 00127 is not a number merely because all its characters are digits. Turning it into a numeric value changes the identifier. Decide which fields are strings at the destination boundary, and test whether the source export has already altered their representation. An importer cannot recover lost leading zeroes with confidence unless an agreed rule supplies that information.

The same caution applies to long reference numbers, postal codes, and telephone labels. When a source has already changed a value, return a correction request rather than pretending the original can always be reconstructed. Display the source value and intended destination type in the preview so the problem is visible before records are committed.

Treat dates and money as explicit conversions

The text 03/04/2026 can represent different dates depending on the chosen convention. Do not guess silently. Ask the producer to use an agreed representation or associate a date interpretation with the mapping. For timestamps, decide whether the source includes a time zone and what a missing zone means. Keep a date-only value distinct from a moment in time.

Set a separate money policy

For monetary values, identify the currency, decimal convention, and rounding policy before converting. A punctuation mark may separate decimals in one export and thousands in another. Test the intended transformation with positive, negative, zero, and blank values. A blank balance should not automatically become zero unless that is a deliberate business rule accepted by the destination owner.

Decide how to handle workbook formulas

A workbook may contain both entered values and formulas. Define whether your chosen parser reads stored results, exposes formulas, or requires another evaluation step. Do not promise that formula results will always be fresh. A source workbook can be saved in a state that does not match the latest underlying data or the destination's calculation environment.

A practical handoff policy is to request a values-only export for fields that must be deterministic. Keep the original workbook available under appropriate access controls when an audit trail is needed. Include a formula-containing sample in acceptance testing so the actual library behavior is observed rather than inferred from what a spreadsheet application displays.

Build a correction-friendly preview

Show a small representative preview after parsing and mapping, while making clear that the full file still needs validation. Include the source row number, destination field name, and any transformation applied. Highlight missing required values and conflicting identifiers. Avoid a preview that makes every cell look successful merely because the parser produced a string.

For the customer-directory example, a useful report might identify row twelve as missing external_id and row twenty as having an unknown country code. Explain the expected value in ordinary language. Let the operator distinguish a spelling correction from a business decision, such as whether two records actually refer to the same customer.

Make duplicates a policy, not a surprise

There are at least two duplicate questions: whether the same identifier appears twice within the file, and whether that identifier already exists in the destination. Handle these separately. An existing destination record might be an intended update, while conflicting rows inside one source file may need review before either is accepted.

Define whether blank fields in an update mean leave unchanged, clear the destination, or reject the record. This decision is easy to overlook and can cause unexpected data loss. Demonstrate it in the import instructions with a small before-and-after example. The mapping and validation guide expands on destination contracts and transformation boundaries.

Test the awkward files before release

Build a fixture collection that contains a quoted comma, a quoted newline, a duplicate header, a blank required identifier, an unexpected sheet name, and a mixed-format date column. Include a file that parses successfully but fails business validation. Keep each fixture small enough that a maintainer can understand its purpose without opening a large workbook.

Test the error report as carefully as the imported records. Row numbers should match the source convention explained to the operator. Ensure the count of accepted and rejected rows reconciles with your definition of the data region. If blank trailing rows are ignored, document that behavior so the reported total does not appear arbitrary.

Conclusion: agree on meaning before automating

A dependable spreadsheet import is a collaboration between the producer, the operator, and the destination application. CSV and XLSX are useful containers, but neither removes the need for an explicit schema or a correction process. Keep identifiers intact, make conversions visible, and choose how updates behave before handling the first real customer file.

Start with the format guides and a small representative export. Once the workflow produces clear results for imperfect source files, automate the repeated steps. The best first milestone is not the largest workbook accepted; it is a predictable import that another person can understand and repair without asking its author.

What Is an Import API? A Practical Guide to Better Data Workflows

Sun, 27 Oct 2024 12:00:00 -0700

An import API is a boundary between data that exists somewhere else and the records an application is prepared to accept. The important work is not simply moving bytes. It is deciding what those bytes mean, whether they belong in the destination, and how to explain anything that could not be imported. A spreadsheet of customers, a collection of product records, and an exported knowledge base all need that boundary, even when their formats differ.

This guide develops a practical design for a small catalog import. The examples are design suggestions rather than a specification for a hosted endpoint. Start with a narrow, understandable workflow, then add scale and automation only after its behavior is clear. The data import overview connects this approach to the other formats covered on ImportAPI.com.

Define the destination before choosing the source

Suppose an application stores products with an external identifier, a display name, a currency, and a price. A source export instead contains columns called SKU, Product, Currency Code, and Retail Price. Before selecting a parser, write down the destination contract. Decide whether an empty price is allowed, whether a product identifier can change, and which currency codes your application accepts. These are product decisions, not questions a parser can answer reliably.

Keep the first contract deliberately small. Importing every source field creates obligations around fields nobody actually uses. Preserve the source separately when appropriate, but make the accepted destination fields explicit. A compact contract is easier to explain to the person preparing the file and easier to review when a source system changes its export.

Separate transport, parsing, and business rules

Transport answers how data reaches a processing boundary. Parsing turns a CSV, JSON document, or another representation into usable values. Business validation decides whether those values are acceptable in the destination. Keep these stages distinguishable in your implementation and in your error reports. A truncated upload, a broken quote, and an unknown currency should not all produce the same message.

This separation also gives you smaller tests. A parser test can inspect escaped text without creating any products. A business-rule test can validate an ordinary object without opening a file. A transport test can check that an interrupted transfer never becomes an accepted job. When failures are isolated, maintainers can change one stage without accidentally redefining another.

Choose an interchange shape with intention

A useful starting envelope contains a schema version, a source label, and an array of records. The source label describes where the records originated; it should not grant permission to write into a destination. Authentication and authorization belong to separate controls. Use record identifiers that remain stable between repeated exports so the destination can distinguish an update from a new product.

The JSON interchange specification, RFC 8259, describes objects, arrays, strings, numbers, booleans, and null. Its interoperability guidance is a useful reference when defining a JSON import contract. In your own design, represent identifiers as strings when leading zeroes or exact character sequences matter. Define money representation explicitly instead of assuming every producer and consumer will make the same numeric choices.

Keep a traceable path through transformations

A mapping might rename SKU to external_id and Retail Price to price_minor. Renaming is straightforward; converting a decimal price into minor units needs an explicit currency and rounding policy. Write the transformation down, attach a version to it, and retain enough provenance to explain which policy produced a destination value. Avoid conversions that silently discard meaning.

A useful preview shows the original value alongside the proposed value. For example, the user should be able to see that the text identifier 00042 remains 00042 rather than becoming 42. A record-level source reference makes later investigation possible without copying the entire original file into every application log. Restrict access to any retained source material.

Validate records before committing changes

Validation should answer concrete questions. Is the required identifier present? Is the product name within the application's length limit? Does the currency match the accepted list? Does another record in this batch use the same identifier? Add cross-record checks where necessary, but do not hide them behind a generic invalid-data message. Someone will need to correct the source.

A dry run is valuable when it applies the same transformations and validation rules as the eventual write. Report proposed inserts, updates, and rejected rows separately. A preview cannot guarantee a future write will succeed if the destination changes between those steps, so decide whether the final commit revalidates or uses a controlled snapshot. Describe that choice in the workflow documentation.

Make error reports a product feature

An actionable error includes a source location, a field, a stable code, and a readable explanation. For the catalog example, a report might identify source row seven, the currency field, an unsupported_currency code, and the allowed values. Avoid placing confidential source values in reports that broader support teams can access. Useful diagnostics do not require unrestricted data exposure.

Keep totals reconcilable

Distinguish a rejected record from a failed job. A file may be readable while containing three unacceptable rows. Conversely, an unreadable file may prevent any record-level evaluation. Provide totals that reconcile: accepted, rejected, and skipped records should have clearly defined meanings. A progress indicator that reaches the end of parsing is not proof that records were written.

Decide what happens when a batch is partly valid

There are two understandable starting policies. An all-or-nothing workflow commits no records until the entire batch passes its requirements. A partial workflow commits acceptable records and returns a report for the rest. Neither is universally better. A product catalog may tolerate partial progress, while a tightly coupled configuration import may require the whole set to change together.

Document the chosen policy before adding a retry button. Under partial processing, retrying an entire corrected file can encounter records already imported. Stable record identifiers and an explicit update policy help avoid accidental duplication. Let the person operating the workflow understand whether the next attempt is a continuation, a replacement, or a completely new job.

Design a useful job lifecycle

For a longer process, a job record can hold the current stage, source reference, mapping version, counts, timestamps, and a concise summary. Suggested stages might include received, validating, ready, committing, completed, and failed. These labels are a design choice; the important property is that each label corresponds to a real, observable condition.

Keep processing attempts separate from the logical job. A worker may retry after an interruption without creating another user-visible import. Record the relationship between attempts and the original job so support teams can follow what happened. The reliable batch import guide explores repeated requests and uncertain outcomes in more detail.

Measure outcomes that matter to the operator

For an initial release, measure completion time, rejected records by reason, and jobs that require manual intervention. A single average duration can conceal a workflow that is fast for tiny files but confusing for larger ones. Compare like with like, and keep metrics definitions stable enough to interpret changes after a release.

Also examine correction effort. An import that rejects bad records clearly can be more useful than one that accepts everything and creates downstream cleanup. Ask a teammate unfamiliar with the implementation to repair a deliberately imperfect catalog. Their questions often reveal ambiguous naming, missing source locations, or undocumented assumptions more effectively than another happy-path test.

Conclusion: make the boundary understandable

A good import API turns an uncertain external dataset into an explainable set of destination changes. Its contract, transformations, validation rules, and failure behavior should all be visible to the people who depend on it. Start with one source, one destination, and a small collection of representative records. Include empty values, duplicate identifiers, and one interrupted attempt before expanding the scope.

Use the developer quickstart to examine a local mapping example, then adapt its concepts to your own application. The goal is not to make every source look identical. It is to preserve important meaning while making the destination's expectations explicit enough to test, operate, and improve.

ImportAPI.com — Import Notes & Guides

ImportAPI.com

Big ideas startwith better imports.

One big idea. Seven ways to bring it in.

Data Import API

Spreadsheet Import API

Website Import API

Document Import API

Image Import API

AI & LLM Import API

AI Prompt Import API

Not sure where to start?

From “I have a file” to “I have a plan.”

Choose your source

Map and validate

Build your workflow

A clearer contract.A better starting point.

Different sources. Real-world workflows.

Customer Onboarding

Data Migration

Content Workflows

Fresh ideas. Smarter imports.

Less guesswork.More clarity.

Your next import starts here.

Import API Types & Workflow Guides

Meet your next starting point.

Data Import API

Spreadsheet Import API

Website Import API

Document Import API

Image Import API

AI & LLM Import API

AI Prompt Import API

Start small. Learn something useful.

Data Import API Guide

Give every record a clear destination.

Start with a small, explicit contract

Make mapping and validation visible

Plan the write and the retry

Turn the idea into a test

A little reading. A better workflow.

Your next import starts here.

Spreadsheet Import API Guide

From spreadsheet chaos to clear records.

Agree on the table before importing it

Preserve the meaning of each cell

Make corrections easy to find

Turn the idea into a test

A little reading. A better workflow.

Your next import starts here.

Website Import API Guide

Import the useful part of the page.

Choose a deliberate source inventory

Keep structure and remove noise

Review revisions and dependencies

Turn the idea into a test

A little reading. A better workflow.

Your next import starts here.

Document Import API Guide

Give your documents a second life.

Understand the collection first

Preserve source and section context

Build a review and reprocessing path

Turn the idea into a test

A little reading. A better workflow.

Your next import starts here.

Image Import API Guide

More than pixels. A better media library.

Define an asset, then accept the file

Create derivatives with a clear recipe

Review description and publication

Turn the idea into a test

A little reading. A better workflow.

Your next import starts here.

AI & LLM Import API Guide

Better context starts before the prompt.

Start with questions and trusted sources

Keep context with every chunk

Evaluate, refresh, and remove deliberately

Turn the idea into a test

A clearer contract.
A better starting point.

Less guesswork.
More clarity.