Upload files and ingest
A file-upload knowledge source takes files over the API. The upload stores them as pending documents and, when it can, starts indexing at once; the ingest route processes anything still pending; the document route shows each file's progress; and a search proves the content is live.
Who can use it: anyone holding an API key with the Documents scope for the upload and document routes and the Load knowledge graph scope for ingest; the sources list needs Knowledge search or Documents. Ingest also requires the key's owner to be the source's creator or to manage its access. Creating a source and uploading into it also require the key's owner to be allowed to create knowledge sources.
Prerequisites
- A file-upload knowledge source. Create it in the platform (Knowledge Sources in Studio, Add Source, then File Upload) or with
POST /public/v1/sourcesusing thefile_uploadconnector type; the File Upload connector page in the help centre lists what it accepts. - Files of an accepted type, each within the size limit your administrator set for the tenant.
Upload, index and follow
-
Find the source id.
GET /public/v1/sourceslists the tenant's sources; pick the one whoseconnector_typeisfile_uploadand keep itsid. -
Upload the files as multipart form data in the
filesfield, one part per file.curl -X POST "https://<your tenant host>/public/v1/sources/<source-id>/upload" \-H "Authorization: Bearer kapi_..." \-F "files=@quarterly-report.pdf" \-F "files=@vendor-policy.docx"The response lists what was stored under
uploaded(each with its documentid,filename,size_bytesandcontent_type), acount, any rejected files underfailedwith areason, and two fields that tell you what happens next:sync_startedsays whether this request started the indexing run, andsync_noteexplains why not when it did not. -
Start indexing only if the upload did not. When
sync_startedis true, indexing is already under way. When it is false andsync_notesays a sync was already running, that run will index the new files. When the note says indexing could not be started, call ingest:curl -X POST "https://<your tenant host>/public/v1/sources/<source-id>/ingest" \-H "Authorization: Bearer kapi_..."A 202 with
statusstartedcarries therun_id; a 200 withno_pending_documentsmeans there was nothing to do; a 409 means a sync is already running and names its run when it can. -
Follow each document.
GET /public/v1/documents/<doc-id>returns the record:processing_statusmoves frompendingthroughprocessingtocompleted, or ends asfailedwithprocessing_error,emptywhen no text could be indexed, oraccess_denied.chunk_countfills in as the document is indexed.import time, requestsBASE = "https://<your tenant host>/public/v1"HEADERS = {"Authorization": "Bearer kapi_..."}with open("quarterly-report.pdf", "rb") as f:up = requests.post(f"{BASE}/sources/<source-id>/upload", headers=HEADERS,files=[("files", ("quarterly-report.pdf", f))]).json()if not up["sync_started"] and "could not be started" in (up.get("sync_note") or ""):requests.post(f"{BASE}/sources/<source-id>/ingest", headers=HEADERS)for doc in up["uploaded"]:while True:rec = requests.get(f"{BASE}/documents/{doc['id']}", headers=HEADERS).json()if rec["processing_status"] in ("completed", "failed", "empty", "access_denied"):print(doc["filename"], rec["processing_status"], rec.get("processing_error") or "")breaktime.sleep(5) -
Prove it. Search with
source_idslimited to the source and a question the file answers; the hit'ssourcenames the document. See Search with citations.
Verify
Each uploaded document reaches completed with a chunk_count above zero, and a search scoped to the source returns a passage from it. In the platform, the source's Documents tab lists the same files with the same status.
Troubleshooting
- Entries under
failed: thereasonstarts withunsupported_type,too_large,empty_fileorread_failed. Fix the file and upload it again; the others were stored. - 413: the request as a whole exceeds the tenant's limit. Upload fewer files per request.
- 400 on upload or ingest: the source is not a file-upload source. Connector-backed sources sync on their own schedule;
POST /public/v1/sources/<source-id>/syncstarts one on demand. - 404 or 403 on ingest: the key's owner did not create the source and may not manage its access.
- 409 on ingest: a sync is running; it picks up pending documents, so wait for it rather than retrying.
- 502 on upload: nothing could be stored because the object store was unavailable. Tell your administrator.
- A document stays
pending: no run has picked it up. Checksync_startedandsync_notefrom the upload, then call ingest.