⚠️ If you’re having issues with Cloudflare blocking our crawlers, read the troubleshooting steps here.

Documents

Create/Update Existing Document

PUT /v2/indices/{index public key}/documents/{document id}

Update existing document, or create a new document with predefined URL or ID.

{
  "custom_fields": {
    "title": "Example product",
    "description": "Description for example product",
    "price_cents": 599,
    "average_customer_rating": 4.5
  }
}

FieldDescriptionTypeNotes
custom_fieldsTop levelA map of custom fields key value pairsSupported data-types: text, integer, and double.

PUT /v2/indices/{index public key}/documents/

Omit the document id in the URL of the request if you include either a document id or a “url” field in the document content.

{
  "url": "https://www.example.com/",
  "id": "8700a03070a37982924597d6baa87be7",
  "thumbnail_external_src": "https://www.example.com/thumbnail-image.jpg",
  "custom_fields": {
    "title": "Example product",
    "description": "Description for example product",
    "price_cents": 599,
    "average_customer_rating": 4.5
  }
}

FieldDescription TypeNotes
urlDocument's URLstring
idDocument's idstringURL encoded as md5 hash.
thumbnail_external_srcThumbnails's URLstringA URL of an image used as a thumbnail for the document by default
Standard fieldAny of the standard fields described here except the URL. Depends on the fieldEach field should be a separate key-value pair.
custom_fieldsTop levelA map of custom fields key value pairs. Supported data-types: text, integer, and double.

If the URL does not contain the document id, the id of the document will be resolved to either:

  1. “id” field present in the document; or if the “id” is not present then
  2. an md5 checksum of the “url” field in the document

Document fields

  • Standard fields: You can find more information on standards fields here.
  • Custom fields: A map of key value pairs for custom fields. Multiple datatypes are supported. See “Custom field data types” below.

The supported data types for the custom fields are:

  • text
  • integer
  • double

Dates should be defined as UNIX timestamps with integer values.

Please note that once you have defined a specific data type for the custom field, the data type cannot be changed.

Indexing unsupported values will fail. If you have defined an unsupported data type, create a new custom field with a different name.

{
  "id": "123",
  "thumbnail_external_src": "https://www.example.com/thumbnail-image.jpg",
  "url": "https://www.example.com/page.html",
  "language": "en",
  "title": "An example article title",
  "main_content": "The text content of the article. The automatic highlight in search results will only be generated from the content of this field.",
  "custom_fields": {
    "image_url": "https://www.example.com/page.png",
    "article_categories": ["Block post", "Article"]
  }
}

Response

On successful queuing of the update, the endpoint returns an HTTP 202. Please note that it might take a few seconds for the changes to become available.