Docent
Reference

CLI

The Docent CLI. sync uploads routes and Markdown knowledge, crawl reads a docs site, with every flag, every limit and what each command prints.

@usedocent/cli is the build-step half of Docent. It runs where your repository is, not in a browser, and it authenticates with the environment's secret key.

npx @usedocent/cli sync ./src/app --docs ./docs --key dk_sec_...
npx @usedocent/cli crawl https://docs.example.com --key dk_sec_...

The key comes from a flag or the environment, never from a file. There is no config file and no home-directory lookup, so a secret key cannot be committed by someone who did not know the tool would find it.

sync

Enumerates your Next.js App Router tree, uploads the route patterns, and optionally uploads a directory of Markdown as the knowledge pack.

npx @usedocent/cli sync <app-dir> [options]

<app-dir> is your App Router directory, such as ./src/app. A project root works too: if the directory you name has no route files of its own, the tool looks in app and src/app and prints which one it used.

FlagEnvironment variableDefaultWhat it does
--docs <dir>DOCENT_DOCSnoneA directory of .md, .mdx or .markdown to upload as the knowledge pack. Without it, only routes go up.
--key <key>DOCENT_SECRET_KEYnoneThe environment's dk_sec_ key, from Settings, Environments in the Console.
--release <name>DOCENT_RELEASEthe version in the app's package.jsonWhich release this route list belongs to.
--endpoint <url>DOCENT_ENDPOINThttps://api.usedocent.comThe Docent API origin.
--dry-runoffPrint what would be uploaded, then stop. No key needed.

It prints the list before it sends it. Route paths are your data, and /admin/pilot-pricing leaks a roadmap, so you see exactly what is about to leave the machine. A route it could not classify is printed with a ? and is not uploaded.

Then it writes what happened: how many routes were uploaded, how many knowledge chunks came from which directory, the knowledge pack version, and whether indexing started. A pack is inert until the index queue embeds it, so the Console's Knowledge page reports chunks and retrievable chunks separately and they match within a minute.

It exits non-zero, with the reason on stderr and nothing uploaded, when:

  • No secret key was given and the run is not a dry run.
  • No release could be determined.
  • The directory could not be read, or holds no routes at all.
  • --docs was given and there is no Markdown under it. Publishing an empty pack would erase what the guide knows, so it refuses rather than doing that.

Markdown only. Pages written as React components are not read; export them to Markdown first, or crawl the published site.

crawl

Reads a published documentation site into a knowledge pack. The fetching happens on Docent's own servers, not on your machine.

npx @usedocent/cli crawl <url> [options]
FlagEnvironment variableDefaultWhat it does
--key <key>DOCENT_SECRET_KEYnoneThe environment's dk_sec_ key.
--endpoint <url>DOCENT_ENDPOINThttps://api.usedocent.comThe Docent API origin.
--max-pages <n>200Page cap, a whole number from 1 to 200.
--dry-runoffPrint the planned pages and the ones robots.txt blocks, then stop. Nothing is started.

The limits are fixed and worth knowing before you point it at a site:

LimitValue
Schemehttps only
HostPublic addresses only. Loopback, private and link-local hosts are refused.
ScopeSame origin as the URL you gave, and under its path
robots.txtHonoured
Pages1 to 200, default 200
Page size2 MB, and a page is abandoned the moment it passes it
Content typetext/html only

Pages that need JavaScript to render are not read, and the job says so rather than reporting an empty crawl as a success. It starts from /sitemap.xml when there is one, following a sitemap index to its children and keeping the pages under the path you gave, and follows same-origin links otherwise.

A crawl publishes a new knowledge pack and replaces the current one, including a pack uploaded with sync. Run one or the other for an app, not both.

A started crawl prints its id and where to watch it. The Console's Knowledge page shows pages and retrievable chunks as they land. One crawl per app at a time: while one is queued or running, a second exits non-zero and says so.

In CI

DOCENT_SECRET_KEY=$DOCENT_SECRET_KEY npx @usedocent/cli sync ./src/app --docs ./docs

Run it on the branch that deploys, with the secret key of the environment being deployed to. The route list is stored per environment, so a production deploy and a staging deploy need their own runs with their own keys. The knowledge pack is stored per app, so one upload covers every environment of that app.

On this page