Skip to main content
Version: 15.2.1

GitHub Connector

The GitHub connector ingests data from a GitHub organization into the Catalog. It runs through the ibdm binary in one of two modes:

  • Sync — pull-based: queries the GitHub REST API and exits.
  • Run — push-based: exposes a webhook endpoint that receives GitHub organization events.

GitHub Enterprise Server is supported by overriding GITHUB_URL.

Commands​

ibdm sync github --include-internal-mappings=all
ibdm run github --include-internal-mappings=all

The commands use every internal mapping of the source. To select only some of them, or to add mappings of your own, see Mappings.

Configuration​

VariableRequiredDefaultDescription
GITHUB_TOKENYes(empty)GitHub personal access token (classic) or fine-grained token with the required scopes.
GITHUB_ORGYes(empty)GitHub organization to synchronize.
GITHUB_URLNohttps://api.github.comBase URL of the GitHub API. Override for GitHub Enterprise Server (e.g. https://github.example.com/api/v3).
GITHUB_HTTP_TIMEOUTNo30sHTTP request timeout (Go duration).
GITHUB_PAGE_SIZENo100Items per API page (1–100).
GITHUB_WEBHOOK_SECRETRun(empty)HMAC secret for webhook signature verification.
GITHUB_WEBHOOK_PATHNo/github/webhookHTTP path for inbound webhook events.

Authentication​

  • Personal Access Token (classic): required scopes repo and read:org.
  • Fine-grained Personal Access Token: required permissions at organization level — Repository → Metadata: Read-only, Organization → Members: Read-only.

Supported data types​

TypeSyncWebhook
repository✅✅
workflow_run✅✅
personal_access_token_request—✅
workflow_dispatch—✅

Webhook actions per type​

TypeActions → UpsertActions → Delete
repositorycreated, edited, renamed, archived, unarchived, transferred, publicized, privatizeddeleted
workflow_runrequested, in_progress, completed—
personal_access_token_requestapproved, createdcancelled, denied
workflow_dispatch(all — no action field)—

GitHub's push webhook event is also handled: it does not add a new data type, but re-upserts the corresponding repository item (refreshing repositoryLanguages). Subscribing only to the Repositories event, as in step 5 below, does not include push events — subscribe to Pushes as well if you want repository items refreshed on every push, not just on repository metadata changes.

Repository languages enrichment​

Whenever a repository item is produced (sync or webhook), ibdm calls GET /repos/{owner}/{repo}/languages and adds a repositoryLanguages field to the mapping context — a JSON object mapping each language to its percentage share, rounded to two decimal places:

{ "Go": 97.50, "Makefile": 2.43, "Dockerfile": 0.07 }

If the languages call fails, the repository item is still emitted without repositoryLanguages.

Internal mappings​

ibdm mappings list github prints the internal mappings of this source, and the Internal Mappings reference lists the Item Type each one writes.

Internal mappingSyncWebhookIts relationships point at
repositories✅✅ on repository and push events—
workflowruns✅ for every repository✅ on workflow_run eventsrepositories

The sync lists the repositories of the organization as soon as either mapping is selected, and fetches the workflow runs of each repository while listing them. Selecting workflowruns alone works, but its items then relate to repository items that nothing creates: select repositories too.

personal_access_token_request and workflow_dispatch have no internal mapping. An external mapping for either type can only be produced by ibdm run github: declare it syncable: false.

Setting up a GitHub webhook​

  1. Go to your GitHub organization's Settings → Webhooks → Add webhook.
  2. Set Payload URL to your public ibdm URL followed by GITHUB_WEBHOOK_PATH (default /github/webhook).
  3. Set Content type to application/json.
  4. Set Secret to the value of GITHUB_WEBHOOK_SECRET.
  5. Subscribe to the events that match the data types you want:
    • Repositories → repository
    • Workflow runs → workflow_run
    • Personal access token requests → personal_access_token_request
    • Workflow dispatches → workflow_dispatch
    • Pushes (optional) → refreshes the corresponding repository item on every push, see note above

See also​