Connectors Overview
Most catalogs are not populated by hand: they are kept in sync with the systems where the entities already live — source-code hosting, the Mia-Platform Console, cloud providers, artifact registries, and security tools. Connectors are what keep the Catalog in step with those systems automatically, so the picture of your platform stays current.
Connectors are powered by an open-source (AGPL-3.0) engine called ibdm (International Berthing and Docking Mechanism), developed at github.com/mia-platform/ibdm. A single connector talks to one upstream system, reads the entities that live there, and writes them into the Catalog as items.
What connectors give you
- Automatic population. Entities are ingested from their systems of record — you don't recreate or maintain them by hand in the Catalog.
- Up to date. Connectors can react to changes as they happen, so a new repository, a new image, or a new cloud resource is reflected shortly after the creation.
- Provenance. Every item written by a connector is associated to that connector, so you can always tell which integration ingested a given entity.
- Relationships out of the box. Beyond the entities themselves, connectors can emit the relationships that wire them together, feeding the Catalog graph rather than a flat list.
Two ways to ingest
Each connector supports one or both of two complementary ingestion styles:
| Style | What it does |
|---|---|
| Pull | Enumerates the supported entities currently in the upstream system in one pass via the source's REST APIs and exits. Run it manually or on a schedule. |
| Push | Starts a long-running process that reacts to events pushed by the source (inbound webhooks, message-queue subscriptions, or similar mechanisms) and updates the Catalog in near real time. |
Not every source supports every mode — see the per-source pages below for details.
There is no built-in "incremental sync" loop: incremental behavior is achieved by combining periodic sync runs with event-driven run mode.
What you can connect
Connectors are available for nine upstream systems, grouped below by the kind of context they bring into the Catalog. Each link points to the source's setup and configuration reference.
Source-code hosting
| Source | Entities brought into the Catalog |
|---|---|
| GitHub | Repositories, workflow runs, access-token requests, workflow dispatches |
| GitLab | Projects, pipelines, access tokens |
| Bitbucket | Repositories, pipelines |
| Microsoft Azure DevOps | Repositories, teams |
Cloud providers
| Source | Entities brought into the Catalog |
|---|---|
| Microsoft Azure | Cloud resources (compute, networking, storage, …) |
| Google Cloud | Cloud resources |
Mia-Platform Console
| Source | Entities brought into the Catalog |
|---|---|
| Mia-Platform Console | Projects, revisions, services, clusters |
Artifact registries
| Source | Entities brought into the Catalog |
|---|---|
| Sonatype Nexus | Docker images |
Security
| Source | Entities brought into the Catalog |
|---|---|
| Sysdig Secure | Container-image vulnerabilities |
Shaping what lands in the Catalog
The shape of the items a connector produces isn't fixed by the connector itself: what a source's data becomes in the Catalog is decided by mapping files. There are two ways to get there:
- Use the ready-made mappings. Each connector ships mappings built for the Catalog's built-in Item Types, ready to use out of the box. They live in the
docs/mappingsfolder of theibdmrepository, one set per source. - Map to your own Item Types. Every source exposes a defined set of data types (for example, GitHub's
repositoryandworkflow_run). Starting from the data a source delivers, you can define custom Item Types and write your own mappings thatibdmmanages.
The mapping-file format and the full setup are covered in the configuration reference.
Authenticating a connector
Each connector authenticates to the Catalog as a service account, registered separately from the Connector item you create in the Catalog App — see Connectors configuration & installation for the full setup.
Where to go next
- Connectors configuration & installation — how to deploy
ibdm, wire it to the Catalog, and write mapping files, plus the per-source configuration references. - Items — the shape of the entities produced by connectors.
- Item Types — how to register custom kinds a connector can populate.
- Relationships — the graph connectors can feed.
- Catalog App → Connectors — how to create and inspect a connector record from the UI.