Moving a Configuration Between Clarive Installations With cla export and cla import

Two commands do most of the work in the new HCL support in Clarive 7.22. cla export writes the configuration of an installation into files. cla import makes another installation match them. The first post in this series covered what the files are and why they name objects instead of numbering them. This one covers the options people will reach for in their first week.

Writing the files

With no options other than the installation to read, cla export writes everything it can into a directory named after the configuration. cla -c prod export writes to cla-objects/prod. The name is there so that a TEST export and a PROD export can't overwrite each other in one Git checkout.

Inside, objects are grouped one file per family, and rules get a file each:

generic_server.hcl            every generic_server
bl.hcl                        every bl
user.hcl                      every user
role.hcl                      every role
resource.GitRepository.hcl    every GitRepository
pipeline.deploy-to-prod.hcl   one rule
event.on-topic-created.hcl    one rule

A file's name is the address of what it holds, minus the object's own label, so nobody has to learn a second naming scheme. Within a grouped file the blocks are sorted by address.

The layout can be changed. --split gives every object its own file, and --split=generic_server,user does it only for the classes named. --pack puts everything in a single clarive-pack.hcl, and --file does the same under a name you choose. --stdout prints the result and writes nothing, which is the quickest way to look at one object:

cla export --stdout --include role.deploy-manager

The command reports as it goes, on standard error, so --stdout still pipes a clean document. At the end it prints a summary like this one, from the documentation:

744 object(s) exported, 0 pulled in as dependencies
  files     147
  written   2
  unchanged 145
  into      /opt/clarive/cla-objects/prod

Only the files that were written are listed by name. A file whose content didn't change is counted and left untouched, so on the second export into a Git working copy the list of written files is the list of what changed.

Choosing what goes in

Most exports after the first are partial. --include decides what is in the set and --exclude takes things out of it. Both take addresses or glob patterns, both can be repeated and both accept a comma-separated list:

cla export --include generic_server
cla export --include 'role.*,project.*'
cla export --exclude resource.DBDest,resource.foo_object
cla export --include generic_server --exclude generic_server.scratch-01

A * matches inside one segment of an address and ** crosses segments. A bare prefix matches everything under it, so --exclude role drops every role. The older --filter option still works, with a leading ! meaning exclude.

Whatever the selection points at comes along with it. Exporting a server that goes through a proxy also exports the proxy, and whatever the proxy needs in turn. Here the documentation exports one project:

$ cla -c prod export --filter 'project.foo'
12 object(s) exported, 9 pulled in as dependencies
  written   12
  unchanged 0
  into      /home/me/cla-objects/prod

Nine of the 12 came along as dependencies of what the pattern matched. That is what lets a partial export be imported somewhere else. It also means --exclude can lose an argument with the dependency walk: if something included points at something excluded, the excluded object is pulled back in. --no-deps switches the walk off when an exclusion has to hold.

Classes that come from a feature or from a plugin outside the core product are left out by default, since Clarive doesn't guess whether their objects mean anything on another installation, or whether that installation even has the feature. They aren't dropped quietly. The summary lists each one it skipped, biggest first, with the option that brings it back. A reference into one of those classes is still written, as an address, but the object behind it isn't exported with it. The summary counts those references, and the installation receiving the files needs to have those objects already.

Put together, moving one project to another machine looks like this:

cla export --include project.acme-web --pack --file acme.hcl
# copy acme.hcl to the other machine
cla import acme.hcl

Reading the plan

cla import doesn't write anything until someone has seen what it is going to do. It reads the files, reads the installation and compares the two after putting both into the same form. An attribute a file leaves out counts as its default, and a reference counts as an address, so the plan lists real changes and not differences in how something is spelled. Then it prints the plan and asks:

  1. CREATE    generic_server.back-office (15 attrs)
  2. UPDATE    generic_server.front-desk (1 attr)
  3. RENAME    generic_server.web01 <- generic_server.oldweb
  4. DELETE    generic_server.gone -- Not present in the imported files
1 create, 1 update, 1 rename, 1 delete
Apply these 4 change(s)? [y/N]

Objects that already match are counted as unchanged and not listed. The answer defaults to no. For a script there is -y, which skips the question, and --dry-run computes and prints everything without writing. --diff adds the old and new value of every attribute that changes:

       - hostname = "front.example.invalid"
       + hostname = "front-new.example.invalid"
       - password = (sensitive)
       + password = (sensitive)

A secret shows as (sensitive) on both sides. By the time a plan exists the secret has been decoded, and printing it would put a live credential on the terminal and in the log of whatever ran the command. The plan still shows that it changed.

Three more actions can appear. SKIPPED, CONFLICT and ERROR each stop something. A conflict is an object declared twice in the files or matched ambiguously on the target, and an error is a block the importer couldn't understand. Either one refuses the whole import. By default so does a reference to something that is neither in the files nor on the target, because a dangling reference is how a migration quietly corrupts the installation it lands on. --allow-missing turns that error into a warning and skips whatever depended on the missing object, and the skip spreads. If B needs a missing A and C needs B, both B and C are skipped.

Rules go through the same save the Rule Designer uses. The steps are compiled before anything is stored, so a rule that doesn't build is refused, and a rule that did change is versioned exactly as an edit in the browser would be. A rule that hasn't changed plans as unchanged, and its version number stays where it was.

Objects are written in dependency order and linked to one another in a second pass once they all exist, so two objects that point at each other need nothing special. There is no rollback. If an apply fails partway, the run stops and reports what was applied and what wasn't:

2 applied, 1 failed, 3 pending
generic_server.front-desk: Cannot connect to the agent
Not applied: generic_server.back-office

The command exits with 3. After the cause is fixed, running it again plans the objects already applied as unchanged and carries on with the rest.

Renames and deletions

Renaming an object changes its address, and without help the importer would see one object disappear and a new one appear. A moved block tells it otherwise:

moved {
  to    = generic_server.web01
  names = ["oldweb", "web-01"]
}

If nothing exists at the new address yet, the importer looks for the most recent old name, then the one before it, and plans a rename for the first it finds. Clarive records renames as it imports them, and cla export writes the history back out as moved blocks, so a renamed object keeps its trail from one installation to the next.

An object is never deleted because the files stopped mentioning it. There are two ways to delete one. A removed block names a single object:

removed {
  at = generic_server.decommissioned
}

The block can stay in the files after the object is gone. On the next import it produces a warning that there is nothing left to remove, and nothing else.

The other way is cla import --prune, which proposes deleting whatever is on the target but absent from the files. It only looks at the kinds of object the files contain, so importing a directory of roles can never propose deleting a server, and a file holding only pipelines can never delete a workflow. A path given to the command that doesn't exist stops it before anything is planned. Under --prune, a mistyped directory name would otherwise read as "delete everything that was in there." The deletions appear in the plan with everything else and need the same yes. The documentation's advice for a first migration is not to pass --prune at all.

The full walkthrough, from a first export on TEST to the checks that run on every commit afterward, is in the command line guide. The next post is about those checks. If you'd like help planning a first migration between two of your installations, get in touch.