Articles

I like CircleCI. And I love orbs. The ability to centralize a library of reusable workflow, job and command logic is awesome.

But an orb doesn’t do everything you might want out of a central library. At its core, a CircleCI orb is just a single yaml file compiled from your repo’s src tree.

You interact with it, and write code in it, using files in directories - you even include scripts into it from files. But when your orb eventually arrives compiled into the consumer’s .circleci/config.yml - all of those “files” are gone. Left behind in the orb repo, or concatenated into the orb.yml you published.

Eventually all of that yaml gets compiled into the consumer’s final .circleci/config.yml - along with the yaml of any other orbs.

So what if you want to have a piece of your orb available to be consumed by a project outside of the scope of the core CircleCI config? (outside of a step)? This might be a config file or a script that needs to be consumed by your project’s tooling, that you’d like to store in a central location. Maybe it’s consumed by orb commands and scripts, or maybe only by the consuming repo’s workflow.

Your normal options all pivot around sticking that resource in some sort of external repository. That might be a docker image or S3 bucket or package repo. Then pull it down into your project. You might populate it from your orb build pipeline, or it might get versioned independently, leaving it out of sync with your library’s changes.

What if you could use your orb as the source of that resource? Passing the “file” from the orb directly into the filesystem of the job that’s being run?

Turns out, there is a way.

You can include a file from the orb repo into an environment variable:

      environment:
          TEMPLATE_CONTENT: <<include(templates/sample.yml)>>

      command: <<include(scripts/example.sh)>>

From here, you can just dump the variable content into a file:

      command: /bin/echo "$TEMPLATE_CONTENT" > /tmp/sample.yml

This is what the completed pattern looks like as a step:

  - run:
      name: "copy sample.yml from orb to the filesystem"
      environment:
          TEMPLATE_CONTENT: <<include(templates/sample.yml)>>
      command: /bin/echo "$TEMPLATE_CONTENT" > /tmp/sample.yml

With this trick, you can’t expand a parameter inside your include statement. To build any kind of utility command out of this pattern would require custom post processing before your orb.yml is compiled (and it wouldn’t be usable outside of your orb itself).

It’s also possible to include files outside of your src/ tree. circleci orb pack will happily resolve your paths:

          TEMPLATE_CONTENT: <<include(../README.md)>>

You can even use this to copy binary files. I successfully copied a small static binary to my remote job - a chmod +x and it ran perfectly. Obviously this isn’t something you’d want to do regularly, but it could be a useful tool for the rare situation.

CircleCI orbs let you put your reusable job and command logic in one easy to access location and share it around through many consuming repositories. However, orbs aren’t a collection of files - they’re one big compiled yaml file.

With the file copy trick, you can copy content from the orb repo directly into your consuming job’s filesystem.

A complete implementation of this in public orb form is available on github:
https://github.com/BrassTack/circleci-poc-orb-file-copy

Along with a sample consumer repo:
https://github.com/BrassTack/circleci-poc-orb-file-copy-consumer

Do you have feedback or thoughts? Feel free to post something on the github issues page for the POC orb or email me directly.

Are you wrestling with a CircleCI or CI/CD challenge? Lets chat! Send me an email, linkedin, or just schedule a free call - Contact Me