I recently published a piece of code called gitea-fly-connector (aka GFC) to simplify small-scale “GitOps” with Gitea and Fly.

It basically just wraps the fly deploy --remote-only command with a Web server that speaks the Gitea webhook language, so a deploy can happen automatically when commits are pushed.

If you don’t have a Gitea instance running on Fly but are interested in the idea, I have another post for you: Hosting Gitea on Fly.

Why?

The project was born from this idea:

Wouldn’t it be nice if Gitea could call the Fly.io API to deploy a new version of our code? Something like this…

UserGiteaFly Remote BuilderMy Fly AppBuilding image…Push ref(s)???Deploys new versionUserGiteaFly Remote BuilderMy Fly App

This would be great! The problem is that it’s not totally clear what should go in that ??? spot.

It may be possible to trigger a Fly deployment through an HTTP API call1, and Gitea has webhook support, so at least they both “speak HTTP” — but Gitea doesn’t know how to compose requests Fly understands. If we want these systems to communicate, something needs to help interpret.

In the long run, one exciting option is to integrate CI/CD into Gitea itself (see, for example, this issue).

But until that’s done, we’ll need some kind of… connector? A Gitea -> Fly connector?

There are a couple issues that the connector might have to deal with:

  1. Since the Fly GraphQL API isn’t particularly well documented, we’ll probably want to shell out to flyctl instead.
  2. Gitea webhook requests normally just include metadata, so the connector will need to get the actual snapshot of code to be deployed.

My solution was to create a lightweight server that receives Gitea webhook requests and facilitates the Fly deployment. This server, which I’ve called gitea-fly-connector, does the following:

  1. Accepts Gitea-formatted HTTP webhook requests.
  2. Checks out the pushed commit into a temporary directory.
  3. Calls fly deploy --remote-only in the directory.

The updated sequence diagram looks like this:

UserGiteagitea-fly-connectorFly Remote BuilderMy Fly AppBuilding image…Push ref(s)WebhookFetch repo (SSH)fly deployWebhook responseDeploys new versionUserGiteagitea-fly-connectorFly Remote BuilderMy Fly App

Advantages

GFC has some advantages over a “real” CD tool (like Jenkins or Argo). Not that GFC is better software; the advantages come from tradeoffs that you couldn’t make for a high-availability, multi-user, extensible build tool. But they’re perfect for my use case.

The major advantages:

  1. Stateless, more or less. All configuration is done via environment variables. There is some state, it’s just kept in-memory only, and nothing bad happens when it’s lost except possibly a failed build.
  2. Hardcoded deployment logic. The only “task” GFC can do is call fly deploy, everything else is handled by the Fly remote builder subsystem. This means GFC itself doesn’t need to containerize anything, and there’s no separate “runner” to deal with users executing arbitrary code.
  3. No public API. GFC can run entirely within a Fly private network and it has no admin dashboard you might need to expose. This is great for a personal solution — just use fly logs to see build output.

Disadvantages

How can I list advantages without listing disadvantages? The two are so often mirrors of each other.

  1. Doesn’t persist any information or communicate across processes. If GFC crashes during a build, you have to resend the webhook using Gitea’s UI.
  2. Not very flexible for different workflows — you need to fork and edit GFC if you want to change most behaviors.
  3. Because there’s no Web interface, GFC build results are only visible via container logs.

It’s funny to put this right after the disadvantages section, but if you want to learn more about GFC, there’s plenty of details and usage instructions in the README!

Reflections and learnings

Babashka

I wrote GFC in Clojure as an experiment with Babashka. Babashka’s instant startup was great, and the built-in libraries were helpful as well. I didn’t need to pull in a single additional dependency. Definitely will reach for Babashka again in the future.

Generating docs with rewrite-clj

I got to learn a bit about rewrite-clj, a cool library available out of the box in Babashka. It probably wasn’t strictly necessary for my use case, but why not learn a new thing? This was also my first experience with zippers, which was nice.

The use case in question: Parsing the GFC source code to generate documentation which then gets inserted into the README.md2.

I wrote a script gendocs.clj that parses the gfc.clj file. It looks for calls to a custom macro in GFC called defenv.

The defenv macro is used like this:

(defenv LOG_LEVEL
   "Defines the log level for the application to use"
   :parse-fn keyword
   :default :info)

When gendocs.clj finds a defenv call, it destructures the form and returns a vector with a map of information about each call, like:

[
 {:name "GFC_LOG_LEVEL"
  :docstring "Defines the log level for the application to use"
  :parse-fn keyword
  :default :info}
 ;; etc...
]

The relevant code is in the get-all-envs function below. It uses rewrite-clj to find and describe every defenv call in gfc.clj. The output of the function is used to automatically generate the Environment variables section of the GFC readme. (Source: gendocs.clj).

(require '[rewrite-clj.zip :as z])

(def zloc (z/of-file "gfc.clj"))

(defn find-next-defenv [zloc]
  (z/find-next zloc #(-> % z/next z/sexpr (= 'defenv))))

(defn get-all-envs []
  (loop [zloc (find-next-defenv zloc) all-envs []]
    (if-not zloc all-envs
      (let [[_ name docstring & {:keys [parse-fn default]}] (z/sexpr zloc)
            env {:name (str "GFC_" name)
                 :docstring docstring
                 :parse-fn parse-fn
                 :default default}]
        (recur (find-next-defenv zloc) (conj all-envs env))))))

Base64-encoded secrets

On an unrelated note, one learning that caused me some considerable pain was creating a Fly secret with an SSH private key in it, like this:

fly secrets set "MY_PRIVATE_KEY=$(cat mykeyfile)"

When GFC tried to use that key (after writing it to a file ssh_private_key and setting 600 permissions), I would get this lovely error:

Load ssh_private_key: error in libcrypto

After trying… a lot of things, I ended up base64 encoding the private key, and that worked:

fly secrets set "MY_PRIVATE_KEY=$(cat mykeyfile | base64 -w0)"

I’m not sure, but I suspect the issue was to do with the newlines. There were newlines in the ssh_private_key file, though…

(It may be worth noting that, despite some minor issues like the above, I really enjoyed the DX of the Fly platform and would recommend them for other folks looking for low-friction personal hosting.)

Footnotes

  1. As far as I can tell, this might be possible using the Fly GraphQL API — perhaps with the saveDeploymentSource() mutation, which accepts repositoryId and ref options, among others. But I haven’t dug into it very deeply. ↩

  2. This is a pattern I’ve used before. ↩