How to create and test a GitHub Action that generates Supabase database types

How to create and test a GitHub Action that generates Supabase database types

·

11 min read

Recently I started a side project Billy, a React Native bill tracker app which uses Supabase for user management & database. Initially, I wasn’t too sure of all the fields that the Bill model should have, so as I developed the project, the data model for Bills changes as well.

🐙 Here's the project repository for Billy.

As features are gradually implemented, I have made around 4 iterative changes to the type of Bill and there's probably more to come in the future. Right now, I have to update the model type manually in my project every time I update the DB schema. Hence, if there is a way to dynamically generate types, that would be great!

Why dynamically generating types from a database is helpful

Below is the evolution process of the type Bill in less than a week 😆

V1

export type Bill = {
  id: string;
  payee: string;
  amount: number;
  category: string;
  deadline: Date;
};

V2

export type Bill = {
  id?: string;
  payee: string;
  amount: number;
  category: string;
  deadline: Date;
  userId?: string;
};
  • Just added POST Bill functionality. Since form data usually don’t have an id in it, I changed id to be optional.
  • Added a test user in Supabase. Since bills are to be related to a user, I added a userId here. However, it’s also optional, because I want users to be able to view/add bills without creating an account.

V3

export type Bill = {
  id: string;
  payee: string;
  amount: number;
  deadline: Date;
  userId?: string;
  category?: string;
  completedDate?: Date | null;
};

interface FormData {
  payee: string;
  amount: string;
  category: string;
  deadline: Date;
}
  • Changed id back to non-optional to follow the DB, and created another type for FormData that doesn’t require the id.
  • Decided to make category optional
  • Added “Mark bill as complete” functionality to the app. So I had to add a new field completedDate for users to update this.

And yes... if I have planned out properly the roadmap, maybe there won’t be a need for so many iterations for the Bill model. But it’s my side project so I get to do whatever I want 😈 However, the process does get tedious.

How to generate types from a Supabase database

At this point of time, I started a #BuildInPublic thread about my Billy project on Twitter, and got to know of a handy solution from Norris.

The idea of being able to generate types on the fly was a big wow moment for me 😍. So I looked up the Supabase documentation, and there is page on generating types. Following the instructions, the script I ran looks something like this.

Note the additional --version=2 parameter. This probably became a mandatory prop after some update to the openapi-typescript tool.

npx openapi-typescript https://{projectId}.supabase.co/rest/v1/?apikey={specialApiKey} --output=types/supabase.ts --version=2

The values for the anon key and URL can be found in the Supabase dashboard.

image.png

Running the script generates a file like this. With the definitions that are generated there, I could replace my current type for Bill with just the code block below.

import {definitions} from '../types/supabase';

export type Bill = definitions['Bill'];

Most of my code still works fine after changing the type. The only code I had to refactor is related to the deadline field which is Date type. The definitions has the field declared as a string, even though they do acknowledge it is a Date format.

Bill: {
    /** Format: date */
    deadline?: string;
}

Personally I think this is ok, since this forces you to convert date objects to appropriate ISO strings, so there is less ambiguity about passing the Date object that you have constructed in JS to Supabase. If you don’t like this you could override it in your self-declared type.

In the same documentation, they also included a section on how to update types automatically with GitHub Actions. This ensures that whenever you update the database, your GitHub action will automatically update the types that you have in your DB. Then, the next time you pull your source code, you would know what are the new changes and how to accomodate them.

In the docs, there’s a script to be added to the package.json as such.

"update-types": "npx openapi-typescript https://your-project.supabase.co/rest/v1/?apikey=your-anon-key --output types/database/index.ts"

However, I wasn’t too sure how to replace the values in the package.json script safely with environmental variables when I push the source code. I’m not very familiar with bash syntax, but I assume the script should look something like this for my project.

"update-types": : "npx openapi-typescript https://${SUPABASE_URL}/rest/v1/?apikey=${ANON_KEY} --version=2 --output types/database/index.ts"

To double confirm the syntax, I decided to use the Sourcegraph Visual Studio Code plugin to look for projects that have implemented this GitHub action.

Cross-reference code with Sourcegraph Visual Studio Code plugin

The Sourcegraph Visual Studio Code plugin can be installed through the Extensions panel.

Once you have it installed, you can find the Sourcegraph icon at the side bar. When you click it, you will see the usual UI to do a universal code search. To find other projects that also generate types, we can pass in a query npx openapi-typescript.

We see quite a few results, and luckily, in the 3rd project called tone-row/flowchart-fun, I see that they have package.json script that looks like a promising candidate. They have variables such as SB_URL and SB_ANON_KEY which resemble Supabase stuff ✨

"generate:types": "export $(cat .env.local | xargs) && npx openapi-typescript \"${SB_URL}/rest/v1/?apikey=${SB_ANON_KEY}\" --output types/database/index.ts",

With this cross-reference and knowledge, we know that we are missing out on wrapping the whole API endpoint in a string with escape characters. Then, we can modify our previous script to match this script.

"update-types": "npx openapi-typescript \"${SUPABASE_URL}/rest/v1/?apikey=${SUPABASE_ANON_KEY}\" --version=2 --output types/database/index.ts"

If we run this script locally via npm run update-types , we can see it generates the same output that we have done previously ✨

Out of curiosity, I also want to know if their GitHub workflow follows the same structure that was given in the example given in Supabase docs. The cool thing is with the Sourcegraph plugin, I can explore the repository in the editor itself.

Clicking on different workflow yml files in the .github/workflows folder

Sadly, this project is not using the generate:types script that they have written in a GitHub workflow. They’re probably only running the npm script locally. Thankfully with the help of the Supabase documentation folks, at least we don’t have to write the action from scratch.

How to create the GitHub workflow

This section has been merged to Supabase documentation here after publication of this article.

To add GitHub action workflows to your project, you need a new folder .github/workflows to store the workflows. For the workflow.yml that consist the steps for the GitHub action workflow, we will be using the sample GitHub action workflow on the Supabase documentation as a base. I have modified it a little to insert some GitHub secrets as environment variables, you can refer to the comments below.

name: Update database types

on:
  schedule:
    # sets the action to run daily. You can modify this to run the action more or less frequently
    - cron: '0 0 * * *'
    # for the workflow to be dispatched manually via GitHub actions dashboards.
  workflow_dispatch:

jobs:
  update:
    runs-on: ubuntu-latest
    env:
        SUPABASE_URL: ${{secrets.SUPABASE_URL}}
        SUPABASE_ANON_KEY: ${{secrets.SUPABASE_URL}}
    steps:
      - uses: actions/checkout@v2
        with:
          persist-credentials: false
          fetch-depth: 0
      - uses: actions/setup-node@v2.1.5
        with:
          node-version: 14
      # -- add echo -- #
      - run: |
          npm run update-types 
          echo "$(cat types/database/index.ts)"
      - name: check for file changes
        id: git_status
        run: |
          echo "::set-output name=status::$(git status -s)"
      - name: Commit files
        if: ${{contains(steps.git_status.outputs.status, ' ')}}
        # -- change git stage filename -- #
        run: |
          git add types/database/index.ts
          git config --local user.email "41898282+github-actions[bot]@users.noreply.github.com"
          git config --local user.name "billy-github-actions[bot]"
          git commit -m "Update database types" -a
      - name: Push changes
        if: ${{contains(steps.git_status.outputs.status, ' ')}}
        uses: ad-m/github-push-action@master
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          branch: ${{ github.ref }}

The changes include:

  • Addition of a trigger workflow_dispatch so that the workflow can be dispatched manually via GitHub actions dashboards if necessary.
  • Addition of an echo statement to see the content of the result file from running update-types script
  • Pass GitHub action secrets into environment variables.
  • Change the filename to be staged

If you are not familiar with GitHub action secrets, think of it as separate environment variables solely for GitHub actions. This is a screenshot of where you can add them at the settings of your GitHub project.

Settings > Secrets > GitHub secrets

To deploy the workflow, simply push the .github/workflows folder.

Then you will see the new workflow available in the Actions tab of your GitHub project. When you click on the additional options button, you can manually dispatch the workflow.

You can also view the logs of each workflow run if you encounter any issues. For example, here in the succeeded job, I can see the src/types/supabase.ts file content that is retrieved since we logged it out as part of the steps of the workflow.

You will see similar content as the definitions file you obtained by manually running the npm script

In the next section, I’ll introduce you to a handy tool that I learnt while building a custom composite GitHub action to test GitHub actions locally.

How to test GitHub Actions locally with act

act is a tool that lets you test GitHub actions locally. The benefit of this is that you get faster feedback without clogging up the GitHub Action Runner history on your project.

Refer to the GitHub repository README.md to see how to install act. You would need Docker installed too. The first time you start up act, you will be asked to choose the image that you want to create. For this GitHub Action, to test the functionality of updating the database types, the Micro image should be sufficient.

3 choices given to you for the image, can refer to docs.

After this config is completed, you can navigate to ~/.actrc to see the config for your docker image.

-P ubuntu-latest=node:16-buster-slim
-P ubuntu-20.04=node:16-buster-slim
-P ubuntu-18.04=node:16-buster-slim

Note that the Micro image does not have the full capability of a GitHub Action runner, so you will see errors related to git commands. I don’t want to explode my computer’s memory with the Large image (~20GB), but you can try if you like.

To run this workflow locally with act, we can run this bash command.

act -j update --secret-file .env
```we can pass in the following parameters:

Explanation of parameters

- `-j ${jobName}`
    - we only have 1 job declared in this workflow, so we can pass the **update** job
- `--secret-file ${filePath}`
    - for the project, I am already using `.env` for all the environment variables, so I can pass this same file for secrets. 
    ```bash
    SUPABASE_URL=your_url
    SUPABASE_ANON_KEY=hehe
- If you rather not have .env files locally for some reason, you can also pass variables manually with the parameter `-s SUPBASE_URL=your_url -s SUPABASE_ANON_KEY=hehe`

After running the bash command, here’s a simplified extract of the terminal output

[Update database types/update]   ✅  Success - actions/setup-node@v2.1.5
[Update database types/update] ⭐  Run npm run update-types
echo "$(cat src/types/supabase.ts)"
[Update database types/update]   🐳  docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/2] user= workdir=
| 
| > billy@0.0.1 update-types /Users/lyqht/self-study/Billy
| > npx openapi-typescript "${SUPABASE_URL}/rest/v1/?apikey=${SUPABASE_ANON_KEY}" --version=2 --output src/types/supabase.ts
| 
| npx: installed 10 in 3.328s
| ✨ openapi-typescript 5.2.0
🚀 ***/rest/v1/?apikey=*** -> /Users/lyqht/self-study/Billy/src/types/supabase.ts [28ms]
| /**
|  * This file was auto-generated by openapi-typescript.
|  * Do not make direct changes to the file.
|  */
| 
| export interface paths {
|   "/": {
|     get: {
|       responses: {
|         /** OK */
|         200: unknown;
|       };
|     };
|   };
|   "/Bill": {
|     get: {
|     .... } // i won't show you all of it, but you will be able to see all these
[Update database types/update]   ✅  Success - npm run update-types
echo "$(cat src/types/supabase.ts)"
[Update database types/update] ⭐  Run check for file changes
[Update database types/update]   🐳  docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/git_status] user= workdir=
| /var/run/act/workflow/git_status: line 2: git: command not found
[Update database types/update]   ⚙  ::set-output:: status=
[Update database types/update]   ✅  Success - check for file changes

If you run this, you can can see the output contains the paths portion for '/Bill' as a result of running the update-types script. This meant that it worked correctly 🎉

⚠️ Remember that these changes happen on the docker instance itself, you will not see any changes on your local folder.

Bonus: In Action

On 7 Apr, I added 2 new tables, and my bot was a good boi.

New definitions added to the src/types/supabase.ts file

Conclusion

You have learnt:

  • Why having types generated from your database can be helpful
  • How to create a GitHub workflow to generate types based on a Supabase database on the fly
  • How to manually dispatch a GitHub workflow
  • How to test GitHub workflows locally with act

That's a wrap folks! 🎉

https://c.tenor.com/eoM1uCVuXtkAAAAM/yay-excited.gif

Thank you for reading, hope you enjoyed the article!

If you find the article awesome, hit the reactions 🧡 and share it 🐦~

To stay updated whenever I post new stuff, follow me on Twitter.

Recently Hashnode introduced a feature for Sponsors ✨ You can buy a cup of tea for me over there if you like!

Did you find this article valuable?

Support Estee's Tech Blog by becoming a sponsor. Any amount is appreciated!