It's been a little while since I've blogged about Pipedream. I'm still a very happy user of the service, I just hadn't had anything to write about recently. That changed earlier this month when they announced GitHub support on thier blog. I decided to test it out and here's what I found. I also recorded a video of it in action so feel free to skip my prose and just skip to the embed at the bottom.

What is it, and what it is not...

First and foremost, the important thing to know is that this feature will sync a copy of your workflow to GitHub. That's probably obvious. I'll show you what that looks like in a moment, but it means you can all the benefits of GitHub (branches, history, etc) and apply it to your workflow.

However - while you can obviously clone your repo locally to get the files, you can't yet run the workflow locally. Now, I know that's been a request for a very long time and my guess (just my guess) is that this support is part of that process and in the future, we'll have that ability as well.

Also note that this feature is for people on the paid tier. Personally, that feels fair to me givin what Pipedream gives out on their free tier. I kinda figure if you're at the level of needing this then you are most likely at the level to be paying for it.

GitHub Support is Project Based

Being that I've been "away" a bit from Pipedream, another feature I hadn't worked with was Projects. Projects serve as a way to group workflows together, and it's something else the service has needed so I'm really happy to see it. You can even include folders within a project for further organization. Pipedream also supports Workspaceswhich groups Projects underneath them.

I'm going to stealborrow this graphic from their docs which shows this in action:

All of the above is just to point out that the GitHub feature is project based.

An Example

To demonstrate this (and remember, I'll have a video at the bottom showing the same), let's start off on the Projects screen:

Clicking on "New project" lets you create, you guessed it, a new project, but also allows you to enable GitHub Sync.

I had already connected Pipedream to GitHub so I was able to quickly select my account. Note two things in particular here. The repository name will default to the project name, but you're allowed to tweak that. Next, by default the repo will be private, but I've unchecked it above. (I'm going to delete the repo after I publish this post as I've already got a completed repo to share.)

Once the project is made, you're dropped into the Project UI:

One thing to note is that you can't start creating workflows. I was a bit confused by that but eventually figured out I needed to click the nice obvious blue "Edit" button on the right. Doing so prompts you to create a branch. All development on your workflows will be done via branches.

Now - let me pause here. I'll be honest and say that while I use GitHub every day, like, every single day, I keep things pretty simple in my repos. I do not make use of branches. Yes, I know that's wrong. That being said, I was a bit worried about all this and the cool thing is that Pipedream's UI makes the entire process braindead simple, even for me.

Anyway, you can take their default branch new, or do as I did, rename it:

At this point you get a "New" button in the project UI for creating stuff:

Ok, so if you make a new workflow here, that process pretty much remains the same. You add steps, you test, etc. The biggest difference is that you get this in the navigation:

Clicking that will give you an awesome list of changes (which for my test with one part to a workflow is pretty small) and one button to do the merge.

The Bits

Ok, so for my real test, I built a workflow with 4 steps:

• An HTTP trigger
• Use Spotify's API to get the top tracks from Depeche Mode
• Manipulate the results to return a smaller set of information for each track
• And finally, return the result in JSON.

If you want, you can hit this workflow here: https://eowg7ft5xg257tw.m.pipedream.net

This workflow is available on GitHub here: https://github.com/cfjedimaster/GHIntegrationTest2

If you visit that repo, you'll see it has a readme and one folder for my one workflow. Going into it shows the contents:

The Yaml file describes my workflow at a high level. In it you can see the steps and metadata:

schema: workflow/2022.04
name: GetDepecheModeTracks
settings:
  error_notification: true
triggers:
- id: hi_ZbHvJ98
steps:
- namespace: get_artist_top_tracks
  runtime: nodejs18.x
  uses: spotify-get-artist-top-tracks@0.1.1
  props:
    spotify:
      authProvisionId: apn_3JhQwL
    artistId: 762310PdDnwsDxAQxzQkfX
    market: US
- namespace: filterResults
  runtime: nodejs18.x
  uses: "./filterResults/entry.js"
- namespace: returnJSON
  runtime: nodejs18.x
  uses: "./returnJSON/entry.js"

For my two code steps, they both have their own folders with one file, entry.js, which just contains my custom code. Here's filterResults/entry.js:

export default defineComponent({
  async run({ steps, $ }) {
    return steps.get_artist_top_tracks.$return_value.map(r => {
      return {
        name: r.name,
        duration:r.duration_ms,
        album:r.album, 
        preview:r.preview_url, 
        external_url:r.external_urls.spotify
      }

    });
  },
})

And that's pretty much it!

The "I want to hear Ray's Smooth Silky Voice" version

As I said above, I recorded a video showing this in action, and you can check it out below. Enjoy!

Photo by Xavier von Erlach on Unsplash