What's new in Roadie

You can now create collections of entities for yourself or to share them with others in your org, you can now register the identity of the user running a software template. You’re now running on Backstage 1.6, but this has no practical effects for you because Roadie takes care of the internal changes for you.

Entities collections through Playlists

In every devs day-to-day work, there’s a handful of websites, APIs, and services that you interact with more often, so it might make sense to group them together for easier access. Or, you may want to curate a subset of entities for your new hires. With the Playlists plugin, you can create collections of entities for yourself or share them with others.

To enable the Playlist plugin, go to Administration > Settings > Playlist

Register user identity in Scaffolder templates

You can now register the user’s identity in any software template. In your software template, you’ll be able to access user.ref as described in the following example:

- id: log-name
      name: Log Message
      action: debug:log
      input:
        message: 'Hello, ${{ user.ref }} ${{ user.entity.metadata.name }} !'

You’re running on Backstage 1.6

We have upgraded all our tenants to Backstage 1.6. Most of the changes in this version are related to internal changes, such as updates in React libraries or Yarn support in the framework. Therefore, you’re not going to be affected by this change.

This week, our engineers have managed to shave up to 38% off the total loading time on the Catalog list page, we continue our efforts on this front. Roadie will be at BackstageCon and KubeCon in Detroit, and we’d love to meet you there. Let us know if you’re going (and if you need a BackstageCon ticket)!

Performance improvements and upcoming work

Every Roadie instance behaves differently and may show different loading times, depending on the Catalog size and configured integrations. Our focus this week has been on removing duplicated API requests and ordering the API calls better. Our improvements are already deployed to your instances so you should be able to see a difference already vs. last week.

The most significant factor introducing an extended loading time is the lack of support for pagination in the Catalog API. We’re working with the Backstage maintainers to make this a reality in the near future. In the meantime, our improvements have managed to shave off up to 38% of the total loading time by cutting down half of the API calls we make.

Following the same approach with API calls, we have also reduced the docs list page significantly. Most pages are affected by the API optimization and should report improvements in their loading time.

We continue to investigate performance. We’ll be working on more optimizations.

Roadie at KubeCon NA + free passes to BackstageCon

Our team will be at BackstageCon and KubeCon in Detroit. If you’re attending, we’d love to meet up with you! Please let us know in the slack channel to make sure we connect over there.

Roadie will have a sponsor table at BackstageCon on Monday 24th, where our staff will deliver two talks. We’ll also have a booth at KubeCon, next to the CNCF Projects Pavillion.

We have two BackstageCon tickets that we’ll to give away to our clients on a first-come, first-serve basis. If you already have a KubeCon ticket and want to attend BackstageCon, let us know and we’ll give you a coupon code for a complimentary BackstageCon pass.

You can now arrange your software templates in groups in the Scaffolder. We have started to review the UI performance issues raised by some of our customers: expect improvements soon. Also, learn more about our new webhooks-based GitHub integration.

Group Scaffolder templates

When you offer more than a few software templates, it becomes trickier for your users to find the one they’re looking for. You can now group templates together by tags that you specify in the Administration panel. To enable this feature, go to Administration > Settings > Scaffolder.

Learn more on how to set up the Scaffolder in our documentation.

Performance optimizations on the way

Our engineering team is currently analyzing the bottlenecks that cause performance issues for some of our customers. Our objective is to reduce the loading times for an improved experience. We have found several points that can be optimized immediately and will begin to ship improvements as soon as possible. We are also filing mid-term and long-term improvements as we discover them.

Edge cases in the new GitHub integration

A few weeks ago, we released a new GitHub integration for the Catalog that lets you immediately see the changes you make in your YAML files reflected in Roadie Backstage. We have written a blog post detailing the tech behind this feature, and describing a couple of cases in which this integration works differently from the previous one. Learn more our blog post.

You can now define drop-down fields with options coming from an external API in your software templates, as well as use randomized valued based on a timestamp in your Scaffolder values. We also added the ability to set up authentication via a service token in the k8s plugin.

Dynamic drop-downs in the Scaffolder

Let’s say your software templates include a step in which the user has to select a database. Instead of having them figure out the DB reference and write it down, you can now give them a drop-down with valid values. We’ve added the ability for you to define drop-down fields in your software templates that show options fetched from an external API. Get started with drop-downs fed from external APIs.

Use randomized values in your Scaffolder template

If you’re writing a software template that generates, for example, a PR in a repo, you want to be sure the name of the branch you generate is unique. You can now use timestamp-based values to randomize data in the Scaffolder using ${{ '' | now }}. Check the following template for reference on how to use it:

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: test-now-filter
  description: test the new now filter
spec:
  type: service
  owner: user:default/kissmikijr
  steps:
    - id: fetch
      name: Fetch Skeleton + Template
      action: fetch:template
      input:
        url: template-url
        values:
          component_id: ${{ '' | now }}
          description: ${{ '' | now }}

Authenticate k8s via a service token

Previously, we only supported AWS EKS and Google Cloud’s GKE for configuring the k8s plugin. However, if you’re self-deploying k8s, the usual mechanism for authenticating is a service account token. Roadie now supports this mechanism in the plugin’s admin panel.

Now you’ll see nearly instant updates in you Catalog thanks to our new webhook-based GitHub integration. Our engineering team went through the recent Backstage Security Audit Report and confirmed the framework’s vulnerabilities are remediated in Roadie. Check out the latest developments of Tech Insights in a video demo by David.

Nearly instant Catalog updates via webhooks

Previously, Roadie used Backstage’s default GitHub integration, which relies on polling for discovering and updating entities. This approach had limitations due to API rate limits and could take time to reflect changes from your YAML files into the Catalog.

Roadie implemented a proprietary GitHub entity provider based on webhooks, so updates are done more efficiently. You’ll get nearly instant updates in the Catalog when you add a new entity, change a YAML file, or delete an entity.

No action is required on your end. For more information, check out our guide on autodiscovery.

Reviewing Backstage’s Security Audit

Our engineering team reviewed the full security audit report commissioned by the Open Source Technology Improvement Fund on Backstage OSS software. Most security issues had been addressed since Backstage 1.3, and all Roadie customers are using Backstage 1.5. Our team confirmed that all issues are mitigated for all our tenants.

Tech Insights demo

We keep working on Tech Insights with the help of our design partners. David has put together a demo to showcase what is currently available in Roadie Tech Insights and the direction that the team is taking. Check out the video demo.

You’ve been upgraded to Backstage 1.5. We keep making progress in Tech Insights and are taking on design partners, check out our new article and reach out to David or Orla if interested.

You’re running on Backstage 1.5

You’re using the latest stable release from the framework, which is mainly about requesting feedback on the refactoring of internal APIs surrounding Backstage’s backend and plugins. Roadie customers are not affected by these internal API changes because our team takes care of adopting the new practices for you.

More progress on Tech Insights

David, Roadie’s CEO, wrote an article outlining the reasons we’re building Tech Insights - Roadie’s first major proprietary feature on top of Backstage. We’re currently taking on design partners who will help us guide this work. Reach out to David or Orla if you’d like to take part.

Read the post

You can now use json:merge in your software templates and set up entity-specific content in the Iframe plugin.

New actions: json:merge

When defining a software template for the Scaffolder, you may find your self having to add options to a configuration file in a JSON format. In those cases, the json:merge action lets you merge an object into another. For example if you want to add a command to it your package.json, you’d do it as below:

- id: add-command
  name: Add command to package json
  action: json:merge
  input:
    path: package.json
    content:
      scripts:
        lsltr: "ls -ltr"

You can already start using json:merge in your templates without having to enable anything.

Iframe plugin v2

The v1 of the Iframe plugin was great for the homepage but it would display the same content for all entity pages. With this upgrade, you can specify entity-specific attributes that will be used to generate the iframe. Check out the guide to set entities Iframes up.

You can now enable the StackOverflow plugin and connect your company’s instance to Roadie. We’ve also added support for the Org plugin, which adds a “My Squads” item to the sidebar.

Connect your Stackoverflow today

We’ve enabled the homepage card for Stackoverflow. It accepts both public and private Stackoverflow questions. The plugin lets you add filters to the query so you can fine-tune what kind of questions and responses are featured on your instance’s homepage. To configure it, please follow the Stackoverflow plugin’s instructions.

Access your squads faster with the Org plugin

We’ve also added the Org plugin to Roadie, which will show logged-in users a sidebar nav item for ‘My Squad’ leading them to their squad page, or ‘My Squads’ if they belong to multiple teams.

You can now ingest Okta users and groups into your Catalog, sign up for our upcoming Backstage Users Unconference (September 8th), and check out progress with the Tech Insights paid add-on.

New feature: onboard Okta users and groups in your Catalog

If Okta is the source of truth for how your teams are organized and who is on which team, you can now make sure that these teams are up to date and represented in the Backstage model. We implemented an entity provider that allows you to ingest Okta users and groups in the Catalog. To enable this feature, refer to our documentation.

Upcoming event: Backstage Users Unconference

Roadie is co-hosting this free event for Backstage users to learn and share about managing an instance, from adoption strategies to domain modeling and everything in between. Sign up today!

More progress on Tech Insights

We’ve received great feedback and enthusiasm about the upcoming Tech Insights paid addon. Most of our engineers are focused on Tech Insights, and we’re making rapid progress. We’ll roll out experimental support for a subset of customers soon. Check out our latest progress demo:

That’s all for this week!

Cheers, Jorge L

You’ve been upgraded to Backstage 1.4, you can now document gRPC APIs, and new label filters in the Catalog.

You’re running on Backstage 1.4

Backstage 1.4 release is centered around security fixes and stabilizing APIs. Read more about the upgrade in our newsletter.

New feature: API Docs for gRPC/protobuf

As requested by some of our customers, we implemented support to render gRPC/protobuf API specs generated by protoc-gen-doc. We contributed the solution upstream and with the new release, we’re making it available to all our customers.

Better UI for labels in the Catalog

We’ve added a labels column to the Catalog. Additionally, you can now filter entities by label.

Roadie’s SOC2 Type 2 Compliance, Roadie 🤝 Spotify, a new way of previewing TechDocs locally, and a peek into TechInsights.

Announcement: Roadie’s achieved SOC2 Type Compliance

We have a set of mature and robust security and availability practices at Roadie and wanted to validate them against industry standards. We see this SOC2 Type 2 compliance achievement as a milestone in our ever-improving security journey. Read more in our blog post.

Roadie’s onsite: Deepening our relationship with Spotify

Last week, we met with Ben Lambert at the Spotify office in Stockholm, Sweden. We talked about our team’s ideas for the upcoming months and possible avenues for collaboration. Last month, we signed a partnership agreement with the team to continue to evolve Backstage and support our community.

New feature: Preview TechDocs locally

You can now generate your docs locally and check how they’ll look when deployed to Roadie. Check out the new feature’s documentation for more details.

Upcoming: A peek into TechInsights

As you may have already heard, Roadie is creating a TechInsights add-on that will allow you to have scorecards for your teams and services. We’ve made solid progress on the first iteration, check out our progress so far in this video demo:

This week we’ve released features to make it easier for you to do more with Backstage. You can now test and preview the structure of your TechRadar using locally hosted content, exclude files from entity validation, and use React 17 in your plugins.

Feature: Test and preview your Tech Radar

Roadie’s TechRadar plugin now allows you to construct Radars using locally hosted content. This feature is useful for finding the right structure for your Tech Radar. Learn more in the Tech Radar documentation.

Enhancement: Exclude files from entity validation by our Github App

Roadie’s Github App checks your PRs to help keep clean and entity YAML files across your codebase. We’ve added a configuration option for you to exclude entity files that you don’t want to us to keep in check. Read the documentation here.

Upgrade: React 17

We have updated React version to v17. This resolves various issues we were seeing as a result of outdated v16 used before.

Fix: Save button when editing props on a widget

The main save button on the menu bar did not always save the props when an admin was editing props on a widget. Now, the save button works correctly.

Thanks to this week’s releases, you’re now using the latest version of Backstage (1.3), you can choose the default kind displayed in the Catalog, you get twice as fast page loads, you won’t experience broken mkdocs links, and you can learn best practices to model your users and groups.

You’re running on the latest Backstage!

All Roadie customers are now using on Backstage 1.3! However, we’ve pinned the Scaffolder as to prevent any disruption to existing templates. We’ll be taking a closer look at compatibility these weeks.

New feature: Customise your Catalog’s default kind

Before, every time you opened the Catalog, you’d see a list of Components by default. Now, you can customise the kind of entities that you see listed as default, whether they’re Systems, APIs, or anything else.

To set the default kind for your Catalog, head to Administration > Setting > Catalog > Default Kind.

UX Improvement: Doubled page load speed

You may have already noticed this: Backstage now loads noticeably faster. We optimised our delivery strategy to reduce the page load time by 50%. We continue to work on improving our UI’s performance.

Fix: Broken mkdocs edit link

If you had set your main mkdocs file to something other than index.md, your edit button would take you to the wrong location. We fixed this error upstream in Backstage and it’s also available for all Roadie customers.

Heads up: if your existing Edit link disappears, you’ll need to add a repo_url to your mkdocs.yaml file.

Documentation: Modelling Users and Groups

We’ve put together best-practices for modelling your organisation’s teams and people to represent ownership in Backstage. We include instructions for doing so via Github Teams or manually through YAML. Check out the documentation.

This week, we finished upgrading all Roadie’s customers to Backstage 1.1.0, updated plantuml, and released quite a few UX improvements.

All Roadie’s tenants have been upgraded to 1.1.0

Upgrading Backstage is one of the most challenging points of adopting Backstage. We’ve heard reports from self-hosted teams spending up to 30% of their efforts managing upgrades.

In this upgrade, our engineers had to deal with breaking changes in core components such as authentication, Kubernetes, and permissions.

Thankfully, as a Roadie customer, you get the upgrades managed for you. All features, no pain!

Updated Plantuml

We updated the plantuml extension for mkdocs. You can now make use of more recent plant uml features in your TechDocs.

UX improvements of the week:

1. Filtering by kind (i.e. component, API, domain, etc.) was confusing for new users because the input had inconsistent styling. We’ve styled the kind filter to be in line with the rest of the filters.
2. Filtering by name used to only filter by metadata.name, but the UI displayed metadata.title, leading to confusing outcomes. Now, the filter consider both fields into account.
3. Sidebars didn’t allow editing, so teams would have to remove and re-add elements if they wanted a change, which could be frustrating. Now, sidebars allow editing their content.
4. The Intercom prompt would sit on top of other UI elements in a few scenarios, obstructing the user from using them. Now, the ‘Talk to us’ button is the last item of the sidebar menu.

This week, we’re releasing two Roadie exclusive features to make it easier to register and update entities and adding a small UX improvement.

Roadie exclusive feature: Entity Previews

At the moment, Backstage isn’t very friendly when it comes to changing your catalog-info.yaml: any modification causes a full override of the current entity state. Thus, you want to make sure your catalog-info renders the right information, and do so without much repetitive work. That’s why we’re excited to introduce Entity Previews to all Roadie’s customers:

With Entity Previews, you can write a catalog-info.yaml file on the spot, validate it, and launch a preview entity. If your catalog-info is good to go, you’ll find a fully formed entity page preview to check if it looks as you expect. Otherwise, you’ll see what went wrong.

You can find the new feature under Administration > Tools > Entity Preview. Learn more about this feature in our documentation.

Roadie exclusive feature: Entity Status

After a change, it can be hard to tell if an entity is still loading or if an error occurred when ingesting or updating it. We’ve introduced error statuses to the entity card to provide you with immediate feedback without having to dig through the locations log and the entity inspector. As a Roadie customer, you get this feature automatically!

Improvement: Custom fields saved on blur

When editing custom props in a component, the UI required you to click a dedicated save prop button before saving the component’s changes. This was confusing and would sometime lead to unsaved props work. Now, we’ve made it so props are saved automatically to prevent this from happening.

We’ve been focussed on added debugging tools to Roadie Backstage so you can figure out why things are not working as you expect.

This theme is an ongoing priority at Roadie, so you should see even more stuff like this next week.

Template previewing tools (feature)

We’ve reorganized the tabs in the administration area to ensure that the most widely useful pages are easiest to access. We’ve also added a new Tools tab.

Inside the Tools tab, you’ll find the old versions page — which is useful to folks who are building custom Backstage plugins — along with a new Template Previewer.

As the name suggests, the template previewer lets you preview scaffolder templates in real time, so you can see what the resulting UI will look like.

This feature was contributed upstream by the prolific Phil Kuang. Here’s the PR. We’re just exposing it.

Phil included this nice demo video which gets the idea across.

We’ll be expanding this TOOLS tab with more catalog debugging tools soon. Watch this space.

Stuck checks (bugfix)

In a small number of very specific repos, some customers experienced issues where Roadie pull request checks were blocking PRs which have nothing to do with Roadie.

We now only add roadie validator checks where a backstage related file has changed. Previously, we added a skipped check.

New Roadie logo

The new Roadie logo has rolled out to all Roadie Backstage instances. You’ll notice a change in the favicon and the sidebar.

We feel this is appropriate now since we are soon beginning work on our first major proprietary features on top of Backstage.

We still do plan to eventually allow you to replace these logos with your own branding. Although I don’t have an ETA for that feature at this time.

Self-service Auto-discovery

Auto-discovery enables Backstage to automatically look in named locations for catalog-info.yaml files.

We’ve always supported auto-discovery, but the configuration was not exposed to Roadie Backstage admins until this release. You can now turn auto-discovery on and off and edit the glob used for finding the catalog-info files.

Our documentation has detailed information explaining how to get the most from this interface.

Login prompt on settings page

Previously, Roadie hid the settings area from non-admins completely. Users who visited this area would only see Versions and Locations.

This turned out to be confusing, and admins would frequently report that they had lost the ability to see the settings. The cause was usually just that they were not logged into GitHub, which meant that the application couldn’t tell if they had the correct permissions to see the settings.

To help with this issue, we now display the settings to everyone, and prompt people to log into GitHub as a reminder.

Once logged in, we can check if the user actually has the permissions to see the settings or not.

Changelog RSS feed

You can now add the Roadie changelog to your Roadie Backstage Home.

This works via the HomePageRSS card which is now available in Roadie Backstage. This can be used to render any RSS feed, but we like to use the following props configuration to render this very Roadie Changelog.

{
  "feedURL": "/api/proxy/roadie-changelog"
}

catalog:register Scaffolder action

You can now use the catalog:register step in your scaffolder templates. It allows the software template to configure the creation of a new catalog item in the Backstage catalog.

We’ve been focussed on small UX improvements and Bugfixes for the past week or two so the changes are coming thick and fast. Expect more of the same for the next few weeks.

New logo

Roadie has a new logo! Within a week, this will replace the Backstage logos found in the application. You may want to warn your teams so they are not surprised.

We are beginning our first major initiatives which will add extra non-Backstage features to Backstage, so we feel it is appropriate to start owning the brand a bit more.

We still plan to allow you to upload your own logos.

Catalog troubleshooting guide (docs)

Debugging catalog ingestion problems isn’t always easy. To help you and your end users, we’ve written a comprehensive troubleshooting guide.

It explains how to use the Locations Log to track down problems and how to deal with common errors.

Scaffolder template creation guide (docs)

This new guide covers the syntax and options available when writing scaffolder templates. Every available action is covered, along with explanations of the arguments they take.

There’s also a troubleshooting section we will build on to cover common problems.

Placeholder secrets (enhancement)

Sometimes you want to make an authenticated HTTP request to a third-party service as part of a scaffolder template or a frontend plugin. We’ve added a few placeholder secrets to the secrets page so you can do this.

At some point we’ll make this page fully dynamic so you can create your own named secrets.

You can read about how to setup a proxy with these tokens in this guide.

Link to docs from the sidebar (enhancement)

You can now link directly to individual TechDocs documents from the sidebar inside Roadie. Previously this would break the application routing.

Prevent duplicate custom plugin cards (bugfix)

It was too easy to accidentally save empty and duplicate custom plugins in Roadie. This caused confusion for users who were writing their own plugins. This is now fixed.

Adding the catalog to the sidebar should not break the app (bugfix)

Adding a link to the catalog into the sidebar would previously render a broken white page for the user. It would also break the Home widget. This should no longer cause a problem.

Shortcut home page widget (feature)

We’re big fans of project management application Shortcut at Roadie. For our own day to day usage, we made a home page widget which shows the tickets currently assigned to the logged in user.

This is currently undocumented but if you’ve added a plugin previously it should be self-evident.

This week we have 3 significant user facing improvements.

Multiple tech radar support

We heard from a couple of customers that they need multiple tech radars because they have independent hardware and software divisions or different orgs with different needs.

You can now add multiple instances of the Tech Radar plugin to the sidebar in Roadie Backstage, and configure them independently. Each configuration now takes an ID, which can then be associated with an instance of the plugin.

The docs have been updated with more in-depth instructions.

Header bar theming

You can now set the main header inside Roadie Backstage to an arbitrary color. Perhaps to match a specific brand your company has.

It’s easy to do, just paste a hex color code into the input on the Theme Customization settings page.

Here’s a blinding organge look… 😀

Generic iframe embed component

We’ve added a widget which can be used to render content into a generic iframe. In the screenshot below, I’m using it to embed an Introduction to Backstage video on the Home page. You might use it to display charts, data or whatever else you need.

It required a few simple properties to set up. The docs are here.

The next improvement we make to this will allow the source URL to be configurable on a per component basis, rather than per component type, which is how it works now. This should make it more useful on the component pages, where you want to display different information for each component.

Azure Plugin

This week we have added support for the Azure DevOps service.

It supports builds and azure repos. For more information on how to get this working for you, please visit this link.

Page load performance improvements

We have improved caching in the Roadie app in order to:

• Removed 50+ extra requests to our backend to fetch static js/css/html files
• Improved the load time of the catalog page from < 10 seconds to just below 2 seconds!