UMB.FYI Gets a New Workflow Engine
5 min read
Every week when I put together UMB.FYI, the weekly Umbraco community newsletter, I found myself repeating the same little tasks over and over again. Checking if a link still worked. Making sure I hadn’t already featured an item. Assigning the right category and icon. Adding tags. Double-checking the author’s name. None of these jobs were complicated, but taken together they were time-consuming, repetitive, and (if I’m honest) a bit boring.
UMB.FYI already leans heavily on automation to gather content — scraping blogs, social media, videos, and events so I don’t have to manually trawl the web every week. But while the news collection process was automated, the editing process was still mostly manual.
That felt wrong.
If the newsletter itself is all about automation in the Umbraco ecosystem, it made sense that its own production should reflect that. Instead of me spending my time clicking through the same checks and fixes each week, I wanted a system that could take on those tasks for me.
The Idea
At first I thought about extending some of the custom property editors I’d already created, but I knew that would quickly turn into a messy tangle of fixes. What I really needed was something purpose-built that I could reuse, extend, and trust to run every week without me babysitting it.
So instead, I built a workflow engine. Each “step” takes an item, does its job, and passes it along — almost like an assembly line. That way I can plug steps together like building blocks and let the pipeline handle the boring checks for me.
Right now it can run checks (making sure links work, flagging duplicates), apply rules (categories, icons, tags), enrich content (author details, summaries), and keep growing over time as I add new steps. It’s not huge or complicated, but it’s flexible, and that makes my weekly editing process feel less like a chore and more like a quick review of work that’s already been done for me.
Before I get into the technical details, here’s a quick look at what it actually feels like to run the workflow and watch a newsletter issue come together.
How It Works
The solution is split into two parts, a workflow engine, and a workflow runner UI component.
The Workflow Engine
The engine itself is written in C# and built to be flexible but lightweight. Rather than hard-coding a set of checks, I created a fluent builder that lets me define workflows in a clean, declarative way. That means I can chain together stages, tasks, and even subtasks, while keeping the definition easy to follow.
It’s fully integrated with Dependency Injection, so tasks can make use of existing services in my solution (HTTP clients, repositories, logging, etc.) without reinventing the wheel. Each workflow is structured hierarchically into stages, tasks, and subtasks, which makes it easy to break down complex processes into smaller, reusable pieces.
As the workflow runs, it streams realtime progress back to the UI using Server-Sent Events (SSE). Each task outputs logs, capturing any warnings or failures along the way. And if a task is marked as critical and it fails, the entire workflow stops immediately. There’s no approval flow or branching logic — a workflow either succeeds or it doesn’t. The goal here is speed and reliability, with enough logging to fix issues before publishing the final output.
public class NewsletterGeneratorWorkflowComposer : IComposer
{
public void Compose(IUmbracoBuilder builder)
{
builder.WithWorkflow<NewsletterGeneratorModel>(
"newsletter-generator",
"Newsletter Generator",
"Generates UMB.FYI newsletter content from collected media items.",
workflowBuilder => workflowBuilder
.AddStage("data-collection", "Data Collection", "Fetch and collect media items from various sources", stageBuilder => stageBuilder
.AddTask<FetchNewsTask, FetchMediaTaskConfig>("fetch-news", "Fetch News Items", "Retrieve news articles from the last 7 days", new FetchMediaTaskConfig
{
Range = TimeSpan.FromDays(-7)
}, isCritical: true)
.AddTask<FetchEventsTask, FetchMediaTaskConfig>("fetch-events", "Fetch Event Items", "Retrieve upcoming events for next 10 days", new FetchMediaTaskConfig
{
Range = TimeSpan.FromDays(10)
}, isCritical: true)
)
.AddStage("data-validation", "Data Validation", "Validate and filter collected media items", stageBuilder => stageBuilder
.AddTask<ValidateAndExtractMarkupTask>("validate-and-extract-markup", "Validate URLs & Extract Markup", "Validate URL accessibility and extract raw HTML markup for blog items", isCritical: true)
.AddTask<FilterDuplicatesTask>("check-duplicates", "Check for Duplicates in Past Editions", "Remove items that appeared in recent newsletters")
.AddTask<CheckEventCancellationTask>("check-event-cancellation", "Check Event Cancellation", "Remove events that have been cancelled or postponed")
.AddTask<CheckEventTbdDescriptionTask, CheckEventTbdDescriptionTaskConfig>("check-event-tbd", "Check Event TBD Descriptions", "Check for TBD/TBC in event descriptions and update from RSS feed or request approval", new CheckEventTbdDescriptionTaskConfig())
)
// Add more stages + tasks here...
);
}
}The Workflow Runner
Of course, an engine is only useful if it’s easy to run. For that, I built a runner UI using Lit and UUI, then exported it as a web component. This way it looks and feels like Umbraco, integrates cleanly with my current Umbraco v13 setup, and is already future-proofed for when I upgrade to v16.
The runner connects to the engine via SSE to show live updates as tasks complete. Tasks are grouped into collapsible stage sections, with logs visible inline so I can see warnings and failures as they happen. A progress bar tracks the overall run, and when the workflow completes, any stages with warnings or errors expand automatically for review. At that point, the runner also fires a completed event with the final model.
Everything is bundled into a single UMD file, which makes distribution simple.
Integration into Umbraco 13 required updating my umbraco-package.json with:
- A context app definition to host the runner.
- A JavaScript include for the runner’s UMD bundle.
- A path to an AngularJS controller that listens for the completed event and converts the workflow’s final model into my newsletter’s Block Grid structure.
- A view for the context app that renders the runner web component, passes in the workflow alias, and wires up the completed event to the controller.
{
"contentApps": [
{
"name": "Workflow",
"alias": "workflow",
"weight": 0,
"icon": "icon-nodes",
"view": "~/App_Plugins/UmbFyi/backoffice/views/apps/workflow.html",
"show": [
"+content/newsletter"
]
}
],
"javascript": [
"~/App_Plugins/UmbFyi/lib/workflow-runner.bundle.umd.js",
"~/App_Plugins/UmbFyi/backoffice/views/apps/workflow.controller.js",
]
}angular.module("umbraco")
.controller("Umb.Fyi.Controllers.WorkflowController", function ($scope, $http, editorState, listViewHelper, udiService, contentEditingHelper, notificationsService) {
var vm = this;
vm.handleWorkflowCompleted = function (event, args)
{
// Get media items from workflow model
var mediaItems = event.detail.model.mediaItems;
// Find the block grid property
var content = angular.copy(editorState.current);
var contentProperty = content.variants[0].tabs[0].properties.find(prop => prop.alias == 'content');
mediaItems.reverse().forEach((itm, idx) => {
// Convert selected media item to a block item
var contentItem = {
udi: udiService.create("element"),
contentTypeKey: "808cbd76-e16f-44cb-93ca-f2b207771514",
itemEmoji: itm.emoji ?? "🔗",
itemTitle: itm.title,
itemDescription: itm.summary,
itemLink: itm.link,
itemSource: itm.source,
itemTags: itm.contentTags
}
contentProperty.value.contentData.push(contentItem);
// Insert a layout item at the right index
var layoutItem = {
contentUdi: contentItem.udi
};
var groupAlias = itm.category.replace(/([a-z])([A-Z])/g, '$1-$2').toLowerCase()
var groupContent = contentProperty.value.contentData.find(x => x.groupAlias == groupAlias);
var groupLayoutIndex = groupContent ? contentProperty.value.layout['Umbraco.BlockList'].findIndex(x => x.contentUdi == groupContent.udi) : -1;
if (groupLayoutIndex >= 0) {
contentProperty.value.layout['Umbraco.BlockList'].splice(groupLayoutIndex + 1, 0, layoutItem);
} else {
contentProperty.value.layout['Umbraco.BlockList'].push(layoutItem);
}
});
// Trigger content update
contentEditingHelper.reBindChangedProperties(editorState.current, content);
// Show notification
notificationsService.success("Import Successful", `Successfully imported ${mediaItems.length} items.`);
}
});<div ng-controller="Umb.Fyi.Controllers.WorkflowController as vm" class="form-horizontal">
<umb-box>
<umb-box-header title="Newsletter Generator Workflow"></umb-box-header>
<umb-box-content>
<umbfyi-workflow-runner
workflow-alias="newsletter-generator"
ng-on-workflow-completed="vm.handleWorkflowCompleted($event)">>
</umbfyi-workflow-runner>
</umb-box-content>
</umb-box>
</div>What My Workflow Automates Today
The workflow is broken down into stages, with each stage handling a different part of the process. Right now, the pipeline looks like this:
Data Collection
- Fetch news items – retrieves all scraped news items from the last week.
- Fetch event items – retrieves upcoming events for the next 10 days.
Data Validation
- Validate & Extract Markup – checks links are active and scrapes their markup for easier processing later.
- Filter Duplicates – checks if a link has appeared in a previous newsletter and logs a warning if it has.
- Check Event Cancellation – verifies whether meetup events have been cancelled and removes them if they have.
- Check Event TBD – if an event contains “TBD” in its content, it re-fetches the event to see if it has been updated.
Data Normalization
- Clean Descriptions – strips markup from descriptions.
- Extract Author From Markup – attempts to determine the author of an item via regex patterns, meta tags, LD-JSON, or common CSS classes.
- Populate Source URLs – determines the source of a news item for attribution.
Content Categorization
- Assign Category – assigns each item to the standard newsletter categories (HQ, Community, Watch & Listen, Events, Training, Packages, Social, Misc).
Content Summarization
- Extract Content – extracts the main body of news articles or transcripts of YouTube videos.
- Assign Summary – for simple news items, copies the description directly.
- Summarize Content – for longer items, sends extracted content to OpenAI for summarization.
Content Enhancement
- Assign Emoji – adds a relevant emoji to each newsletter item.
- Assign Tags – assigns tags based on a predefined regex-matched list (more reliable than AI in my testing).
First Impressions in Practice
I’ve only used it on one newsletter so far, but it’s already made a difference.
The workflow immediately flagged a few URLs that had gone stale — the kind of thing that would easily slip through in a manual process. Tagging also worked really well. Using regex patterns instead of AI has made it more reliable, and it saves me from having to skim entire articles just to figure out their context. On top of that, categories, emojis, and other metadata are applied consistently, which eliminates the small inconsistencies that inevitably creep in when doing things by hand.
There are still gaps, of course. The duplicate detection missed one link from the previous week, which I’ll need to tighten up. And while AI summaries generally worked, the model occasionally altered people’s names, so I’ll need to tweak the prompt to keep it from rewriting proper nouns.
Even with those issues, the workflow is already improving reliability. Some tasks — like link validation and tagging — feel solved. Others — like duplicate detection and summarization — need iteration, but the foundation is solid.
Looking Ahead
Building this engine was about more than saving time — it was about making the newsletter process more reliable and more enjoyable for me to work on. It’s already paying off, but like any tool, it will get better the more I use it and refine it. Over the next few issues I’ll get a clearer picture of how much time it really saves, and I’ve already spotted areas to tighten up, like improving duplicate detection and refining the AI prompt so it doesn’t tinker with people’s names. I’m also keen to experiment with new enrichment steps, such as calculating read time or even summarising podcast content.
In short, the foundations are there, and now it’s about iterating and extending.
Until next time 👋