DK Development

I built my own Coding Agent harness and orchestrator — Part 1

2026-08-20T00:00:00+00:00

Three things lined up over the past few months and pushed me into building this. Token costs kept creeping upwards the more I leaned on cloud coding agents. Open weight models got genuinely good — good enough that “just run it locally” stopped sounding like a compromise. And underneath both of those, I just wanted to know for myself what’s actually possible when you cut the cloud out entirely, rather than take someone else’s word for it.

None of those three on their own would’ve been enough to get me off the couch. Together, they were.

This is part one of a series on localagent-box — something I built to answer that question. It’s a self-hosted orchestration layer that hooks OpenCode up to Ollama and GitHub. You give it a repo and a prompt, and it goes off and clones the code, creates a branch, runs an AI coding session with a local model, then commits and pushes the result back to GitHub. All on your own hardware. No cloud APIs, no usage bills.

The three things that pushed me here

The token bills kept climbing. Not one shocking invoice — a slow drift. The more I folded cloud-hosted agents into my day-to-day, the more sessions I was running, and the more each session cost as I asked them to do bigger chunks of work. Nothing that broke the budget, but enough that I started mentally flinching before kicking off a bigger job. That’s not a great place to be if you want to use these tools freely.

Open weight models stopped being the fallback option. A year ago, running a local model for coding felt like a novelty — something you’d do to prove a point, not to get real work done. Models like qwen2.5-coder changed that calculus. They’re not frontier-model good, but they’re good enough for a real slice of the work I throw at an agent, and that slice is only growing.

And I just wanted to see for myself. I could read benchmarks and other people’s takes all day, but the only way to actually know what a 7B model running on my own GPU can and can’t handle unsupervised is to build the harness and let it loose on real repos. This whole project is as much an experiment as it is a tool.

Why not just use OpenCode directly?

OpenCode is great — it’s a terminal-first AI coding tool. But it’s built for interactive sessions where you’re sitting there at the keyboard. I wanted something I could fire-and-forget: kick off a job via the web UI or an API call, go do something else, and come back to a branch ready to review.

LocalAgent-box is the orchestration layer in the middle — it talks to OpenCode, handles the git plumbing, and keeps concurrent jobs from stepping on each other.

How it works

The core flow is straightforward:

Job comes in — via the API or the web UI
Clone the repo — fresh workspace, checks out the base branch
Create a feature branch — so the agent has a clean place to work
Run OpenCode — pointed at the workspace, Ollama as the model provider, using whatever prompt you gave it
Commit and push — stages everything, commits with your message, pushes back to GitHub

That’s the default “batch” mode — single prompt, single run, done. No back-and-forth, no follow-ups. Just a PR branch waiting for you when it’s finished.

What’s under the hood

The server is a Node.js/TypeScript HTTP API that manages the job queue, spawns worker processes for each agent run, and exposes a REST API for creating and monitoring jobs.

The UI is a React app (Vite + Tailwind) that shows you what’s running, lets you kick off new jobs, and streams the worker logs so you can see what the agent is actually doing.

The main bits:

OpenCode CLI — opencode run does the actual coding work
Ollama — local model server; OpenCode talks to this instead of a cloud API
GitHub App — generates short-lived installation tokens per repo so workers can read and write to private repos without PATs kicking around everywhere
Open Code Review - Open source code review harness

That last one is worth a mention. I’ve done this kind of GitHub App integration before so it was the natural choice — the server authenticates as a GitHub App and mints a fresh scoped token for each job rather than sitting on a hardcoded PAT. A bit more setup the first time, but it works cleanly.

What a job actually looks like

Kick off a batch agent via the API:

curl -X POST http://localhost:8081/api/v1/agents \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "repoId": "myorg-my-app",
    "prompt": "Add input validation to the login form",
    "baseBranch": "main",
    "agentBranch": "agent/login-validation",
    "commitMessage": "Agent: add input validation to login form",
    "push": true
  }'

You get an agent ID back immediately. Then it’s just polling GET /api/v1/agents/:id for status or tailing logs if you want to watch it run. Status goes queued → running → completed (or failed if something blows up).

So, did it work?

Once the hardware’s paid for, each agent run on qwen2.5-coder:7b costs basically nothing. No token counting, no surprise bills, no throttling at peak hours. That’s the cost side of the equation sorted — but the more interesting answer is how much real work it can actually do.

Give it something specific and scoped — “add input validation to this form handler” — and it nails it, most of the time. Ask it to “improve the error handling in this service” with no further context and you get something that technically compiles but isn’t what you wanted. It’s a floor lower than GPT-5 or Claude, and you feel that floor the moment your prompt gets vague.

There’s a bonus I wasn’t chasing but noticed anyway: privacy. Your code never leaves your network. If you’re working on anything sensitive, or just don’t love the idea of your whole codebase passing through a third-party API, running everything locally sidesteps that entirely.

So the honest answer, a few weeks in: local models won’t replace a frontier model for the hard stuff, but they’ve earned a real spot in the rotation for well-scoped, repetitive work. Which, if I’m being honest about how I spend most of my agent budget, is most of it.

What’s next: interactive sessions

Batch mode is great for well-scoped tasks where you know exactly what you want. But sometimes you want to poke at the agent while it’s running — redirect it, ask it to extend something, course-correct before it goes too far down the wrong path.

Part 2 is about adding interactive mode: instead of opencode run and exit, the worker starts up an opencode serve instance and keeps it running. You get a proper chat UI with a live transcript, can send follow-up messages, and hit Finish when you’re happy — which is when the commit and push actually happens.

The tricky part is that OpenCode’s server API wasn’t built with an orchestration wrapper in mind. There’s a bunch of work mapping its SSE event stream into something the UI can render properly, plus some XDG environment variable gymnastics to make sure concurrent agents don’t step on each other’s sessions.

More implementation detail in the next post — worker architecture, the GitHub App setup, and diving into interactive sessions. The code is (or will shortly be) open source at github.com/dkarzon/localagent-box; I’m splitting this into a series to walk through the build, not because it’s still half-finished.

Building Nomii Without the Builder

2026-08-17T00:00:00+00:00

We’ve been building Nomii, a family memory keepsake app for parents, for a while now. How we build it has changed a lot in that time. We started in a low-code app builder. These days it’s React Native in a repo we own, with Cursor and a pile of AI tools doing most of the typing.

It wasn’t a dramatic break, more a gradual shift. The product got clearer, our needs got more specific, and the tooling around us caught up to the point where the next step felt natural instead of risky.

Why a builder was the right place to start

Nomii isn’t a project we can give forty hours a week. It’s something we build in the gaps: evenings, weekends, the occasional stolen hour between day jobs. At that stage, low-code made a lot of sense.

Draftbit gave us a visual way to lay out screens, wire up navigation, and iterate on UI without bootstrapping a full React Native project from scratch. For a small team with mobile experience in the room but no one living in that world day to day, that’s a reasonable trade. We weren’t avoiding code so much as avoiding setup, while we figured out what Nomii actually was.

And it worked. The first version came together faster than we expected — screens, flows, basic data wiring, all the stuff that usually eats the first week of a mobile project. Draftbit let us focus on the product idea: how parents capture milestones, stories, voice recordings, and the bits of family life that scatter across camera rolls and group chats.

That early phase is exactly what a builder is good for. Get something in front of people, learn, move on.

When the project started asking for more

As Nomii grew, so did the gap between what we wanted to build and what felt comfortable inside the builder.

The product vision got more specific: richer memory types, templates and prompts, multi-user accounts, a legacy access system we’re designing for the long term. The flows got more interconnected too. The UI stopped being a collection of screens and started being a system, with navigation, state, and media handling that needed to evolve together.

Around the same time, a few things shifted in our toolchain. Expo had matured to the point where the old mobile setup tax (Xcode rabbit holes, SDK path drama, an evening lost to pod install) was mostly gone. Cursor and AI coding agents had gone from novelty to something we were already using seriously on other projects. pen.dev gave us a way to bring our existing Figma designs into Cursor as implementation artifacts, which was basically the visual handoff we liked about the builder, minus staying inside one.

Draftbit’s move to v2 was part of that picture too, a reminder that the platform was heading in a different direction, with more emphasis on AI assistance and less on the no-code workflow we’d built the early Nomii prototype in. It wasn’t some villain origin story, it just sharpened a question we’d already been circling: are we still in the phase where a builder is the best fit, or have we moved past it?

We had.

Expo made the switch feel possible

The thing that kept us on a builder longest wasn’t fear of React Native. It was the feedback loop.

We needed to change something, see it on a phone, show it to the rest of the team, and keep going, without a build pipeline getting in the way.

Expo flattened a lot of that. One project template, a dev server, a QR code on your phone, and you’re looking at the app in minutes. Hot reload. Over-the-air updates when we need them. A testing loop that matches how we actually work on Nomii now: implement, run in Expo Go, test on a device, iterate.

That’s what made the switch possible. The setup cost we’d been paying the builder to absorb was suddenly cheap enough to carry ourselves, especially with AI handling a fair chunk of the boilerplate.

Cursor, agents, and owning the codebase

Moving off the builder didn’t mean going back to hand-coding every screen like it was 2017. The whole point was still speed. We just wanted speed with source code we could read, diff, refactor, and grow as Nomii gets more ambitious.

Cursor is where Nomii gets built now. Same pattern we use on other projects: an AGENTS.md with conventions, clear scope boundaries, and enough context that the agent isn’t guessing at architecture every session.

In practice that looks like:

Describing a screen or flow in plain language, then iterating on the generated React Native components
Using the agent for the tedious glue (navigation wiring, form validation, API client scaffolding) while we stay in the director’s seat on product decisions
Fixing the subtle misses in review: state that works on the happy path but breaks on back navigation, accessibility gaps, the UI that’s technically correct but not quite Nomii

The output lives in our repo. We can run it locally, test it in Expo Go, rip out a component that isn’t working, and prompt our way to a better version. As the product gets more complex, that ownership matters more. Builders aren’t bad, they just don’t give you a home you control, and mature products tend to need one.

From Figma to code via pen.dev

We were already designing Nomii in Figma. That’s where the team iterates on layout, flow, and visual polish. What we didn’t have was a clean way to get those designs in front of Cursor as something the agent could actually work from.

Describing UI in prose works fine for simple screens. For anything with real visual intent (spacing, hierarchy, component structure) it turns into a game of telephone. You end up reviewing generated code that sort of matches the design, then spending as long fixing it as you would have spent just building it.

pen.dev bridged that gap. We drop Figma designs in as artifacts alongside the prompt, and Cursor uses them as reference during implementation. The agent isn’t guessing at what “the memory capture screen” should look like; it can see the design we already agreed on.

The workflow is design in Figma, reference in Pencil, implement in Cursor, run in Expo, test on a device. Each step has a clear handoff. We’re not asking the AI to be a designer and a developer at the same time — we’re giving it the same brief a human developer would get.

What changed, and what didn’t

Different tools, different tradeoffs — not a strict upgrade in every dimension.

We took on more responsibility. Responsive layouts, component libraries, navigation patterns, media handling — the builder had been absorbing some of that thinking for us. Going native-ish means we own those problems again, which is more work but also more flexibility as Nomii’s feature set grows.

There’s a learning curve to prompting well. A builder UI is constrained; you can only do what the widgets allow, which is frustrating but also guard-railed. Cursor will happily build exactly what you asked for, including the parts you forgot to mention. Review becomes part of the craft.

And cost isn’t zero: Cursor subscription, Expo tooling, our time reviewing generated code. But it’s a cost structure that fits where Nomii is now — pay for tools, own the output, no platform ceiling on what we can build next.

What we’d tell ourselves at the start

If we were starting from scratch today, knowing what we know now, we’d probably still begin with a builder, or at least something equally fast and visual, and plan to graduate once the product vision solidified. That early phase was worth it. We didn’t waste time wrestling a toolchain; we spent it learning what parents actually need from a memory keepsake app.

We’d just write the AGENTS.md earlier when the switch happened. The first week of agent-assisted development involved a lot of re-explaining conventions that should have been captured once. Same lesson as every other AI-native project we’ve worked on: the context doc is the product.

Where Nomii is now

Nomii is still very much in progress — we’re on the waitlist phase while we build out the app. But the build loop feels right for this stage: fast iteration on real devices, a codebase that can grow with the product, and AI doing the parts of mobile development we were happy to delegate anyway.

Draftbit got us moving when speed mattered most. Expo, Cursor, and Pencil are how we build now that Nomii knows what it wants to be.

We’ll write more about Nomii itself as we get closer to launch. For now, we’re just enjoying building a mobile app with agents, on our terms, in a repo that’s finally ours.

ScaleBop: From Vibe to Viable

2026-07-07T00:00:00+00:00

I’ve been building ScaleBop for a while now, and the more I talk to people who ship with AI coding tools, the clearer the problem gets: we’ve never had it easier to generate an app, and we’ve never had more confusion about what happens after the prototype works on localhost.

Lovable, Bolt, Replit, Claude — pick your tool — they’ll get you a full-stack app in an afternoon. That part of the story is well told. What they don’t tell you is what comes next.

The gap nobody talks about

Hosting models. Which AWS services you actually need at your current scale. Infrastructure-as-code when you’ve never written infrastructure. CI/CD, secrets, environments — without copying a tutorial that doesn’t match your repo. Whether the generated code is safe to expose to the internet.

That’s the gap between “I have an app” and “I have a production system”. Projects stall there. Founders burn weeks in AWS documentation rabbit holes. Agencies quietly eat margin doing work the AI tools implied was already done.

ScaleBop is my attempt to close that gap — not by becoming another black-box host, but by meeting you at the codebase and giving you a credible first pass at architecture, infrastructure, and delivery that you own.

What ScaleBop does

The tagline is from vibe to viable. In practice, that’s a short, repeatable workflow:

Import your project — connect a GitHub repo, upload a ZIP, or point at an AI-generated project.
Detect the stack — frameworks, languages, build tools, databases, deployment patterns. ScaleBop reads what’s actually in the repo, not what you think you built.
Synthesize an architecture — a rule-based AWS deployment blueprint tuned for cost, scalability, and simplicity. Next.js maps differently than Express; a SQLite prototype gets different guidance than Postgres.
Generate delivery assets — starter AWS CDK scaffolding and GitHub Actions workflows oriented toward OIDC and environment promotion. You merge these into your repo; ScaleBop doesn’t silently deploy into your account without you in the loop.
Scan the source — a static security pass on the same snapshot, so you can triage risky patterns before you wire things up to the public internet.

The output isn’t magic. It’s scaffolding — a serious starting point you can review, adapt, and grow. That’s intentional. Whether you’re an enterprise team or an indie founder, you need artifacts you control, not another platform that holds the keys.

The education problem

Here’s the part I find most interesting, and the reason ScaleBop isn’t just a codegen tool.

A growing audience of non-technical builders — founders, operators, designers, product people — are using AI to create real software. They can describe what they want, iterate on UI and flows, and get surprisingly far. Then someone mentions Lambda, OIDC, or a VPC, and the conversation turns opaque. They either pretend to understand, pay someone without context, or abandon the project.

We can fix that — not by turning everyone into a cloud architect, but by translating as we go.

Every architecture recommendation comes with rationale: why this backend shape, why this database choice, what tradeoff you’re making. AWS service names in the blueprint get paired with plain-language explainers. What is CloudFront? Why would a Next.js app use Lambda@Edge versus a container? What does OIDC mean in a GitHub Actions workflow, and why should you care? Next steps land as a checklist, not a wall of jargon — connect AWS, bootstrap CDK, open a PR with generated files, run the pipeline.

You already have a codebase you care about. ScaleBop connects each infrastructure decision back to your stack. That’s a much better teaching surface than a generic AWS course.

If you’re a non-technical builder: you don’t need to master every service. You need enough literacy to ask good questions, spot when something sounds wrong, and collaborate with a developer or consultant without flying blind.

If you’re a developer: you still get speed. The blueprint and generated IaC are starting points, not gospel. But your stakeholder can finally see why the architecture looks the way it does.

Problems I set out to solve

When I started, I wrote down a few problems I wanted ScaleBop to address directly:

“I have code but no cloud plan” — stack detection plus a deployment blueprint from the actual repo
“AWS is powerful and intimidating” — opinionated defaults with explanations, not a blank console
“I don’t know what IaC to write” — generated CDK bundle matched to detected deployables
“CI/CD feels like a separate project” — GitHub Actions workflow with OIDC-oriented setup docs
“AI-generated code might be unsafe” — advisory security scan on import, visible alongside analysis
“I want to own my infra, not rent a black box” — downloadable artifacts; customer-owned AWS account; no hidden deploy

What ScaleBop is not, at least not today: a fully managed Vercel-style platform that runs your production for you. That’s a deliberate boundary. Trust, security review, and operational ownership matter — especially for anyone learning their way into this space.

How I built it

ScaleBop is a TypeScript monorepo.

Next.js handles the web app — import flows, architecture views, IaC and pipeline previews, billing, admin. The API is Node.js — raw node:http locally, the same handlers packaged as AWS Lambda in production. No Express, no framework magic; one handler shape everywhere.

PostgreSQL for application data; S3 for code snapshots at scale. The real business logic lives in packages/core: analyzer, blueprint synthesis, IaC and pipeline generation, security findings normalisation. The API and web app stay thin. AWS CDK defines the ScaleBop platform itself — API Gateway, Lambda, RDS, buckets, secrets.

The analyzer and blueprint engine are deterministic today — heuristics and rules, not an LLM guessing your architecture. That was a conscious MVP choice. Predictable output you can debug beats impressive-sounding hallucinations when someone is trying to learn.

The tools that made it possible

Building a product like this solo — or nearly solo — would have been unrealistic a few years ago. Three tools shaped how I work.

Cursor

Cursor is where almost all of the code gets written. Not as a replacement for thinking — as a force multiplier for implementation.

The ScaleBop codebase has strict conventions: shared types in packages/core, handlers in domain folders, SQL in query modules, Zod validation on request bodies, no console.log in the API. Cursor helps me move fast within those guardrails — refactoring across workspaces, extending the blueprint rules, writing migrations, keeping tests aligned.

The AGENTS.md file in the repo is effectively a briefing doc for the AI: stack, patterns, things not to do. That meta-layer — human rules plus AI execution — is how a single builder can maintain a monorepo spanning web, API, core logic, and CDK without drowning in context switching.

Honest take: Cursor doesn’t remove the hard product decisions. It removes friction on the decisions already made.

Linear

Linear is the source of truth for what to build next. Issues are tracked as units of work — Customer-owned AWS connections, security scan UX, CDK bootstrap automation, billing paths, custom domains, and dozens of vertical slices in between.

Why Linear and not just GitHub Issues? Speed and clarity. Linear’s workflow matches how I think about product work: a crisp issue, a plan doc when scope is fuzzy (docs/plans/ often links back to a Linear ticket), ship, close. For a builder wearing product, engineering, and ops hats, that lightweight structure keeps momentum without enterprise ceremony.

When something breaks in production or a user report comes in, it becomes a ticket. When an idea needs exploration — AWS reseller models, deployment architecture tradeoffs — it gets a plan doc and a Linear issue so it doesn’t live only in my head.

GitHub

GitHub shows up twice: as infrastructure for ScaleBop the product, and as part of what ScaleBop generates for customers.

For the product: source control, code review, GitHub OAuth for repo imports, GitHub App flows for listing repositories and opening PRs with generated IaC and pipeline files, CI workflows on push and pull request.

For customers: ScaleBop generates GitHub Actions workflows designed around OIDC — short-lived credentials, no long-lived AWS keys in the repo. That’s the modern standard, and it’s exactly the kind of thing non-technical builders never know they need until something goes wrong.

Your code lives here, your pipeline runs here, your AWS account stays yours.

What I’ve learned so far

The prototype economy is real. AI app generation isn’t a fad; it’s a new on-ramp to software. The downstream tooling lag is the opportunity.

Ownership beats convenience for the first deploy. People will tolerate more steps if they understand them and retain control. Black-box deploy is a later-stage feature, not an MVP wedge.

Education is a feature, not a blog sidebar. Explainers in the UI convert better than explainers in a PDF. Builders stay when they understand their architecture.

Deterministic beats flashy for trust. A rule-based blueprint you can trace beats an LLM essay that sounds confident and wrong.

Solo building is a workflow problem. Cursor plus Linear plus GitHub isn’t hype — it’s an operating system for one person shipping a credible SaaS.

Where it goes next

ScaleBop today delivers the core loop: ingest, analyze, blueprint, generate, scan. The roadmap includes deeper AWS connection flows, richer cost visibility, better security findings UX, and continued work on making the “first deploy” path approachable without crossing the trust boundary.

If you’re building with AI and stuck between “it works on my machine” and “it runs in prod,” that’s the problem space I’m obsessed with. ScaleBop is my answer so far — free import and stack detection on the starter tier, enough to see if it understands your app before you commit to anything.

I’m still figuring out what parts of the journey feel most opaque. If you’ve hit that wall, I’d like to hear about it.

The Rise of the Engineering Agent Manager

2026-04-07T00:00:00+00:00

There’s a conversation happening in every engineering org right now, whether it’s out loud or just quietly in people’s heads: what exactly is my job now?

For a long time, the path was well understood. Junior developers write code, senior developers write better code and review it, tech leads set patterns and mentor, and engineering managers coordinate people and delivery. Everyone had a lane. AI has blown that model up, not by replacing people, but by fundamentally changing where the value is.

The engineers and leaders who are thriving right now aren’t the ones who are best at writing code. They’re the ones who are best at directing, reviewing, and shaping the work that AI agents are producing.

From Producer to Director

Here’s the mental shift that took me a while to internalise: your job is no longer to be the best implementer. It’s to be the best decision-maker about what gets implemented and why.

Think of it like the difference between being a craftsman and being an architect. A craftsman’s value is in their ability to produce the work with their hands. An architect’s value is in their ability to understand the brief, envision the outcome, and communicate that clearly enough for others (or now, agents) to execute it faithfully.

The coding work itself hasn’t disappeared. There’s more of it than ever. But the constraint has shifted from how fast can humans write code to how well can humans define, review, and guide what agents produce. That changes everything about what a strong engineering leader looks like.

Architecture, Strategy, Vision: Now the Whole Job

When an agent can scaffold a feature in minutes, the bottleneck becomes the quality of what you gave it to work from. Vague requirements produce vague code. Poorly considered architecture decisions, when handed to an agent that will faithfully implement them at scale, produce large amounts of faithfully bad code.

This is where Engineering Managers and tech leads are finding renewed purpose. Spending time deeply on:

System design and architecture: The decisions that are hard to reverse, that constrain everything downstream, that require experience and judgment.
Product strategy: What should we actually be building? What’s the simplest version of this that delivers value?
Technical vision: Where is the codebase going in 6 months? What patterns should agents be following? What should they never do?
Context documents: More on this below, but writing the documentation that puts agents in the right headspace is now a core leadership skill.

The engineers who are leaning into this are finding that their impact is multiplied. One person with a clear vision and a well-configured agent workflow can ship what used to take a team.

Controlling Agents Through Context

One of the more interesting things about working with AI coding agents (tools like GitHub Copilot, Cursor, and others) is that they’re heavily influenced by the context you give them. This has spawned a new type of document that’s becoming a standard part of engineering workflows: the AGENTS.md file, custom instruction files, and project-scoped prompt documents.

These documents are essentially your communication layer with your agents. A well-written AGENTS.md might define:

The coding style and conventions for the project
Which patterns to use and which to avoid
How to structure components, name things, and organise files
Testing expectations
Scope boundaries: what the agent should and shouldn’t touch

Tech leads who are writing these documents well are discovering something important: it forces you to articulate things that usually live only in your head. The conventions that you’d catch in a code review but never wrote down anywhere. The architectural decisions that are obvious to you but invisible to a new team member, or an agent.

There’s also a feedback loop here. When an agent produces something that misses the mark, the right response isn’t just to fix the output. It’s to update your context documents so the agent gets it right next time. This is a fundamentally different kind of engineering leadership work, and it’s a skill that takes practice.

Linear and the Agent-Friendly Backlog

Project management tooling is catching up fast. Linear has become the tool of choice for a lot of engineering teams, and it’s easy to see why when you start working in an AI-native way.

Where a traditional Jira ticket might be a title and two sentences, Linear has nudged teams toward writing richer, more structured issue descriptions. And now with AI agents that can be pointed at a Linear ticket to execute on, the quality of that ticket has a direct impact on the quality of what gets built.

The mental model is shifting from assign this to a developer to can an agent execute on this ticket as written? If the answer is no (too vague, missing acceptance criteria, or lacking technical context), that’s a signal the issue needs more work before it goes anywhere.

For engineering managers and leads, this creates a new responsibility around backlog hygiene. Grooming a ticket used to be about making sure a human developer had enough information to get started. Now it’s about making sure that information is structured clearly enough that an agent can run with it autonomously, with a human reviewing the output rather than doing the work line by line.

Some teams are even going further, using agents to draft tickets from higher-level briefs, then having leads review and refine those before they’re executed. The human judgment is still in the loop, just at a different stage.

The Review Layer Is the New Craft

If you’re an engineering leader and you’re worried that AI is devaluing your expertise, I’d push back on that. What’s changing is where you apply it.

The code review is becoming more important, not less. When agents are generating code, the review is where human expertise catches the subtle misalignments: the technically correct implementation that misses the intent, the approach that works for this feature but creates problems for the next one, the security concern that the agent didn’t have enough context to see coming.

This requires:

Deep familiarity with the codebase and its constraints
Clear communication about why something isn’t right, not just that it isn’t
The ability to update context documents so the same mistake doesn’t happen again
Trust in your own judgment about when to approve and when to push back

These are the skills that separate a great engineering leader from a mediocre one, and they haven’t changed. They’ve just been relocated to a new part of the workflow.

The Leaders Who Are Struggling

It’s also worth being honest about where the friction is. Not everyone is finding this transition easy.

The engineers who built their identity around being great implementers — who took pride in hammering out clean code quickly — are having to find a new source of professional satisfaction. That’s genuinely hard. The craft is real and it matters, and being asked to step back from it to direct others (or agents) doing it can feel like a demotion even when it isn’t.

There’s also a learning curve to working effectively with agents that a lot of people underestimate. It’s not as simple as asking a question and accepting whatever comes back. It requires developing intuition for when to trust the output, when to scrutinise it, and how to give feedback that actually improves future results.

The leaders who are investing time in building this intuition are pulling ahead. The ones who are treating AI tools as a productivity shortcut but not changing how they think about their role are finding the benefits much more limited.

Getting Started

If you’re an Engineering Manager or tech lead who’s still finding your footing with this shift, a few practical starting points:

Write an AGENTS.md for one of your projects. Even if it’s imperfect, the act of writing down your conventions is valuable regardless of agents.
Try completing a full feature with an agent, staying in the director seat. Define the work clearly, review what it produces, iterate through feedback. Notice where you struggle to articulate what you want.
Audit your Linear (or equivalent) backlog. Would an agent have enough context to work from these tickets? Where are the gaps?
Share the experience with your team. The best teams right now are learning together, sharing prompt patterns that work, updating shared context docs based on what agents get wrong.

The shift is real and it’s happening fast. The good news is that the skills that make a great engineering leader (clear thinking, good judgment, strong communication, technical depth) are exactly what this new way of working demands. They just need to be applied in new places.

The Engineering Agent Manager is already here. The question is whether you’re ready to step into that role.

Using Schema.org structured data to improve SEO

2025-12-03T00:00:00+00:00

Search engine optimization has long been a cornerstone of the web, continually evolving as search engines and open‑web initiatives shape how information is discovered.

Today, the rise of AI tools and AI‑powered search is accelerating that evolution, making structured data more critical than ever. In this post, I’ll explore how adding schema.org markup can enrich your content, improve visibility across both traditional search and AI‑driven platforms, and ultimately create a better experience for users.

Schema.org is a set of extensible schemas that enables webmasters to embed structured data on their web pages for use by search engines and other applications. For more details, see the homepage.

Why Use Structured Data?

Adding structured data to your website provides several benefits:

Rich Search Results: Your content can appear with enhanced visuals in search results, including star ratings, images, prices, and availability information
Better Click-Through Rates: Rich snippets stand out in search results, leading to higher engagement
Voice Search Optimization: Search assistants use structured data to provide spoken answers
Knowledge Graph Integration: Your content can be featured in Google’s Knowledge Panel
AI Agent Compatibility: Modern AI systems can better understand and utilise your content (more on this below)

How to Implement Structured Data

There are 3 main ways to include structured data in a website’s markup:

Microdata
JSON-LD
RDFa

Microdata adds additional attributes to HTML elements to identify the data on the page. The data is embedded directly within your existing HTML.

<article itemscope itemtype="https://schema.org/BlogPosting">
  <h1 itemprop="headline">My Blog Post Title</h1>
  <span itemprop="author" itemscope itemtype="https://schema.org/Person">
    <span itemprop="name">Damian Karzon</span>
  </span>
  <time itemprop="datePublished" datetime="2025-12-04">December 4, 2025</time>
  <div itemprop="articleBody">
    <p>The content of the blog post goes here...</p>
  </div>
</article>

JSON-LD (JavaScript Object Notation for Linked Data) adds a JSON block to the page where the structured data is output in a single location. This is Google’s recommended format as it’s easier to implement and maintain.

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "BlogPosting",
  "headline": "My Blog Post Title",
  "author": {
    "@type": "Person",
    "name": "Damian Karzon"
  },
  "datePublished": "2025-12-04",
  "dateModified": "2025-12-04",
  "image": "https://example.com/images/post-banner.jpg",
  "publisher": {
    "@type": "Organization",
    "name": "DK Development",
    "logo": {
      "@type": "ImageObject",
      "url": "https://example.com/logo.png"
    }
  }
}
</script>

RDFa (Resource Description Framework in Attributes) like Microdata, adds additional attributes to HTML elements to identify the data on the page. It’s more verbose but offers greater flexibility.

<article vocab="https://schema.org/" typeof="BlogPosting">
  <h1 property="headline">My Blog Post Title</h1>
  <span property="author" typeof="Person">
    <span property="name">Damian Karzon</span>
  </span>
  <time property="datePublished" datetime="2025-12-04">December 4, 2025</time>
  <div property="articleBody">
    <p>The content of the blog post goes here...</p>
  </div>
</article>

Which Format Should You Use?

JSON-LD is the recommended approach for most implementations because:

It’s easier to add and maintain (no need to modify existing HTML)
It keeps your markup cleaner
It’s Google’s preferred format
It can be dynamically generated server-side or via JavaScript

Popular Schema Types

A comprehensive list of schema types can be found at schema.org/docs/schemas.html.

The vocabulary currently consists of 792 Types, 1447 Properties, 15 Datatypes, 83 Enumerations and 445 Enumeration members.

Here are some of the most commonly used schema types:

Schema Type	Use Case
Article/BlogPosting	Blog posts and news articles
Product	E-commerce product pages
LocalBusiness	Physical business locations
Organization	Company information
Person	Individual profiles
Recipe	Cooking recipes
Event	Upcoming events
FAQPage	Frequently asked questions
HowTo	Step-by-step guides
Review	Product or service reviews
BreadcrumbList	Site navigation breadcrumbs
VideoObject	Video content

AI Agents and Structured Data

With the rise of AI assistants like ChatGPT, Claude, Gemini, and others, structured data has taken on a new dimension of importance. Here’s how AI agents interact with schema.org markup:

Enhanced Content Understanding

AI agents parse structured data to better understand the context and meaning of web content. When an AI encounters a page with proper schema markup, it can:

Accurately identify the type of content (article, product, recipe, etc.)
Extract key information like authors, dates, prices, and ratings
Understand relationships between entities on the page

AI-Powered Search and Assistants

Many AI systems now power search experiences and virtual assistants:

Retrieval-Augmented Generation (RAG): AI systems that search and synthesise information benefit from structured data to provide accurate, well-attributed responses
Voice Assistants: Siri, Alexa, and Google Assistant use structured data to answer questions directly
AI Search Engines: Perplexity, Bing Copilot, and Google’s AI Overviews leverage schema markup to generate accurate summaries

Agentic Web Browsing

As AI agents become more autonomous in browsing and interacting with websites:

Structured data helps agents understand page purpose and available actions
Schema types like Action and EntryPoint can indicate how to interact with a service
Well-marked content is more likely to be correctly interpreted and cited

Best Practices for AI Compatibility

To optimise your structured data for AI agents:

Be comprehensive: Include as many relevant properties as possible
Use accurate data: AI systems may cross-reference your claims
Include sameAs properties: Link to authoritative sources (Wikipedia, social profiles)
Add speakable schema: Indicates content suitable for text-to-speech
Implement WebPage and WebSite schemas: Helps AI understand your site structure

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "WebSite",
  "name": "DK Development",
  "url": "https://dkdevelopment.net",
  "potentialAction": {
    "@type": "SearchAction",
    "target": "https://dkdevelopment.net/search?q={search_term_string}",
    "query-input": "required name=search_term_string"
  }
}
</script>

Testing Your Structured Data

Before deploying structured data to production, validate it using these tools:

Google Rich Results Test: Tests if your page is eligible for rich results
Schema.org Validator: Validates your markup against the schema.org specification
Google Structured Data Markup Helper: Helps generate markup for your pages

A Real-World Blog Example

Here’s how I’ve implemented structured data on this very blog using JSON-LD:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "BlogPosting",
  "mainEntityOfPage": {
    "@type": "WebPage",
    "@id": "https://dkdevelopment.net/seo-structured-data-2025/"
  },
  "headline": "Using Schema.org structured data to improve SEO",
  "description": "How to add Schema.org markup to your websites to improve users search experience.",
  "image": "https://dkdevelopment.net/img/Banners/book.jpg",
  "author": {
    "@type": "Person",
    "name": "Damian Karzon",
    "url": "https://dkdevelopment.net"
  },
  "publisher": {
    "@type": "Organization",
    "name": "DK Development",
    "logo": {
      "@type": "ImageObject",
      "url": "https://dkdevelopment.net/img/favicon.ico"
    }
  },
  "datePublished": "2025-12-04",
  "dateModified": "2025-12-04"
}
</script>

Recap

Structured data is no longer just about improving traditional search results—it’s becoming essential infrastructure for how AI systems understand and interact with web content. By implementing schema.org markup:

Choose JSON-LD as your primary format for ease of implementation
Start with the basics: Add Article/BlogPosting schema to your blog posts
Expand to business schemas: Organization, LocalBusiness, and Person for your about pages
Test thoroughly before deployment using Google’s validation tools
Think about AI: Your structured data helps AI agents accurately represent your content

As AI continues to transform how users discover and consume web content, well-structured data will become a competitive advantage—not just for SEO, but for ensuring your content is accurately understood and represented across an increasingly AI-mediated web.

My Home Assistant Docker Compose file

2022-05-19T00:00:00+00:00

Over the last few months I have been playing around with a Home Assistant instance in Docker to run all of my home automation and over time I have slowly added more and more to it (more integrations, more services, more devices, etc.).

Some info about my setup before we get into it.

Host machine running Windows 10 Pro
Docker Desktop using WSL2 backend
.wslconfig configured with 30GB Ram and 5 Processors

Since I don’t run Home Assistant Operating System I don’t have the ability to use the supervisor and install services through that which means I’ve had to setup my own docker containers for additional services. The containers I have running here are:

Home Assistant - The standalone Home Assistant Core install
PostgreSQL - Database storage for Home Assistant recorder (instead of using filesystem)
Node Red - Low-code programming tool for wiring up events
Mosquitto - An MQTT message broker (to allow devices to use a publish/subscribe model of messaging)
Tasmoadmin - Administration platform for Tasmota devices (to allow OTA updates of devices without direct internet access)
Chrony - A Network Time Protocol (NTP) Server (to allow devices without internet access to synchronise time)
InfluxDB - A Scalable datastore for metrics, events, and real-time analytics (used for long term storage of sensor data)
Grafana - Open source data visualisation platform (to create dashboards from the data in InfluxDB)

So let’s take a look at the docker-compose.yml file:

version: '3.1'

services:

  home-assistant:
    container_name: home-assistant
    image: homeassistant/home-assistant:2022.5.4
    restart: always
    networks:
      - localnet
    volumes:
      - /c/local-docker/home-assistant/config:/config
      - /c/local-docker/home-assistant/media:/media
    depends_on:
      - mosquitto
    ports:
      - 8123:8123
      - 1400:1400
    environment:
      TZ: "Australia/Brisbane"

  postgres:
    container_name: postgres
    image: postgres:12.6
    restart: always
    networks:
      - localnet
    ports:
      - 5432:5432
    environment:
      POSTGRES_DB: homeassistant
      POSTGRES_USER: ha_user
      POSTGRES_PASSWORD: CrazyStr0ngPa$$word

  nodered:
    container_name: nodered
    image: nodered/node-red
    restart: always
    networks:
      - localnet
    ports:
      - 1880:1880
    volumes:
      - /c/local-docker/home-assistant/nodered:/data
    depends_on:
      - home-assistant
      - mosquitto
    environment:
      TZ: "Australia/Brisbane"

  mosquitto:
    container_name: mosquitto
    image: eclipse-mosquitto
    restart: always
    networks:
      - localnet
    ports:
      - 1883:1883
      - 1884:1884
    volumes:
      - /c/local-docker/home-assistant/mosquitto/config:/mosquitto/config
      - /c/local-docker/home-assistant/mosquitto/data:/mosquitto/data
      - /c/local-docker/home-assistant/mosquitto/log:/mosquitto/log
    environment:
      TZ: "Australia/Brisbane"

  tasmoadmin:
    container_name: tasmoadmin
    image: ghcr.io/tasmoadmin/tasmoadmin:v1.8.0
    restart: always
    networks:
      - localnet
    ports:
      - 8124:80
    volumes:
      - /c/local-docker/home-assistant/tasmoadmin:/data/tasmoadmin

  chrony:
    container_name: chrony
    image: cturra/ntp
    restart: always
    networks:
      - localnet
    environment:
      - NTP_SERVERS=time.windows.com
    ports:
      - 123:123

  influxdb:
    container_name: influxdb 
    image: influxdb:latest
    restart: always
    networks:
      - localnet
    environment:
      - DOCKER_INFLUXDB_INIT_MODE=setup
      - INFLUXDB_DB=home_assistant
      - DOCKER_INFLUXDB_INIT_USERNAME=influx_user
      - DOCKER_INFLUXDB_INIT_PASSWORD=CrazyStr0ngPa$$word
      - DOCKER_INFLUXDB_INIT_ORG=ha-org
      - DOCKER_INFLUXDB_INIT_BUCKET=ha-bucket
    ports:
      - 8086:8086
    volumes:
      - /c/local-docker/home-assistant/influxdb:/var/lib/influxdb2

  grafana:
    container_name: grafana
    image: grafana/grafana:latest
    restart: always
    networks:
      - localnet
    depends_on:
      - influxdb
    environment:
      - GF_SECURITY_ADMIN_USER=grafana_user
      - GF_SECURITY_ADMIN_PASSWORD=CrazyStr0ngPa$$word
    ports:
      - 8125:3000
    volumes:
      - /c/local-docker/home-assistant/grafana:/var/lib/grafana

networks:
  localnet:
    driver: bridge

Let’s start with a few of the common parts to it.

Commons

container_name - I’ve given all of the containers specific names instead of the default naming of <folder name>_<service name>_1
networks - All the containers have been added to a custom network called localnet which is also defined at the top level with the driver set to bridge. This is to make sure all of the containers can properly talk to each other and get around some network oddities I was experiencing with my setup.
restart - All of the containers are also set to always restart so whenever docker starts up they will also startup. So if the host is every restarted the containers will come back automatically.
depends_on - I have a few containers that set the depends on to ensure other containers are successfully loaded before them but this isn’t necessary.

Now into the containers themselves…

home-assistant

The main Home Assistant Core image, here I have given it a specific version so that I can easily see which version I’m using and update when I need to. Using latest here would work as well but may need to make sure if pulls down latest instead of using the existing cached version.

I also have setup some mapped volumes for this so I can keep my config files in a folder on the Windows host machine. /c/local-docker/home-assistant/config:/config This one maps the Windows folder C:\local-docker\home-assistant\config to the /config directory within the container which is where the main config yaml files are kept. This allows me to more easily edit the config files from the host machine as well as persisting the config across rebuilds of the container. The other volume entry is for media which allows me to easily add media to the server for local playback on media players. The TZ environment variable here sets the timezone of the container to make sure it’s running in my local timezone.

postgres

I’ve gone with using a PostgreSQL instance here for the recorder because my home assistant container has it’s config folder as a mounted volume which on Windows has a bit of a performance hit and since I run these folders from my local C Drive I wanted to make sure my home assistant wasn’t slowing anything else running on the host machine too much. The recorder integration also supports MySQL, MariaDB and SQLite (default), more info here: https://www.home-assistant.io/integrations/recorder/. The container I actually use is in a separate docker-compose file because I use it for other things as well but I included it here for completeness.

nodered

I am not setting a version for this image so it will just use the latest available when it is started (I haven’t had any issues with this so far). To connect the node red instance to home assistant you need to make sure you install the relevant home assistant packages:

node-red-contrib-home-assistant
node-red-contrib-home-assistant-websocket

Running node red in docker like this I found that installing packages from the UI didn’t install the latest from npm, because it installed them from the node red flows library (https://flows.nodered.org/). To install the latest versions here I had to run the npm install commands in the /data directory on the container itself. This container has a volume setup for the /data directory so that I can backup the config and the flow json files and state is again persisted between rebuilds of the container.

mosquitto

Using the eclipse-mosquitto broker for MQTT (latest version works fine here) because it seems to be the best supported one for home assistant. For this one I have 3 different volumes to map the config, data and log directories to local folders, mainly so I can backup the config and ensure state is persistent between container rebuilds.

tasmoadmin

The image for this is no longer being uploaded to docker hub which is why it is pulling from github packages (ghcr.io/tasmoadmin/tasmoadmin:v1.8.0). The mapped volume here lets me keep a backup of my config and devices as well as helping to persist state between container rebuilds.

chrony

Not too much to configure for this one, there is an environment variable to set the time server to use (NTP_SERVERS) this can be set to what ever you want. I went with time.windows.com because my host machine is running windows and wanted to have the 2 running against the same time server.

influxdb

Took me a little bit to decipher the documentation to get the right config I needed for this one since I hadn’t used influxdb before but got all the environment variables setup for it to work. Most of these are pretty straight forward for things like username, password, org and bucket, DOCKER_INFLUXDB_INIT_MODE=setup is used to ensure the automated setup is run the first time it is started. I also have a volume mapped here so that it persists between container builds mainly so that I don’t have to go about reconfiguring the integration with home assistant which requires the organisation id (not the same as the org environment variable) and the API token (generated through the automated setup).

grafana

This one I haven’t used too much yet but so this config may change as I use it more. At the moment I just have a mapped volume for backups and persistent state.

Bonus Content!

Since these services are all installed manually via docker I miss some of the integration points that comes with installing them through the HA supervisor. One of those is adding them to the side menu in home assistant, to do this we can make use of the panel_iframe integration in home assistant, here is an example of that integration in configuration.yaml.

panel_iframe:
  nodered:
    title: "Node Red"
    url: "http://192.168.1.5:1880/"
    icon: mdi:file-tree
  tasmoadmin:
    title: "TasmoAdmin"
    url: "http://192.168.1.5:8124/"
    icon: mdi:home-automation

Basic remote PC monitoring through Home Assistant

2021-12-15T00:00:00+00:00

I’ve had a server running at home for a while to run various things for work and personal use and recently I put Home Assistant on it to have a play with. If you haven’t heard of Home Assistant before it is an Open source home automation that puts local control and privacy first.

One thing that I wanted to do was be able to monitor the server status itself, things like storage, memory usage and temps then feed that into Home Assistant so I could try out a notification pipeline. The problem was since my HA instance runs as a docker container it doesn’t have direct access to those APIs in windows to give out that kind of info.

To get around this limitation I figured I had a look at a few tools that could surface that info as a Rest API that HA could consume but none did quite what I wanted easily so I decided to write my own light weight api to surface this data (also because it was fun). Then using the built in rest integration within HA I could set it up to read this data in as sensors.

I did this in .net core using the System.IO.DriveInfo.GetDrives() method which gets me all the details I need about the attached hard drives. Once I had that I filtered out to only should fixed drives (ie. non-removable storage) and mapped it to a model object so that I could output it as JSON without circular reference issues. I also outputted it as a dictionary where the drive letter was the key, this is to make it easier to pick up the specific drives later in HA.

Here is the code for the controller:

using Microsoft.AspNetCore.Mvc;
using System.IO;
using System.Linq;

namespace WinSysMon.Controllers
{
    public class HomeController : Controller
    {
        [Route("drives")]
        public IActionResult Drives()
        {
            DriveInfo[] drives = DriveInfo.GetDrives();

            var driveModels = drives
                .Where(d => d.DriveType == DriveType.Fixed)
                .Select(d => DriveInfoModel.Create(d))
                .ToDictionary(d => d.Name.Split(":").First());

            return Json(driveModels);
        }
    }

    public class DriveInfoModel
    {
        public long AvailableFreeSpace { get; set; }
        public string DriveFormat { get; set; }
        public string Name { get; set; }
        public long TotalFreeSpace { get; set; }
        public long TotalSize { get; set; }
        public string VolumeLabel { get; set; }

        public static DriveInfoModel Create(DriveInfo drive)
        {
            return new DriveInfoModel
            {
                AvailableFreeSpace = drive.AvailableFreeSpace,
                DriveFormat = drive.DriveFormat,
                Name = drive.Name,
                TotalFreeSpace = drive.TotalFreeSpace,
                TotalSize = drive.TotalSize,
                VolumeLabel = drive.VolumeLabel
            };
        }
    }
}

Once that was done and the API was working I needed to setup the sensors in HA using the rest integration. I setup 3 sensors per drive: availableFreeSpace, totalSize and volumeLabel which gave me enough info to do what I needed. The rest integration can be configured via the yaml file in HA and it allows you to setup multiple sensors from a single endpoint it also lets you set the interval times and authentication.

Using host.docker.internal as the url so that my HA instance running in docker can reference the host PC. Also doing some calculations on the values here since the API outputs the values in bytes so that I can just read them in as GB with 2 decimal places which is enough for me but can be changed easily. Here is my yaml configuration for the rest sensors setting

rest:
  - scan_interval: 600
    resource: http://host.docker.internal:55555/drives
    sensor:
      - name: "Server Drive C availableFreeSpace"
        value_template: "{{(value_json.C.availableFreeSpace | float / 1024 / 1024 / 1024) | round(2)}}"
        unit_of_measurement: GB
        
      - name: "Server Drive C totalSize"
        value_template: "{{(value_json.C.totalSize | float / 1024 / 1024 / 1024) | round(2)}}"
        unit_of_measurement: GB
        
      - name: "Server Drive C volumeLabel"
        value_template: "{{value_json.C.volumeLabel}}"
        

      - name: "Server Drive D availableFreeSpace"
        value_template: "{{(value_json.D.availableFreeSpace | float / 1024 / 1024 / 1024) | round(2)}}"
        unit_of_measurement: GB
        
      - name: "Server Drive D totalSize"
        value_template: "{{(value_json.D.totalSize | float / 1024 / 1024 / 1024) | round(2)}}"
        unit_of_measurement: GB
        
      - name: "Server Drive D volumeLabel"
        value_template: "{{value_json.D.volumeLabel}}"

This will add some new entities to Home Assistant for us which look something like this in the dev tools.

Now that we have the sensor data in HA we can start using it, for now I have put it into a lovelace dashboard but it can also be used in automation rules and setup as triggers etc. I created a simple gauge card to test it setting severity with red as under 10%, orange as under 50% and green over 50%.

type: gauge
entity: sensor.server_drive_c_availablefreespace
min: 0
max: 233
severity:
  green: 115
  yellow: 23
  red: 0
needle: true
name: 'C: Drive Usage'

From here next steps would be to look at integrating more sensors into the pipeline such as memory and temperature which can be done by querying with System.Manamgent.ManagementObjectSearcher.

Google Lighthouse CI Server Automation

2021-09-28T00:00:00+00:00

Website performance is one of those things that is “important” to everyone but it is sometime hard to test and keep track of the results in a way that is easy to navigate and doesn’t require too much effort. Recently it has become even more important for SEO (Search Engine Optimization) with the big search engines starting to use these metrics more to affect ranking (See: Understanding page experience in Google Search results).

Google Lighthouse is a well known performance testing tool for websites that is built into Google Chrome and used by Google PageSpeed Insights. But there is also Lighthouse CI which is an open source CLI tool for automating running lighthouse for a CI/CD environment.

Lighthouse CI also has a server component that let’s you save reports and compare them which can be very useful.

To setup the Lighthouse CI server through docker there are 3 parts to it.

Dockerfile

Setup the docker image to run the Lighthouse CI server with the appropriate config.

FROM node:12-buster-slim

WORKDIR /usr/src/lhci
COPY package.json .
RUN npm install
COPY lighthouserc.json .

EXPOSE 8000
CMD [ "npm", "start" ]

package.json

The package.json file used to install the libraries and configure the server start.

{
  "name": "lhci",
  "version": "0.0.0",
  "scripts": {
    "start": "lhci server --config=./lighthouserc.json"
  },
  "dependencies": {
    "@lhci/cli": "^0.8.1",
    "@lhci/server": "^0.8.1",
    "mysql2": "^2.1.0",
    "pg": "^7.12.1",
    "pg-hstore": "^2.3.3",
    "sqlite3": "^4.0.6"
  }
}

lighthouserc.json

The configuration file for the server, using PostgreSQL storage and daily CRON task to collect reports from PageSpeed Insights. Check out the Lighthouse CI Server docs for more info on this. To enable the PageSpeed Insights cron task you will need to get an API Key from the official documentation.

{
    "ci": {
        "server": {
            "port": 8000,
            "storage": {
                "storageMethod": "sql",
                "sqlDialect": "postgres",
                "sqlConnectionUrl": "postgres://lhci_user:<password>@host.docker.internal:5432/lighthouseci_db"
            },
            "basicAuth": {
                "username": "damian",
                "password": "<PASSWORD>"
            },
            "psiCollectCron": {
                "psiApiKey": "<psiApiKey>",
                "sites":[
                    {
                        "urls": ["https://dkdevelopment.net/", "https://dkdevelopment.net/2019/05/25/working-remotely/", "https://dkdevelopment.net/resume/"],
                        "schedule": "0 0 * * *",
                        "projectSlug": "dkdevelopment.net"
                    }
                ]
            }
        }
    }
}

Once the server is up and running you then need to create a project on the server, to do this run: lhci wizard to create a new project on the server and answer the questions it asks. Save the tokens it generates (you will need these later).

With the project setup and the PSI CRON tasks running you will then be able to see the daily reports coming in with the ability to compare reports over time.

Tips and tricks for Github Pages

2020-07-26T00:00:00+00:00

This blog has been hosted on Github Pages for a long time, a very very long time, according to my git history since Feb 2012 (before that it was on Wordpress).

I decided to make the change for a few reasons:
-To learn something new (Jekyll/Liquid were new to me)
-Have it feel more like a dev blog (writing posts in a dev IDE pushing to git etc.)
-Also the Pretzel Code52 project had just kicked off and I was interested in getting involved in that (https://github.com/Code52/pretzel).

Now, 8 years later I am trying to get into blogging again so I am breathing new life into my Github Pages blog as well as building a new website for a podcast using Github Page so I thought I’d share some of the interesting things it can do.

Docker

I’m not a ruby developer and I didn’t want to have to install all of the ruby dev tools just to be able to add blog posts. Previously I was using a prebuilt version of pretzel locally which worked pretty well, there were a few things that worked differently or were not supported but I got by. After doing a fair bit of work with docker I figured I should just be able to run jekyll on docker, run the same version that runs on Gihub Pages without all the overhead.

Luckily there are already github pages images setup so I didn’t need to do much to get this working, the one I went with is Starefossen/docker-github-pages. After finding that it was as easy as creating a powershell script to run it, which also mounts the directory to the docker container so it will auto update when changes are made.

docker run -it --rm -v ${PWD}:/usr/src/app -p "1000:4000" starefossen/github-pages jekyll serve -d /_site --drafts --watch --force_polling -H 0.0.0.0 -P 4000

For each loops

Github pages and Jekyll use the Liquid templating language which lets you do some fun things, one of them is for loops. The below code loops through the posts on the site (getting the first 12) to display links for each. {% for post in site.posts limit: 12 %} - site.posts being the collection to iterate over. for post in defines a post as the individual item being iterated. limit: 12 sets the loop to only iterate over the first 12 items, this can also be used with offset: 12 which sets the iteration to start at item 12 in the collection.

{% for post in site.posts limit: 12 %}
    <a href="{{ post.url }}">
        {{ post.title }}
    </a>
{% endfor %}

Combine and Minify CSS

This one is a bit of a hack but it allows you to combine and (somewhat) minify CSS files using some of the liquid templating. Starting with the capture block which lets you store a result in a variable without rendering it. Using the capture we then include the relevant css files inside the capture here using the include_relative tag. Finally we render the newly captured variable making sure to strip out the new lines and multiple spaces.

---
permalink: /css/dkdev.min.css
---
{%- capture sitecss -%}
    {% include_relative bootstrap.min.css %}
    {% include_relative fontawesome-free.min.css %}
    {% include_relative clean-blog.css %}
{%- endcapture -%}
{{- sitecss | strip_newlines | replace: "  ", "" -}}

Data files

Jekyll allows you to add other data files that can be used by various pages. These are just yml, json or csv files that can be put in the _data folder that can be accessed from any page or layout using site.data.

- name: Phili J Fry
  title: Delivery Boy

- name: Bender
  title: Company Chef

- name: Turanga Leela
  title: Captain

- name: John Zoidberg
  title: Staff doctor

This data file can then be used by a for loop to display all the items:

{% for character in site.data.characters %}
    <li>
        {{ character.name }} - {{ character.title }}
    </li>
{% endfor %}

Syntax Highlighting

Github Pages has built in syntax highlighting that uses the Rouge Ruby syntax highlighter library. It let’s you wrap your code with {% highlight %} and {% endhighlight %} and it will do some formatting with it which allows you to easily style it. All you need to do is include a rouge or pygments compatible stylesheet.

Here is what mine looks like for reference: https://dkdevelopment.net/css/pygments_style.css

Getting started with Singer and Docker

2020-06-14T00:00:00+00:00

I’ve been doing a lot of work with data recently and I wanted a way to be able to join some of our platform data to our traffic data from Google Analytics.

After trying a few methods to get data out of Google Analytics into our PostgreSQL database with not much success I discovered singer.io a “Simple, Composable, Open Source ETL” tool. So I started playing around with the Google Analytics tap and the PostgreSQL target (https://www.singer.io/tap/google-analytics/postgresql/). After a lot of digging through various documentations and source code I was able to get it working!

Now that I have it running it was time to try and automate it somehow, to do this I tried to set it up as a docker container that way it can be run in the cloud (In our case AWS) on a schedule to pull in data daily from Google Analytics.

Here is the Docker file that puts it all together.

FROM python:3.7.6-stretch

RUN pip install singer-target-postgres

COPY ./tap-google-analytics-master/ ./external/tap-google-analytics-master/
WORKDIR /external/tap-google-analytics-master
RUN pip install -e .

WORKDIR /
COPY ./singer/tap_google-analytics_config.json ./tap_google-analytics_config.json
COPY ./singer/client_secrets.json ./client_secrets.json
COPY ./singer/tap_google-analytics_reports.json ./tap_google-analytics_reports.json
COPY ./singer/target_postgres_config.json ./target_postgres_config.json

CMD tap-google-analytics -c ./tap_google-analytics_config.json | target-postgres -c ./target_postgres_config.json >> state.json

A few things here to call out to get this working. Installing the PostgreSQL target was straight forward but for the Google Analytics tap I used an unofficial library and built it from the source at https://gitlab.com/meltano/tap-google-analytics. After that we have the various config files.

tap_google-analytics_config.json This is the config for the Google Analytics tap, key file location is name of the config file that details the service account credentials. The reports property is a json file that defines the reports to pull from Google Analytics. The start date and end date here define the date range to pull the report data for.

{
  "key_file_location": "client_secrets.json",
  "view_id": "111111111",
  "reports": "tap_google-analytics_reports.json",
  "start_date": "2020-06-01T00:00:00Z",
  "end_date": "2020-06-15T00:00:00Z"
}

client_secrets.json this file contains the service account credentials, to get these follow the instructions defined in https://gitlab.com/meltano/tap-google-analytics#creating-service-account-credentials.

tap_google-analytics_reports.json this file defines the report data being pulled from Google Analytics. A full list of the available dimensions and metrics can be found here: https://ga-dev-tools.appspot.com/dimensions-metrics-explorer/. Here is an example of a report for pageviews.

[{
    "name": "ga_pageviews",
    "dimensions": [
        "ga:date",
        "ga:pagePath",
        "ga:source",
        "ga:medium",
        "ga:sourceMedium"
    ],
    "metrics": [
        "ga:pageviews"
    ]
}]

target_postgres_config.json This file is the config for the PostgreSQL target, mostly straight forward connection config.

{
  "postgres_host": "localhost",
  "postgres_port": 5432,
  "postgres_database": "ga_data",
  "postgres_username": "db_user",
  "postgres_password": "db_treasure",
  "postgres_schema": "public",
  "after_run_sql": ""
}

Now that we have everything setup to run in docker to truly automate this all that is left to do is being able to update the start and end date properties in the Google Analytics config file dynamically. This can be done with a bit of bash.

Working Remotely (and how)

2019-05-25T00:00:00+00:00

I work remotely and have been full time for over 6 months now and I wanted to take some time to talk about my experience with it. I quite often joke about this, for example here is a post from my Instagram after my 2nd week at another fully remote workplace.

damiankarzon

Damian Karzon (@damiankarzon) on Mar 8, 2019 at 4:42pm PST

What people think of when I say I work from home vs. what it's actually like.

But this is actually something I take very seriously and I am constantly asked about it. When I tell people I work from home they often ask me a question like “How often do you have to go into the office?” or “Where is their head office?”. My current workplacehas no office, none at all. The entire company works remotely which is sometimes a strange concept for people.

When I first started working remotely there were a few things that I was worried about:

Distractions
Cabin fever
Isolation
Work-life balance

Again, this post is specifically about my experience with remote work and, as I have learned throughout this time, everyone is different and depending on personalities are affected in different ways when it comes to this sort of thing.

Distractions

When I first started I thought the distractions would be a big problem for me (from past experience working from home in a non-full time way) so I set out to make sure I could minimise it. The main thing I did for this was to separate my home and work life both physically and mentally. I had a spare room at my place which I have converted into my home office that I use exclusively for working, I also have a dedicated work computer that I don’t use for anything else (ie. no games on it). To separate my home and work life mentally I try to go for a walk every day when I finish work, this helps to break up the work day from the non-work day (it’s also good to get some exercise in).

Cabin fever

This wasn’t something I had actually considered when I started since I am quite comfortable staying at home for long periods of time. However the problem came up often when I found myself being stuck in a problem that I couldn’t solve. I would start to get irritated very easily by my surroundings and not being able to solve the problem in front of me so I would end up walking up and down the hallway for no reason. In this situation now I quite often turn to someone else on the team and jump of a quick call just to talk things out or I leave the house and go for a walk to get some fresh air.

Isolation

I often hear people talk about isolation being their biggest problem when it comes to working remotely but this hasn’t been a massive problem for me, probably due to living alone for the last 6 years. This is also helped by the virtual meetings we have which are all video calls and always start with a bit of a chat before getting into business. A great example of this is our weekly whole company meetings where we start by everyone having a chance to share something personal (maybe they did something exciting on the weekend or during the week) or they can share a learning from the previous week.

Work-life balance

“Do you find yourself working more since it’s all just there?” or “Do you sometimes just go and watch tv instead of working?” other questions I sometimes get asked. Both are on opposite sides of the work-life balance, on one hand you have the potential to over work since it is so close and on the other you could do no work at all because you are not confined to an office. My answer to both of these questions is normally “Yes but…”, yes sometimes I do find myself working more, yes sometimes I do take a random break during the day if I’m feeling drained or stuck. It’s all about having the balance between the 2, it is much easier when working from home to over work or to under work but if you are motivated to do the work and have the discipline to keep working when you need to then you won’t have a problem. Motivation can be hard to find sometimes so it’s important to have work that you enjoy and are passionate about and to have a team that work well together. It is also important to remember that no job is awesome all the time and work will sometimes get you down but you need to keep at it anyway and work on making it better where you can.

Environment

I have put a lot a lot of time, effort and investment into getting my home office setup and it’s at a point where I am more productive there than any office I have ever worked at.

Desk: SKARSTA sit/stand desk from IKEA (120x70 cm)
Chair: DX Racer Classic Series
Laptop: Microsoft Surface Book 2 (13” i7, 16GB Ram, 512GB SSD) with Surface dock
Monitor: Dell P2715Q 27” 4K
Audio: Steelseries Arctis 3 Bluetooth headset & JBL Charge 2 Bluetooth speaker
Keyboard: Steelseries Apex 350
Mouse: Steelseries Rival 110

Tools

It is also super important to find the right tools and team process to be able to work remotely. The majority of our communication is text based and for that we use Slack mostly (some emails here and there when dealing with external parties). Sometimes it’s harder to communicate things over text, when this happens we tend to jump on a quick call using either a phone call, Slack call or a Zoom video call. We currently run a “scrum-like” development process which includes daily virtual stand-ups, for this we use Zoom. We also use video calls for larger team meetings such as the weekly whole company or product team meetings, this makes discussions much easier and allows people to share screens if required.

There are also times when the team (or at least part of the team) will meet up in person, for example the Brisbane dev team try to get together once a month and spend the day working together, this is either at a co-working space or a shared space (such as a library). There are also times when all of the workers in Brisbane will get together for a team lunch, this greatly heaps everyone get to know each other more and to feel more comfortable with each other.

Well that’s enough ramblings from me about working remotely, let me know your thoughts on it or if you have any questions feel free to reach out to me (@dkarzon on Twitter).

Building a Discord voice bot with Discord.NET

2019-02-18T00:00:00+00:00

I've been playing around with the Discord api recently, I wanted to see what could actually be done with it. I set a goal to make a discord bot that slightly annoys people by joining voice channels and randomly say Zapp Brannigan quotes.

For those who don't know Discord is an:

All-in-one voice and text chat for gamers that's free, secure, and works on both your desktop and phone. - https://discordapp.com/

I got started with Discord.NET and was able to get the bot connected very quickly. Discord.NET is an unofficial .NET API wrapper for discord which seem to be the most popular and widely used library for it so I went with it. Be sure to check out the documentation for details. For this to work with voice connections there are a few native dlls that you need to download and put with your application. These are simple enough to get from the voice documentation (or check out my full source link at the end).

First thing you need to do is get an instance of DiscordSocketClient for my project I am doing a .NET Core 2.2 console app with the built in dependency injection.

_discord = _services.GetRequiredService<DiscordSocketClient>();
await _discord.LoginAsync(TokenType.Bot, _botToken);
await _discord.StartAsync();

That is all the code you need to have your bot connect to discord!

I wanted my bot to detect when a user joined a voice channel and connect to that channel (if it wasn't already). To do that I used the UserVoiceStateUpdated event on the DiscordSocketClient which can tell us when a users voice state updates (join/leave/move voice channels).

// Add this before calling StartAsync
_discord.UserVoiceStateUpdated += OnVoiceStateUpdated;

The code to handle the event then looks like this:

private async Task OnVoiceStateUpdated(SocketUser user, SocketVoiceState state1, SocketVoiceState state2)
{
    // Check if this was a non-bot user joining a voice channel
    if (user.IsBot)
        return;

    if (state1.VoiceChannel == null && state2.VoiceChannel != null)
    {
        ConnectToVoice(state2.VoiceChannel).Start();
    }
}

private async Task ConnectToVoice(SocketVoiceChannel voiceChannel)
{
    if (voiceChannel == null)
        return;

    Console.WriteLine($"Connecting to channel {voiceChannel.Id}");
    var connection = await voiceChannel.ConnectAsync();
    Console.WriteLine($"Connected to channel {voiceChannel.Id}");
}

Important to note here is that we don't await ConnectToVoice, this is because if we await that call from inside the UserVoiceStateUpdated event it will cause a deadlock and never return from connecting to the voice channel.

Now we are connecting to a voice channel when we detect a user joins it so lets look at having the bot actually send audio through. To do this the documentation for the library has an example using ffmpeg to read the sound files and pipes that into a stream that then gets sent to Discord.

var psi = new ProcessStartInfo
{
    FileName = "ffmpeg",
    Arguments = $@"-i ""{sound.Filename}"" -ac 2 -f s16le -ar 48000 pipe:1",
    RedirectStandardOutput = true,
    UseShellExecute = false
};
var ffmpeg = Process.Start(psi);

var output = ffmpeg.StandardOutput.BaseStream;
var discord = connection.CreatePCMStream(AudioApplication.Voice);
await output.CopyToAsync(discord);
await discord.FlushAsync();

Here we start a new process passing in start info that opens up ffmpeg.exe (also included in the project) passing in a relative path to out sound file and a few other command line args to play the sound and pipe the output which we then copy into a stream into discord.

This is the basics of creating a voice capable discord bot with Discord.NET, for a more advanced implementation take a look at the full source for this project at https://github.com/dkarzon/DiscordZapBot.