Without fail, in every consulting engagement I start, there is a common theme – that failure to articulate and scope “the software development work” pushes wide and varying dysfunction through organisations.

It’s universal. It happens to everyone. And it’s nothing to be ashamed about.

Working out what to do, how hard it is, and communicating it effectively is a huge proportion of getting work done. Entire jobs (project management, delivery management) and methodologies exist around trying to do just that.

What is slightly astonishing is how prevalent poor estimation and failed projects are in technology – especially when there’s so much talk about agile, and data driven development, and measurements.

In this piece, we’re going to talk about why we estimate, what we estimate, and how you should actually estimate, hopefully to remove some of the stress and toil from what can often seem like theatrics and process.

Why we estimate.

No more than you would hire a plumber to fix up your bathroom if they couldn’t indicate to you how long it would take or what it would cost, should you start a software project without understanding its complexity. Estimation is a necessary burden that exists when people must pay for work to get done and estimation means different things for different people.

For a team, estimation is a tool we use to understand how much work we can get done without burning out. For your C-level, estimation is a tool they use to understand what promises they feel comfortable making.

In both cases, estimation is about reducing or mitigating risk by making sure we don’t promise things we can’t deliver. At an even wider scale, estimation is simply about us understanding what we can and cannot do.

Estimation also does not exist in a vacuum - estimations are only valid for the specific set of people that make them. If you change the composition of a team or the skills it possesses, the amount and types of work that team can accomplish are going to change.

Estimation works by presuming that in fixed period, the same people should be able to do the same amount of work, of roughly similar or known types, that they have previously. It’s based around a combination of a stable team, knowledge, and experience.

Estimates are owned by the team; they are the team’s measure.

What we estimate

From the mid-2000s, User Stories have been the more-or-less-standard way of articulating work that needs to be done in software, they are meant to be an artifact that tracks a stand-alone and deployable change to a piece of software.

They should be articulated from the position of user following the mostly standard format of:

As a <type of user>
I want <some outcome>
So that <reason that this is important>

In traditional agile workflows, user stories tended to be written on index cards, with any associated acceptance criteria written on the back of the index card. When all the acceptance criteria are met, your story is complete.

Over time, the idea of tasking stories grew in popularity, and tasks were often stuck to the story cards using sticky notes. This was a useful physical constraint, as stories could only get so big before the sticky notes would literally fall off the cards.

To estimate, we estimate the complexity of each of these discrete pieces of work, in isolation. Our estimates are meant to be the measure of how much effort it would take the whole team to get the entire story into production.

It’s not uncommon to find different teams have different ideas about what “done” means for their stories – but over time almost everyone agrees that “in production” is what qualifies a story as done.

Some organisations may think they need to complete additional work in release process automation before this feels like an achievable goal, but I would always rather an estimate be honest and capture the existing complexity related to releasing software, if there is any.

Ultimately, for estimation to provide an accurate picture of your teams’ capability, any work the team performs should be captured as user stories and estimated.

Nothing ships to production without estimation.

Estimation Scales

Estimation in agile projects takes place using abstract measurements that represent the amount of effort it takes a given team to complete a user story.

Over the last twenty years, people have suggested different estimation scales for work.

These are:

• Time (usually in half days)
• T-Shirt Sizes (XS, S, M, L, XL, XXL)
• XPs “doubling numbers” (1,2,4,8,16,32…)
• The Fibonacci sequence (1, 2, 3, 5, 8, 13, 21…)
• The Modified Fibonacci sequence (1, 2, 3, 5, 8, 13, 20, 40 and 100)
• A few others not worth articulating.

The different methods offer different pros and cons.

Time is a generally poor estimation scale because it is both concrete and fixed. There is no room for manoeuvre when you estimate in time, and no negotiation. Estimating in time is the quickest way to disappoint your stakeholders the first time you are wrong.

Estimating in T-Shirt sizes is often useful for long-term projections and roadmap planning where the objective is to just get a feeling for the relative size of a piece of work.

XP (eXtreme Programming) originally suggested the doubling of numbers for its abstract estimation scale to capture the fact that work got more complex the bigger it became, and it was very easy for work to grow too big.

Later, the Fibonacci sequence was suggested as a more accurate way of capturing that same concept, but instead as an exponentially increasing curve, to capture the escalation of complexity. This sequence was later modified and capped off with 20, 40 and 100, for practical reasons (nobody wants to endlessly debate which enormous number is the right one to pick).

Most teams that do estimation end up using the Modified Fibonacci sequence because it captures the increasing complexity of unknown, unknowns, along with being simple to remember, and explicitly not being a measure of time.

How to estimate

In a planning meeting, the entire team that will be working on the work get together and read through the stories that have been written for them. These stories are usually prioritised by a product manager or product owner, but estimation doesn’t require this to be useful.

As a team, short 2–3-minute discussion about each story take place, sharing with each other any context or information. During this discussion, people can note down the kind of tasks that completing the work would require, discuss special knowledge they have about the story, or provide any additional context.

As a rule of thumb, it’s a good idea to list the kinds of activities involved - “some front-end work, maybe a new API, some configuration, some new deployment scripts, some UI design” as a way of highlighting hidden complexity.

This is because the more moving parts there are to a story, the more complex it will become.

At the end of this brief discussion and crucially at the same time using either planning poker cards or just by holding the pre-requisite number of fingers up as if playing a game of stone paper scissors, the team should all estimate the complexity of a story.

The team estimates simultaneously to prevent the cognitive bias of anchoring, where whoever gets in first with a number will “set the tone” for future estimates, consciously or not.

The estimate is then noted on the story card, and everyone moves on.

It’s exceptionally important that the entire team estimates every card, and the estimates are for the whole amount of effort required of the entire team.

One of the more common mistakes in estimation involves people estimating only the work for their specific function (Front-end and back-end estimates, dev and QA estimates, design and dev estimates, are all equal anti-patterns). This is problematic, because it hinders a shared understanding of complexity by all team members, along with occasionally driving people towards the anti-pattern of maintaining estimates for individual team functions.

The team estimates as a whole to increase its coherence, to share it’s understanding of the complexity of its work, and most importantly to replace the mental model of a single contributor with that of the whole team.

There are a few common points of conflict that happen during estimation sessions that are worth calling out.

The team can disagree on the score they are settling on.

This will happen frequently – imagine you’re estimating using modified Fibonacci and half the team believes the story to be worth 5 points, while the other half believes the story to be an 8-point story. You can resolve this conflict by always taking the higher estimate.

There is often resistance to this approach from product owners responsible for trying to maximise for team throughput but the truth to this is very simple – it’s always easier to under-promise and over-deliver than to do the opposite and disappoint people.

If a team over-estimates some work, they’ll have plenty of capacity to schedule in subsequent work anyway. If the larger estimate was true, and the smaller number was chosen, it jeopardises work that may have been promised outside of the team.

The range of story points expressed by the team in planning poker is too wide.

Another common scenario is where vastly different numbers are selected by team members. Perhaps there’s an outlier where one team member thinks a piece of work is a 1-point story against an average of 5 points from everyone else.

You can resolve this conflict with a short discussion – perhaps that team member knows something that wasn’t shared in the prior discussion. Estimates can be discussed and revised at this stage by negotiation.

The team argues a story is inestimable, or just too big.

If the team feels the story is not refined enough to be estimated as actionable work, they may push back on the grounds that the story is too big for estimation.

When this occurs, the team should either, collectively, during planning, decompose the story into smaller stories that can be estimated or, defer the story to do the same in another meeting.

If stories are frequently inestimable, unclear, or lacking sufficient detail, schedule a refinement session to occur every other day for 45 minutes. In this session, the team can collectively refine and write stories, until the backlog of work to be done becomes more approachable. The team should choose to what extent these sessions are required based on their comfort levels with the work present in the backlog.

For particularly sticky stories that appear to get repeatedly rebuffed for being inapproachable, it might be advisable to write up a spike. A spike is a time-boxed story that results in further information and research as its output and should inform the refinement and rework of stories deemed too complicated or unapproachable.

If you find yourself writing spikes, be sure to articulate and write acceptance criteria on the spike that state the exact expected outcome of the work.

Planning and estimating work often starts from a place mired in detail and complexity, but with the use of frequent refinement sessions and lightweight estimation conversations, it should mature to be more predictable and simpler as your team gains an increasing coherence.

Estimation and planning with newly formed teams will often take a few iterations to level out, as the team members slowly gain a greater understanding of the kinds of work that happens in the team.

Understanding velocity

Velocity is the term used to describe the rolling average of story points completed during iterations.

A teams velocity is an average, and represents the work of the team in it’s current configuration. If your team makeup changes (the number of people, the distribution of skills, holidays) your velocity will change accordingly.

Velocities are used during planning exercises to understand how much work a team can realistically expect to get done in an average iteration. It’s important to understand that velocity is not a target to be exceeded – it’s an abstract number that represents the sustainable pace of the team.

Velocity is for the team and is a non-comparable measure.

Two teams, even two teams in the same organisation, reporting the same velocity number will always be talking about different things. While it is often tempting to try and baseline or compare teams based on velocity metrics, you are not comparing like for like and any statistics gleamed from them will be false.

Velocity is designed to be used in the following scenarios:

• Understanding how much work the team can do in each iteration.
• Planning for holidays and sickness by deliberately reducing velocity
• Understanding the predictability of a team – inconsistent velocity is a key indicator of poor story writing hygiene or an unpredictable environment.

The goal of a team is to have a stable velocity because a stable velocity means a team can make promises about when stories will be delivered.

Estimating roadmaps, and projection

Estimation is often desired during long term planning exercises such as company roadmaps, quarterly reviews, or other long term vision activities.

It’s useful to understand that estimation is like a shotgun – it is very accurate at short-range and decreases in accuracy the further away your plan or your schedule is. Consequently, detailed estimates become useless at range.

This frequently leads to tension between business leaders that want to communicate to their investors, the public, or their other departments and technologists, who embrace the uncertainty of high-level planning exercises.

Road mapping and other high level estimation processes are an entirely valid way for a business to state it’s intent and its desires but what they are not, and cannot be, are schedules.

We can use high level estimating to help these processes with a few caveats. Firstly, we should always use a different estimation scale for high level estimates. This is important to ensure that wires do not get crossed and estimations conflated.

Secondly**,** high-level estimation processes should and will lack implementation detail, and as a result will end up with items often categorised broadly as either easy, average, large, or extra-large.

This idea of vague complexity is important for a business to understand the order of magnitude size of a piece of work, rather than day-and-date estimation of expected delivery. Once this rough size is established, further work will be required to produce more useful, team centric estimates.

An alternative approach to detailed roadmap projections, and I promise I am not joking, is counting.

Usually, roadmaps are vast and contain many themes and pieces of work, and it’s often enough to simple count the number of items and ask the honest question “if your entire company was working on this roadmap, could you do one thing a day”. If the answer is no, repeat the question for two days, and three days until you work out what you think your “multiplication factor” is.

Often, this number is accurate enough to understand how long a roadmap might take, in conjunction with some simple “small / medium / large” estimates over the roadmap items.

Estimation and dates

If estimates were dates, they’d be called dates, not estimates.

Estimates exist to help teams understand their sustainable pace and velocity. Using estimates to reverse engineer dates is a dangerous (if sometimes possible) non-science.

Rather than attempting to convert story points back to days-and-dates, teams should focus on providing stable velocity and release windows.

A team with a stable velocity should be able to predict the earliest point at which a feature should be delivered by combining their estimates, their capacity, and the expected end of the iteration that work is prioritised into.

The earliest point work could be expected, should be that date.

#NoEstimates

“But I read on twitter that estimates never work! All the cool kids are talking about #NoEstimates!”

Over time, healthy and stable teams refine their stories down to consistently shaped manageable chunks. As these teams cut through their work, they often find that their estimates for stories are always the same leading to a scenario where “everything is a 5!” or “everything is a 3!”.

This is amazing for predictability, because what those consistent numbers really represent are a shared mental model and a known sustainable pace.

Now, because estimates are deliberately abstract and deliberately the teams own internal measure if everything is a 5, and all stories are always the same size, and they’re always a 5. Then, well, everything could just as easily be a 1.

When everything is the same size, then the goal of estimation has been reached.

As a result, you may hear people from highly functioning, long term and consistent teams advocating for no estimates because they are at the far end of this journey and have probably developed internal mechanisms for accounting for team capacity changes.

In contrast to this, teams that do not share a mental model, and a coherent internalised idea of average story size absolutely require estimation to help them walk to that place of predictability and safety.

It’s unrealistic to expect teams to magically develop a shared idea of story size and capability without giving them tools – like estimation – to measure their progress on the way.

When your team reaches a comfortable place of coherence, you’ll know when you can drop estimates.

FAQs

I feel like I’ve been talking about estimation and planning for a decade and a half in various forms, and people have plenty of questions about the topic. Here are some of the most common ones.

Should we estimate tasks and subtasks?

You estimate anything at all that the team must work on, to do some work.

There’s certainly an anti-pattern supported by digital tools with a lack of physical constraints (you can’t stick loads of subtasks onto an index card!) that leads to explosions of subtasks.

Often this is people using subtasks as a specification, rather than doing the work to split up and refine a story that is too large to reason about.

I would always prefer any tasks be present as notes on a story card, rather than things that can move, be reallocated, and block on their own. They’re either an integral part of the story, or they are not.

Either way, yes, you must point them if they are work.

How do we estimate technical tasks?

This comes up a lot – and there are two answers.

Either:

• The technical task as part of the work of a user facing story, and its “estimate” is just part of that work.
• You articulate a technical user persona, who has desires of the system, and the “technical task” is a user story that meets a need for that user.

In both cases, you estimate the story like any other, based on the complexity for the whole team to complete that change in production.

Should we estimate bugs?

Bugs are only bugs when they’re on production – before that point, they’re just unfinished work.

The effort it takes to fix any bugs you find during development is part of the estimation of the story and observations made by testers while you’re building is just part of the work.

Once a bug has made it to production? Well, it’s just another story to fix it, and it should be estimated, like any other story and these stories will count towards your velocity, like any other work.

Should we estimate front-end and back-end tasks separately?

No. Estimates account for the work of the entire team, communication overhead, and delivery. Any work that people do should be accounted for in the teams estimate.

Why are spikes time-boxed and not pointed?

Good catch! Spikes are the anomaly in this process because they are timeboxed.

They are timeboxed because they do not result in work shipped to production but result in more information. They should be treated as if whoever is doing the work is removed from the iteration for the duration of the spike, and the amount of work done in your iteration should be reduced accordingly.

Over-reliance on spikes is an anti-pattern – they’re here to help you break down meaty pieces of work, not to push analysis downwards into the team.

If we don’t finish a story at the end of a sprint, should we re-estimate it at the start of the next one?

It doesn’t really matter a huge amount, you either leave the story on your board, and reduce incoming work until everything is finished, or you re-estimate your story for “work remaining”.

I don’t like re-estimating stories once they’re being worked on and seeing as it’s less work to just do nothing, I’d recommend doing nothing and allowing the fact that velocity is a rolling average to deal with any discrepancy in your reporting.

You need different tools at different times.

Estimation and planning are probably some of the more frequently debated agile methods.

The good news is that you get better at estimation and planning by practicing it, and by refining your stories, and by functioning increasingly as a team rather than a group of individuals.

As you work towards a level of coherence where estimates become less useful, it’s important to remember that at different levels of maturity, teams need different things, and that’s totally, totally fine.

The purpose of estimation is to help give you a sense of predictability. Do not be afraid to change the process if it’s not helping you achieve that goal.

While we cannot predict the future, we can get better at estimating it with practice.

Programming is full of dogma.

Programmers are so often self-reflective and obsessed with the 'new way of doing things'. It can make programming feel like a cottage industry dedicated to best practice and leadership, patterns and practices and trying to write the 'right' code.

Amongst all the noise, it’s easy to lose sight of why people practice certain disciplines. There’s no shortage of people desperate tell you exactly what you should be doing to make your code “good”, but it is less common for people toarticulate why their approach is better.

The pursuit of good code framed as “application architecture” or “patterns and practices” often pushes codebases towards “tried and tested design patterns”.

These design patterns are model answers – examples of how you can solve specific problem in your language and environment. They exist to help people understand how a good, well factored, solution to a problem should look.

At some point, the collective hive mind decided that remembering these patterns was a mark of experience and status, while at the same time often losing the knowledge of why those solutions – the patterns - were reached in the first place.

Unfortunately, because so much of software is context sensitive, there are many times where blindly applying design patterns makes software much more difficult to reason about and increases the cognitive load of understanding what a program does.

So how can we make sure we’re writing our programs in maintainable, sustainable, and most importantly, understandable, ways?

Without patterns, how do we know what good looks like?

Once you have seen enough codebases, in enough languages, you will realise that most of the arbitrary stylistic qualities of code that people often focus on (“tabs vs spaces!”, “brace style!”) are mostly irrelevant, and instead there are a couple of non-negotiable qualities you look for in a codebase that are far less regimented than your average JavaScript style guide.

Good code is:

• Code that is easy for someone with minimal “domain context” to read Code that your average developer can understand is good code.

• Code that focuses on developer-experience, debuggability, and usability Code that you cannot debug meaningfully, is not good code.

• Code where the intent takes up more visual space than the language syntax. Code that buries what it does under the programming language, is not good code.

Regardless of language, or style, or intent, all programmers interact with codebases in the same way – they glance at it to understand what it does, and they read intently to understand how it does it.

Good code, regardless of style, makes that easy.

You don’t like design patterns?

A design pattern is a commonly accepted approach to a specific type of problem.

The original works about design patterns (the famous “Gang of Four” book) managed to capture battle tested solutions that were required so often by the programming languages and runtimes at the time, that they became the “model answers”.

These design patterns were useful because they exposed shortcomings in the languages of the time. They identified common pieces of code that were written time and time again, to solve problems that existed in multiple systems. As languages have changed and evolved, the design patterns that remain relevant have mutated and changed with them.

The reason design patterns are often read as good code is because they act as a shorthand for a shared understanding that teams can reach.

This shared understanding comes with an amount of gatekeeping, and with gatekeeping, a trend for people to believe that they are the only legitimate solutions to some problems.

Because of this, people will reach for the patterns they know in inappropriate contexts, believing that is the only way. People will reach for them to prove they “know good code”. People will reach for them and end up introducing complexity instead of clarity.

I love design patterns and known good solutions when they fit, but only when the need for them can be precisely articulated.

Care about the cost of abstraction

At a really meta level, all code is constructed from abstractions and encapsulation. Regardless of programming language, and regardless of era, this has always been true.

The programming languages, runtimes and frameworks with which we write higher level code go through an enormous amount of compilation, translation, and interpretation to execute.

Programming languages themselves are common abstractions over operating system APIs, which are abstractions over assembly languages of microarchitectures, which are abstractions over physical hardware.

This trend continues upwards; the functions you write, the types you define, and the classes and files you create that make up your application are all examples of both abstraction and encapsulation in action.

For the most part we are blissfully unaware of the huge amount of complexity that exists underneath your programming environment because the value that that abstraction brings vastly outweighs the complexity of the code running underneath it.

It is orders of magnitude easier to comprehend this (in pseudocode here):

File.Open(“testFile.txt”);

than the following roughly equivalent x64 Linux sample (stolen with love from StackOverflow)

; Program to open and write to file
; Compile with:
;     nasm -f elf64 -o writeToFile64.o writeToFile64.asm
; Link with:
;     ld -m elf_x86_64 -o writeToFile64 writeToFile64.o
; Run with:
;     ./writeToFile64
;========================================================
; Author : Rommel Samanez
;========================================================
global _start

%include 'basicFunctions.asm'

section .data
fileName:  db "testFile.txt",0
fileFlags: dq 0102o         ; create file + read and write mode
fileMode:  dq 00600o        ; user has read write permission
fileDescriptor: dq 0
section .rodata    ; read only data section
msg1: db "Write this message to the test File.",0ah,0
msglen equ $ - msg1
msg2: db "File Descriptor=",0

section .text
_start:
    mov rax,2               ;   sys_open
    mov rdi,fileName        ;   const char *filename
    mov rsi,[fileFlags]       ;   int flags
    mov rdx,[fileMode]        ;   int mode
    syscall
    mov [fileDescriptor],rax
    mov rsi,msg2
    call print
    mov rax,[fileDescriptor]
    call printnumber
    call printnewline
    ; write a message to the created file
    mov rax,1                 ; sys_write
    mov rdi,[fileDescriptor]
    mov rsi,msg1
    mov rdx,msglen
    syscall
    ; close file Descriptor
    mov rax,3                 ; sys_close
    mov rdi,[fileDescriptor]
    syscall

    call exit

You never have to worry about all that stuff going on, because “File.Open” and its equivalents reduce the cognitive load on the developer by taking complex implementation details and removing them from your code.

As you implement your own software, sadly it is not a truth that all abstractions reduce cognitive load, and it’s often a very subtle process to understand what does and doesn’t make code easier to work with.

The cognitive trade-off in extracting functions

One of the hardest challenges when working out exactly how to design your code, is understanding when the right time is to introduce abstraction or refactor out code into functions.

Consider this example:

public class Customer
{
    public List<string> Names { get; set; } = new List<string>();
}

public static class Program
{
    public static void Main()
    {
        var input1 = "My Customer Name";
        var customer = new Customer();
        customer.Names = input1.Split(' ').ToList();
    }
}

One of the more common refactorings that could be applied to this code is to extract a method from the logic that computes the customer names based on the input.

public class Customer
{
    public List<string> Names { get; set; } = new List<string>();
}
public static class Program
{
    public static void Main()
    {
        var input1 = "My Customer Name";
        var customer = new Customer();
        ExtractName(customer, input1);
    }

    private static void ExtractName(Customer customer, string input1)
    {
        customer.Names = input1.Split(' ').ToList();
    }
}

This refactoring, in a codebase only 3 lines long, adds nothing for the syntactic weight it adds. Reading the code requires an additional hop on behalf of the reader – and introduces an entire extra concept (parameter passing) for it’s supposed benefit (providing a name to the extraction operation).

In small examples of code, where you can rationalise and read the code in a single pass, extracting this function adds cognitive load rather than removes it.

If you read that example carefully, it relies on the fact that input1 is a well-formed string, with spaces between the Names. A good counter example, where factoring would improve readability, is the following sample that deals with UpperCamelCaseNames.

public class Customer
{
    public List<string> Names { get; set; } = new List<string>();
}

public static class Program
{
    public static void Main()
    {
        var input = "MyCustomerName";
        var customer = new Customer();

        var parts = new List<string>();
        var buffer = "";
        foreach (var letter in input)
        {
            if (char.IsUpper(letter))
            {
                if (!string.IsNullOrWhiteSpace(buffer))
                {
                    parts.Add(buffer);
                }

                buffer = "";
            }

            buffer += letter;
        }

        if (!string.IsNullOrWhiteSpace(buffer))
        {
            parts.Add(buffer);
        }

        customer.Names = parts;
    }
}

By subtly changing the requirements, the amount of code required to solve this problem explodes in size – the cyclomatic complexity (number of conditionals in the code) increases, and with it, the ability to glance and sight read the code.

Refactoring that sample by extracting a well named Split function is vital to keeping the code legible.

public static class Program
{
    public static void Main()
    {
        var input = "MyCustomerName";
        var customer = new Customer();
        customer.Names = SplitOnCapLetters(input);
    }

    private static List<string> SplitOnCapLetters(string input)
    {
        var parts = new List<string>();
        var buffer = "";
        foreach (var letter in input)
        {
            if (char.IsUpper(letter))
            {
                if (!string.IsNullOrWhiteSpace(buffer))
                {
                    parts.Add(buffer);
                }

                buffer = "";
            }

            buffer += letter;
        }

        if (!string.IsNullOrWhiteSpace(buffer))
        {
            parts.Add(buffer);
        }

        return parts;
    }
}

Some of the most illegible and hard to follow codebases I’ve seen fall foul of this anti-pattern of prematurely extracting implementation details from the place they’re most important.

Always try and remember the Principle of locality (borrowed from physics) – “an object is directly influenced only by its immediate surroundings” and keep the implementation of logic close to the place it’s used.

Over deconstruction of implementation can hamper readability because it forces people to context switch. If we consider the first example again –

public static class Program
{
    public static void Main()
    {
        var input1 = "My Customer Name";
        var customer = new Customer();
        ExtractName(customer, input1);
    }
}

With the removal of the implementation of ExtractName, you would be forgiven for thinking that it contained complex or lengthy logic. Since its implementation is elsewhere, it forces you to check that it does not in order to accurately understand the code. It is an abstraction that adds no value.

Extract functions when they enhance readability, rather than just incur the cost of a context switch.

The cognitive trade-off in moving code physically

If moving code between functions has a chance of both increasing or decreasing cognitive load, then moving code between files amplifies this difference.

Files and directories act as a larger context switch – you should expect that things that exist in different files to contain difference concepts inside of your domain, and that things in different modules describe entirely different and complete concepts.

The inverse of this is also true – splitting out code into different modules, that are tightly coupled to concepts used elsewhere in your application, dilutes the readability of the code. Therefore, making apprehending your codebase more complicated

Understanding this is the path towards feature based code organisation, as the natural conclusion is that defining code closest to where it is used in the application or system is often the right answer.

The right time to introduce abstraction.

So far, we’ve covered the cost of context switching when dealing with abstractions, but do not think the message here is “don’t use abstractions in your software”. Abstractions are the thing you use to build up the internal language of your software, they’re what you use to craft your APIs, they exist to enhance clarity, not reduce it.

Let’s think about a few rules to follow:

Introduce abstractions to fully describe concepts in a codebase.

These abstractions can be files, directories, or types (this’ll vary by programming language). You should absolutely create files and types to capture core-domain concepts like “Users” and not have that logic leak throughout your code.

These encapsulations should be kept as near to the parts of the code that use them as possible.

The more distinct a concept is from the code that calls it, the further away it should be physically located in your codebase.

For example, the type or file related to “Users” should be located near the implementation of features that interact with “Users” – not arbitrarily split into another package or assembly, because the concept of “Users” is application specific and doesn’t need to be used elsewhere.

Beware prematurely creating shared Domain or Core packages because they often become dependency magnets that get inappropriately reused and are hard to untangle. Resist the urge to extract the core concepts of your application from the application itself. The moment you extract this code into its own assembly, you introduce the chance that a breaking change impacts another application that use it, which increases cognitive load.

Conversely, abstractions that describe concepts unrelated to what your application does should be kept further and further from where you use them. Don’t bury abstractions relating to infrastructure deep inside feature code and accidentally tie your applications behaviours to its hosting environment.

When things get big – moving to repositories and published packages.

As your application grows, it will become increasingly less maintainable by virtue of its size. This isn’t your fault, it’s an inevitability. The larger something becomes, the harder it is to fit the thing inside your head, and with this will come talk of decomposition.

There are two main ways of decomposing an application:

• You split out full vertical slices of its functionality into different “bounded contexts” (this is the original microservice vision)

• You split out lower-level concepts into their own packages or modules and maintain them separately from the application.

Decomposing an application has value – it allows you to fit more people around a problem and scale your teams, but as you sub-divide something that was once a single thing, complexity and interdependencies add to cognitive load.

If you’re lucky, and the original codebase was feature-organised, then splitting the application up should be a relatively simple task - but it will inevitably result in either one of two things:

• Shared component libraries are extracted from the original application.
• Code is duplicated between the two new applications.

There’s a natural tension between those two approaches – duplication of the code gives the owners of these respective codebases total control over their entire application but introducing shared dependencies will add a layer of complexity where the applications may require coordination in order to grow.

This cost of code sharing is real and significant and should be considered carefully.

It is important that the shared code encapsulates significant complexity to be worthy of the additional logistical costs of sharing and maintenance. Smaller pieces of shared code should be trivially duplicated to avoid this cost.

The cost of reuse

But reuse is good!

I hear you cry, sobbing in the distance.

The SOLID principles!

DRY!

I’m not listening.

In the eternal tension between DRY (don’t repeat yourself) and SRP (single responsibility), SRP should nail DRY into a coffin.

Extracting something out to be an independent package means that you need to make sure that re-use justifies the cost of:

• Maintaining a repository
• Ensuring the repository always has an owner.
• Running a pull request process for that component.
• Writing documentation
• Creating build pipelines to build, test, and release, that component.
• The extra leap it takes to change this component in any system that use it.

Reuse is great because it stops you solving the same problems twice, but as soon as something gets promoted to being its own first-class concept in your software, it needs to be maintained like one.

This is work. It’s not always complicated work, and you may have good patterns and automation to help you deal with this process, but reusing components isn’t as free as you think it is.

In fact, the cost of change with re-usable code is an order of magnitude higher that just putting some code in your application.

It’s ok! We’ll use a mono-repository!

In languages and ecosystems that were born in the age of package management (Node and Python especially) it has been the norm for years to split dependencies into lots of very small, often open source and reusable packages.

While at first this separation of concerns felt like a good thing, the over-eagerness to divide out applications caused increasing debugging and development friction, to the point where entire tools, like Facebook’s Lerna, were created just to… copy files back into the applications that they should never have been removed from in the first place.

Because those “packages” were never real, they were just the application code.

In a sense, you can look to the proliferation of the term “mono-repository” as something that tracks the presence of codebases so DRY that they might turn to dust.

The real answer is neither “use a mono-repo”, nor “split out all the things”, but in fact is this – whatever moves together, and is versioned together, and deploys together, is the same thing, and lives together.

Thoughtful Code

There’s clearly a cost and benefit to each of the refactorings you can make to your codebase. The cost of introducing abstractions and encapsulations has to be lower than leaving the code where it is or repeating it in another location.

As a guiding, thoughtful principle, it’s always worth thinking “what am I gaining from moving this piece of code or changing the way it describes a concept?” and frequently, that little bit of time and thoughtfulness is enough to combat dogma in your own work.

If you can’t articulate why you are moving something, then you probably shouldn’t do it. Here, in contrast, are some good reasons for introducing abstractions -

Creating an abstraction to describe concepts more clearly.

Extracting to provide some meaningful syntax over what the code does, rather than how it achieves it, is a good way to make your code easier to understand. Much like the name parsing example above, make sure that you’re encapsulating enough behaviour for this to be valuable.

Splitting out something that is expected to grow large.

In many codebases you start building something that is clearly a stub of an implementation, a placeholder for something that’ll become bigger. Appreciating that this is an entirely different concept than the rest of the code it is near is a good reason to either provide an extensibility point or move your nascent implementation into another file somewhere else.

Increasing testability or developer ergonomics.

It’s a common and good thing to extract out particularly significant pieces of code so that they can be independently isolated and tested. It’s vital to keep important code factored out and free from dependencies of the system around it.

For safety.

In some cases, it makes sense to physically move important components out of your application and into a module or package if you explicitly want to introduce ceremony around its modification – either for peer review or safety reasons.

To reduce visual noise at the call-site.

Sometimes it’s important to factor code out to reduce the cognitive load in reading the code that depends upon it. Anything that improves readability and clarity in your application is an excellent reason.

To encourage reuse.

While it’s useful to understand the reasons that you wouldn’t want to reuse code, it’s just as important to understand that reuse is a powerful tool. It can help keep alignment across systems, and to prevent waste in fixing the same problems time and time again.

Appropriate reuse should be encouraged, just so long as you’re aware of the associated costs.

Does this make my code good?

Whenever you introduce anything into code, being thoughtful about the reasons why, and the developer experience of the resulting code should always be your guide.

There isn’t really such a thing as “good code” – there’s just code that works, that is maintainable, and is simple. There’s a good chance you’ll end up in a better place if you spend time and effort making sure the cognitive load of your application is as low as possible while you code.

And on the way, you might realise that removing some of the abstractions in your codebase is a simpler path than over-design.

One of the things I see absolutely plague developers is time management. There is never enough time in the day to both do work, and communicate it. This counts extra in an increasingly distributed world where clear and accurate communication is more important than ever.

The other side of this desire for communication, is people clinging to filling diaries up with meetings as a way to "keep people connected", but if you're not careful, this can significantly backfire.

We've all been there, and we've all seen it:

• "too many meetings!"
• "I can't get my work done because of all these meetings!"
• "I don't feel like I have control over my time!"

Here are a few things I've done, to make sure I can get things actually done, over the years.

1. Diarise your own time.

Your calendar doesn't exist purely for other people to fill your time up. It is yours. You can use it.

During consulting gigs where people bombard you with requests, literally the moment someone asks me to do something, or look at something, I'll block out a slot in my calendar of at least one hour to actually do the work and think about the problem.

This:

• Helps you remember to do the thing
• Organises your day
• Stops meetings preventing work

This is a great trick to help set expectations of when people will expect to see results from you and embraces the fact that doing any work takes time and space.

This is perhaps the easiest way to make sure meetings don't get in the way of doing.

2. Politely reject meetings that do not have agendas or stated outcomes.

Many meetings are formless & ineffective, becoming unstructured thought & conversation. All meetings requests should come with either an agenda or an explicit goal.

Help by making sure your own invites do.

3. Leave meetings where you are adding or gaining nothing.

I suspect the biggest cost-waste in most organisations are meetings with passive participants.

Respect your own time by observing the law of two feet and politely excusing yourself to get other things done.

4. Be present in the meetings you do attend.

No laptops unless you're taking notes or doing some meeting related activity. It's simple, it's respectful.

If you feel like you have to brain space to do other things, see point 3 - observe the law of two feet and get up and leave.

5. Get into the habit of circulating meeting outcomes

There is no need to minute or notate the vast majority of meetings - but any outcomes - decisions, should be circulated to the exact invite group of the meeting after the fact.

This gives people the confidence that if they cannot attend a meeting in real time because of time conflicts, that they will be able to understand the outcomes regardless.

This is part of the respectful contract that not everyone you wish to attend a meeting, will be able to.

6. Decline meeting invites if you cannot, or do not want to, attend.

To solidify the human contract that people will circulate conclusions and communicate effectively, you need to let people know if you're not coming.

You don't have to tell them why, but it'll help them plan.

7. Don't miss mandatory team communication meetings.

There's a pattern of fatigue that develops when meetings start to feel rote or useless. Don't respond by not attending, respond by "refactoring your meetings".

Discuss cancelling pointless recurring events to free up time.

8. Schedule formless catchup meetings

It's easy for meetings to devolve into chatter, especially in 2020, where we're all craving human contact. Set meetings up with this goal.

A social meeting is not a crime, it's optional, and people that are craving contact will thank you.

9. Consider doing work in meetings rather than talking about it.

Many meetings can be replaced with mob programming sessions, where you do the thing, instead of discussing it.

Doing the work is the best way to know if it's right & valuable.

Prefer this format where possible.

Changing a meeting culture

It's easy to think that you have no control over your time, or that the meeting culture of your organisation won't change - this is a real concern, but there are a few things you can do to counter it.

Regardless of the meeting culture of your organisation, you can make sure your own meeting requests follow these guidelines. Encourage your peers to follow suit. This goes doubley if you're in a position of power or responsibility, like a team lead, or a head of. You can normalise good meeting discipline.

You can also help nudge meeting culture out of the way by talking to meeting organisers about the above as meetings appear. Challenge the meeting requests with friendly questions, offer to help improve meeting invites so people can be more prepaired.

Effective meetings are wonderful, people are wonderful, and getting work done requires brain space and time.

Don't let meetings crush your work and motivation 🖤

Work in software long enough and you'll meet people that absolutely adore functional programming, and the families of languages that self-select as "functional" - even if you might not realise what it means.

"In computer science, functional programming is a programming paradigm where programs are constructed by applying and composing functions. It is a declarative programming paradigm in which function definitions are trees of expressions that each return a value, rather than a sequence of imperative statements which change the state of the program."

From Wikipedia

Originally coined as a term to describe an emerging family of languages - Haskell, LISP, Standard ML, and more recently applied to languages that follow in their spiritual footsteps like F# and Clojure - functional programming languages are very much the mathematical end of programming.

"Functional programming languages are a class of languages designed to reflect the way people think mathematically, rather than reflecting the underlying machine. Functional languages are based on the lambda calculus, a simple model of computation, and have a solid theoretical foundation that allows one to reason formally about the programs written in them."

From "Functional Programming Languages", Goldberg, Benjamin

This style of programming leads to applications that look very dissimilar to procedural or object-orientated code bases, and it can be arresting if you're not used to it as a style.

Let's take a look at an example of a program written in Clojure

user=> (if true
         (println "This is always printed")
         (println "This is never printed"))

and that same logic using C-like syntax

if (user) {
    println("This is always printed");
} else {
    println("This is never printed")
}

These two programs do exactly the same thing but the way that control flows through the programs are different. On smaller programs, this might not seem like a significant difference in style, but as a program grows in size, the way that control flow and state is managed between functional and procedural programming languages differs greatly, with larger functional programming styled programs looking more like large compound equations than programs that you "step through".

Consider this example - a port of some of the original "Practical LISP" samples into Clojure

This is a good, and well factored sample.

Any programming language that you don't know will feel alien at first glance, but consider the style of programming as you see it here - with functions composed as the unions of smaller, lower level functions.

With C-like languages (C, C++, Java, JavaScript, C#, Ruby, et al) being the dominant style of programming, if you've not seen functional code before, it at least appears different on a surface level.

Why do people do this?

Ok, so just because it's different, doesn't make it bad ??

Functional programming has got pretty popular because state management and mutability are hard, and cause bugs.

What does that mean?

Well, if your program operates on some variables that it keeps in memory, it's often quite difficult to keep track of which parts of your program are modifying your variables. As programs grow, this becomes and increasingly difficult and bug prone task.

People that love functional programming often cite the "immutability" as one of its biggest benefits - the simplest way of thinking about it, is that the functions that comprise of a functional program, can only modify data inside of them, and return new pieces of data. Any variables defined never escape their functions, and data passed to them is doesn't have its state changed.

There's a really nice benefit to this, in that each individual function is easy to test, and reason about in isolation, it really does help people build code with less bugs, and the bugs that are present often are contained only to single functions rather than sprawled across a codebase.

Functional programs often remove your need to think about the global state of your program as you're writing it, and that's a pleasant thing for a programmer.

Let's make everything functional!

So yes, some people reach that conclusion - and actually over the last decade or so, there's been a trend of functional syntactic constructs (that's a mouthful) being introduced into non-functional languages.

This probably started with C#'s Linq (Language Integrated Query - no, I don't understand quite how they reached that acronym either), and continued with functional constructs in Kotlin (probably strongly inspired by C#), Swift, the general introduction of Lambdas and "functions as data" into a lot of programming languages, and more recently, hugely popularised by React.js being an implementation of "functional reactive programming".

This subtle creep of functional style programming into more mainstream programming environments is interesting, because more and more people have been exposed to this style of programming without necessarily either studying it's nuance, or having to buy into a whole ecosystem switch.

It's lead to codebases with "bits of functional" in them, and in some of the more extreme cases, the idea of a "functional core" - where core programming logic is expressed in a functional style, with more traditional procedural code written to deal with data persistence, and bits at the edges of the application.

As more hybrid-functional code exists in the real world, there's an interesting antipattern that's emerged - functional code can be exceptionally difficult to comprehend and read.

But this isn't the functional utopia you promised!

Reading code is a distinctly different discipline to being able to write it.

In fact, reading code is more important than writing it, because you spend more of your time reading code, than you do writing it, and functional programming suffers a lot at the hands of "the head computing problem".

Because functional programming optimises for the legibility of reading a single function and understanding that all of its state is contained, what it's far worse at is helping you comprehend the state of the entire application. That's because to understand the state of the entire application outside of running the code, you have to mentally maintain the combined state of the functions as you read.

This isn't particularly different to code written in a more linear procedural style, where as you read code, you imagine the state as it's mutated, but with functional applications, you're also trying to keep track of the flow of control through the functions as they are combined - something that is presented linearly in a procedural language, as you follow the flow of control line-by-line.

Holding these two things in your mind at once involves more cognitive load than just following the control flow through a file in a procedural codebase, where you're just keeping track of some piece of state.

Proponents of functional programming would probably argue that this comprehension complexity is counter balanced by how easy it is to test individual functions in a functional application, as there's no complicated state to manage or build up, and you can use a REPL to trivially inspect the "true value".

Reading difficulty spikes however, are the lived experience of many people trying to read functional code, especially in apps-with-functional-parts.

What we really have is a context problem

This leads to an oft repeated trope that "functional is good for small things and not large applications" - you get the payoff of easy state free comprehension without the cognitive dissonance of trying to comprehend both the detail of all the functions, and their orchestration at once.

As functional codebases get bigger, the number of small functions required to compose a full application increases, as does the corresponding cognitive load. Combine this with the "only small parts" design philosophy of many functional languages, and if you're not careful about your application structure and file layout, you can end up with an application that is somehow at once not brittle, but difficult to reason about.

Compared to Object Oriented approaches - OO code is often utterly fixated on encapsulation, as that's kind of the point - and a strong focus on encapsulation pushes a codebase towards patterns of organisation and cohesion where related concepts are physically stored together, in the same directories and files, providing a kind of sight-readable abstraction that is often easy to follow.

OO codebases are forced into some fairly default patterns of organisation, where related concepts tend to be captured in classes named after their functionality.

But surely you can organise your code in functional languages?

Absolutely!

I suppose I just feel like OO code and its focus on encapsulation pushes code towards being organised by default. By comparison, FP code revels in its "small and autonomous" nature, perhaps a little to its detriment.

It often is trying to be not be everything OO code is, and actually a little bit of organisational hierarchy by theme and feature benefits all codebases in similar ways.

I absolutely believe that FP code could be organised in a way that gives it the trivial sight-readability of "class-ical" software, combined with the state free comprehension benefits of "small FP codebases", but it definitely isn't the default stylistic choice in much of the FP code I've seen in industry.

There's a maths-ish purism in functional code that I find both beautiful, and distressing, all at once. But I don't think it helps code comprehension.

I don't think people look at maths equations and succeed at "head computing" them without worked examples, and in the same way, I think programmers often struggle to "head compute" functional programs without a REPL or step debugger, because the combined effects of the functions is opaque, and what a computer is good at.

I absolutely also appreciate that some people love this exact same thing I find an un-expressive chore with my preference for languages with more expressive and less constrained syntax

We're really talking about what the best abstraction for reading is

There's a truism here - and it's that hierarchy and organisation of code is the weapon we have to combat cognitive load, and different programming styles need to take advantage of it in different ways to feel intuitive.

All abstractions have a cost and a benefit - be it file and directory hierarchy, function organisation and naming, or classical encapsulation. What fits and is excellent for one style of interaction with a codebase might not be the best way to approach it for another.

When chatting about this on twitter, I received this rather revealing comment -

"Whenever I've seen functional in the wild there is inevitably large chunks of logic embedded in 30-line statements that are impossible to grok. I think it is due to being such a natural way to write-as-you-think but without careful refactoring it is a cognitive nightmare"

Stephen Roughley (@SteBobRoughley)

This is many peoples truth about encountering functional code in the wild, and I don't think it has to be this way - it really shares a lot in common with all the "write once read never" jokes thrown at Regular Expressions - which are really just tiny compounded text matching functions.

Does functional programming make your code hard to read then?

Like everything in programming, "it depends".

Functional code is beautiful, and side effect free, but for it to be consumed by people that aren't embedded in it, it often needs to meet its audience half way with good approaches to namespacing, file organisation, or other techniques to provide context to the reader so they feel like they can parse it on sight, rather than having to head compute its flow control.

It's easy to get swept away in the hubris of programming style without considering the consumers of the code, and I think well organised functional codebases can be just as literate as a procedural program.

Functional parts of hybrid apps have a steeper challenge - to provide the reader with enough context during the switch from another programming style that they do not feel like they're lost in myopic and finely grained functions with no structure.

Beware cognitive load generators :)

JavaScript is everywhere, and TypeScript is JavaScript with some cool extra features.

You've probably heard of it, it's exceptionally popular, with lots of really mainstream JavaScript libraries and frameworks being built in TypeScript.

We're going to go through what a type is, why they're useful, and how you can use them without getting lost in configuration and tools.

First, let's understand what TypeScript is -

TypeScript extends JavaScript by adding types.

By understanding JavaScript, TypeScript saves you time catching errors and providing fixes before you run code.

Any browser, any OS, anywhere JavaScript runs. Entirely Open Source.

TypeScript is a programming language that is a superset of JavaScript - any valid JavaScript, is valid TypeScript - and it adds additional language features that get compiled down to vanilla JavaScript before it runs in your web browser. The most notable thing it adds to the language are types.

What are types?

The TypeScript pitch is pretty simple - "JavaScript with Types, to help prevent you making mistakes in your code" - but when you start to google around what Types are, you end up with things like the wikipedia page on computational type theory.

But we should translate this into simpler English - a Type let's you tell the computer that you expect data in a specific "shape", so that it can warn you if you try to use data that isn't in the correct format.

For example, this is an interface:

inteface Animal {
    numberOfLegs: number,
    numberOfEyes: number
}

This interface is a Type definition - that says:

• Animals have two properties.
• numberOfLegs, which is a number
• numberOfEyes, which is a number

In TypeScript you can just put an interface like that in your .ts files.

A .ts file? Well that is identical to a regular JavaScript .js file - that also has TypeScript code in it.

When we create a JavaScript object that contains the properties or functions that we've declared in our interface, we can say that our object implements that interface. Sometimes you'll see people say the "object conforms to that interface".

In practice, this means that if you create an object, for it to be an Animal and be used in your code in places that require an animal, it must at least have those two properties.

// Just some object

const notAnAnimal = {
    blah: "not an animal"
};

// Cats are animals

const cat = {
    numberOfLegs: 4,
    numberOfEyes: 2
};

// You can even tell TypeScript that your variable
// is meant to be an animal with a Type Annotation.

const cat2: Animal = {
    numberOfLegs: 4,
    numberOfEyes: 2
};

We'll work some examples later on, but I'd rather look at what TypeScript can do for you.

Let's start by working out how we're going to run our TypeScript code in our browser.

Running TypeScript in our browser with snowpack

Snowpack is a frontend development server - it does similar things to CreateReactApp if you're familiar with React development. It gives you a webserver that reloads when you change your files.

It's built to help you write your webapps using ES Modules - that's where you can use import statements in your frontend code, and the browser does the work of loading JavaScript files from your server and making sure that requests don't get duplicated.

It also natively, and transparently supports TypeScript - this means you can add TypeScript files (with the extension .ts) and load them as if they're just plain old JavaScript. This means if you have all your code in a file called index.ts, you can reference it from a HTML file as index.js and it'll just work without you doing anything at all.

Setting up snowpack

snowpack is available on NPM, so the quickest way we can create a project that uses snowpack is to npm init in a new directory.

First, open your terminal and type

npm init

Just hit enter a few times to create the default new node project. Once you have a package.json, we're going to install our dependencies

npm install snowpack typescript --save-dev

That's it!

Snowpack just works out of the current directory if you've not configured anything.

We can just go ahead and create HTML, JavaScript or TypeScript files in this directory and it'll "just work". You can run snowpack now by just typing

npx snowpack dev

ES Modules, the simplest example

Let's take a look at the simplest possible example of a web app that uses ES Modules

If we were to have a file called index.html with the following contents

<!DOCTYPE html>
<html lang="en">

<head>
    <title>Introduction to TypeScript</title>
    <script src="/index.js" type="module"></script>
</head>

<body>
    Hello world.
</body>

</html>

You'll notice that where we're importing our script, we're also using the attribute type="module" - telling our browser that this file contains an ES Module.

Then an index.js file that looks like this

console.log("Oh hai! My JavaScript file has loaded in the browser!");

You would see the console output from the index.js file when the page loaded.

Oh hai! My JavaScript file has loaded in the browser!

You could build on this by adding another file other.js

console.log("The other file!");

and replace our index.js with

import "./other";

console.log("Oh hai! My JavaScript file has loaded in the browser!");

Our output will now read:

The other file!
Oh hai! My JavaScript file has loaded in the browser!

This is because the import statement was interpreted by the browser, which went and downloaded ./other.js and executed it before the code in index.js.

You can use import statements to import named exports from other files, or, like in this example, just entire other script files. Your browser makes sure to only download the imports once, even if you import the same thing in multiple places.

ES Modules are really simple, and perform a lot of the jobs that people were traditionally forced to use bundlers like webpack to achieve. They're deferred by default, and perform really well.

Using TypeScript with snowpack

If you've used TypeScript before, you might have had to use the compiler tsc or webpack to compile and bundle your application.

You need to do this, because for your browser to run TypeScript code, it has to first be compiled to JavaScript - this means the compiler, which is called tsc will convert each of your .ts files into a .js file.

Snowpack takes care of this compilation for you, transparently. This means that if we rename our index.js file to index.ts (changing nothing in our HTML), everything still just works.

This is excellent, because we can now use TypeScript code in our webapp, without really having to think about any tedious setup instructions.

What can TypeScript do for you right now?

TypeScript adds a lot of features to JavaScript, but let's take a look at a few of the things you'll probably end up using the most, and the soonest. The things that are immediately useful for you without having to learn all of the additions to the language.

TypeScript can:

• Stop you calling functions with the wrong variables
• Make sure the shape of JavaScript objects are correct
• Restrict what you can call a function with as an argument
• Tell you what types your functions returns to help you change your code more easily.

Let's go through some examples of each of those.

Use Type Annotations to never call a function with the wrong variable again

Look at this addition function:

function addTwoNumbers(one, two) {
    const result = one + two;
    console.log("Result is", result);
}

addTwoNumbers(1, 1);

If you put that code in your index.ts file, it'll print the number 2 into your console.

We can give it the wrong type of data, and have some weird stuff happen - what happens if we pass a string and a number?

addTwoNumbers("1", 1);

The output will now read 11 which isn't really what anyone was trying to do with this code.

Using TypeScript Type Annotations we can stop this from happening:

function addTwoNumbers(one: number, two: number) {
    const result = one + two;
    console.log("Result is", result);
}

If you pay close attention to the function parameters, we've added : number after each of our parameters. This tells TypeScript that this function is intended to only be called with numbers.

If you try and call the function with the wrong Type or paramter - a string rather than a number:

addTwoNumbers("1", 1); // Editor will show an error here.

Your Visual Studio Code editor will underline the "1" argument, letting you know that you've called the function with the wrong type of value - you gave it a string not a number.

This is probably the first thing you'll be able to helpfully use in TypeScript that'll stop you making mistakes.

Using Type Annotations with more complicated objects

We can use Type annotations with more complicated types too!

Take a look at this function that combines two coordinates (just an object with an x and a y property).

function combineCoordinates(first, second) {
    return {
        x: first.x + second.x,
        y: first.y + second.y
    }
}

const c1 = { x: 1, y: 1 };
const c2 = { x: 1, y: 1 };

const result = combineCoordinates(c1, c2);

Simple enough - we're just adding the x and y properties of two objects together. Without Type annotations we could pass objects that are completely the wrong shape and crash our program.

combineCoordinates("blah", "blah2"); // Would crash during execution

JavaScript is weakly typed (you can put any type of data into any variable), so would run this code just fine, until it crashes trying to access the properties x and y of our two strings.

We can fix this in TypeScript by using an interface. We can decalre an interface in our code like this:

interface Coordinate {
    x: number,
    y: number
}

We're just saying "anything that is a coordinate has an x, which is a number, and a y, which is also a number" with this interface definition. Interfaces can be described as type definitions, and TypeScript has a little bit of magic where it can infer if any object fits the shape of an interface.

This means that if we change our combineCoordinates function to add some Type annotations we can do this:

interface Coordinate {
    x: number,
    y: number
}

function combineCoordinates(first: Coordinate, second: Coordinate) {
    return {
        x: first.x + second.x,
        y: first.y + second.y
    }
}

And your editor and the TypeScript compiler will throw an error if we attempt to call that function with an object that doesn't fit the shape of the interface Coordinate.

The cool thing about this type inference is that you don't have to tell the compiler that your objects are the right shape, if they are, it'll just work it out. So this is perfectly valid:

const c1 = { x: 1, y: 1 };
const c2 = { x: 1, y: 1 };

combineCoordinates(c1, c2);

But this

const c1 = { x: 1, y: 1 };
const c2 = { x: 1, bar: 1 };

combineCoordinates(c1, c2); // Squiggly line under c2

Will get a squiggly underline in your editor because the property y is missing in our variable c2, and we replaced it with bar.

This is awesome, because it stops a huge number of mistakes while you're programming and makes sure that the right kind of objects get passed between your functions.

Using Union Types to restrict what you can call a function with

Another of the really simple things you can do in TypeScript is define union types - this lets you say "I only want to be called with one of these things".

Take a look at this:

type CompassDirections = "NORTH" | "SOUTH" | "EAST" | "WEST";

function printCompassDirection(direction) {
    console.log(direction);
}

printCompassDirection("NORTH");

By defining a union type using the type keyword, we're saying that a CompassDirection can only be one of NORTH, SOUTH, EAST, WEST. This means if you try call that function with any other string, it'll error in your editor and the compiler.

Adding return types to your functions to help with autocomplete and intellisense

IntelliSense and Autocomplete are probably the best thing ever for programmer productivity - often replacing the need to go look at the docs. Both VSCode and WebStorm/IntelliJ will use the type definitions in your code to tell you what parameters you need to pass to things, right in your editor when you're typing.

You can help the editors out by making sure you add return types to your functions.

This is super easy - lets add one to our combineCoordinates function from earlier.

function combineCoordinates(first: Coordinate, second: Coordinate) : Coordinate {
    return {
        x: first.x + second.x,
        y: first.y + second.y
    }
}

Notice at the end of the function definition we've added : Coordinate - this tells your tooling that the function returns a Coordinate, so that if at some point in the future you're trying to assign the return value of this function to the wrong type, you'll get an error.

Your editors will use these type annotations to provide more accurate hints and refactoring support.

Why would I do any of this? It seems like extra work?

It is extra work! That's the funny thing.

TypeScript is more verbose than JavaScript and you have to type extra code to add Types to your codebase. As your code grows past a couple of hundred lines though, you'll find that errors where you are providing the wrong kind of data to your functions or verifying that API calls return data that is in the right shape dramatically reduce.

Changing code becomes easier, as you don't need to remember every place you use a certain shape of object, your editor will do that work for you, and you'll find bugs sooner, again, with your editor telling you that you're using the wrong type of data before your application crashes in the browser.

Why is everyone so excited about types?

People get so excited, and sometimes a little bit militant about types, because they're a great tool for removing entire categories of errors from your software. JavaScript has always had types, but it's a weakly typed language.

This means I can create a variable as a string

let variable = "blah";

and later overwrite that value with a number

variable = 123;

and it's a perfectly valid operation because the types are all evaluated while the program is running - so as long as the data in a variable is in the "correct shape" of the correct type - when your program comes to use it, then it's fine.

Sadly, this flexibility frequently causes errors, where mistakes are made during coding that become increasingly hard to debug as your software grows.

Adding additional type information to your programs reduces the likelihood of errors you don't understand cropping up at runtime, and the sooner you catch an error, the better.

Just the beginning

This is just the tip of the iceberg, but hopefully a little less intimidating than trying to read all the docs if you've never used TypeScript before, without any scary setup or configuration.

In 2020, I’m finding myself writing TypeScript as much as I’m using C# for my day-to-day dev. I’ve found myself experimenting, building multiplayer browser-based games, small self-contained PWAs and other “mostly browser-based things” over the last year or so.

One of the most frustrating things you have to just sort of accept when you’re in the browser, or running in node, is the frequently entirely incoherent and flaky world of node and JavaScript toolchains.

Without wanting to labour the point too much, many of the tools in the JavaScript ecosystem just don’t work very well, are poorly maintained, or poorly documented, and even some of the absolutely most popular tools like WebPack and Babel that sit underneath almost everything rely on mystery meat configuration, and fairly opaque error messages.

There is a reason that time and time again I run into frontend teams that hardly know how their software is built. I’ve spent the last year working on continual iterations of “what productive really looks like” in a TypeScript-first development environment, and fighting that healthy tension between tools that want to offer plenty of control, but die by the hands of their own configuration, to tools that wish to be you entire development stack (Create React App, and friends).

What do I want from a frontend development stack?

In all software design, I love tools that are correct by default and ideally require zero configuration.

I expect hot-reload, it’s the fast feedback cycle of the web and to accept the inconsistencies of browser-based development without the benefit it’s a foolish thing.

I want native TypeScript compilation that I don’t have to think about. I don’t want to configure it, I want it to just work for v.current of the evergreen browsers.

I want source maps and debugger support by default.

I want the tool to be able handle native ES Modules, and be able to consume dependencies from npm.

Because I’ve been putting a lot of time into hosting websites as Azure Static Web Apps, I also want whatever tool I use to play nicely in that environment, and be trivially deployable from a GitHub Action to Azure Static Web Apps.

Enter Snowpack

Snowpack is a modern, lightweight toolchain for faster web development. Traditional JavaScript build tools like webpack and Parcel need to rebuild & rebundle entire chunks of your application every time you save a single file. This rebundling step introduces lag between hitting save on your changes and seeing them reflected in the browser.

I was introduced to snowpack by one of it’s contributors, an old friend, when complaining about the state of “tools that don’t just work” in the JavaScript ecosystem as a tool that was trying to pretty much do all the things I was looking for, so I’ve decided to use it for a couple of things to see if it fits the kind of projects I’ve been working on.

And honestly, it pretty much just works perfectly.

Setting up snowpack to work with Azure Static Web Apps

Last month I wrote about how Azure Static Web Apps are Awesome with a walkthrough of setting up a static web app for any old HTML site, and I want to build on that today to show you how to configure a new project with snowpack that deploys cleanly, and uses TypeScript.

Create a package.json

First, like in all JavaScript projects, we’re going to start by creating a package.json file.

You can do this on the command line by typing

npm init

We’re then going to add a handful of dependencies:

npm install npm-run-all snowpack typescript --save-dev

Which should leave us with a package.json that looks a little bit like this

{
    "name": "static-app",
    "version": "",
    "description": "",
    "repository": "http://tempuri.org",
    "license": "http://tempuri.org",
    "author": "",
    "dependencies": {},
    "devDependencies": {
        "npm-run-all": "^4.1.5",
        "snowpack": "^2.9.0",
        "typescript": "^4.0.2"
    }
}

Add some build tasks

Now, we’ll open up our package.json file and add a couple of tasks to it:

{
    ...
    "scripts": {
        "start": "run-p dev:api dev:server",
        "dev:api": "npm run start --prefix api",
        "dev:server": "npx snowpack dev",
        "build:azure": "npx snowpack build"
    },
    ...
}

What we’re doing here, is filling in the default node start task – using a module called npm-run-all that allows us to execute two tasks at once. We’re also defining a task to run an Azure Functions API, and the snowpack dev server.

Create our web application

Next, we’re going to create a directory called app and add an app/index.html file to it.

<html>
<head>
    <title>Hello Snowpack TypeScript</title>
    <script src="/index.js" type="module"></script>
</head>

<body>
    Hello world.
</body>
</html>

And we’ll create a TypeScript file called app/index.ts

class Greeter {
    private _hasGreeted: boolean;

    constructor() {
        this._hasGreeted = false;
    }

    public sayHello(): void {
        console.log("Hello World");
        this._hasGreeted = true;
    }
}

const greeter = new Greeter();
greeter.sayHello();

You’ll notice we’re using TypeScript type annotations (Boolean, and : void in this code, along with public access modifiers).

Configuring Snowpack to look in our APP directory

Next, we’re going to add a snowpack configuration file to the root of our repository. We’re adding this because by default, snowpack works from the root of your repository, and we’re putting our app in /app to help Azure Static Web Apps correctly host our app later.

Create a file called snowpack.config.json that looks like this:

{
    "mount": {
        "app": "/"
    },
    "proxy": {
        "/api": "http://127.0.0.1:7071/api"
    }
}

Here we’re telling snowpack to mount our content from “app” to “/”, and to reverse proxy “/api” to a running Azure Functions API. We’ll come back to that, but first, let’s test what we have.

npm run dev:server

Will open a browser, and both in the console and on the screen, you should see “Hello World”.

Snowpack has silently Transpiled your TypeScript code, into a JavaScript file with the same filename, that your webapp is referencing using ES Module syntax.

The cool thing here, is everything you would expect to work in your frontend now does. You can use TypeScript, you can reference npm modules in your frontend code and all this happens with next to no startup time.

You can extend this process using various snowpack plugins, and it probably supports the JavaScript tooling you’re already using natively – read more at snowpack.dev

Create our Azure Functions API

Because Azure Static Web Apps understand Azure functions, you can add some serverless APIs into a subdirectory called api in your repository, and Azure Oryx will detect and auto-host and scale them for you as part of it’s automated deployment.

Make sure you have the Azure Functions Core Tools installed by running

npm install -g azure-functions-core-tools\@3

Now we’re going to run a few commands to create an Azure functions app.

mkdir api  
cd api  
func init --worker-runtime=node --language=javascript

This generates a default javascript+node functions app in our API directory, we just need to create a function for our web app to call. Back in the command line, we’ll type (still in our /api directory)

func new --template "Http Trigger" --name HelloWorld

This will add a new function called HelloWorld into your API directory.

In the file api/package.json make sure the following two tasks are present:

  "scripts": {
    "prestart": "func extensions install",
    "start": "func start"
  },

If we now return to the root of our repository and type

npm run start

A whole lot of text will scroll past your console, and snowpacks live dev server will start up, along with the Azure Functions app with our new “HelloWorld” function in it.

Let’s add a little bit of code to our app/index.html to call this

The cool thing, is we can just do this with the app running, and both the functions runtime, and the snowpack server, will watch for and hot-reload changes we make.

Calling our API

We’re just going to add some code to app/index.ts to call our function, borrowed from the previous blog post. Underneath our greeter code, we’re going to add a fetch call

…
const greeter = new Greeter();
greeter.sayHello();

fetch("/api/HelloWorld")
    .then(response => response.text())
    .then(data => console.log(data));

Now if you look in your browser console, you’ll notice that the line of text

“This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.”

Is printed to the window. That’s the text returned from our “HelloWorld” API.

And that’s kind of it!

Really, that is it – you’ve now got a TypeScript compatible, hot-reloading dev server, with a serverless API, that is buttery smooth to develop on top of. But for our final trick, we’re going to configure Azure Static Web Apps to host our application.

Configuring Static Web Apps

First, go skim down the guide to setting up Azure Static Web Apps I put together here - https://www.davidwhitney.co.uk/Blog/2020/07/29/azure_static_web_apps_are_awesome

You’re going to need to push your repository to GitHub, go and signup / login to the Azure Portal, and navigate to Azure Static Web Apps and click Create.

Once you’re in the creation process, you’ll need to again, authenticate with GitHub and select your new repository from the drop-downs provided.

You’ll be prompted to select the kind of Static Web App you’re deploying, and you should select Custom. You’ll then be faced with the Build Details settings, where you need to make sure you fill in the following:

App Location: /
API location: api
App artifact location: build

Remember at the very start when we configured some npm tasks in our root? Well the Oryx build service is going to be looking for the task build:azure in your scripts configuration.

We populated that build task with “npx snowpack build” – a built in snowpack task that will compile and produce a build folder with your application in it ready to be hosted.

This configuration let’s Azure know that our final files will be available in the generated build directory so it knows what to host.

When you complete this creation flow, Azure will commit a GitHub action to your repository, and trigger a build to deploy your website. It takes around 2 minutes the first time you set this up.

That’s it.

I’ve been using snowpack for a couple of weeks now, and I’ve had a wonderful time with it letting me build rich frontends with TypeScript, using NPM packages, without really worrying about building, bundling, or deploying.

These are the kind of tools that we should spend time investing in, that remove the nuance of low-level control, and replace it with pure productivity.

Give Azure Static Sites with Snowpack a go for your next project.

I woke today with an interesting question in my inbox, about the effectiveness of remote work and communication, especially during the global pandemic:

Diego Alto: Here's the statement

"Working remotely has an added benefit (not limited to software) of forcing documentation (not only software!) which helps transfer of knowledge within organizations. (I am not just talking about software).

Example: a decision made over a chat in the kitchen making coffee is inaccessible to everyone else who wasn't in that chat. But a short message in slack, literally a thumbs up in a channel everyone sees, makes it so.

Discuss"

Let's think about what communication really means in the context of remote work, especially while we're all forced to live through this as our day-to-day reality.

Tools are not discipline

High quality remote work requires clear, concise, and timely communication, but I'm not sure that it causes or facilitates it.

I think there's a problem here that comes from people learning to use tools rather than the disciplines involved in effective communication - which was entirely what the original agile movement was about.

DHH talks about this a lot - about how they hire writers at Basecamp, their distributed model works for them because they have a slant towards hiring people that use written communication well.

I find this fascinating, but I also have a slightly sad realisation that people are unbelievably bad at written communication, and time is rarely invested in making them better at it.

This means that most written communication in businesses is waste and ineffective and may as well not have happened. Most of the audit trails it creates either get lost or are weaponised - so it becomes a case of "your mileage may vary".

Distinct types of written communication are differently effective though, and this is something we should consider.

Different forms of communication for different things

Slack is an "ok form of communication" but it's harmed by being temporal - information drifts over time or is repeated.

But it's better than email! I hear you cry.

Absolutely, but it's better because it's open and searchable. Email is temporal AND closed. Slack is an improvement because it's searchable (to a point) and most importantly a broadcast medium of communication.

It's no surprise that slack and live chat has seen such a rise in popularity - it's just the 2010's version of the reply-all email that drove all of business through the late 90s and early 2000s.

Slack is basically just the millennial version of reply-all email chains.

Both forms of communication are worse than structured and minimal documentation in a known location though.

"Just enough" documentation - a record of decisions, impacts, and the why, is far more effective that sifting through any long-form communication to extract details you might just miss.

Co-location of information

I'm seeing a rise in Architecture Decision Records (ADRs) and tooling inside code repositories to support and maintain them for keeping track of decisions.

An architectural decision record (ADR) is a document that captures an important architectural decision made along with its context and consequences.

There's a tension between the corporate wiki, which is just rotten and useless, and "just read the code bro".

"Just read the code" is interesting, as it's a myth I've perpetuated in the past. It's an absolute fact that the code is the only true and honest source of understanding code, but that doesn't mean "don't give me a little bit of narrative here to help me".

Just enough

I don't want you to comment the code, I want you to tell me its story. I want you to tell me just enough.

I've done a bunch of stuff with clients about what "just enough" documentation looks like, and almost every time, "just enough" is always "co-located with the thing that it describes".

Just enough means, please tell me -

• What this thing is
• Why it exists
• What it does
• How to build this thing
• How to run this thing
• Exactly the thing I'll get wrong when I first try and use it

Any software or libraries that don't ship with this information, rot away, don't get adopted, and don't get used.

I'm glad the trend has taken hold. ADRs really fit into this "just enough" minimal pattern, with the added benefit of growing with the software.

The nice thing about ADRs, is they are more of a running log than a spec - specs change, they're wrong, they get adapted during work. ADRs are meant to be the answer to "why is this thing why it is".

Think of them as the natural successor to the narrative code comment. The spiritual sequel to the "gods, you gotta know about this" comment.

Nothing beats a well-formed readme, with its intent, and a log of key decisions, and we know what good looks like.

Has remote work, and a global pandemic, helped this at all?

On reflection, I'm not sure "better communication" is a distributed work benefit - but it's certainly a distributed work requirement.

I have a sneaking suspicion that all the global pandemic really proved was that "authoritarian companies that didn't support home work was nothing more than control freakery by middle management".

There's nothing in the very bizarre place the world finds itself that implicitly will improve communication, but we're now forced into a situation where everyone needs the quality levels of communication that was previously only required by remote workers.

Who needs to know how things work anyway?

People that can't read code have no concern knowing how it works, but every concern in understanding why it works and what it does.

Why. What. Not how.

Understanding the how takes a lot of context.

There's a suitable abstraction of information that must be present when explaining how software works and it's the job of the development teams to make this clear.

We have to be very aware of the "just enough information to be harmful" problem - and the reason ADRs work well, in my opinion, is they live in the repository, with the code, side by side with the commits.

This provides a minimum bar to entry to read and interpret them -and it sets the context for the reader that understanding the what is a technical act.

This subtly barrier to entry is a kind of abstraction, and hopefully one that prevents misinterpretation of the information.

Most communication is time-sensitive

There's a truth here, and a reason - the reason slack and conversations are so successful at transmitting information; Most information is temporal in nature.

Often only the effects communication needs to last, and at that point, real-time-chat being limited by time isn't a huge problem.

In fact, over communication presents a navigation and a maintenance burden - with readers often left wondering what the most-correct version of information is, or where the most current information resides, while multiple-copies of it naturally atrophies over time.

We've all seen that rancid useless corporate wiki, but remember it was born from good intentions of communication before it fell into disrepair.

All code is writing

So, remember that all code is writing.

It's closer to literature than anything. And we don't work on writing enough.

Writing, in various styles, with abstractions that should provide a benefit.

And that extends to the narrative, where it lives, and how it's updated.

But I believe it's always been this way, and "remote/remote first" rather than improving this process (though it can be a catalyst to do so), pries open the cracks when it's sub-par.

This is the difficulty of remote. Of distributed team management.

It's ironically, what the agile manifestos focus on co-location was designed to resolve.

Conflict is easier to manage in person than in writing.

You're cutting out entire portions of human interaction around body language, quickly addressing nuance or disagreement, by focusing on purely written communication. It's easier to diffuse and subtly negotiate in person.

The inverse is also true, thoughts are often more fully formed in prose. This can be both for better or worse, with people often more reticent to change course once they perceive they have put significant effort into writing a thing down.

There is no one way

Everything in context, there is no one way.

The biggest challenge in software, or teamwork in general, is replacing your mental model with the coherent working pattern of a team.

It's about realising it's not about you.

It's easy in our current remote-existence to think that "more communication" is the better default, and while that might be the correct place to start, it's important that the quality of the communication is the thing you focus on improving in your organisations.

Make sure your communication is more timely, more contextual, more succinct, and closer in proximity to the things it describes.

Over the last 3 months or so, I’ve been building a lot of experimental software on the web. Silly things, fun things. And throughout, I’ve wrangled with different ways to host modern web content.

I’ve been through the ringer of hosting things on Glitch for its interactivity, Heroku to get a Node backend, even Azure App Services to run my node processes.

But each time it felt like effort, and cost, to put a small thing on the internet.

Everything was somehow a compromise in either effort, complexity, or functionality.

So when Microsoft put out the beta of static web apps a couple months ago, I was pretty keen to try them out.

They’re still in beta, the docs are a little light, the paint is dripping wet, but they’re a really great way to build web applications in 2020, and cost next to nothing to run (actually, they're free during this beta).

I want to talk you through why they’re awesome, how to set them up, and how to customise them for different programming languages, along with touching on how to deal with local development and debugging.

We need to talk about serverless

It is an oft-repeated joke – that the cloud is just other people’s computers, and serverless, to extend the analogy, is just someone else’s application server.

While there is some truth to this – underneath the cloud vendors, somewhere, is a “computer” – it certainly doesn’t look even remotely like you think it does.

When did you last dunk a desktop computer looking like this under the sea?

While the cloud is “someone else’s computer”, and serverless is “someone else’s server” – it’s also someone else’s hardware abstraction, and management team, and SLA to satisfy, operated by someone else’s specialist – and both the cloud, and serverless, make your life a lot easier by making computers, and servers, somebody else’s problem.

In 2020, with platforms like Netlify and Vercel taking the PaaS abstraction and iterating products on top of it, it’s great to see Microsoft, who for years have had a great PaaS offering in Azure, start to aim their sights at an easy to use offering for “the average web dev”.

Once you remove the stupid sounding JAMSTACK acronym, shipping HTML and JavaScript web apps that rely on APIs for interactivity, it’s a really common scenario, and the more people building low-friction tools in this space, the better.

Let’s start by looking at how Azure Static Web Apps work in a regular “jamstack-ey” way, and then we’ll see how they’re a little bit more magic.

What exactly are Azure Static Web Apps?

Azure Static Web Apps is currently-beta new hosting option in the Azure-WebApps family of products.

They’re an easy way to quickly host some static files – HTML and JavaScript – on a URL and have all the scaling and content distribution taken care of for you.

They work by connecting a repository in GitHub to the Azure portal’s “Static Web Apps” product, and the portal will configure your repository for continuous delivery. It’s a good end-to-end experience, so let’s walk through what that looks like.

Creating your first Static Web App

We’re going to start off by creating a new repository on GitHub -

And add an index.html file to it…

Great, your first static site, isn’t it grand. That HTML file in the root is our entire user experience.

Perfect. I love it.

We now need to hop across to the Azure portal and add our new repository as a static site.

The cool thing about this process, is that the Azure portal will configure GitHub actions in our repository, and add security keys, to configure our deployment for us.

We’re just giving the new site a resource group (or creating one if you haven’t used Azure before - a resource group is just a label for a bunch of stuff in Azure) and selecting our GitHub repository.

Once we hit Review + Create, we’ll see our final configuration.

And we can go ahead and create our app.

Once the creation process has completed (confusingly messaged as “The deployment is complete”) – you can click the “Go to resource” button to see your new static web app.

And you’re online!

I legitimately think this is probably the easiest way to get any HTML onto the internet today.

Presuming you manage to defeat the Microsoft Active Directory Boss Monster to login to Azure in the first place ;)

What did that do?

If we refresh our GitHub page now, you’ll see that the Azure Create process, when you gave it permission to commit to your repositories, used them.

When you created your static web app in the Azure portal, it did two things:

1. Created a build script that it committed to your repository
2. Added a deployment secret to your repository settings

The build script that gets generated is relatively lengthy, but you’re not going to have to touch it yourself.

It configures GitHub actions to build and push your code every time you commit to your master branch, and to create special preview environments when you open pull requests.

This build script is modified each time to reference the deployment secret that is generated by the Azure portal.

You will notice the secret key lines up in your repository.

Is this just web hosting? What makes this so special?

So far, this is simple, but also entirely unexciting – what makes Azure Static Web Apps so special though, is their seamless integration with Azure Functions.

Traditionally if you wanted to add some interactivity to a static web application, you’d have to stand up an API somewhere – Static Web Apps pulls these two things together, and allows you define both an Azure Static Web App, and some Azure functions that it’ll call, in the same repository.

This is really cool, because, you still don’t have a server! But you can run server-side code!

It is especially excellent because this server-side code that your application depends on, is versioned and deployed with the code that depends on it.

Let’s add an API to our static app!

Adding an API

By default, the configuration that was generated for your application expects to find an Azure Functions app in the /api directory, so we’re going to use npm and the Azure functions SDK to create one.

At the time of writing, the Functions runtime only supports up to Node 12 (the latest LTS version of node) and is updated tracking that version.

You’re going to need node installed, and in your path, for the next part of this tutorial to work.

First, let’s check out our repository

Make sure you have the Azure Functions Core Tools installed by running

npm install -g azure-functions-core-tools@3

Now we’re going to run a few commands to create an Azure functions app.

mkdir api
cd api
func init --worker-runtime=node --language=javascript

This generates a default javascript+node functions app in our API directory, we just need to create a function for our web app to call. Back in the command line, we’ll type (still in our /api directory)

func new --template "Http Trigger" --name HelloWorld

This will add a new function called HelloWorld into your API directory

These are the bindings that tell the Azure functions runtime what to do with your code. The SDK will generate some code that actually runs…

Let’s edit our HTML to call this function.

We’re using the browsers Fetch API to call “/api/HelloWorld” – Azure Static Web Apps will make our functions available following that pattern.

Let’s push these changes to git, and wait a minute or two for our deployment to run.

If we now load our webpage, we’ll see this:

How awesome is that – a server-side API, without a server, from a few static files in a directory.

If you open up the Azure portal again, and select Functions, you’ll see your HelloWorld function now shows up:

I love it, but can I run it locally?

But of course!

Microsoft recommends using the npm package live-server to run the static portion of your app for development, which you can do just by typing

npx live-server

From the root of your repository. Let’s give that a go now

Oh no! What’s going on here.

Well, live-server is treating the /api directory as if it were content, and serving an index page locally, which isn’t what we want. To make this run like we would on production, we’re also going to need to run the azure functions runtime, and tell live-server to proxy any calls to /api across to that running instance.

Sounds like a mouthful, but let’s give that a go.

cd api
npm i
func start

This will run the Azure functions runtime locally. You will see something like this

Now, in another console tab, let’s start up live-server again, this time telling it to proxy calls to /api

npx live-server --proxy=/api:http://127.0.0.1:7071/api

If we visit our localhost on 8080 now, you can see we have exactly the same behaviour as we do in Azure.

Great, but this all seems a little bit… fiddly… for local development.

If you open your root directory in Visual Studio Code, it will hint that it has browser extension support for debugging and development, but I like to capture this stuff inside my repository really so anyone can run these sites from the command line trivially.

Adding some useful scripts

I don’t know about you, but I’m constantly forgetting things, so let’s capture some of this stuff in some npm scripts so I don’t have to remember them again.

In our /api/package.json we’re going to add two useful npm tasks

This just means we can call npm run start on that directory to have our functions runtime startup.

Next we’re going to add a package.json to the root of our repository, so we can capture all our live server related commands in one place.

From a command prompt type:

npm init

and hit enter a few times past the default options – you’ll end up with something looking like this

And finally, add the npm-run-parallel package

npm install npm-run-all –save-dev

We’re going to chuck a few more scripts in this default package.json

Here we’re setting up a dev:api, dev:server and a start task to automate the command line work we had to incant above.

So now, for local development we can just type

npm run start

And our environment works exactly how it would on Azure, without us having to remember all that stuff, and we can see our changes hot-reloaded while we work.

Let’s commit it and make sure it all still works on Azure!

Oh No! Build Failure!

Ok, so I guess here is where our paint is dripping a little bit wet.

Adding that root package.json to make our life easier, actually broke something in our GitHub Actions deployment pipeline.

If we dig around in the logs, we’ll see that something called “Oryx” can’t find a build script, and doesn’t know what to do with itself

As it turns out, the cleverness that’s baked into Azure static web apps, is a tool called Oryx, and it’s expecting frameworks it understands, and is running some language detection.

What’s happened is that it’s found our package.json, presumed we’re going to be specifying our own build jobs, and we’re not just a static site anymore, but then when we didn’t provide a build task, it’s given up because it doesn’t know what to do.

The easiest way I’ve found to be able to use node tooling, and still play nicely with Azure’s automated deployment engine is to do two things:

1. Move our static assets into an “app” directory
2. Update our deployment scripts to reflect this.

First, let’s create an app directory, and move our index.html file into it.

Now we need to edit the YAML file that Azure generated in .github/workflows

This might sound scary, but we’re only really changing one thing – in the jobs section, on line ~30 of the currently generated sample there are three configuration settings –

We just need to update app_location to be “app”.

Finally, we need to update the npm scripts we added to make sure live-server serves our app from the right location.

In our root package.json, we need to add “app” to our dev:server build task

We’re also going to add a task called build:azure – and leave it empty.

In total, we’ve only changed a few files subtly.

You might want to run your npm run start task again now to make sure everything still works (it should!) and commit your code and push it to GitHub.

Wonderful.

Everything is working again.

“But David! You’re the TDD guy right? How do you test this!”

Here’s the really cool bit I suppose – now we’ve configured a build task, and know where we can configure an app_artifact_location – we can pretty much do anything we want.

• Want to use jest? Absolutely works!
• Want to use something awesome like Wallaby? That too!

Why not both at once!

You just need to npm install the thing you want, and you can absolutely test the JavaScript in both your static site and your API.

You can install webpack and produce different bundled output, use svelte, anything, and Microsoft’s tooling will make sure to host and scale both your API and your web app.

My standard “dev” load-out for working with static web sites is

1. Add a few dev dependencies

   

2. Add this default babel.config.js file to the root of my repository

   

This allows jest to use any language features that my current version of node supports, and plays nicely with all my Visual Studio Code plugins.

I’ll also use this default Wallaby.conf.js configuration *for the continuous test runner Wallaby.js – which is similar to NCrunch but for JavaScript and TypeScript codebases.

You mentioned TypeScript?

Ah yes, well, Azure Functions runtime totally supports TypeScript.

When you create your API, you just need to

func init --worker-runtime=node --language=typescript

And the API that is generated will be TypeScript – it’s really that simple.

Equally, you can configure TypeScript for your regular static web app, you’ll probably want to configure WebPack to do the compiling and bundling into the assets folder, but it works absolutely fine.

When your functions are created using TypeScript, some extra .json metadata is created alongside each function that points to a compiled “dist” directory, that is built when the Azure functions runtime deploys your code, complete with source-maps, out of the box.

But let’s go wild, how about C# !

You can totally use C# and .NET Core too!

If you func init using the worker dotnet, the SDK will generate C# function code that works in exactly the same way as it’s JavaScript and TypeScript equivalent.

You can literally run a static web app, with an auto-scaled C# .NET Core API backing it.

Anything that the Azure Functions runtime supports is valid here (so python too).

I Think This is Really Awesome

I hope by splitting this out into tiny steps, and explaining how the GitHub Actions build, interacts with both the Functions runtime and the Oryx deployment engine that drives Azure Static Web Apps has given you some inspiration for the kinds of trivially scalable web applications you can build today, for practically free.

If you’re a C# shop, a little out of your comfort zone away from ASP.NET MVC, why not use Statiq.Web as part of the build process to generate static WebApps, that use C#, and are driven by a C# and .NET Core API?

Only familiar with Python? You can use Pelikon or Lector to do the same thing.

The Oryx build process that sits behind this is flexible, and provides plenty of hooks to customise the build behaviour between repository pulling, and your site getting served and scaled.

These powerful serverless abstractions let us do a lot more with a lot less, without the stress of worrying about outages, downtime, or scaling.

You can really get from zero to working in Azure static sites in five or ten minutes, and I legitimately think this is one of the best ways to host content on the internet today.

Whenever I'm looking to put out a new library I find myself configuring everything in repetitively simple ways. The default File -> New Project templates that Visual Studio ships with never quite get it right for my default library topology.

Almost every single thing I build looks like this:

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----        04/05/2020   1:16 PM                .github               # GitHub actions build scripts
d-----        04/05/2020   1:10 PM                adr                   # Architecture decision register
d-----        04/05/2020   1:05 PM                artifacts             # Build outputs
d-----        04/05/2020   1:05 PM                build                 # Build scripts
d-----        04/05/2020   1:05 PM                docs                  # Documentation markdowns
d-----        04/05/2020   1:05 PM                lib                   # Any non-package-managed libs
d-----        04/05/2020   1:05 PM                samples               # Examples and samples
d-----        04/05/2020   4:23 PM                src                   # Source code
d-----        04/05/2020   4:21 PM                test                  # Tests
-a----        03/09/2019   8:59 PM           5582 .gitignore
-a----        04/05/2020   4:22 PM           1833 LibraryTemplate.sln
-a----        04/05/2020   1:02 PM           1091 LICENSE
-a----        04/05/2020   3:16 PM            546 README.md
-a----        04/05/2020   1:08 PM              0 ROADMAP.md

So I spent the afternoon deep diving into dotnet templating and creating and publishing a nuget package to extend .NET new with my default open source project directory layout.

Installation and usage

You can install this template from the command line:

dotnet new -i ElectricHead.CSharpLib

and then can create a new project by calling

dotnet new csghlib --name ElectricHead.MySampleLibrary

Topology and conventions

This layout is designed for trunk based development, against a branch called dev. Merging to master, triggers publishing.

• Commit work to a branch called dev.
• Any commits will build and be tested in release mode by GitHub Actions.
• Merge to master will trigger Build, Test and Publish to NuGet
• You need to setup your NuGet API key as a GitHub Secret called NuGetApiKey

You need to use the csproj to update your SemVer version numbers, but GitHubs auto-incrementing build numbers will be appended to the build parameter in your version number, so discrete builds will always create unique packages.

The assembly description will be set to the SHA of the commit that triggered the package.

GitHub Actions

Pushing the resulting repository to GitHub will create builds and packages for you as if by magic. Just be sure to add your NuGet API key to your repositories Secrets from the Settings tab, to support publishing to NuGet.org

For the longest time, I've relied on, and recommended Azure's Kudu deployment engine as the simplest and most effective way to deploy web apps into Azure App services on Windows. Sadly, over the last couple of years, and after it's original author changed roles, Kudu has lagged behind .NET Core SDK versions, meaning if you want to run the latest versions of .NET Core for your webapp, Kudu can't build and deploy them.

We don't want to be trapped on prior versions of the framework, and luckily GitHub actions can successfully fill the gap left behind by Kudu without much additional effort.

Let's walk through creating and deploying an ASP Net Core 3.1 web app, without containerisation, to Azure App services and GitHub in 2020.

This entire process will take less than 5-10 minutes to complete the first time, and once you understand the mechanisms, should be trivial for any future projects.

Create Your WebApp

• Create a new repository on GitHub.
• Clone it to your local machine.
• Create a new Visual Studio Solution
• Create a new ASP.NET Core 3.1 MVC WebApplication

Create a new Azure App Service Web App

• Visit the Azure Portal
• Create a new web application
• Select your subscription and resource group

Instance details

Because we're talking about replacing Kudu for building our software - the Windows native deployment engine in AppServices, we're going to deploy just our code to Windows. It's worth noting that GitHub actions can also be used for Linux deployments, and Containerised applications, but this walkthrough is intended as a like-for-like example for Windows-on-Azure users.

• Give it a name
• Select code from the publish options
• Select `.NET Core 3.1 (LTS) as the runtime stack
• Select Windows as the operating system
• Select your Region
• Select your App service plan

Configure it's deployment

Now we're going to link our GitHub repository, to our AppServices Web app.

• Visit the Deployment Centre in for your newly created application in the Azure Portal.
• Select GitHub actions (preview)
• Authorise your account
• Select your repository from the list

A preview of a GitHub action will be generated and displayed on the screen, click continue to have it automatically added to your repository.

When you confirm this step, a commit will be added to your repository on GitHub with this template stored in .github/workflows as a .yml file.

Correct any required file paths

Depending on where you created your code, you might notice that your GitHub action fails by default. This is because the default template just calls dotnet build and dotnet publish, and if your projects are in some (probably sane) location like /src the command won't be able to find your web app by default.

Let's correct this now:

• Git pull your repository
• Customise generated .yml file in .github/workflows to make sure it's paths are correct.

In the sample I created for this walkthrough, the build and publish steps are changed to the following:

- name: Build with dotnet
  run: dotnet build ANCWebsite.sln --configuration Release

- name: dotnet publish
  run: dotnet publish src/ANCWebsite/ANCWebsite.csproj -c Release -o ${{env.DOTNET_ROOT}}/myapp
  

Note the explicit solution and CSProj paths. Commit your changes and Git push.

Browse to your site!

You can now browse to your site and it works! Your deployments will show up both in the Azure Deployment Center and the GitHub Actions list.

This whole process takes less than ~2-3 minutes to setup, and is reliable and recommended. A reasonable replacement for the similar Kudu "git pull deploy" build that worked for years.

How it works under the hood

This is all powered by three things:

• An Azure deployment profile
• A .yml file added to .github/workflows
• A GitHub secret

As you click through the Azure Deployment center setup process, it does the following:

• Adds a copy of the templated GitHub Actions Dot Net Core .yml deployment file from https://github.com/Azure/webapps-deploy to your repository
• Downloads the Azure publishing profile for your newly created Website and adds it as a GitHub secret to your GitHub repositories "Secrets" setting.
• Makes sure the name of the secret referenced in .github/workflows/projectname_branchname.yml matches the name of the secret it added to the respository.

The rest is taken care of by post-commit hooks and GitHub actions automatically.

You can set this entire build pipeline up yourself by creating the .yml file by hand, and adding your secret by hand. You can download the content for your Publish profile by visiting

Deployment Center -> Deployment Credentials -> Get Publish Profile

In the Azure portal.

It's an XML blob that you can paste into the GitHub UI, but honestly, you may aswell let Azure do the setup for you.

Next Steps

You've just built a simple CI pipeline that deploys to Azure WebApps without any of the overhead of k8s, docker, or third party build systems.

Some things you can now do:

• Consider running your tests as part of this build pipeline by adding a step to call dotnet test
• Add an additional branch specification to deploy test builds to different deployment slots

The nice thing about this approach, is that your build runs in a container on GitHub actions, so you can always make sure you're using the versions of tools and SDKs that you desire.

You can find all the code used in this walkthrough here.